Machine Learning Frameworks and Model Standards

Reading Time: 3 minutes

Submitted by Oliver Hamilton – Director of Computer Vision – COSMONiO

Machine learning has almost been around nearly as long as the computer itself [samuel-53]. The 1998 paper by Yann LeCun [lecun-98] lay the groundwork for deep learning, however the processing power was still severely lacking to make it a reality so it all went a bit quiet. In 2012 Alex Krizhevsky kicked things off again [AlexNet], with thanks to the gaming industry pushing GPU technology to new highs, it was trivial to get hold of incredible number crunching power in any PC.

Since 2014 we’ve seen the number of machine learning frameworks explode. Check out this chart from Andrej Karpathy (Director of AI at Tesla) showing the unique mentions of deep learning frameworks in arxiv papers over time to the start of 2018.

Continuing on this analysis we can clearly see PyTorch has seen an incredible uptake by the research community over the last 2 years.

[Source] Framework mentions on arxiv papers. Data from RISELab, graphic by Ben Lorica.

While the number of available frameworks has been fantastic for researchers, allowing them to jump-ship to whichever one currently offers the best features or support for experimentation, this is far from ideal for industry. For companies to leverage cutting edge models coming out of academia, they would also have to change frameworks just as often – a scary prospect if you have a large established product or codebase.

For image processing hidden behind an API this is less important, the backend can be switched out with little difference. Deploying machine learning models to the edge is a different story. In these situations, the model usually has to be optimized to run on a fixed hardware platform in order to meet performance requirements such as number of images processed per second, size, weight, power consumption and heat output. 

With all these different frameworks and hardware solutions we need to develop one universal standard…wait.

The good news is that things aren’t quite that bad.

People saw this coming, there are now converts such as Microsoft’s MMdnn and Intel’s OpenVINO that allow for models from all of the major platforms to be converted into a common format and back again in some cases.

While this is a significant step in the right direction to create a standardised model representation, it is still early days with caveats.

Some network architectures rely on mathematical operations that don’t easily optimise or run very well on specialised hardware. OpenVINO tackles this issue by executing different parts of the network on different bits of hardware, for example falling back to the CPU for the layers that are not supported by specialised accelerator hardware.

However, if your edge deployment device is lacking the hardware resources required to fallback to you could still be limited in which models you can deploy.

Get started quickly with OpenVINO, Intel® Movidius Myriad X VPU and ADLINK Edge software.

To get started quickly with OpenVINO try the ADLINK Vizi-AI an EdgeAI Machine Vision DevKit – The Vizi-AI starter devkit includes an Intel Atom® based SMARC computer module with Intel® Movidius Myriad X VPU and 40 pin connector, Intel® Distribution of OpenVINO and ADLINK Edge™ software.

Stay in touch

Sign up to our email list to be notified of the latest industry news