With Google officially announcing Cloud IoT Edge and Edge TPUs at Google Next 2018, all major cloud players are now providing a machine learning service on the edge. It remains to be seen whether or not the IoT market is ready for extensive ML offerings at the edge, but major industry support is starting to drive an ecosystem centered around edge processing.

As cloud computing pushes ML to new heights with automated feature engineering and AutoML, what do these IoT and ML offerings mean for the future of ML on the edge?  

AWS Greengrass

Being the first to market back in 2016, AWS leveraged its rich AWS Lambda ecosystem to create an event-driven, function-focused approach to Edge Intelligence. Earlier this year, AWS Greengrass ML Inference also became generally available, allowing users to deploy machine learning models on AWS Greengrass devices.

In terms of features, AWS is still unmatched, providing local execution of Lambda functions, IoT device shadow capabilities (AWS IoT Core), local messaging, and Over-The-Air (OTA) updates. It’s interesting to note that Greengrass doubled-down on Lambda instead of supporting Docker containers. In typical AWS fashion, Greengrass ML Inference took the route of providing many pre-built ML frameworks (TensorFlow, MXNet, Chainer, Caffe2, and even Microsoft Cognitive Toolkit).

This approach seems to focus on bringing models that used to run in the cloud down to the edge. From my experience working at Leverege, most ML applications in IoT, outside of the typical deep-learning use cases (i.e. NLP, computer vision), are running simple linear regression and classification models. AWS seems well-suited to encourage companies running such models to migrate to the edge, especially if they were using AWS Lambda or SageMaker.

Azure IoT Edge

Shortly after AWS Greengrass went GA, Microsoft announced a partnership with Qualcomm at Microsoft Build to complement its Azure IoT Edge and Azure Machine Learning capabilities with Qualcomm’s AI hardware. Unlike AWS, Azure focused on supporting container-based modules for deployment.

This may have been a great strategic move to allow customers to quickly containerize their applications (which arguably is easier than converting the code into functions) and test/train both in the cloud (perhaps with kubeflow) then deploy it to Azure IoT Edge (now even with Kubernetes support).

The debate between containers and functions are outside the scope of this article, but Azure’s container approach, complemented by its decision to open source the platform, further signal Microsoft’s strategy in engaging with more partners to grow the edge ecosystem.

Another important announcement from Build 2018 was the launch of Project Brainwave, a new deep learning acceleration platform designed for ultra-low latency use cases. Project Brainwave reaffirms Microsoft’s commitment and belief that running ML models on FPGAs through industry partners (e.g. Intel Stratix FPGA) will be a superior approach than building custom chips specialized for ML on the edge. Of course, this approach is uniquely possible given Microsoft’s business model and the dominance it held in the PC era.

Cloud IoT Edge

Last to the party was Google. At Next 2018, Injong Rhee announced Cloud IoT Edge and Edge TPUs. Cloud IoT Edge mostly extends Google’s IoT Core services and allows ML inferences of TensorFlow Lite models on AndroidThings OS or Linux. Although the features for Google’s IoT offerings are lacking in comparison to AWS and Azure, Google’s announcement of Edge TPUs, its custom ASIC chips specialized for running ML models, reveal its big plans to eat the entire AI stack.

It’s no secret that Google’s ML talent and services are vastly superior. The ecosystem that it has created with open-sourcing Tensorflow and acquiring Kaggle cannot be overlooked in attracting those researchers to use Cloud IoT Edge, especially if they are already using Cloud ML Engine to train models in the cloud with TPUs or simply using its vast suite of ML products.

Matthew Lynley explains in detail in his TechCrunch post how Google is using its custom chip and TensorFlow lite for faster processing and less power consumption. We will have to wait until Edge TPUs are available to actually compare performance with Azure’s FPGAs, but if Google is right, their vision for the edge may go beyond simply extending cloud capabilities to the edge as they seek to dominate the entire AI/ML stack from hardware to software.