Edge AI Gains Momentum: Cloud Leaders AWS and Microsoft Show Integrated Inference & Management Solutions

Deploying AI directly on edge devices delivers instant responses and cuts the cost of sending large data streams to the cloud. Recent hardware advances—dedicated ML accelerators, powerful microcontrollers, and refined models—enable richer inference within tight power budgets. Yet, even a fully local pipeline still benefits from cloud‑based device management, continuous model training, and secure update mechanisms.

AWS IoT

At Embedded World Digital 2021, AWS and Microsoft highlighted how their ecosystems support edge AI. AWS Principal Specialist Solutions Architect Rajeev Muralidhar explained that while running TensorFlow Lite or other ML models on the device is one capability, the real advantage lies in managing the entire device lifecycle securely and at scale. He emphasized the importance of rolling out firmware and model upgrades that keep fleets both up‑to‑date and secure.

AWS delivers this capability through the AWS IoT platform, which is composed of three core components: device‑side software (FreeRTOS or AWS Greengrass), control and connectivity services (AWS IoT Core), and analytics and event‑driven services.

AWS offers comprehensive support for edge AI devices through its IoT platform (Image: AWS)

FreeRTOS is an open‑source OS designed for microcontrollers and is backed by a two‑year long‑term support (LTS) program that guarantees security patches, feature upgrades, and bug fixes. Its OTA update framework lets operators push firmware changes to thousands of devices securely and at scale. The FreeRTOS kernel can communicate directly with a gateway running AWS Greengrass, creating a seamless bridge between local inference and cloud services.

AWS IoT Core serves as the entry point for data flowing into the cloud. It provides a message broker that can enforce rules on incoming data—storing, routing to a database, or sending it to analytics tools such as SageMaker for machine learning. The platform also offers fleet‑management tools, device‑management capabilities, and event‑driven automation that detect anomalies and trigger responses across a large network of IoT endpoints.

Muralidhar highlighted that secure, end‑to‑end device life‑cycle management is essential when operating at scale. Rotating security credentials and applying operating‑system updates without compromising fleet integrity are key to maintaining a trustworthy ecosystem.

He envisions a continuous loop where data sent to the cloud is used for model refinement, after which newer, more accurate models are pulled back onto devices. “That way, the devices running in your vehicles or on the shop floor are more capable and can react faster and more accurately,” he said.

Azure Percept

Microsoft’s Azure Percept is a hardware‑and‑software platform designed to make edge AI accessible to developers without deep specialization. The system leverages Azure’s cloud services for device management, model development, and analytics, while providing pre‑built hardware modules that incorporate cutting‑edge AI accelerators.

Microsoft’s Azure Percept platform combines hardware modules—including a Trusted Platform Module, an audio module, and a vision module—with Azure’s cloud‑backed services (Image: Microsoft)

The current hardware development kit features the Azure Percept Vision module, which uses Intel’s Movidius Myriad X accelerator for computer‑vision workloads, and an Azure Percept Audio module, details of which are forthcoming. The vision module is designed to simplify training, deploying, and managing AI models directly at the edge.

Microsoft also ties the platform to Azure IoT Hub, ensuring secure, bidirectional communication between devices and the cloud. Future plans include expanding the range of third‑party Percept‑certified devices, allowing developers to port solutions from the development kit to production‑ready hardware.

>> This article was originally published on our sister site, EE Times Europe.