Deploying AIoT on Microcontrollers: A Practical Guide

In a recent post I explored how artificial intelligence is increasingly woven into Internet‑of‑Things devices, giving rise to intelligent AIoT solutions that learn from data and make autonomous decisions. These systems exhibit human‑like interactions with their surroundings, enhancing product functionality and user experience.

AI and IoT together have unlocked fresh markets for microcontrollers (MCUs). By pairing simple MCUs with AI acceleration, developers can embed intelligence into devices that once relied solely on basic sensors. Modern AI‑enabled MCUs combine digital signal processing (DSP) for compute with machine‑learning (ML) inference, enabling applications such as keyword spotting, sensor fusion, vibration analysis, and voice recognition. Higher‑performance MCUs now support vision and imaging tasks—face recognition, fingerprint analysis, autonomous robots—bringing what was once exclusive to MPUs and GPUs to the edge.

AI Technologies in Edge Devices

Machine Learning (ML): ML algorithms build models from representative data, allowing devices to detect patterns automatically. Vendors supply algorithms, APIs, and training tools that enable embedded systems to run pre‑trained models for inference. Common use cases include sensor hubs, keyword spotting, predictive maintenance, and classification.

Deep Learning: A subset of ML that trains multilayer neural networks to extract high‑level features from complex inputs. Deep learning excels with large, diverse datasets, continually improving with each iteration. Applications span image processing, chatbots, and face recognition.

Natural Language Processing (NLP): NLP enables systems to understand and respond to human language—text or speech—making decisions based on that input. It powers speech recognition, machine translation, and predictive typing.

Computer Vision: This field trains machines to capture, interpret, and act on image data. By leveraging deep learning, systems can identify and classify objects in real‑time. Use cases include manufacturing fault detection, medical diagnostics, retail face recognition, and autonomous vehicle testing.

AIoT on MCUs

Historically, AI resided in MPUs and GPUs, where large memory and powerful cores could handle heavy workloads. The shift toward edge intelligence now places AI on MCUs, driven by the need for lower latency, reduced bandwidth, and tighter power budgets. Recent MCUs now feature enough compute and memory to host lightweight neural network (NN) frameworks tailored for constrained devices, making real‑time inference feasible on the edge.

A neural network consists of layers of nodes that process weighted inputs to produce outputs. During training—typically performed in the cloud—data flows through the network, and back‑propagation adjusts weights to minimize prediction error. Once trained, the fixed weights are stored in flash memory, allowing the MCU to perform inference with minimal RAM.

Inference requires far less compute than training, making MCUs ideal hosts for pre‑trained models. The weights remain static, and the MCU can deliver instantaneous decisions based on new sensor data.

Implementing AI on MCUs

The typical workflow involves four stages:

1. Data Capture: Collect extensive datasets from the target device or application.
2. Model Training: Use cloud‑based tools from AI vendors (e.g., TensorFlow Lite, Caffe) to train a neural network that satisfies application requirements.
3. Model Conversion: Convert the trained model to a flat‑buffer format with the vendor’s converter, optionally applying quantization to reduce size. The flat buffer is then translated into C code and bundled into the MCU’s firmware.
4. Deployment & Updates: Flash the firmware onto the MCU. When new data classes appear, the model can be retrained in the cloud and updated on the device via over‑the‑air (OTA) firmware upgrades.

There are two common architectural paths for MCU‑based AI solutions, assuming an Arm Cortex‑M core:

1. Software‑Only Execution: The flat‑buffer model runs on the Cortex‑M CPU, accelerated by CMSIS‑NN libraries. This lightweight setup is suitable for tasks like keyword spotting and vibration analysis.
2. Hardware‑Accelerated Execution: Incorporate a micro neural processing unit (u‑NPU) or NN accelerator. The u‑NPU handles most common audio, speech, image classification, and object detection workloads, falling back to the CPU for unsupported operations. This approach delivers higher throughput and lower power consumption.

As MCUs evolve toward the performance envelope of MPUs, we anticipate the emergence of on‑device learning—lightweight training algorithms running directly on the MCU. This will open new markets and spur significant investment from MCU manufacturers.

AI on the Edge Is the Future

Edge AI on resource‑constrained MCUs will grow exponentially, unlocking new applications as performance boundaries blur between MCUs and MPUs. The proliferation of thin NN models designed for embedded contexts will drive adoption across industries.