Edge AI: Why Processing Is Shifting to the Device Layer

Key takeaways:

• The rapid rise of connected devices and data volumes has made distributed computing indispensable for speed, performance, and data security.
• Strategic moves by NVIDIA and VMware now enable data‑intensive AI and machine‑learning workloads to run directly at the edge.
• Edge processing offers tangible benefits across a spectrum of industries—from industrial IoT to consumer applications.

With billions of sensors and smart devices worldwide, traditional centralized cloud architectures are reaching their limits. While cloud computing revolutionized startup agility and enterprise scalability, emerging workloads—such as high‑definition video, autonomous vehicles, and real‑time logistics—demand sub‑second latency and secure, on‑premises data handling.

“Customers are realizing that offloading too much processing to the cloud is no longer viable,” says Markus Levy, head of AI technologies at NXP Semiconductors, in a recent discussion on embedded AI. “The edge is where the real opportunity lies.”

Market data from MarketsandMarkets projects the edge‑computing market to grow from $3.6 billion in 2020 to $15.7 billion by 2025—a more than four‑fold expansion driven by the need for low‑latency, high‑bandwidth processing.

Market Moves Accelerate Edge Adoption: NVIDIA

NVIDIA’s $40 billion acquisition of Arm in September 2020 is a landmark step toward bringing GPU‑accelerated AI closer to the device. While NVIDIA has long dominated data‑center GPUs, Arm’s dominance in mobile processors—supplied to Apple, Qualcomm, Broadcom and others—provides a powerful platform for edge AI.

Zeus Kerravala, analyst at an industry research firm, notes, “If you see AI as the future, powered by GPUs and CPUs, NVIDIA’s ability to build end‑to‑end systems has just surged—especially in the edge computing arena.”

Regulators are closely monitoring the deal for potential antitrust implications, given Arm’s extensive licensing network and NVIDIA’s growing influence in mobile CPU markets.

SmartNICs: Moving Intelligence to the Edge

At VMworld 2020, VMware unveiled Project Monterey—a collaboration with NVIDIA that leverages SmartNICs and NVIDIA’s BlueField DPUs to accelerate AI workloads at the edge. By offloading hypervisor, networking, security and storage functions from the host CPU to the DPU, Monterey frees up to 30 % of CPU cycles for application logic.

Alexander Harrowell, senior analyst at Omdia, observes, “The trend is clear: moving computation closer to data improves performance and resource utilization. SmartNICs shift the hypervisor’s responsibilities to the network, unlocking a Harvard‑style computing model.”

Greg Lavender, VMware’s senior VP and CTO, explains, “Offloading I/O and compute to the Arm core in SmartNICs liberates CPU resources, giving applications additional compute and memory to operate faster.”

For the connected‑things ecosystem, this shift enables real‑time AI—turning passive IoT devices into intelligent, responsive systems, as Levy emphasizes.

For more coverage on Industrial IoT, join Industrial IoT World this December.