Edge AI: Accelerating On-Device Intelligence and Transforming Consumer and Enterprise Markets

In 2020, analysts forecasted over 750 million edge AI chips—devices that perform or accelerate machine learning directly on the hardware—would be sold, generating approximately US$2.6 billion in revenue. The edge‑AI segment is expected to outpace the broader semiconductor industry, with unit sales projected to exceed 1.5 billion by 2024, implying an annual growth rate of at least 20% versus a 9% CAGR for the overall market.

These chips are increasingly found in high‑end smartphones, tablets, smart speakers, wearables, and a growing array of enterprise equipment, including robots, cameras, sensors, and general IoT devices.

While the consumer segment remains larger, it is projected to grow at a 18% CAGR from 2020 to 2024, whereas the enterprise segment is expected to accelerate at 50% over the same period.

In 2020, consumer devices represented more than 90% of the edge‑AI chip market by volume and value. High‑end smartphones accounted for over 70% of the consumer units in use, and more than a third of the 1.56 billion smartphone shipments that year were likely to include an edge‑AI chip.

Historically, AI workloads were handled in data centers because of their computational intensity. Edge‑AI chips, however, are smaller, less expensive, consume far less power, and produce minimal heat—enabling their integration into handheld devices and robots. By processing data locally, they reduce latency, improve usability, and enhance privacy and security by keeping personal information on the device.

When device resources are insufficient, a hybrid approach—combining on‑device and cloud processing—can be employed. The optimal mix depends on the specific AI workload.

Figure 1: Locations where intelligence can be embedded (Image: Deloitte Insights)

The economics of edge AI in smartphones

Although smartphones dominate the market, other categories—tablets, wearables, and smart speakers—also incorporate edge‑AI chips. In 2020, tablets were largely stagnant, while smart speakers and wearables together were expected to ship only 125 million units, limiting their impact on overall sales. Nonetheless, penetration rates for these devices are already high.

Figure 2: The Edge AI chip market (Image: Deloitte Insights)

Currently, only the most expensive smartphones—those in the top third of the price spectrum—tend to feature edge‑AI chips, yet the incremental cost is minimal.

By examining silicon die shots from Samsung, Apple, and Huawei, we can approximate the share of a SoC devoted to AI. Samsung’s Exynos 9820, for example, dedicates roughly 5% of its die area to an NPU. With a total SoC cost of about US$70.50, the AI portion represents roughly 5% of that cost, or about US$3.50 per chip.

Figure 3: A die shot of Samsung’s Exynos 9820 showing ~5% AI area (Image: ChipRebel, Annotation: AnandTech)

Apple’s A12 Bionic allocates about 7% of its die to machine‑learning, translating to an estimated cost of US$5.10. Huawei’s Kirin 970 dedicates 2.1% of its area to the NPU, suggesting a cost between US$1.10 and US$1.42, depending on whether area or transistor count is used as the metric.

Figure 4: Apple’s A12 Bionic dedicates ~7% of its die to machine‑learning (Image: TechInsights / AnandTech)

Even with a wide cost range, an average of US$3.50 per NPU is a reasonable estimate. Multiplied across half a billion smartphones—plus tablets, speakers, and wearables—the market volume is substantial. Importantly, adding a $1 manufacturing cost typically translates to a $2 price increase for consumers, allowing edge‑AI NPUs to appear in phones priced as low as US$250 without a significant margin impact.

Sourcing AI chips: in‑house or third‑party?

Device makers choose sourcing strategies based on product tier and geography. Companies like Qualcomm and MediaTek supply SoCs to many manufacturers, capturing roughly 60% of the smartphone SoC market in 2018. Their higher‑end lines—Snapdragon 845, 855, and MediaTek’s Helio P60—usually incorporate edge‑AI capabilities.

Apple, by contrast, designs its own SoCs (A11–A13 Bionic) and embeds AI throughout. Samsung and Huawei adopt a hybrid model, purchasing some SoCs from third parties while deploying their own Exynos and Kirin chips for higher‑tier devices.

Over 50 AI accelerator companies vying for edge AI in enterprise and industry

While smartphone‑grade chips perform well for consumer use, many enterprise and industrial applications require more focused, power‑efficient solutions. An NPU that occupies only the AI portion of a SoC uses 95% less power and is significantly cheaper than the full SoC.

Today, more than 50 firms are developing standalone edge‑AI accelerators. In 2019, developers could purchase single chips for roughly US$80 each. In bulk, manufacturers can acquire them for as little as US$1 to a few dollars. Using the smartphone NPU as a proxy, we’ll assume an average cost of US$3.50.

Standalone chips are not only inexpensive but also small and low‑power, drawing between 1 and 10 W. In contrast, a 16‑GPU data‑center cluster costs US$400 k, weighs 350 lb, and consumes 10 kW.

Edge AI empowers enterprises to process data locally, reducing the need to transfer vast datasets to the cloud, cutting costs, simplifying architecture, and mitigating security risks.

Data security and privacy

Local processing minimizes exposure to cyber threats and regulatory scrutiny. For example, smart cameras can analyze footage on‑device, sending only relevant segments to the cloud, while smart speakers can refine wake‑word detection to reduce unnecessary audio capture.

Low connectivity

Devices that operate in network‑poor environments—such as autonomous drones—benefit from on‑device AI, enabling real‑time decision making without relying on connectivity.

Massive data volumes

IoT deployments can generate terabytes of data daily. Edge AI can filter and pre‑process this data, transmitting only actionable insights, thereby reducing storage and bandwidth costs.

Power constraints

Low‑power accelerators enable small‑battery devices, like inhalers that analyze usage patterns locally and send concise metrics to a smartphone. Efforts such as Google’s TensorFlow Lite for microcontrollers push deep learning into micro‑controllers, enabling “tiny machine learning.”

Low latency requirements

On‑device inference eliminates round‑trip latency to data centers, which can be critical for autonomous vehicles, robotics, and other safety‑critical applications.

The bottom line: edge AI will be vital for data‑heavy applications

Consumers will benefit from features like secure device unlocking, offline voice assistants, and high‑quality photography without constant connectivity. For enterprises, edge AI can unlock new business models, improve operational efficiency, and transform industries such as manufacturing, logistics, agriculture, and energy by enabling real‑time, on‑device analytics.

Duncan Stewart and Jeff Loucks are with Deloitte’s Center for Technology, Media and Telecommunications. This article is based on an article originally published by Deloitte for its TMT Predictions 2020 report.