Imagine real-time translation on a wearable, or continuous health sensing, running entirely on the device. No round-trip to the cloud, and power so low the battery runs for days.
That’s the direction Edge AI is heading, and my colleagues at Google Research just took a big step toward it.
The problem has always been fragmentation. Edge AI is split across hundreds of proprietary chips and toolchains, so ML engineers end up rewriting kernels, re-quantizing models and debugging vendor compilers for every target. It’s painful.
This week, the team completed the open 32-bit Coral NPU architecture, an ML-first RISC-V design, and released the Coral Compiler v1.0, built on IREE/MLIR. You can take a PyTorch, JAX or TensorFlow model, optimize it once, and deploy it across silicon built on the standard. Synaptics is already shipping it: the Astra SR200 MCU (400 GOPS) for battery-powered wearables, and the 1 TOPS Coralboard SL for robotics and IoT.
The new Matrix Execution Unit scales up to 1 TOPS and targets about 10mW average power for inference. For perspective, that’s around 1% of what your phone draws when it’s active. Yet it’s enough to bring compact transformers like Gemma into devices that used to handle only simple tasks like keyword spotting.
All of this is already real: on Synaptics Coralboard, their open-source Torq toolchain runs Gemma 3 270M at up to 12 tok/s (INT8) and 16 tok/s (4-bit). That’s faster than most people read, fully on-device.
Why this matters to me: in places where connectivity is unreliable and power is expensive, AI that runs locally isn’t a nice-to-have. It’s a game changer. Edge-first, with the cloud as a fallback for heavy reasoning, is the right architecture for most of the world.
🎓 To the CS and engineering students in my network: study the architecture diagram below. On the left, a scalar RISC-V core runs ordinary C/C++ control logic. On the right, vector and matrix engines run the neural network math, all in the same pipeline. That’s the world you’re graduating into, where software engineering and AI are fused down to the silicon. The RTL and compiler are open source, so go read them. 🛠️
To entrepreneurs in my region and across the Global South: pay attention to this, and if you’re not a techie, share it with one who is. The future is full of opportunities.
Coral NPU (RTL & architecture): https://github.com/google-coral/coralnpu
Coral Compiler v1.0: https://github.com/google-coral/coralnpu-compiler
Blog: https://developers.googleblog.com/unifying-edge-ai-how-coral-npus-open-standard-brings-portable-energy-viable-intelligence-from-model-to-silicon/
Synaptics (incl. Gemma benchmark): https://www.synaptics.com/company/news/synaptics-and-google-research-expand-the-coralboard-ecosystem
Coralboard SL on DigiKey: https://www.digikey.com/en/products/detail/synaptics-coralboard/720-KDVSL2619-11/29784108