· The Ensemble E4/E6/E8 are the industry’s first MCUs to provide hardware acceleration for transformer networks, enabling local generative AI inference on edge and endpoint devices
· Together with Arm, Alif demonstrated at the PyTorch Conference generative AI models compiled with the ExecuTorch Runtime running on the Ensemble E8
San Francisco, CA – Alif Semiconductor®, the leading global supplier of secure, connected, power efficient Artificial Intelligence and Machine Learning (AI/ML) microcontrollers (MCUs) and fusion processors, today announced that developers can now use the ExecuTorch Runtime, a quantization extension of the popular PyTorch ML framework, for AI applications built to run on its Ensemble E4/E6/E8 series of MCUs and fusion processors.
The ExecuTorch extension enables developers of AI models aimed at resource-constrained hardware targets such as MCUs to use the PyTorch framework to create lightweight models which support low-latency, high-accuracy inference. Since the Ensemble E4, E6 and E8 products include the Arm® Ethos™-U85 NPU, which supports transformer-based ML networks, developers can now use PyTorch and the ExecuTorch Runtime to build generative AI applications to run locally in battery-powered endpoint devices such as smart glasses, human-to-computer interfacing, healthcare and diagnostics, robotics, transportation, toys and education, smart homes, and smart city equipment.
At the PyTorch Conference (Moscone West, San Francisco CA, 22-23 October), Arm demonstrated examples of generative AI applications compiled with ExecuTorch and running on Alif’s Ensemble E8 fusion processor, including:
· A small language model running on the Ensemble E8, capable of generating stories suitable for children in response to visual prompts, and extendable to verbal prompts.
· Real-time on-device speech-to-text models, suitable for integration into wearables such as smart glasses. The demo performs real-time transcription of speech to enable live captioning.
Reza Kazerounian, President of Alif Semiconductor, said: “Alif’s continuous innovation keeps us at the forefront of edge AI, now extending into the era of generative AI. With native support for the ExecuTorch Runtime on our Ensemble microcontrollers, we’re expanding what’s possible for AI at the MCU level. For embedded device manufacturers building next-generation intelligent products, Alif remains the trusted leader in bringing AI to the edge.”
Paul Williamson, Senior Vice President and General Manager, IoT Business at Arm said: “Generative AI at the edge is enabling a new class of intelligent, battery-powered devices that can understand and respond in real time. Using the ExecuTorch framework, developers can efficiently deploy PyTorch models on Alif’s Ensemble MCUs for low-power, on-device inference. This collaboration demonstrates how a unified ecosystem can scale intelligence across the edge, accelerating real-world AI innovation.”
The Alif DK-E8 development board used in the PyTorch Conference demonstrations, which supports development on the entire E4/E6/E8 series, is available to purchase now.
To learn more about the Ensemble E4, E6, and E8 GenAI MCUs, visit alifsemi.com or email [email protected].
