Significant Milestones in ARM’s Initiative
Yesterday, ARM announced remarkable progress in its Total Design Initiative, a program launched a year ago with the vision of accelerating custom silicon development for data centers. This initiative encourages collaboration among industry partners, and it has expanded significantly, featuring nearly 30 participating companies. Recent additions to this impressive roster include Alcor Micro, Egis, PUF Security, and Semifive.
A Collaborative AI CPU Chiplet Platform
A key highlight of this progress is ARM’s partnership with Samsung Foundry, ADTechnology, and Rebellions to create a revolutionary AI CPU chiplet platform. This innovative collaboration aims to efficiently address workloads related to cloud computing, high-performance computing (HPC), and AI/machine learning (ML). By merging Rebellions’ AI accelerator technology with ADTechnology’s compute chiplet, implemented using Samsung’s cutting-edge 2nm GAA FET technology, they expect significant efficiency gains—reportedly a 2-3x improvement for generative AI workloads, particularly for models like Llama3.1, which boasts an impressive 405 billion parameters.
Supporting the Complete AI Stack
ARM emphasizes the crucial role of CPU compute in powering the entire AI stack, from data pre-processing and orchestration to advanced methodologies like retrieval-augmented generation (RAG). Their Compute Subsystems (CSS) are specifically designed to cater to these diverse needs, providing partners with a solid groundwork to develop various chiplet solutions. Notably, companies such as Alcor Micro and Alphawave have already unveiled plans to create CSS-powered chiplets tailored for AI and high-performance applications.
Moreover, the initiative places significant focus on software readiness, ensuring that major frameworks and operating systems seamlessly integrate with ARM-based systems. Recent advancements include the introduction of ARM Kleidi technology, optimizing CPU-based inference for popular open-source projects like PyTorch and Llama.cpp. As Google points out, a vast majority of AI workloads are currently being handled on CPUs, underscoring the necessity of developing the most efficient and performant CPUs for AI applications.
