AMD acquires Taalas to accelerate AI inference with specialized chips

AMD acquired startup Taalas as part of its effort to challenge Nvidia's dominance in AI hardware. The company etches machine learning model weights directly into silicon, enabling significantly higher inference throughput. Initial tests demonstrated the HC1 chip's ability to serve Meta's Llama 3.1 8B model at 16,960 tokens per second using TSMC's 6nm fabrication process, achieving 48 times higher throughput than Nvidia GPUs and 8.5 times that of Cerebras accelerators when launched. The upcoming HC2 chip, expected this summer, will support 20 billion parameters. AMD's strategy involves pairing Taalas accelerators with its Helios rack infrastructure, where Instinct GPUs focus on computationally intensive prompt processing while offloading token generation to Taalas-based processors.