Startups pursue alternatives to transformers for next-generation language models

A wave of startups is investigating next-generation language models beyond the dominant transformer architecture. While transformers have been fundamental to current large language model success, researchers have identified fundamental limitations that are being worked around through partial solutions in recent models. Startups like Subquadratic are exploring alternative architectures that might resolve these technical bottlenecks. While some of these efforts will fail, the wave of innovation represents an opportunity for smaller companies to compete as the field evolves beyond transformer dominance.