OpenAI expands storage platform to support 1 billion ChatGPT users

OpenAI has expanded its storage platform, Habitat, to support over 1 billion users globally. Originally a Python library, Habitat has evolved into a distributed system handling more than 70 million requests per second. Launched in mid-2024, the platform now manages over 500 petabytes of data across 40 regions. The team faced challenges scaling the system to meet demand, requiring a shift from a client-side library to a centralized service. This change allowed for better control over deployments and security, while also enabling more efficient management of storage resources like Azure Cosmos DB. Despite performance trade-offs, the decision to keep Habitat in Python was strategic, leveraging advancements in coding models to simplify future migrations. The transition to a service model improved reliability and reduced operational complexity. However, it introduced new challenges, such as managing tail latencies and ensuring efficient resource utilization. By optimizing asyncio scheduling and load balancing, the team mitigated issues like high latency and uneven request distribution. These efforts highlight the complexities of scaling infrastructure to support massive user growth while maintaining performance and security standards.