Openai's Habitat platform scales to serve over 1 billion users with azure cosmos db

OpenAI has scaled its Habitat storage platform to serve over 1 billion users, leveraging Azure Cosmos DB to manage unprecedented growth. The platform, which supports more than 500 petabytes of data, was developed to provide fast and reliable access to information for OpenAI products. Habitat evolved from a Python client into a globally distributed storage system, addressing challenges in reliability, performance, and scalability. The company faced significant challenges with Azure Cosmos DB, including noisy neighbor issues and throughput limitations. Initially, OpenAI used a single container, but this led to performance bottlenecks. They shifted to a multi-tenant model, allowing for better resource management and isolation. However, this approach required careful planning and led to operational complexities. In 2024, a regional outage highlighted the need for closer collaboration with Azure. OpenAI now operates one of the largest Azure Cosmos DB accounts, storing tens of petabytes of data and handling over 20 million queries per second. The partnership with Azure has been crucial in managing the scale and complexity of their storage needs.