Baseten and partners launch open-weight AI safety initiative

Baseten, an AI inference provider, has launched a new safety infrastructure standard through its Base Labs research division, collaborating with Hugging Face and Goodfire AI to develop safety evaluation and monitoring tools for open-weight models. The initiative comes as debates intensify over the safety risks of open models, which can be compromised through a technique called abliteration, with Hugging Face listing over 6,000 such models. Base Labs, established earlier this year, aims to create a transparent standard for open models, integrating safety measures into training and deployment rather than adding them later. The company emphasized that openness enhances AI safety by increasing model transparency and enabling actionable controls. Goodfire, specializing in model interpretability, is likely responsible for embedding safety features into the models. Baseten, which raised $1.5 billion in June, is inviting developers to contribute to the framework, aiming to build a safe and accessible ecosystem for open models. The partnership underscores the growing importance of safety in the open-source AI landscape.