Digital Sabotage, Flawed Optimizers, and AI for Human Flourishing

Researchers uncovered Fast16, a virus over two decades old, engineered to sabotage high-precision calculation software used in civil engineering, physics, and environmental modeling. The malware introduced systematic errors into simulations, degrading critical scientific and technological programmes. The discovery raises concerns about how advanced actors might use similar techniques to constrain technological progress. Tilde Research identified significant flaws in the Muon optimizer, an algorithm used to train large language models. The tool causes neurons to die in multi-layer perceptron layers during learning rate warmup, compromising model quality. The team developed Aurora, a leverage-aware optimizer for rectangular matrices, as a solution. In experiments with 1.1 billion-parameter transformers, Aurora outperformed Muon, with particularly strong gains on knowledge-intensive benchmarks like MMLU. The AI alignment research community is proposing positive alignment, a paradigm extending beyond traditional safety research. Rather than focusing solely on harm prevention, this framework aims to build systems that actively support human and ecological flourishing. Researchers from Oxford, Google DeepMind, OpenAI, Anthropic, and other institutions advocate for decentralized governance reflecting the plurality of human values instead of centralized control. Prime Intellect researchers demonstrated that contemporary language models can autonomously optimize the training of other models, setting records in optimization challenges. However, these systems show clear limitations: while competent at hyperparameter search and optimization methods, they lack the creativity to generate truly novel ideas.