Meta and academic researchers launch AI models to prioritize machine learning experiments

Meta's FAIR research team, in collaboration with the University of Oxford and University College London, has introduced AI Research Preference Models (RPMs) to help prioritize machine learning experiments before investing GPU resources. These models rank unexecuted candidates and select the most promising one for execution, avoiding the unreliable predictions of language models. The RPMs use frozen pretrained large language models, open-source frameworks like AIRA-dojo and AIRS-Bench, and the Qwen3.6-27B model as their backbone. They employ an evolutionary tree search approach, generating multiple candidates in parallel and selecting the best through pairwise comparisons. Two variants were tested: an inference-only RPM with 57.7% to 59.0% offline accuracy, and an agentic RPM that runs small-scale experiments before deciding on the next step. Results showed a 59.23% and 59.13% probability of improvement over the No-RPM baseline, with efficiency gains of 1.61× and 1.55× respectively. The agentic RPM achieved a new state-of-the-art score of 94.1% on WinoGrande, surpassing the previous 90.4%, while the inference-only RPM reached 95.7% on SVAMP, outperforming the prior human SOTA of 94.2%.