Microsoft introduces local AI model support for GitHub copilot

Microsoft announced at a press event in San Francisco on October 7 that GitHub Copilot will support local AI model inference by the end of October. Developers will be able to switch automatically or manually between cloud and device-based models. GitHub Copilot, an AI coding assistant developed by Microsoft, integrates with tools like Visual Studio Code and CLI, offering suggestions for code completion, explanation, and refactoring based on context. Currently, Copilot relies on cloud-hosted models, with requests routed by a coordinator based on performance, cost, and accuracy. Microsoft plans to expand this system to allow Copilot to automatically select between cloud and local models. The company also introduced MAI Code 1.1 Flash, a hybrid model with 137 billion total parameters and 68 billion active parameters, optimized for speed and memory usage through quantization and speculative decoding. On the Surface Laptop Ultra, the model achieved a peak throughput of up to 63 tokens per second during decoding tests. The local model support includes options for specifying providers, models, or endpoints, with developers able to use Windows ML to select MAI Code 1.1 Flash or connect to OpenAI-compatible local endpoints. Microsoft emphasized that the model’s performance in complex tasks showed significant improvements in memory usage, token efficiency, and processing speed. IT Home provided images of the test results, highlighting the model’s capabilities.