Linkup unveils SPARSEUP: a 149m-parameter sparse embedding model for open-source use

Linkup Research has introduced SPARSEUP, an open-source sparse embedding model with 149 million parameters, designed for efficient retrieval tasks. The model is built on a ModernBERT backbone and released under the Apache 2.0 license, with weights available on Hugging Face. SPARSEUP achieves an average nDCG@10 score of 56.4 on the BEIR-13 benchmark, positioning it as the strongest public vocabulary-based sparse encoder under 150M parameters. Unlike dense models, SPARSEUP generates vectors over a vocabulary, mapping each dimension to a real token, which aligns with inverted indexes and improves readability. The model was developed in response to LightOn’s DenseOn and LateOn releases, filling the gap in sparse retrieval methods. Training involved LateOn-unsupervised checkpoints, with fine-tuning using LightOn’s mixture and contrastive learning, resulting in a single H100 GPU training setup. On BEIR-13, SPARSEUP outperforms DenseOn in certain tasks like ArguAna and Touché but lags in semantic sets such as FiQA. On decontaminated BEIR, the gap to DenseOn narrows to 0.17 points, though results on NQ and MS MARCO are considered noisy due to limited query counts. SPARSEUP also achieves high recall with Seismic search, reaching over 97% in 380 microseconds per query.