MongoDB - Build AI That Scales
Mini toolsSubQ 1.1 Small
By Subquadratic
SubQ 1.1 Small uses Subquadratic Sparse Attention (SSA), replacing O(n2) dense attention with a content-dependent sparse formulation that scales linearly. Trained primarily at 1M tokens with staged extension to 2M, it generalizes retrieval to 12M tokens without retraining. Key benchmarks: GPQA Diamond 85.4%, LiveCodeBench v6 pass@4 89.7%, RULER 99.12% at 128K, Needle-in-a-Haystack 100% at 1M and 2M. Continued pretrained on approximately 1 trillion tokens of long-form artifacts including books, documents, and repository-scale code. Designed for workloads requiring reasoning over complete artifacts such as full codebases, financial filings, and legal contracts. At 1M tokens, requires 64.5x less compute than dense attention and runs 56x faster than FlashAttention-2. Results independently verified by Appen. Currently deploying with select design partners.
New
Text
Gen 7
Overview
SubQ 1.1 Small is a long-context language model built on Subquadratic Sparse Attention (SSA), a learned sparse mechanism with linear compute cost. It supports context windows up to 12M tokens with near-perfect retrieval, using 64.5x less compute than dense attention at 1M tokens and running 56x faster than FlashAttention-2.
About Subquadratic
Industry:
Artificial Intelligence
Company Size:
16
Location:
Wilmington, Delaware, US
Website:
subq.ai
Tools using SubQ 1.1 Small
No tools found for this model yet.
Last updated: June 17, 2026
