Gopher

Gopher marked DeepMind’s first flagship LLM release, scaling a standard Decoder-only Transformer to very large size and training it on a filtered, book- and web-heavy dataset to boost factual coverage and style control. The model showed strong gains on open-domain QA, reading comprehension, and general knowledge, with decent coding and writing skills for its time. It also exposed limits that later work addressed, such as arithmetic and step-by-step reasoning, brittleness under distribution shift, and safety concerns like biased or hallucinated content. Follow-on research, including compute-optimal scaling and data strategy studies, built on lessons from Gopher and influenced the next generation of models that traded sheer parameter count for better token budgets and efficiency.