Wispr Flow - Don’t type, just speak
Mini toolsSequential Hidden Decoding 8B n8
By Tencent
Sequential-Hidden-Decoding-8B-n8 is Tencent’s n=8 implementation of Sequential Hidden Decoding, a method for scaling sequence length by interleaving eight independent embeddings of the same token sequence into a longer input for the same Transformer model. The Hugging Face card lists Qwen3-8B-Base as the base model, 5.6B additional embedding parameters, 187B training tokens, and states that the model is not instruction-tuned, but intended for benchmarking, text completion, and downstream fine-tuning. The method uses preceding hidden embeddings as continuous latent reasoning steps, and the card reports gains over the 8B baseline on BBH, MMLU, MATH, GSM8K, and other benchmarks.
Text
Gen 7
Overview
Sequential-Hidden-Decoding-8B-n8 is Tencent’s base language model variant that extends effective sequence length by 8x using multiple embedding passes while keeping the same Transformer backbone. Built on Qwen3-8B-Base, it is aimed at benchmarking, text completion, and fine-tuning research, with hidden latent steps acting like implicit reasoning before each predicted token.
About Tencent
Tencent is a world-leading internet and technology company that develops innovative products and services to improve the quality of life of people around the world.
Industry:
Technology, Information and Media
Company Size:
110558
Location:
Shenzhen, CN
Website:
tencent.com
Last updated: March 31, 2026
