A 2-billion-parameter dense hybrid language model pretrained from scratch on 500B–2T tokens. BOREAL-2B is the first model in the BOREAL family intended for actual downstream use — the one you download, fine-tune, quantize, and build on. It carries forward the Gated DeltaNet architecture validated by BOREAL-250M and scales it to a size where benchmarks become meaningful. Targets competitive performance against Qwen3-1.7B and SmolLM2-1.7B while offering native 64K context — 4x what pure Transformers at this scale can practically support. Built on Crucible — RUPS skyline weighting + EESD submodular diversity selection with formal (1 — 1/e) guarantees — and the DDM analyzer that models reasoning as Ornstein-Uhlenbeck evidence accumulation. The same pipeline that produced Harmonic-9B and Ornstein-27B, where 799 DDM-curated rows outperformed datasets 20–100x larger. BOREAL-2B targets parity with Qwen3-1.7B while supporting 4x the native context length and using roughly half the inference memory at long context. Every model trained in Canada. Every weight learned from random init. Apache 2.0. Built for Canadian researchers, startups, and institutions. No strings. No API keys. No foreign…
Модальности:
Генерация текста
Задача: Генерация текста
Автор: GestaltLabs
Теги: boreal, deltanet, hybrid, linear-attention, swiglu, rmsnorm, rope, gqa
Лайков: 3 | Загрузок: 0
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
