Inferno-Exp-1.6 is a 393.6M-parameter decoder-only base language model trained from scratch by AlgoDriveAI on about 4B tokens. It is a math-first model: its training emphasizes arithmetic and math word problems, and it is geared toward the AxiomicLabs small-language-model benchmarks used in SLM challenges, ArithMark 3.0. Future versions will build on this math foundation with improvements aimed at broader language understanding and logical reasoning. Parameter accounting: 32,833,536 tied token-embedding parameters + 360,775,680 parameters across 32 transformer blocks + 1,024 final normalization parameters = 393,610,240 unique trainable parameters. The tied LM head is not counted twice. This is a base model (no instruction tuning). The model code ships in the repo, so load it with trustremotecode=True. Same architecture family, about 5.17× the parameters and 32 layers instead of 14. Word-problem arithmetic and every language benchmark improve; bare-equation arithmetic (ArithMark 2.0) is lower than in Inferno-Exp-1.5. The easiest way to play with the model is a small web app with a prompt box, sampling sliders, live streaming, and a Stop button: 1. Open a new notebook at…
Модальности:
Генерация текста
Задача: Генерация текста
Автор: AlgoDriveAI
Теги: inferno, small-language-model, gqa, custom_code, arithmark, en, model-index
Лайков: 3 | Загрузок: 616
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
