İvme (Turkish: acceleration) is a series of stupidly small language models built to punch above their weight. Ivme-Coder-v1 is the first release in a new branch of the family: a ~50M parameter decoder-only model trained from scratch on Python source code. Coder’s architecture deliberately mirrors the same core recipe as the Conversate line — RoPE, SwiGLU, RMSNorm, tied embeddings, Muon-optimized body weights. The point of keeping the recipe constant across both lines is to isolate what changes with domain-specific data and a narrower target distribution, not to introduce a second set of untested architectural choices at the same time. Unlike a general-text model, code has a far narrower and more rigid target distribution — Python’s syntax is small, unambiguous, and repetitive relative to open-domain English. That changes the token math: Coder deliberately targets well past strict compute-optimal scaling for a model this size, but nowhere near the scale a general-purpose model in the same parameter class would need, since there is comparatively little left to learn once syntax, common idioms, and standard-library usage patterns are well covered. Trained on an RTX PRO 6000 Blackwell…
Модальности:
Генерация текста
Области применения:
Генерация кода
Языки программирования:
Python
Задача: Генерация текста
Автор: IvmeLabs
Теги: ivme_coder, language-model, transformer, rope, swiglu, muon, from-scratch, tiny
Лайков: 3 | Загрузок: 129
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
