q-project/Q-164M - Каталог нейросетей
Генерация текста

q-project/Q-164M

Добавлено:
q-project/Q-164M

Q-164M A 164.6M-parameter ternary text model with exact tool calls > [!IMPORTANT] > This is a research release, not a product. It is small and often factually wrong outside its trained > tool-calling domain. See Results below and the > blog post before using it for anything beyond > experimentation. Q-164M is a 164,648,648-parameter decoder-only model. Its weights are ternary ({-1, 0, +1}, quantization-aware from step one of training), and instead of a trained embedding table it uses a frozen 512-bit fingerprint per token plus a small learned correction, covering the whole 32,832-token vocabulary. A fixed circuit intercepts expr spans during generation and force-decodes the exact result from a deterministic evaluator — the model’s job is only ever to decide when and which tool to call (arithmetic, dates, unit conversion, sort, compare, gcd/lcm, primality, and 12 other kinds), never to compute the answer itself. Pretrained from scratch on a single 16GB V100, no cluster: 200,000 steps, ~92.4 hours, 13.1B tokens, zero NaNs over the whole run. An instruction-tuned version, Q-U-164M, is available as its own repo — chat-SFT then RL, fine-tuned specifically for dialogue. Code for both is…

Модальности:
Генерация текста

Области применения:
Вызов функций (Tool use)


Задача: Генерация текста
Автор: q-project
Теги: qagent, causal-lm, custom-architecture, ternary-weights, bitnet, frozen-embeddings, exact-circuits, tool-use
Лайков: 3  |  Загрузок: 162

Открыть на HuggingFace →

Описание основано на материалах HuggingFace. Перевод выполнен автоматически.