A 9.9M-parameter general-purpose base model trained from scratch on 10.16B tokens, as a plain Llama (LlamaForCausalLM): no custom code. It succeeds TinyBrainBot-10M-Base and is built like TinyBrainBot-3M-v2-Base: web text, Q&A, facts and commonsense first, with arithmetic as a small part of the mix. Zero-shot, full test sets, accnorm. The (raw acc)* row shows plain accuracy. The Intelligence Index is the Open SLM Leaderboard’s; the other models’ numbers come from that leaderboard. Arithmetic sets the model never trained on (Gemma and Llama were never used to write training data), all acc_norm: GGUF files for llama.cpp, LM Studio and Jan: tinybrainbot-10m-v2-base-F16.gguf (20 MB) and tinybrainbot-10m-v2-base-Q80.gguf` (11 MB). This is a base model: it continues text and does not chat. At this size expect short, simple text. 10.16B tokens: 8.57B of pretraining at constant learning rate, then a 1.59B-token decay. The arithmetic comes from several sources so the skill is not tied to one wording: drills generated by our own programs (number formats, number words, whitespace variants, negatives, order of operations, tables, unknown positions, distractors; every answer computed in…
Модальности:
Генерация текста
Задача: Генерация текста
Автор: nkthebass
Теги: gguf, llama, tiny, small-language-model, from-scratch, en, model-index, text-generation-inference
Лайков: 3 | Загрузок: 571
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
