MiniBananaMind-v3-9M is a small causal language model trained from scratch on FineWeb-Edu and FineMath. The model has about 8.9M parameters and uses a custom 8k-token byte-level BPE tokenizer with digit-aware tokenization. This is a base language model, not an instruction-tuned chat assistant. Digits are kept as separate tokens so numbers do not collapse into large number tokens during tokenization. The training mix used both general educational web text and math-heavy text. Formal benchmark results for this checkpoint are not included yet. This model uses custom architecture code, so load it with trustremotecode=True. — dosample=False — repetitionpenalty=1.1 — maxnewtokens=64 to 128 — dosample=True — temperature=0.6 — topp=0.9 — repetitionpenalty=1.1 — maxnewtokens=64 to 128`
Модальности:
Генерация текста
Области применения:
Математика
Задача: Генерация текста
Автор: BananaMind
Теги: minibanana, causal-lm, language-model, small-language-model, bananamind, fineweb-edu, finemath, math
Лайков: 4 | Загрузок: 48
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
