> Note: New Millennium Artificial Intelligence (NMAI) has been renamed > OpenCerebral. The organization, models, and maintainers are unchanged — > only the name is new. Older references to NMAI (including the previous > KSP-NMAI repository paths) refer to OpenCerebral. Boris-1.3-125M is a 125 million-parameter language model created by OpenCerebral. It extends the original Boris-125M base checkpoint with additional continued pretraining aimed at closing gaps found in Boris-125M’s own benchmark results (see Continued pretraining below). This is a base (pretrained) model. It has not been instruction-tuned and does not follow instructions or hold a conversation — it continues text. For an instruction-following version, see opencerebral/Boris-1.3-125M-Instruct. The Boris-125M base checkpoint was trained on 2.50B tokens of FineWeb-Edu for 33h 38m 48s on one RTX 3060. Boris-125M’s benchmark results showed the same FineWeb-Edu-driven gap seen at 75M. Boris-1.3-125M adds seven sequential continued-pretraining passes on top of the base checkpoint, each with a re-warmed learning rate, extending total training by roughly 2.66B tokens (~106% more than the original 2.50B-token pretraining…
Модальности:
Генерация текста
Задача: Генерация текста
Автор: opencerebral
Теги: gpt2, boris, opencerebral, 125M, en, text-generation-inference, endpoints_compatible
Лайков: 4 | Загрузок: 454
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
