sz14/cRia-LM-75M-Instruct - Каталог нейросетей
Генерация текста

sz14/cRia-LM-75M-Instruct

Добавлено:
sz14/cRia-LM-75M-Instruct

> Cria (noun): a baby llama, alpaca, vicuña, or guanaco. Pronounced ˈkrē-ə. > > ~ Merriam-Webster cRia-LM-75M-Instruct is a 75.7M-parameter instruction-tuned language model built as a Relaxed Recursive Transformer (RRT). It uses a shared 11-layer recurrent block evaluated twice, with pass-specific LoRA parameters on the second traversal. The model starts from cRia-LM-75M, then adds supervised instruction tuning and preference optimization. It ships with a native chat template using and markers. cRia-LM-75M-Instruct keeps the base model’s 13 unique Transformer layers: This gives an effective depth of 24 Transformer layers while storing the main parameters for 13 unique layers. The recurrent block shares its base weights across both passes. Rank-172 LoRA updates provide separate capacity on the second pass. The full architecture and base-model history are documented on the cRia-LM-75M model card. Post-training starts from cRia-LM-75M. Base-model training details are kept on that model card. The first part uses the full training split of HuggingFaceTB/smol-smoltalk. Only assistant turns contribute to the language-model loss. HuggingFaceTB/SmolLM2-360M-Instruct supplies…

Модальности:
Генерация текста

Области применения:
Диалог / чат Следование инструкциям


Задача: Генерация текста
Автор: sz14
Теги: rrt, recursive-transformer, small-language-model, knowledge-distillation, custom_code, instruct-tuned, dpo, conversational
Лайков: 3  |  Загрузок: 226

Открыть на HuggingFace →

Описание основано на материалах HuggingFace. Перевод выполнен автоматически.