Novi-Nano-Instruct is a tiny instruction-tuned causal language model developed by Novi-AI. It is based on Novi-Nano-Base and fine-tuned on a small instruction dataset to experiment with instruction following and conversational behavior at an extremely small scale. ⚡ 1.26M parameters · 500 training examples · 256-token context Novi-Nano-Instruct was trained from Novi-Nano-Base using a small instruction dataset containing 510 examples. Training loss was applied specifically to the assistant responses, allowing the model to focus on learning how to respond to user instructions. Because the validation set contains only 10 examples, these metrics should be considered experimental rather than a comprehensive benchmark. Novi-Nano-Instruct uses the custom tokenizer developed for Novi-Nano. The original tokenizer vocabulary was 8,192 tokens, with additional tokens already present in the tokenizer. 🔬 Research and experimentation 🧪 Small-model instruction-tuning experiments 🎓 Educational purposes 💬 Tiny conversational-model experiments 💻 Lightweight local inference 🛠️ Experimenting with extremely small instruction-tuned models As an experimental 1.26M-parameter model, it is not intended…
Модальности:
Генерация текста
Области применения:
Диалог / чат Следование инструкциям
Задача: Генерация текста
Автор: Novi-AI
Теги: gpt2, novi, novi-nano, novi-nano-instruct, causal-lm, from-scratch, instruction-tuning, chatml
Лайков: 3 | Загрузок: 568
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
