Please refer to https://huggingface.co/uzlm/alloma-1B-Instruct for updated model. Our Llama-3.2-1B-Instruct-uz (experimental) model has been continually pretrained with context length of 2048 tokens, on 1.2B tokens (80% English, 20% Uzbek), then SFT fine-tuned. Our customized tokenizer averages 1.7 tokens per Uzbek word vs. ~3.5 in the original Llama models, meaning 2x faster inference and longer effective context length on Uzbek text. You’ll be able to run this model on just 2 GB of VRAM (with quantization), perfect for small GPUs, edge devices, or even mobile scenarios. —————————— | —-: | —-: | —-: | —-: | —-: | —-: | —-: | The results show that our Uzbek-optimized models consistently outperform their base counterparts in translation benchmarks (BLEU and COMET) on the FLORES+ Uz-En / En-Uz evaluation datasets and sentiment analysis in Uzbek language. Also, on the MMLU benchmark, which measures general language understanding across multiple tasks in English, and News classification tasks, our Uzbek optimized model showed slight decline because of catastrophic forgetting of original English instruction following. (The official Llama model’s MMLU…
Модальности:
Генерация текста
Области применения:
Диалог / чат Перевод Суммаризация Ответы на вопросы Следование инструкциям
Задача: Генерация текста
Автор: bxod
Теги: llama, uzbek, uzbekllm, uzbeknlp, translation, summarization, question-answering, tokenizer
Лайков: 4 | Загрузок: 44
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
