This is quantized version of Lyte/Llama-3.2-3B-Overthinker created using llama.cpp — Training Data: This model was trained on a dataset with columns for initial reasoning, step-by-step thinking, verifications after each step, and final answers based on full context. Is it better than the original base model? Hard to say without proper evaluations, and I don’t have the resources to run them manually. — Context Handling: The model benefits from larger contexts (minimum 4k up to 16k tokens, though it was trained on 32k tokens). It tends to «overthink,» so providing a longer context helps it perform better. — Performance: Based on my very few manual tests, the model seems to excel in conversational settings—especially for mental health, creative tasks and explaining stuff. However, I encourage you to try it out yourself using this Colab Notebook. — Dataset Note: The publicly available dataset is only a partial version. The full dataset was originally designed for a custom Mixture of Experts (MoE) architecture, but I couldn’t afford to run the full experiment. — Acknowledgment: Special thanks to KingNish for reigniting my passion to revisit this project. I almost abandoned it after my…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: QuantFactory
Теги: gguf, text-generation-inference, unsloth, llama, trl, sft, en, endpoints_compatible
Лайков: 3 | Загрузок: 373
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
