This repository contains quantized GGUF versions of Meno-Lite-0.1. All variants were produced using an importance matrix computed on the train split of the rullmcalibration dataset, and are intended to be run with llama.cpp. Quality was measured on the test split of the Ru LLM Calibration dataset using the llama-perplexity utility. The original FP16 model served as the reference. How to interpret these metrics: — Mean PPL (Q): Lower is better. Shows the average perplexity of the quantized model. — PPL correlation: Closer to 100% indicates the quantized model behaves almost identically to FP16. Values above 99.5% are considered excellent. — Mean KLD: Measures the divergence between the output probability distributions. Lower is better; 0 means identical distributions. — Same top p: The percentage of tokens where the quantized model’s top prediction matches the FP16 model. Higher is better – it reflects how often the model’s first-choice token remains unchanged. For more details on available parameters, see the llama.cpp documentation. Meno-Lite-0.1 is a 7B model based on Qwen2.5, fine-tuned for RAG, document QA, information extraction, and knowledge graph construction. Read more…
Модальности:
Генерация текста
Области применения:
Диалог / чат RAG (поиск + генерация) Суммаризация Распознавание сущностей Ответы на вопросы
Задача: Генерация текста
Автор: bond005
Теги: gguf, rag, ner, information-extraction, summarization, question-answering, document-qa, long-context
Лайков: 3 | Загрузок: 694
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
