MeltdownQ8.gguf is a 3B local agent model, exported as Q80 GGUF. Runs fully offline — no API key required at inference time. Use temperature 0.2 and load systemprompt.txt as the system message. OpenAI-compatible API: http://127.0.0.1:8080/v1/chat/completions`. Measured locally on NVIDIA RTX 3060 12GB with lm-evaluation-harness v0.4.12 and llama-server (20 examples per task). Frontier scores are vendor-published references (see frontierreference.json`). While the benchmark scores themselves are low, it is due to image-related benchmarks. For chat, the APE format overrides the base chat template, which is why it scores low in chat. In day to day agentic workflows, with abbreviated MCP tools and unified instructions, Meltdown performs exceptionally. Escpecially while utilizing local Agentic harnesses like iRun’s ReAct and APE. Meltdown is optimized for local agent work (privacy, zero API cost, offline use), not for beating frontier models on broad knowledge benchmarks. Derived from Qwen2.5-Coder-3B-Instruct. See the base model license.
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: iRunStudio
Теги: gguf, llama-cpp, tool-calling, agent, local, qwen2, endpoints_compatible, conversational
Лайков: 3 | Загрузок: 16
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
