ahtx/AH-Nanbeige4.3-33B-IQ3_XXS-GGUF - Каталог нейросетей
Генерация текста

ahtx/AH-Nanbeige4.3-33B-IQ3_XXS-GGUF

Добавлено:
ahtx/AH-Nanbeige4.3-33B-IQ3_XXS-GGUF

An IQ3_XXS quantization of Nanbeige4.3-33B by AH, made with bartowski’s imatrix. A 33B agent model in one 13.1 GB file. Our goal is high accuracy for agents on small, low-cost systems: tool calls, terminal work and code on one consumer GPU. This file is a quantization of Nanbeige/Nanbeige4.3-33B by the Nanbeige team. We did not change the weights by training. Their work makes this release possible. The importance matrix (imatrix) is from bartowski. Thank you, bartowski. We quantized it from bartowski’s Q80 with bartowski’s imatrix, and we used a Q5K output tensor so that Kindi can run it. Then we measured each candidate file against the original model, token by token. We kept the file that stayed closest to the original at this size. This is our initial test: 115 tasks, one seed per task, the same settings for both models. — kbench: 65 terminal and coding tasks for agents. The model works in the pi agent harness, and hidden tests check each result. — 50 recent LiveCodeBench problems. Both models are almost at the limit of kbench. The model has much more room to show its strength on harder tests. We are making those tests now. With llama.cpp (use —jinja for the chat template and…

Модальности:
Генерация текста

Области применения:
Диалог / чат


Задача: Генерация текста
Автор: ahtx
Теги: gguf, llama.cpp, iq3_xxs, agent, tool-calling, kindi, endpoints_compatible, imatrix
Лайков: 3  |  Загрузок: 487

Открыть на HuggingFace →

Описание основано на материалах HuggingFace. Перевод выполнен автоматически.