SmallAICreator/GRAFT-10B-A2B-GGUF - Каталог нейросетей
Генерация текста

SmallAICreator/GRAFT-10B-A2B-GGUF

Добавлено:
SmallAICreator/GRAFT-10B-A2B-GGUF

GRAFT-10B-A2B is a ~10B-parameter mixture-of-experts chat model from UltraLabs. Only about 2B parameters are active per token, so it generates text at roughly the speed of a 2B model while carrying the knowledge of a much bigger one. It was made by shrinking Qwen3-30B-A3B-Instruct-2507 to a third of its size with GRAFT, our own model-shrinking method, and then healing, distilling and chat-tuning the result. — Made by: UltraLabs (SmallAICreator) — Format: GGUF, GRAFT-10B-A2B-Q4KM.gguf (6.27 GB) — Architecture: Qwen3-MoE, 48 layers, 40 experts (5 active per token), hidden size 2048 — Chat template: ChatML with tool calling (embedded in the GGUF) — Context: 32K tokens (perfect 60/60 needle-in-a-haystack recall all the way to 32K, see below) — Language: English first — License: Apache 2.0 Use a system prompt. Without one, the model sometimes introduces itself with its base model’s name. A one-line system prompt fixes that: ChatML. Tool calls use the format, and the embedded template handles OpenAI-style tools`. 60/60: every code returned exactly, at every depth, all the way to 32K tokens. GRAFT-10B played a full game of Go Fish against GPT-OSS-20B and won 9 books to 4. How it played…

Модальности:
Генерация текста

Области применения:
Диалог / чат


Задача: Генерация текста
Автор: SmallAICreator
Теги: gguf, qwen3_moe, mixture-of-experts, llama.cpp, conversational, tool-calling, graft, ultralabs
Лайков: 3  |  Загрузок: 576

Открыть на HuggingFace →

Описание основано на материалах HuggingFace. Перевод выполнен автоматически.