Imatrix-quantized GGUF build of AlexWortega/SIQ-1-35B — a reasoning/agentic finetune of Qwen3.6-35B-A3B (MoE, 35B total / ~3B active, 256 experts, Qwen3 template). Quantized with an importance matrix and tuned for fast local inference on Intel Arc. > 🙌 The model is AlexWortega’s — AlexWortega/SIQ-1-35B. > This repo is only a fast GGUF build of it. The reasoning, the agentic behavior, all of it is his post-training. > If you use this, go ★ the original — it deserves the reach. > This release ships Q5KM`** — the quant that holds all measurable reasoning (20/20 gate) while > full-offloading on a 30 GiB B70. Q4 / Q6 / Q8 builds are evaluated and will be > added to this repo later. Measured on the B70: flash-attention on, q80 KV cache, full offload (ngl 99`). — +9.8% from the Turbo fusions: decode 85.2 → 93.6 t/s vs stock llama.cpp SYCL, same GGUF weights (fusions on/off, 20/20 reasoning gate held). This is the controlled engine win. — Trained context 256K; decode holds at depth (81 t/s @16K). — Prefill +53% — -ub 4096 un-starves the MoE expert GEMMs: 874 → 1334 t/s at 131072 ctx / q80 KV (20/20 reasoning gate held). +73% at small-ctx / f16-KV bench. — Deterministic decode, free —…
Модальности:
Генерация текста
Области применения:
Логика и рассуждение Диалог / чат
Задача: Генерация текста
Автор: Frosty40
Теги: gguf, llama.cpp, imatrix, moe, reasoning, qwen3, endpoints_compatible, conversational
Лайков: 3 | Загрузок: 50
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
