GGUF + importance-matrix quantizations of Nex-N2-mini — the Qwen3.5-35B-A3B MoE (~3B active, multimodal reasoning) by the Nex-AGI team — tuned and measured for fast local inference on the Intel Arc Pro B70 (Battlemage) via llama.cpp’s SYCL backend. The model is Nex-N2-mini, built by the Nex-AGI team — the reasoning, multimodal capability, and training are entirely theirs; this is an independent community quantization that only repackages their model to run fast on Intel Arc, with sincere thanks to Nex-AGI for releasing it openly. The base model is © Nex-AGI (Apache-2.0), post-trained on Qwen3.5-35B-A3B-Base © Qwen Team (Apache-2.0); these quants are redistributed under Apache-2.0 and the names identify the upstream work. Code, kernel patch, full methodology, and reproducible benchmarks: https://github.com/newjordan/NexN2-B70-Turbo Recommendation: Q5KM (best accuracy under the Q6K reference, near-fastest) or Q4KM for max speed. On SYCL, Q4K / Q5K carry the optimized kernels and lead the decode column — they’re the speed picks. Accuracy is KL-divergence + top-1 agreement vs a Q6K reference (PPL 6.572, wikitext-2, 100 chunks). For vision, add mmproj-f16.gguf`. Q5KM/Q4KM decode…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: Frosty40
Теги: gguf, llama.cpp, quantized, imatrix, moe, intel-arc, sycl, nex-n2
Лайков: 3 | Загрузок: 199
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
