Derivative of Qwen3.8-27B, quantized using MagicQuant hybrid evolutionary per-tensor search. Sibling repo with AMD-native (ROCmFPX fork-only) builds: lmcoleman/Qwen3.8-27B-ROCmFPX-GGUF. This is a derivative of Qwen3.8-27B. All credit for the base model architecture and weights goes to the original authors. The base model’s license applies to this derivative. Quantized using MagicQuant hybrid evolutionary per-tensor quantization, based on the methodology by magiccodingman: — Tensors are classified into sensitivity groups (Embeddings, Head, Query, Key, Output, FFN Up/Down, MoE Experts, Router) — An evolutionary search finds the optimal quantization type per group, balancing size vs. perplexity — Q4/Q5/Q6 tier targets are searched, and each one ships only if it earns its place (see below) — Small-row tensors and sensitivity-critical layers (embeddings, output head, router) are kept at F32/F16/BF16 — This is NOT a uniform quantization — each tensor group gets its own optimal type A tier name here is a size band, not a promise that every tensor uses that exact type. A «Q5» is whatever mix of schemes landed in the Q5 size band with the lowest measured perplexity loss — which is the…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: lmcoleman
Теги: llama.cpp, gguf, quantized, magicquant, en, endpoints_compatible, conversational
Лайков: 3 | Загрузок: 175
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
