Unofficial 4-bit GPTQ quantization of Altworld/Hemmingway-1 for low-VRAM inference, with the MTP (multi-token-prediction) draft head deliberately kept unquantized. Produced with gptqmodel 7.3.2 on an Intel Arc Pro B70 (Xeon/e-core host, 30.3 GiB VRAM). This model is not affiliated with, endorsed by, or supported by Altworld, hemmingway.io, or Alibaba Cloud. It is a community quantization; the Apache-2.0 terms of the base models below continue to apply (see License and attribution). The quantizeconfig.json follows the same contract as the community reference artifact SergiioB/Qwen3.8-27B-GPTQ-Int4-sym-G128-MTP-BF16 (W4A16, g128, sym, descact=false, MTP excluded), applied here to the Hemmingway-1 fine-tune instead of the base Qwen3.8-27B. — All quantizable linear projections of the language model → GPTQ INT4 (W4A16, g128, sym, descact=false). — mtp. draft tensors (15) kept in their original BF16 dtype; the dynamic exclusion {«-:.mtp.»: {}} in quantizeconfig.json records this. The source’s separate model-mtp.safetensors shard is merged into the output checkpoint (model-00005-of-00005.safetensors; confirmed via model.safetensors.index.json). — Added: quantizeconfig.json, quantlog.csv…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: bjonor
Теги: qwen3_5_text, gptq, int4, w4a16, 4-bit, quantized, vllm, intel-xpu
Лайков: 3 | Загрузок: 275
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
