Hemmingway-1-GPTQ-Int4-sym-G128-MTP-BF16
Unofficial 4-bit GPTQ quantization of Altworld/Hemmingway-1 for low-VRAM inference, with the MTP (multi-token-prediction) draft head deliberately kept unquantized....
Unofficial 4-bit GPTQ quantization of Altworld/Hemmingway-1 for low-VRAM inference, with the MTP (multi-token-prediction) draft head deliberately kept unquantized....