A 4-bit ROCmFP4STRIX` quant of poolside/Laguna-S-2.1 (118B-A8B), built and measured on an AMD Ryzen AI Max+ 395 (Strix Halo, gfx1151, 128 GB unified memory). 58.4 GiB — Laguna-S-2.1-Q40ROCMFP4STRIX.gguf — 4.27 achieved bpw** Companion to raulvidis/Laguna-S-2.1-ROCmFP4-COHERENT-GGUF. Prefer this one: identical measured quality, 3.4 GiB smaller, slightly faster. ROCmFP4 uses GGUF tensor types 100/101, unknown to upstream llama.cpp. You need charlie12345/ROCmFPX. Use the chat template from the base repo’s auxiliary files. Note yarnattnfactor must be 1.0 (poolside’s value) — unsloth-lineage Laguna GGUFs carry 1.4852, which is wrong; this build has the correct value baked in. INVALID LANGUAGE PAIR SPECIFIED. EXAMPLE: LANGPAIR=EN|IT USING 2 LETTER ISO OR RFC3066 LIKE ZH-CN. ALMOST ALL LANGUAGES SUPPORTED BUT SOME MAY HAVE NO CONTENT Without the norm actually applied in the encoder graph, the drafter fuses six unnormalized* feature blocks: output is still…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: raulvidis
Теги: gguf, rocm, rocmfp4, strix-halo, gfx1151, speculative-decoding, dflash, endpoints_compatible
Лайков: 4 | Загрузок: 155
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
