ROCmFP4 and ROCmFPX quantizations of MiMo-V2.5 (310B total / 15B active MoE), built for AMD Strix Halo (Ryzen AI Max+ 395, gfx1151) with the ROCmFPX fork of llama.cpp. There are two ways to run this model on Strix Halo, and they want different files: The single-node option being faster is not a mistake. llama.cpp’s RPC layer split runs the nodes sequentially, so a second box buys capacity, not speed. If the model fits on one node, one node wins. > Note on quality. The ROCmFP4 and ROCmFP4FAST variants are not > imatrix-calibrated; Unsloth’s UD-Q4KXL is, and is tensor-aware on top of that. > They win on speed and size — benchmark both if marginal output quality matters > more to you than tok/s. > > Q20ROCMFPX-COHERENT is imatrix-calibrated (it has to be — see below). But > it is 2.5 bpw against FAST’s 4.26, and that reduction is real. It passes coherence > checks on code, factual recall, arithmetic and JSON, but no pass@1 evaluation has > been run, so treat the quality gap as unquantified rather than absent*. Q40ROCMFP4 and Q40ROCMFP4FAST are quantization types defined by the ROCmFPX fork. Upstream llama.cpp, Ollama, LM Studio and every downstream that vendors mainline ggml cannot…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: christopher-kapic
Теги: gguf, rocm, rocmfp4, rocmfpx, strix-halo, gfx1151, moe, endpoints_compatible
Лайков: 3 | Загрузок: 43
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
