Official llama.cpp prebuilt binaries without this compatibility patch will fail to load this GGUF. Reason: unpatched official llama.cpp builds cannot recognize the pruned Step35 / Step 3.7 MoE layout where the routed expert count differs by layer. They still read step35.expertcount as a single u32`, while this pruned MoE stores expert counts as a per-layer array. For example, llama-b9892-bin-win-vulkan-x64 fails during hyperparameter loading with: This is not a corrupted-shard issue and is not caused by the MTP sidecar. Use a patched Step 3.7 / LynnStyle-compatible build, or apply the patch and rebuild. If you are not comfortable rebuilding llama.cpp, give the Patch and Usage notes links below to Codex or another coding agent, and ask it to apply the patch to your target llama.cpp source tree and rebuild. Reference runtime: llama.cpp 8c146a8366304c871efc26057cc90370ccf58dad; the release tests used CUDA llama-server built from the patched llama.cpp tree. For newer official source trees or binaries such as b9892, apply the same patch logic and rebuild; re-downloading GGUF shards will not fix this error. > Important GGUF parser note: Hugging Face / ModelScope GGUF auto-detection may…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: nerkyor
Теги: llama.cpp, gguf, lynnstyle, moe, quantization, glm52, gpt55, endpoints_compatible
Лайков: 3 | Загрузок: 243
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
