> 📖 Research White Paper: 📄 Read Whitepaper (PDF) | 🌐 View Online | 📝 Full MI300X Serving Blog High-efficiency, hardware-benchmarked GGUF releases of DeepSeek-V4.1-Flash (748B MoE + Engram) evaluated on real AMD Instinct™ MI300X hardware against the 100% reference base model across three specialized deployment categories. Generated artifacts — shipped in artifacts/ as proof-of-work: Raw run data (benchmark JSON + server log): evidence/run20261005/` Standard low-bit quantization collapses 384-expert MoE models because 2-bit quantization flips gating decisions. Our engine introduces Decision Surface Consistency (DSC) to eliminate the cliff: — 🟢 zralddeepseek-v4.1-accuracy (Q4KM): Enterprise Zero-Tolerance Workhorse. Holds 92.88% – 99.06% accuracy retention with full code pass-rate fidelity. — 🔵 zralddeepseek-v4.1-balance (Q3KM): The Pareto Sweet Spot. Retains 86.32% – 97.41% accuracy retention while cutting memory footprint by 150 GB. — 🟣 zralddeepseek-v4.1-compressed (Q2KDEEPSEEK): The 2-Bit Cliff Slayer. Completely eliminates the 33% cliff, locking the router at Q80 ( > This repo ships the compressed production rung only (the 7 compressed- GGUF shards…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: Zrald
Теги: gguf, llama.cpp, quantization, deepseek, moe, mi300x, endpoints_compatible, conversational
Лайков: 3 | Загрузок: 423
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.