Better quant here: https://huggingface.co/el4/GRM-3.2-Sky-ONYX-GGUF Tier Name Target Size Middle Layer Strategy 💎 quality~23 GBIQ4XS ⚖️ balanced~25 GBQ5K 📦 compact~18 GBQ3K 🚀 mini~14 GBIQ2S 🔬 micro~11 GBIQ1M 🤏 nano~12 GBIQ2XXS 🛡️ exp-minimalist~16 GBIQ2_XXS (Protected Edges) 📚 Credits 👉 APEX Quantization MethodEttore Di Giacinto & Richard Palethorpe (LocalAI Team). OPAL adapts the layer-wise precision gradients and MoE-aware tensor classification outlined in the APEX technical paper. 👉 Bartowski and LamimFor the excellent semantic imatrix calibration dataset that powers OPAL’s activation scaling. 👉 llama.cppGeorgi Gerganov and contributors for the foundational inference and quantization engine. 👉 HuggingFace AccelerateFor the initemptyweights() context manager that makes the 0-RAM «Ghost Model» possible. 👉 JackrongFor the sexy markdown readme inspo. Support the Project A coffee in Ethereum would be cool! Although I don’t drink coffee—I think it tastes like burnt water—but a pink lemonade would be fire! 🔥 0xDEE7fa8C421BD038D32e4441ea1aDe72fE973706
Модальности:
Генерация текста
Области применения:
Логика и рассуждение Диалог / чат
Задача: Генерация текста
Автор: el4
Теги: gguf, llama.cpp, imatrix, moe, reasoning, qwen3, opal, endpoints_compatible
Лайков: 3 | Загрузок: 121
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
