🪨 ONYX Adaptive Precision Engine This repo contains ONYX Quants of nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16 ⚙️ The ONYX Architecture 🧠 Dynamic Layer Sensitivity Replaces hardcoded edge boundaries with real activation variance measurements. ONYX autonomously identifies critical layers (like mid-network attention blocks) and protects them dynamically. 🎯 Router-Weighted Imatrix Captures MoE router probabilities and multiplies them into activation scales. This forces the quantizer to aggressively crush «cold» experts while fiercely protecting «hot» ones within the same tensor block. 🏗️ Architecture-Agnostic Dynamically reads HuggingFace modules and the generated F16 GGUF to map tensors. No hardcoded regex. Works out-of-the-box on Llama, DeepSeek, and custom hybrid SSM/MoE architectures. Tier Name Target Quality Target Size Middle Layer Strategy 🪨 qualityQ8 Match~28.2 GBIQ4XS ⚖️ balancedQ6 Match~28.4 GBQ5K 📦 compactQ4 Match~20.1 GBQ3K 🚀 miniQ2 Match~17.9 GBIQ2XXS 📚 Credits & Foundations 👉 APEX Quantization MethodEttore Di Giacinto & Richard Palethorpe (LocalAI Team). ONYX evolves the layer-wise precision gradients and MoE-aware tensor classification outlined in the…
Модальности:
Генерация текста
Области применения:
Логика и рассуждение Диалог / чат
Задача: Генерация текста
Автор: el4
Теги: gguf, ollama, llama.cpp, imatrix, moe, reasoning, frontier, qwen3
Лайков: 3 | Загрузок: 106
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
