> Original fine-tune by yuxinlu1 (gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2); this repo is an NVFP4 (MLX) requant for fast local inference on Apple Silicon. A mixed-precision NVFP4 conversion of the Gemma‑4‑12B agentic fine‑tune (optimized for tool‑use / τ²‑bench). The weights are plain MLX safetensors; built for and tested with Krill — a pure Swift + MLX inference engine for Apple Silicon (no Python, no GGUF at inference). Loads in ~1.7 s at ~6.8 GB. — Bulk weights are NVFP4 (4‑bit float, groupsize 16); attention oproj and the vision/audio projectors are kept at 8‑bit affine (groupsize 64). This «protected» mixed recipe recovers the quality uniform 4‑bit loses on those sensitive modules while keeping 4‑bit speed and size. — Conversion:** bf16 safetensors → key‑remap to MLX layout → NVFP4 requant. No GGUF round‑trip. This is an MLX checkpoint — not GGUF, not a HF/transformers checkpoint. To load it an engine needs (1) the gemma4unified architecture (text+vision+audio) and (2) the mixed‑precision NVFP4 config (top‑level nvfp4 + per‑module 8‑bit overrides). Today that means Krill; it is not drop‑in for vanilla mlxlm/mlxvlm, and not** loadable by llama.cpp/Ollama (GGUF) or…
Модальности:
Генерация текста
Области применения:
Диалог / чат Вызов функций (Tool use)
Задача: Генерация текста
Автор: srv-sngh
Теги: mlx, gemma4_unified, nvfp4, krill, gemma4, apple-silicon, agentic, tool-use
Лайков: 3 | Загрузок: 214
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
