NVFP4 GGUF quantizations of google/gemma-4-12B-it, for use with llama.cpp. The dense FFN tensors (all 48 layers × 3 projections = 144 tensors) are quantized to NVFP4 — NVIDIA’s 4-bit float with FP8-E4M3 block scale over 16-element groups. The remaining tensors (attention, embeddings, output) use a conventional GGUF quant; three variants are provided. NVIDIA shipped official NVFP4 weights for Gemma-4-31B and the 26B-A4B MoE, but not the dense 12B — so we calibrated it ourselves with NVIDIA ModelOpt (cnndailymail, NVFP4 MLP-only, matching NVIDIA’s 31B recipe). The ModelOpt source checkpoint is at LibertAIDAI/Gemma-4-12B-IT-NVFP4**. LibertAI is a decentralized AI platform — private inference, an OpenAI-compatible API, and a chat UI, all running on community GPUs over Aleph Cloud instead of a single company’s servers. No accounts required to chat, no logs sent home, and the same models you’d self-host are available behind a sovereign endpoint. If you want to put this model to work as an autonomous agent without running your own infrastructure, check out LiberClaw — Hermes-style agents hosted on Aleph Cloud with LibertAI inference. Free tier: 2 agents, no credit card, 5 minutes to…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: LibertAIDAI
Теги: gguf, llama.cpp, nvfp4, blackwell, gemma4, en, endpoints_compatible, conversational
Лайков: 4 | Загрузок: 838
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
