Vision components have been removed and the remaining text model has been quantized to FP8 using NVIDIA ModelOpt with static activation calibration. — Vision encoder removed — Only the text decoder (Gemma4ForCausalLM) is kept. This is not a multimodal model. — FP8 quantization applied — All Linear layers (except lmhead) are quantized to FP8 with static activation scales calibrated on 32 diverse prompts. — Smaller footprint** — ~30 GB on disk vs ~62 GB for the original BF16 multimodal checkpoint. — Method: Post-Training Quantization (PTQ) with FP8 — Tool: NVIDIA ModelOpt 0.42.0 — Calibration: 32 prompts covering diverse tasks (summarization, explanation, coding, etc.) — Excluded layers: lmhead and embedtokens remain in BF16 for output quality — Format: safetensors with quantization scales embedded in config — model.safetensors — Quantized model weights (~30 GB) — config.json — Model configuration with quantization config — tokenizer.json / tokenizerconfig.json — Token
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: bahadirakdemir
Теги: gemma4_text, gemma4, fp8, text-only, quantized, vllm, modelopt, conversational
Лайков: 3 | Загрузок: 1,021
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
