EAGLE speculative decoding drafter for Gemma 4 26B-A4B Uncensored (TrevorJS) — a 42-layer E4B (EAGLE for Blackwell) model quantized to NVFP4 AWQ using NVIDIA ModelOpt 0.42.0. Designed for EAGLE-based speculative decoding on NVIDIA DGX Spark (GB10, SM 12.1) and other Blackwell-architecture GPUs. 1. Target model — Any NVFP4-quantized Gemma 4 26B MoE (e.g., AEON-7/Gemma-4-26B-A4B-it-Uncensored-NVFP4) 2. This drafter model — Download below 3. Three vLLM patches — Required for Gemma 4 speculative decoding (see Required Patches) 4. Pre-built container — ghcr.io/aeon-7/vllm-spark-gemma4-nvfp4-awq:latest Three patches to vLLM 0.19.1 are required for Gemma 4 speculative decoding. Download from the DECKARD 31B GitHub repo: On the DGX Spark’s unified memory keep —gpu-memory-utilization at 0.6-0.7; above ~0.8 the shared CPU+GPU pool page-thrashes and stalls the box, and a spec-decode drafter’s verify buffers are not counted by the fraction so leave headroom (0.65 here). Discrete-VRAM GPUs can run higher. Speculative decoding with Gemma 4 requires three patches to vLLM 0.19.1. Without these, the server will crash on startup. vLLM 0.19.1 calls raiseifmultimodal() which blocks ALL multimodal…
Модальности:
Генерация текста Компьютерное зрение Мультимодальность
Области применения:
Генерация кода Логика и рассуждение Следование инструкциям Диалог / чат Вызов функций (Tool use)
Задача: Генерация текста
Автор: AEON-7
Теги: gemma4, image-text-to-text, 4-bit, aarch64, abliterated, aeon, aeon-7, agentic
Лайков: 4 | Загрузок: 900
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.