I’ve been trying to get as much performance as possible out of a single DGX Spark with Gemma 4 models. In my local setup, I can run LilaRest/gemma-4-31B-it-NVFP4-turbo at about 8-9 tok/sec on an OpenClaw benchmark suite I built for the Spark. My hope was that if the Gemma 4 E2B model was accurate enough, I could use it as a draft model and materially improve generation speed when paired with the 31B model. After getting the E2B draft + 31B target setup running in memory on the Spark with a 128k context window, I got a perfect score on my benchmark suite and about 18-20 tok/sec. That was better than I expected, so I’m sharing it here for others to test and build on. This release is focused on the text serving path used in that validated setup. This repository contains an NVFP4-quantized draft/speculative model derived directly from: It was prepared to be used as a speculative decoding draft model with: — Validated target model: LilaRest/gemma-4-31B-it-NVFP4-turbo — google/gemma-4-E2B-it is the source/base model — LilaRest/gemma-4-31B-it-NVFP4-turbo is the validated pairing/target model — This repository is not derived from the LilaRest 31B model — ✅ Validated for the text serving…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: mikeytag
Теги: gemma4, image-text-to-text, gemma-4, gemma-4-e2b, nvfp4, modelopt, speculative-decoding, draft-model
Лайков: 4 | Загрузок: 20
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
