———— | ———————————————————————————————- | The following layers are not quantized to preserve model quality: Tested on a single NVIDIA DGX Spark (GB10, 128GB LPDDR5X, SM121): > Note: The DFlash drafter was trained on the original Qwen3.6-35B-A3B. Acceptance rate on the abliterated variant may be lower than on the original model. — Use —quantization moewna16 for Marlin INT4 kernel (SM121 compatible via SM120 binary compat) — FP8 KV cache is not compatible with GDN non-causal attention layers; use —kv-cache-dtype auto — NVFP4 falls back to Marlin W4A16 on SM121 (missing cvt.e2m1x2 PTX instruction) — Runtime FP8 (—quantization fp8) is not compatible with DFlash (drafter inherits FP8 config and crashes) — —language-model-only skips vision encoder profiling for text-only inference — —performance-mode Users are solely responsible for all…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: YuYu1015
Теги: qwen3_5_moe, image-text-to-text, qwen3.6, moe, 4-bit, int4, auto-round, gptq
Лайков: 3 | Загрузок: 393
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
