This checkpoint preserves the hybrid Qwen3.5 DeltaNet + softmax architecture and MTP (Multi-Token Prediction) head from the BF16 source, quantizing most linear layers to FP8 W8A8 while keeping the most sensitive projections and sidecar components in BF16. Local export and sanity-check evaluation were verified on 2026-04-07 on a single NVIDIA RTX PRO 6000 Blackwell Workstation Edition (96 GB) with: — transformers==5.3.0 — llm-compressor==0.14.1.dev24 — vllm==0.17.1 — FP8 export completed successfully via llm-compressor — MTP weights are included in the main safetensors file — The checkpoint loads in vLLM and generates correct output — Quick perplexity sanity check: 7.67 (FineWeb-Edu, 50 samples) — modeltype=qwen35 — 64 text lay — MTP weights are embedded in the main model.safetensors file (no separate model.mtp.safetensors). — The model includes a vision encoder (loaded but unused for text-only…
Модальности:
Генерация текста
Области применения:
Логика и рассуждение Диалог / чат
Задача: Генерация текста
Автор: mconcat
Теги: qwen3_5, image-text-to-text, qwen3.5, reasoning, quantized, fp8, fp8-dynamic, compressed-tensors
Лайков: 3 | Загрузок: 26
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
