NVFP4-quantized version of Qwen/Qwen2.5-72B-Instruct, produced by Enfuse. — Full NVFP4 (W4A4): Requires NVIDIA Blackwell GPU (B200, GB200, RTX 5090) for native FP4 tensor core support — Weight-only FP4: Older GPUs (H100, A100) can load the model but will only apply weight quantization, not activation quantization — Recommended: 2x B200 with tensor parallelism for optimal throughput This model was quantized using LLM Compressor (v0.10.0) with the NVFP4 scheme: — Method: Post-training quantization (PTQ) with calibration — Calibration data: 512 sample Evaluated with —applychattemplate —fewshotasmultiturn flags,…
Модальности:
Генерация текста
Области применения:
Диалог / чат Следование инструкциям
Задача: Генерация текста
Автор: enfuse
Теги: qwen2, nvfp4, quantized, 4-bit, blackwell, qwen2.5, compressed-tensors, vllm
Лайков: 4 | Загрузок: 875
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.