This is an FP8-quantized version of dnotitia/DNA-2.0-14B, optimized for efficient inference by DLM (Data Science Lab., Ltd.). FP8 (8-bit floating point) quantization with static per-tensor scaling reduces model size by approximately 35% while maintaining near-original accuracy. Fully compatible with vLLM for high-throughput production serving. — Method: Static FP8 quantization via llm-compressor oneshot — Precision: FP8E4M3 for weights, FP8E4M3 for input activations — Strategy: Per-tensor symmetric scaling with MinMax observer — Calibration: 512 samples from HuggingFaceH4/ultrachat200k (trainsft split), max sequence length 2048 — Format: compressed-tensors (safetensors) — Preserved layers: lmhead kept in full precision (BF16) — Targets: All Linear layers (except lmhead) — Thinking mode: Add /think to enable detaile
Модальности:
Генерация текста
Области применения:
Диалог / чат Мультиязычность
Задача: Генерация текста
Автор: dataslab
Теги: qwen3, fp8, quantized, vllm, conversational, compressed-tensors, llmcompressor, ko
Лайков: 4 | Загрузок: 12
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.