dataslab/DLM-2.0-14B-FP8 - Каталог нейросетей
Генерация текста

dataslab/DLM-2.0-14B-FP8

Добавлено:
dataslab/DLM-2.0-14B-FP8

This is an FP8-quantized version of dnotitia/DNA-2.0-14B, optimized for efficient inference by DLM (Data Science Lab., Ltd.). FP8 (8-bit floating point) quantization with static per-tensor scaling reduces model size by approximately 35% while maintaining near-original accuracy. Fully compatible with vLLM for high-throughput production serving. — Method: Static FP8 quantization via llm-compressor oneshot — Precision: FP8E4M3 for weights, FP8E4M3 for input activations — Strategy: Per-tensor symmetric scaling with MinMax observer — Calibration: 512 samples from HuggingFaceH4/ultrachat200k (trainsft split), max sequence length 2048 — Format: compressed-tensors (safetensors) — Preserved layers: lmhead kept in full precision (BF16) — Targets: All Linear layers (except lmhead) — Thinking mode: Add /think to enable detaile

Модальности:
Генерация текста

Области применения:
Диалог / чат Мультиязычность


Задача: Генерация текста
Автор: dataslab
Теги: qwen3, fp8, quantized, vllm, conversational, compressed-tensors, llmcompressor, ko
Лайков: 4  |  Загрузок: 12

Открыть на HuggingFace →

Описание основано на материалах HuggingFace. Перевод выполнен автоматически.