— Model Architecture: NemotronHForCausalLM — Input: Text — Output: Text — Model Optimizations: — Weight quantization: FP8 — Activation quantization: FP8 — Release Date: 9/30/2025 — Version: 1.0 — Model Developers: Red Hat — ModelCar Storage URI: oci://registry.redhat.io/rhelai1/modelcar-nvidia-nemotron-nano-9b-v2-fp8-dynamic:1.5 — Validated on RHOAI 2.25: quay.io/modh/vllm:rhoai-2.25-cuda — Validated on RHAIIS 3.2.2: http://registry.redhat.io/rhaiis/vllm-cuda-rhel9:3.2.2 — Va This optimization reduces the number of bits per parameter from 16 to 8, reducing the disk size and GPU memory requirements by approximately 50%. Only the weights and activations of the linear operators within transformers blocks are quantized. This model was created with llm-compressor by running the code snippet below. The model was evaluated on the OpenLLMv1 leaderboard task, using lm-evaluation-harness, and on reasoning tasks using lighteval. vLLM was used for all evaluations. Category Metric nvidia/NVIDIA-Nemotron-Nano-9B-v2…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: RedHatAI
Теги: nemotron_h, nvidia, fp8, quantized, llm-compressor, compressed-tensors, red hat, conversational
Лайков: 3 | Загрузок: 1,586
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
