This repository contains an NVFP4-quantized version of DeepSeek-V2-Lite, prepared using TensorRT Model Optimizer. The model has approximately 15.6B total parameters and is quantized to NVIDIA 4-bit floating-point precision (NVFP4). — Developed by: Krisakorn Chanthasang — Model type: Large Language Model for text generation — Languages: English, Chinese — License: Apache 2.0 — Base model: deepseek-ai/DeepSeek-V2-Lite — Quantization format: NVFP4 — Quantization tool: TensorRT Model Optimizer For details about the original model architecture, training data, and intended usage, see the official base model page: This model requires hardware and inference software with NVFP4 support. — NVIDIA Blackwell GPU or newer — Tested/quantized using: NVIDIA RTX PRO 6000 Blackwell Workstation Use this model with an inference engine that supports NVFP4 quantized models. Compatibility depends on your runtime, GPU architecture, and TensorRT/NVIDIA software stack. This is a quantized derivative of DeepSeek-V2-Lite. Accuracy, throughput, and memory usage may differ from the original model depending on the inference engine and hardware used.
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: Beambutbetter
Теги: deepseek_v2, conversational, custom_code, en, zh, 8-bit, modelopt
Лайков: 3 | Загрузок: 55
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
