— Model Architecture: Qwen3ForCausalLM — Input: Text — Output: Text — Model Optimizations: — Weight quantization: INT4 — Intended Use Cases: — Reasoning. — Function calling. — Subject matter experts via fine-tuning. — Multilingual instruction following. — Translation. — Out-of-scope: Use in any manner that violates applicable laws or regulations (including trade compliance laws). — Release Date: 05/05/2025 — Version: 1.0 — Model Developers: RedHat (Neural Magic) This model was obtained by quantizing the weights of Qwen3-1.7B to INT4 data type. This optimization reduces the number of bits per parameter from 16 to 4, reducing the disk size and GPU memory requirements by approximately 75%. Only the weights of the linear operators within transformers blocks are quantized. Weights are quantized using a asymmetric per-group scheme, with group size 64. The GPTQ algorithm is applied for quantization, as implemented in the llm-compressor library. This model can be deployed efficiently using the vLLM backend, as shown in the example below. vLLM aslo supports OpenAI-compatible serving. See the documentation for more details. Creation details This model was created with llm-compressor by…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: RedHatAI
Теги: qwen3, neuralmagic, redhat, llmcompressor, quantized, INT4, conversational, text-generation-inference
Лайков: 3 | Загрузок: 535
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
