Pairwise Rotation Quantization for Efficient Reasoning LLM Inference ParoQuant is the state-of-the-art INT4 quantization for LLMs. It closes the accuracy gap with FP16 while running at near-AWQ speed. Supports NVIDIA GPUs (vLLM, Transformers) and Apple Silicon (MLX). For more information, see https://github.com/z-lab/paroquant. z-lab/Qwen3-14B-PARO is a 4-bit Qwen/Qwen3-14B quantized with ParoQuant. Check out other ParoQuant models from the Hugging Face collection. For vLLM, you can directly use vllm serve to serve ParoQuant models: > [!NOTE] > The following commands map the local cache directory to the container in order to persist kernel cache across runs. Remove -v … to disable this behavior.
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: z-lab
Теги: qwen3, mlx, conversational, text-generation-inference, endpoints_compatible, 4-bit, paroquant
Лайков: 3 | Загрузок: 55
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
