TheBloke/Venus-120b-v1.2-AWQ - Каталог нейросетей
Генерация текста

TheBloke/Venus-120b-v1.2-AWQ

Добавлено:
TheBloke/Venus-120b-v1.2-AWQ

Chat & support: TheBloke’s Discord server Want to contribute? TheBloke’s Patreon page TheBloke’s LLM work is generously supported by a grant from andreessen horowitz (a16z) — Model creator: John Smith — Original model: Venus 120B V1.2 This repo contains AWQ model files for John Smith’s Venus 120B V1.2. AWQ is an efficient, accurate and blazing-fast low-bit weight quantization method, currently supporting 4-bit quantization. Compared to GPTQ, it offers faster Transformers-based inference with equivalent or better quality compared to the most commonly used GPTQ settings. AWQ models are currently supported on Linux and Windows, with NVidia GPUs only. macOS users: please use GGUF models instead. — Text Generation Webui — using Loader: AutoAWQ — vLLM — version 0.2.2 or later for support for all model types. — Hugging Face Text Generation Inference (TGI) — Transformers version 4.35.0 and later, from any code or client that supports Transformers — AutoAWQ — for use from Python code AWQ model(s) for GPU inference. GPTQ models for GPU inference, with multiple quantisation parameter options. 2, 3, 4, 5, 6 and 8-bit GGUF models for CPU+GPU inference John Smith’s original unquantised…

Модальности:
Генерация текста


Задача: Генерация текста
Автор: TheBloke
Теги: llama, not-for-all-audiences, en, text-generation-inference, 4-bit, awq
Лайков: 3  |  Загрузок: 14

Открыть на HuggingFace →

Описание основано на материалах HuggingFace. Перевод выполнен автоматически.