TheBloke/vigogne-2-70B-chat-AWQ - Каталог нейросетей
Генерация текста

TheBloke/vigogne-2-70B-chat-AWQ

Добавлено:

Chat & support: TheBloke’s Discord server Want to contribute? TheBloke’s Patreon page TheBloke’s LLM work is generously supported by a grant from andreessen horowitz (a16z) — Model creator: bofeng huang — Original model: Vigogne 2 70B Chat This repo contains AWQ model files for bofeng huang’s Vigogne 2 70B Chat. These files were quantised using hardware kindly provided by Massed Compute. AWQ is an efficient, accurate and blazing-fast low-bit weight quantization method, currently supporting 4-bit quantization. Compared to GPTQ, it offers faster Transformers-based inference with equivalent or better quality compared to the most commonly used GPTQ settings. — Text Generation Webui — using Loader: AutoAWQ — vLLM — Llama and Mistral models only — Hugging Face Text Generation Inference (TGI) — AutoAWQ — for use from Python code AWQ model(s) for GPU inference. GPTQ models for GPU inference, with multiple quantisation parameter options. 2, 3, 4, 5, 6 and 8-bit GGUF models for CPU+GPU inference bofeng huang’s original unquantised fp16 model in pytorch format, for GPU inference and for further conversions For my first release of AWQ models, I am releasing 128g models only. I will…

Модальности:
Генерация текста

Области применения:
Диалог / чат


Задача: Генерация текста
Автор: TheBloke
Теги: llama, LLM, llama-2, finetuned, conversational, fr, text-generation-inference, 4-bit
Лайков: 2  |  Загрузок: 15

Открыть на HuggingFace →

Описание основано на материалах HuggingFace. Перевод выполнен автоматически.