Chat & support: TheBloke’s Discord server Want to contribute? TheBloke’s Patreon page TheBloke’s LLM work is generously supported by a grant from andreessen horowitz (a16z) These files are GPTQ 4bit model files for Jon Durbin’s Airoboros 65B GPT4 1.3. It is the result of quantising to 4bit using GPTQ-for-LLaMa. Note from model creator Jon Durbin: This version has problems, use if you dare, or wait for 1.4. 4-bit GPTQ models for GPU inference 2, 3, 4, 5, 6 and 8-bit GGML models for CPU+GPU inference * Unquantised fp16 model in pytorch format, for GPU inference and for further conversions Please make sure you’re using the latest version of text-generation-webui 1. Click the Model tab. 2. Under Download custom model or LoRA, enter TheBloke/airoboros-65B-gpt4-1.3-GPTQ. 3. Click Download. 4. The model will start downloading. Once it’s finished it will say «Done» 5. In the top left, click the refresh icon next to Model. 6. In the Model dropdown, choose the model you just downloaded: airoboros-65B-gpt4-1.3-GPTQ 7. The model will automatically load, and is now ready for use! 8. If you want any custom settings, set them and then click Save settings for this model followed by Reload…
Модальности:
Генерация текста
Задача: Генерация текста
Автор: TheBloke
Теги: llama, text-generation-inference, 4-bit, gptq
Лайков: 3 | Загрузок: 10
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
