From: https://huggingface.co/ehartford/Wizard-Vicuna-13B-Uncensored merged with SuperHOT 8K. This is an experimental new GPTQ which offers up to 8K context size The increased context is tested to work with ExLlama, via the latest release of text-generation-webui. It has also been tested from Python code using AutoGPTQ and trustremotecode=True. 1. Click the Model tab. 2. Under Download custom model or LoRA, enter localmodels/Wizard-Vicuna-13B-Uncensored-SuperHOT-8K-GPTQ. 3. Click Download. 4. The model will start downloading. Once it’s finished it will say «Done» 5. Untick Autoload the model 6. In the top left, click the refresh icon next to Model. 7. In the Model dropdown, choose the model you just downloaded: Wizard-Vicuna-13B-Uncensored-SuperHOT-8K-GPTQ 8. To use the increased context, set the Loader to ExLlama, set maxseqlen to 8192 or 4096, and set compressposemb to 4 for 8192 context, or to 2 for 4096 context. 9. Now click Save Settings followed by Reload 10. The model will automatically load, and is now ready for use! 11. Once you’re ready, click the Text Generation tab and enter a prompt to get started! Then run the following code. Note that in order to get this to work,…
Модальности:
Генерация текста
Задача: Генерация текста
Автор: localmodels
Теги: llama, custom_code, endpoints_compatible
Лайков: 3 | Загрузок: 10
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
