These files are GPTQ model files for Meta’s Llama 2 7b Chat. Multiple GPTQ parameter permutations are provided; see Provided Files below for details of the options provided, their parameters, and the software used to create them. GPTQ models for GPU inference, with multiple quantisation parameter options. 2, 3, 4, 5, 6 and 8-bit GGML models for CPU+GPU inference * Unquantised fp16 model in pytorch format, for GPU inference and for further conversions Multiple quantisation parameters are provided, to allow you to choose the best one for your hardware and requirements. Each separate quant is in a different branch. See below for instructions on fetching from different branches. — In text-generation-webui, you can add :branch to the end of the download name, eg TheBloke/Llama-2-7b-Chat-GPTQ:gptq-4bit-32g-actorderTrue` — With Git, you can clone a branch with: — In Python Transformers code, the branch is the revision parameter; see below. Please make sure you’re using the latest version of text-generation-webui. It is strongly recommended to use the text-generation-webui one-click-installers unless you know how to make a manual install. 1. Click the Model tab. 2. Under Download custom…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: gsaivinay
Теги: llama, facebook, meta, llama-2, en, text-generation-inference, endpoints_compatible
Лайков: 2 | Загрузок: 28
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
