, —, —, . You can ask it anything you want and it will do its best to give you accurate and relevant information. ‘ quantized_by: TheBloke , —, —, . You can ask it anything you want and it will do its best to give you accurate and relevant information. shell python3 python -m vllm.entrypoints.api_server —model TheBloke/Uncensored-Jordan-13B-AWQ —quantization awq python from vllm import LLM, SamplingParams prompts = [ «Tell me about AI», «Write a story about llamas», «What is 291 — 150?», «How much wood would a woodchuck chuck if a woodchuck could chuck wood?», ] prompt_template=f»’This is a conversation with your Uncensored AI assistant Jordan. Jordan can discuss with you at length about various topics such as —, —, —, . You can ask it anything you want and it will do its best to give you accurate and relevant information. prompts = [prompt_template.format(prompt=prompt) for prompt in prompts] llm = LLM(model=»TheBloke/Uncensored-Jordan-13B-AWQ», quantization=»awq», dtype=»auto») for output in outputs: prompt = output.prompt generatedtext = output.outputs[0].text print(f»Prompt: {prompt!r}, Generated text: {generatedtext!r}») shell —model-id…
Модальности:
Генерация текста
Задача: Генерация текста
Автор: TheBloke
Теги: llama, en, text-generation-inference, 4-bit, awq
Лайков: 2 | Загрузок: 10
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.