TheBloke/Uncensored-Frank-7B-AWQ

, —, —, . You can ask it anything you want and it will do its best to give you accurate and relevant information. ‘ quantized_by: TheBloke , —, —, . You can ask it anything you want and it will do its best to give you accurate and relevant information. shell python3 python -m vllm.entrypoints.api_server —model TheBloke/Uncensored-Frank-7B-AWQ —quantization awq python from vllm import LLM, SamplingParams prompts = [ «Hello, my name is», «The president of the United States is», «The capital of France is», «The future of AI is», ] samplingparams = SamplingParams(temperature=0.8, topp=0.95) llm = LLM(model=»TheBloke/Uncensored-Frank-7B-AWQ», quantization=»awq») for output in outputs: prompt = output.prompt generatedtext = out python from awq import AutoAWQForCausalLM from transformers import AutoTokenizer model = AutoAWQForCausalLM.fromquantized(modelnameorpath, fuselayers=True, trustremotecode=False, safetensors=True) tokenizer =…

Модальности:
Генерация текста

Задача: Генерация текста
Автор: TheBloke
Теги: llama, en, text-generation-inference, 4-bit, awq
Лайков: 3  |  Загрузок: 12

Открыть на HuggingFace →

Описание основано на материалах HuggingFace. Перевод выполнен автоматически.