This is a 5.0bpw EXL2 quant of sophosympatheia/Midnight-Miqu-70B-v1.0 Details about the model and the merge info can be found at the above mode page. I have not extensively tested this quant/model other than ensuring I could load it and chat with it. Below is what I used to run this model on a dual 3090 Linux server. I have not tested inference above a couple K tokens. If the model blows out past 8k, consider a 8192 context without cache_8bit set. Included is a Tavern format character card created by Midnight Miqu for chat. The card was created using a character creator helper bot using a single prompt for the base card, another prompt asking for specific conversation examples and then asking it to provide a text to image portrait prompt. Being able to faithfully follow the character creator bot to create this card demonstrates a pretty high level of intelligence. Standing at approximately five feet six inches tall, Seraphina presents herself as a breathtakingly beautiful woman with long, cascading silver hair that reaches down to her waist. It flows freely around her, framing a face defined by high cheekbones and full lips curved into a perpetual smile. Her emerald green eyes…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: Dracones
Теги: llama, mergekit, merge, conversational, text-generation-inference, endpoints_compatible
Лайков: 3 | Загрузок: 22
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.