Quantized to 4bpw and tested using the mlxlm` utility on a 64GiB URAM M1 Max. Looking for the 103B version? You can get it from FluffyKaeloky/Midnight-Miqu-103B-v1.5. This is a DARE Linear merge between sophosympatheia/Midnight-Miqu-70B-v1.0 and migtissera/Tess-70B-v1.6. This version is close in feel and performance to Midnight Miqu v1.0 but I think it picked up some goodness from Tess. Their EQ Bench scores are virtually the same and their post-EXL2 quant perplexity scores were the same too. However, Midnight Miqu v1.5 passes some tests I use that Midnight Miqu v1.0 fails, without sacrificing writing quality. This model is uncensored. You are responsible for whatever you do with it. This model was designed for roleplaying and storytelling and I think it does well at both. It may also perform well at other tasks but I have not tested its performance in other areas. You can run this model out to 32K context with alpha_rope set to 1, just like with Miqu. I recommend using Quadratic Sampling (i.e. smoothing factor) for creative work. I think this version performs best with a smoothing factor close to 0.2. I recommend using Min-P. Experiment to find your best setting. You can enable…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: heni86
Теги: llama, mergekit, merge, conversational, text-generation-inference, endpoints_compatible, 4-bit
Лайков: 3 | Загрузок: 14
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
