ZeroWw/Mistral-7B-Instruct-v0.3-SILLY

ZeroWw ‘SILLY’ version. The original model has been quantized (fq8 version) and a percentage of it’s tensors have been modified adding some noise. Full colab: https://colab.research.google.com/drive/1a7seagBzu5l3k3FL4SFk0YJocl7nsDJw?usp=sharing Fast colab: https://colab.research.google.com/drive/1SDD7ox21di_82Y9v68AUoy0PhkxwBVvN?usp=sharing Original reddit post: https://www.reddit.com/r/LocalLLaMA/comments/1ec0s8p/imadeasillytest/ I created a program to randomize the weights of a model. The program has 2 parameters: the percentage of weights to modify and the percentage of the original value to randmly apply to each weight. At the end I check the resulting GGUF file for binary differences. In this example I set to modify 100% of the weights of Mistral 7b Instruct v0.3 by a maximum of 15% deviation. Since the deviation is calculated on the F32 weights, when quantized to Q8_0 this changes. So, in the end I got a file that compared to the original has: The cool thing is that chatting with the model I see no apparent difference and the model still works nicely as the original. Since I am running everything on CPU, I could not run perplexity scores or anything computing intensive. As…

Модальности:
Генерация текста

Области применения:
Диалог / чат Следование инструкциям

Задача: Генерация текста
Автор: ZeroWw
Теги: gguf, en, endpoints_compatible, conversational
Лайков: 3  |  Загрузок: 7

Открыть на HuggingFace →

Описание основано на материалах HuggingFace. Перевод выполнен автоматически.