The Magnum v4 series is complete, but here’s something a little extra I wanted to tack on as I wasn’t entirely satisfied with the results of v4 72B. «SE» for Special Edition — this model is finetuned from meta-llama/Llama-3.3-70B-Instruct as an rsLoRA adapter. The dataset is a slightly revised variant of the v4 data with some elements of the v2 data re-introduced. The objective, as with the other Magnum models, is to emulate the prose style and quality of the Claude 3 Sonnet/Opus series of models on a local scale, so don’t be surprised to see «Claude-isms» in its output. This model is intended for creative writing and roleplay purposes. It may show biases similar to those observed in contemporary LLM-based roleplay, in addition to those exhibited by the Claude 3 series of models and the base model. All outputs should be considered fiction, as this model is not intended to provide factual information or advice. The following hyperparameters were used during training: — learningrate: 4e-05 — trainbatchsize: 1 — evalbatchsize: 1 — seed: 42 — distributedtype: multi-GPU — numdevices: 8 — totaltrainbatchsize: 8 — totalevalbatchsize: 8 — optimizer: Use pagedademamix8bit and the args are:…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: DS-Archive
Теги: peft, llama, axolotl, generated_from_trainer, conversational, en
Лайков: 4 | Загрузок: 3
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
