This model is a result of merging Orca2-13B with itself using ‘mergekit-legacy’. Merge parameters were —weight 0.5 —density 0.5 This merged model showed marginal improvement in perplexity scores: The following table summarizes the model performance across a range of benchmarks: Interestingly the GSM8K performance more than doubled with the first self merge. Second self merge resulting in the 39B model did not produce any further gains.
Модальности:
Генерация текста
Задача: Генерация текста
Автор: vmajor
Теги: llama, merge, text-generation-inference, endpoints_compatible
Лайков: 2 | Загрузок: 52
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.