Model Architecture COKAL-DPO_test-v2 is an auto-regressive 13B language model based on the LLaMA2 transformer architecture. Training Dataset — DPO training dataset: DopeorNope/DPO-Ko-Dataset — private This dataset was constructed by directly collecting and reorganizing data by DopeorNope, obtaining insights from «lvwerra/stack-exchange-paired» to create a paired dataset. (It means I do not use stack-exchange-paired; I just got an insight from it.) — SFT training dataset: DopeorNope/OrcaNearDedup-v2 — private This dataset is based on «kyujinpy/OpenOrca-KO» and has been processed using the Near Dedup algorithm to remove items with a Jaccard Similarity threshold of 0.8 or higher. In addition, inconsistent inputs have been cleaned and modified. Training I developed the model in an environment with four RTX 3090 GPUs running Ubuntu 18.04. It seems that when uploading the model directly to a repository from a Linux server, there may be an issue causing the model to appear to have more parameters. However, this model is based on a 13B architecture. — 이 모델은 과학기술정보통신부·광주광역시가 공동 지원한 ‘인공지능 중심 산업융합 집적단지 조성사업’으로 지원을 받아 수행된 연구 결과입니다. — This model was supported by Artificial intelligence…
Модальности:
Генерация текста
Задача: Генерация текста
Автор: DopeorNope
Теги: llama, ko, text-generation-inference, endpoints_compatible
Лайков: 2 | Загрузок: 14
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.