This repository contains the Mecellem-Qwen3-4B-TR model, as presented in the paper Mecellem Models: Turkish Models Trained from Scratch and Continually Pre-trained for the Legal Domain. — GitHub Repository: newmindai/mecellem-models — Paper: arXiv:2601.16018 Mecellem-Qwen3-4B-TR is a Turkish legal language model adapted through Continual Pre-training (CPT) on Turkish legal and official texts. The model is based on Qwen3-4B decoder architecture (4B parameters) and trained using a single-phase, large-scale CPT process. Unlike the 1.7B model’s four-phase curriculum learning, this model employs a single-phase training strategy on a comprehensive dataset, demonstrating that larger model capacity can benefit from direct large-scale domain adaptation. Key Features: — Continual pre-training on approximately 270.8 billion tokens in a single phase — Single-phase large-scale CPT process (270,791,712,595 tokens) — Dataset includes Turkish legal sources (Yargıtay, Danıştay, YÖKTEZ) and general Turkish web data (FineWeb2, CulturaX) — Preserves general language capabilities while injecting domain-specific legal knowledge Model Type: Decoder-only Language Model Parameters: 4B Base Model:…
Модальности:
Генерация текста
Области применения:
Диалог / чат Юриспруденция
Задача: Генерация текста
Автор: newmindai
Теги: qwen3, turkish, legal, turkish-legal, mecellem, qwen, decoder-only, continual-pretraining
Лайков: 3 | Загрузок: 88
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
