We present Tri-7B-Base, a foundation language model that serves as the pre-trained base for our Tri-7B model family. This model represents our commitment to efficient training while establishing a strong foundation for downstream fine-tuning and adaptation. Foundation Architecture: State-of-the-art transformer architecture optimized for efficiency Multi-lingual Foundation: Pre-trained on diverse data in Korean, English, and Japanese Efficient Training**: Optimized training methodology for computational efficiency — Type: Causal Language Model — Training Stage: Pre-training — Architecture: Transformer Decoder with RoPE, SwiGLU, RMSNorm — Number of Parameters: 7.76B — Number of Layers: 32 — Number of Attention Heads: 32 — Contex
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: trillionlabs
Теги: llama, pretrained, base-model, conversational, en, ko, ja, text-generation-inference
Лайков: 4 | Загрузок: 40
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
