YASIN utilizes a modern Transformer architecture optimized for stability and inference efficiency. Its design is inspired by high-performance models like Llama but customized for the Persian script. 1. RoPE (Rotary Positional Embeddings): Implements dynamic rotary embeddings for better long-range dependency handling and positional awareness within the context window. 2. RMSNorm (Root Mean Square Layer Normalization): Used at the start of each transformer block to improve training stability and prevent gradient explosion. 3. SwiGLU (SiLU) Activation: Employs the SiLU activation function within the MLP (Gated Linear Unit) layers, which has been shown to outperform standard ReLU in LLMs. 4. Byte-level BPE Tokenizer: A specialized tokenizer designed to handle the zero-width non-joiner (ZWNJ) and unique Persian characters without excessive token fragmentation. Since YASIN uses a custom architecture, ensure you have the modelingyasin.py and configurationyasin.py files in your directory or use trustremotecode=True. Knowledge Cutoff: The model’s knowledge is limited to its training data timeframe. Hallucinations: Like all LLMs, YASIN may occasionally generate factually incorrect…
Модальности:
Генерация текста
Задача: Генерация текста
Автор: ysn-rfd
Теги: yasin, persian, decoder-only, fa
Лайков: 3 | Загрузок: 25
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
