kl3m-003-1.7b is a small language model (SLM) trained on clean, legally-permissible data. Originally developed by 273 Ventures and donated to the ALEA Institute, kl3m-003-1.7b was part of the first LLM family to obtain the Fairly Trained L-Certification for its ethical training data and practices. The model is designed for legal, regulatory, and financial workflows, with a focus on low toxicity and high efficiency. Given its small size and lack of training data for instruction alignment, kl3m-003-1.7b is best suited for use either in SLM fine-tuning or as part of training larger models without using unethical data or models. — Architecture: GPT-NeoX (i.e., ~GPT-3 architecture) — Size: 1.7 billion parameters — Hidden Size: 2048 — Layers: 32 — Attention Heads: 32 — Intermediate Size: 8192 — Max Position Embeddings: 8192 — Context Window: 8,192 tokens (true size, no sliding window) — Tokenizer: kl3m-001-32k BPE tokenizer (32,768 vocabulary size with unorthodox whitespace handling) — Language(s): Primarily English — Training Objective: Next token prediction — Developed by: Originally by 273 Ventures
Модальности:
Генерация текста
Области применения:
Юриспруденция
Задача: Генерация текста
Автор: alea-institute
Теги: gpt_neox, kl3m, kl3m-003, legal, financial, enterprise, slm, gpt-neox
Лайков: 4 | Загрузок: 276
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
