asinha08/slm-125m-base is a 125M parameter, Llama-compatible, decoder-only causal language model trained from scratch on a legal-first English corpus. It is a base language model, not an instruction-tuned or chat-aligned assistant. The extra chat-style tokens are present in the tokenizer, but the model was not instruction tuned. — Research on small language model pretraining. — Legal-domain language modeling experiments. — Continued pretraining, supervised fine-tuning, or evaluation pipelines. — Text continuation/completion tests. — Legal advice or compliance decisions. — Production legal automation without task-specific evaluation and guardrails. — Chatbot use without instruction tuning and safety tuning. — Factual question answering where accuracy is required. The corpus was built from three public Hugging Face datasets with a legal-first mix: — Basic line/document cleaning. — OCR/noise filtering for case-law data. — Exact duplicate removal. — MinHash near-duplicate removal. — Decontamination against held-out evaluation sources: coastalcph/lexglue and casehold/casehold. — Byte-level BPE tokenizer training on the cleaned corpus. — Packed fixed-length token windows with sequence…
Модальности:
Генерация текста
Области применения:
Юриспруденция
Задача: Генерация текста
Автор: asinha08
Теги: llama, causal-lm, legal, base-model, from-scratch, bfloat16, modal, en
Лайков: 3 | Загрузок: 21
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
