llm-jp/llm-jp-13b-instruct-lora-jaster-v1.0 - Каталог нейросетей
Генерация текста

llm-jp/llm-jp-13b-instruct-lora-jaster-v1.0

Добавлено:

This repository provides large language models developed by LLM-jp, a collaborative project launched in Japan. Checkpoints format: Hugging Face Transformers (Megatron-DeepSpeed format models are available here) — torch>=2.0.0 — transformers>=4.34.0 — tokenizers>=0.14.0 — accelerate==0.23.0 — peft==0.5.0 — Model type: Transformer-based Language Model — Total seen tokens: 300B — Pre-training: — Hardware: 96 A100 40GB GPUs (mdx cluster) — Software: Megatron-DeepSpeed — Instruction tuning: — Hardware: 8 A100 40GB GPUs (mdx cluster) — Software: TRL, PEFT, and DeepSpeed The tokenizer of thi The vocabulary entries were converted from llm-jp-tokenizer v2.1 (50k). Please refer to README.md of llm-ja-tokenizer for details on the vocabulary construction procedure. — Model: Hugging Face Fast Tokenizer using Unigram byte-fallback model which requires tokenizers>=0.14.0 — Training algorithm: SentencePiece Unigram byte-fallback — Training data: A subset of the datasets for model pre-training — Vocabulary size: 50,570 (mixed vocabulary of Japanese, English, and source code) The models have been pre-trained using a blend of the…

Модальности:
Генерация текста

Области применения:
Следование инструкциям


Задача: Генерация текста
Автор: llm-jp
Теги: peft, en, ja
Лайков: 2  |  Загрузок: 27

Открыть на HuggingFace →

Описание основано на материалах HuggingFace. Перевод выполнен автоматически.