rustformers/gpt-j-ggml

GPT-J 6B is a transformer model trained using Ben Wang’s Mesh Transformer JAX. «GPT-J» refers to the class of model, while «6B» represents the number of trainable parameters. * Each layer consists of one feedforward block and one self attention block. † Although the embedding matrix has a size of 50400, only 50257 entries are used by the GPT-2 tokenizer. The model consists of 28 layers with a model dimension of 4096, and a feedforward dimension of 16384. The model dimension is split into 16 heads, each with a dimension of 256. Rotary Position Embedding (RoPE) is applied to 64 dimensions of each head. The model is trained with a tokenization vocabulary of 50257, using the same set of BPEs as GPT-2/GPT-3.

Модальности:
Генерация текста

Языки программирования:
Rust

Задача: Генерация текста
Автор: rustformers
Теги: llm-rs, ggml, en, endpoints_compatible
Лайков: 2  |  Загрузок: 29

Открыть на HuggingFace →

Описание основано на материалах HuggingFace. Перевод выполнен автоматически.