dvruette/gidd-unif-10b - Каталог нейросетей
Генерация текста

dvruette/gidd-unif-10b

Добавлено:
dvruette/gidd-unif-10b

Dimitri von Rütte, Janis Fluri, Antonio Orvieto, Omead Pooladzandi, Bernhard Schölkopf, Thomas Hofmann —|——:|————:|:—————|:——————| This model is trained on diffusion language modeling with either masking or uniform noise. During pre-training, 20% of sequences had a random fraction of the context unperturbed, making this model capable of conditional generation (prompt completion). The training data is a random subset of Nemotron-CC without quality filtering. The tokenizer is a BPE tokenizer trained on a 256 GB subset of Nemotron-CC using the HF tokenizers library. Please refer to the paper for further training details. Benchmarks run with lm-evaluation-harness on ARC-E, ARC-C, WinoGrande, PIQA, OpenBookQA, BoolQ, GSM8k; likelihood-based multiple-choice except GSM8k. This is a base model trained on Nemotron-CC, a large-scale web corpus. Due to this, it can reproduce social biases and generate toxic or harmful content. Performance on math/coding is limited in the reported setup due to lack of dedicated math/coding data.

Модальности:
Генерация текста


Задача: Генерация текста
Автор: dvruette
Теги: gidd, custom_code, en
Лайков: 4  |  Загрузок: 82

Открыть на HuggingFace →

Описание основано на материалах HuggingFace. Перевод выполнен автоматически.