Bielik-PL-Minitron-7B-v3.0-Instruct is a generative text model featuring 7.35 billion parameters. It is an instruct-aligned version of the pruned Bielik-11B-v3-Base-20250730 model, and after replacing its tokenizer to the APT4 tokenizer optimized specifically for the Polish language. By leveraging NVIDIA’s Minitron methodology, the team employed the NVIDIA Model Optimizer and the NVIDIA NeMo Framework to execute a sophisticated two-stage compression process involving structured pruning and knowledge distillation. This technical synergy allowed for a significant reduction in parameter count while maintaining high-tier performance benchmarks. Developed and trained on a massive multilingual corpus spanning 32 European languages, with a specific emphasis on Polish data curated by the SpeakLeash team, this project utilizes Poland’s large-scale computing infrastructure within the PLGrid environment. The training was conducted on the Athena and Helios supercomputers at ACK Cyfronet AGH, supported by computational grant PLG/2024/016951. This access to cutting-edge NVIDIA hardware and software resources was essential for the complex machine learning processes required to produce a model of…
Модальности:
Генерация текста
Области применения:
Диалог / чат Мультиязычность Следование инструкциям
Задача: Генерация текста
Автор: speakleash
Теги: llama, conversational, pl, en, multilingual, text-generation-inference, endpoints_compatible
Лайков: 4 | Загрузок: 93
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
