Project: https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/ This is a pruned variant of OpenAI’s GPT-OSS-20B model, reduced to 32 experts per layer based on activation patterns from the AmanPriyanshu/GPT-OSS-20B MoE Expert Activations dataset. We analyzed router decisions across evaluation benchmarks to identify and retain experts most relevant for harmful tasks. ⚠️ Experimental Model: This is an experimental pruned model that may not work well — check the examples below to see if the outputs meet your needs before use. This pruning approach reduces the model size while attempting to preserve performance on the target domain. Mixture-of-Experts models contain multiple specialized sub-networks (experts) per layer. During inference, only a subset of experts are activated for each token. Expert pruning involves: 1. Analyzing Usage Patterns: Tracking which experts activate most frequently for specific tasks 2. Removing Underutilized Experts: Discarding experts with low activation rates for the target domain 3. Preserving Router Functionality: Maintaining the routing mechanism with fewer available experts — Data-Driven Selection: Used activation patterns from harmful…
Модальности:
Генерация текста
Области применения:
Логика и рассуждение Диалог / чат
Задача: Генерация текста
Автор: AmanPriyanshu
Теги: gpt_oss, mixture-of-experts, moe, expert-pruning, gpt-oss, openai, reasoning, harmful
Лайков: 3 | Загрузок: 21
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
