You can use gpt-oss-120b and gpt-oss-20b with Transformers. If you use the Transformers chat template, it will automatically apply the harmony response format. If you use model.generate directly, you need to apply the harmony format manually using the chat template or use our openai-harmony package. To get started, install the necessary dependencies to setup your environment: Once, setup you can proceed to run the model by running the snippet below: Alternatively, you can run the model via Transformers Serve to spin up a OpenAI-compatible webserver: vLLM recommends using uv for Python dependency management. You can use vLLM to spin up an OpenAI-compatible webserver. The following command will automatically download the model and start the server. To learn about how to use this model with PyTorch and Triton, check out our reference implementations in the gpt-oss repository. If you are trying to run gpt-oss on consumer hardware, you can use Ollama by running the following commands after installing Ollama. If you are using LM Studio you can use the following commands to download. You can download the model weights from the Hugging Face Hub directly from Hugging Face CLI: You can…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: MuXodious
Теги: gguf, vllm, heretic, uncensored, decensored, abliterated, endpoints_compatible, conversational
Лайков: 3 | Загрузок: 57
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
