Requirements • Quick Start • Available Environments • Documentation & Resources • Community & Support • Citations

NeMo Gym is a library for building reinforcement learning (RL) training environments for large language models (LLMs). It provides infrastructure to develop environments, scale rollout collection, and integrate seamlessly with your preferred training framework.

🏆 Why NeMo Gym?

• Scaffolding and patterns to accelerate environment development: multi-step, multi-turn, and user modeling scenarios
• Contribute environments without expert knowledge of the entire RL training loop
• Test environments and throughput end-to-end, independent of the RL training loop
• Interoperable with existing environments, systems, and RL training frameworks
• Growing collection of training environments and datasets for Reinforcement Learning from Verifiable Reward (RLVR)

🔗 Ecosystem

NeMo Gym is part of NVIDIA NeMo, NVIDIA's GPU-accelerated platform for building and training generative AI models. NeMo Gym integrates with a growing number of RL training frameworks and environment libraries; see the Ecosystem page for full details and tutorials.

Training Frameworks: NeMo RL • OpenRLHF • TRL • Unsloth • more →

Environment Libraries: Reasoning Gym • Aviary • more →

📋 Requirements

NeMo Gym is designed to run on standard development machines:

Additional Requirements

• API Keys: OpenAI API key with available credits (for the quickstart examples)
  • Other model providers supported (Azure OpenAI, self-hosted models via vLLM)
• Ray: Automatically installed as a dependency (no separate setup required)

🚀 Quick Start

Install NeMo Gym, start the servers, and collect your first verified rollouts for RL training.

Setup

# Clone the repository
git clone git@github.com:NVIDIA-NeMo/Gym.git
cd Gym

# Install UV (Python package manager)
curl -LsSf https://astral.sh/uv/install.sh | sh
source $HOME/.local/bin/env

# Create virtual environment
uv venv --python 3.12
source .venv/bin/activate

# Install NeMo Gym
uv sync --extra dev --group docs

Configure Your API Key

Create an env.yaml file that contains your OpenAI API key and the policy model you want to use. Replace your-openai-api-key with your actual key. This file helps keep your secrets out of version control while still making them available to NeMo Gym.

echo "policy_base_url: https://api.openai.com/v1
policy_api_key: your-openai-api-key
policy_model_name: gpt-4.1-2025-04-14" > env.yaml

Start Servers

Terminal 1 (start servers):

# Start servers (this will keep running)
config_paths="resources_servers/example_single_tool_call/configs/example_single_tool_call.yaml,\
responses_api_models/openai_model/configs/openai_model.yaml"
ng_run "+config_paths=[${config_paths}]"

Terminal 2 (interact with agent):

# In a NEW terminal, activate environment
source .venv/bin/activate

# Interact with your agent
python responses_api_agents/simple_agent/client.py

Collect Rollouts

Terminal 2 (keep servers running in Terminal 1):

# Create a simple dataset with one query
echo '{"responses_create_params":{"input":[{"role":"developer","content":"You are a helpful assistant."},{"role":"user","content":"What is the weather in Seattle?"}]}}' > weather_query.jsonl

# Collect verified rollouts
ng_collect_rollouts \
    +agent_name=example_single_tool_call_simple_agent \
    +input_jsonl_fpath=weather_query.jsonl \
    +output_jsonl_fpath=weather_rollouts.jsonl

# View the result
cat weather_rollouts.jsonl | python -m json.tool

This generates training data with verification scores!

Clean Up Servers

Terminal 1 with the running servers: Ctrl+C to stop the ng_run process.

Next Steps

Now that you can generate rollouts, choose your path:

• Start training — Train models using NeMo Gym with your preferred RL framework. See the Training Tutorials.

• Use an existing environment — Browse the Available Environments below to find an environment that matches your goals.

• Build a custom environment — Implement or integrate existing tools and define task verification logic. Get started with the Creating a Training Environment tutorial.

📦 Available Environments

NeMo Gym includes a curated collection of environments for training and evaluation across multiple domains:

Example Environment Patterns

Purpose: Demonstrate NeMo Gym patterns and concepts.

Environments for Training & Evaluation

Purpose: Training-ready environments with curated datasets.

📖 Documentation & Resources

• Documentation - Technical reference docs
• Training Tutorials - Train with NeMo Gym environments
• API Reference - Complete class and function reference

🤝 Community & Support

We'd love your contributions! Here's how to get involved:

• Report Issues - Bug reports and feature requests
• Contributing Guide - How to contribute code, docs, new environments, or training framework integrations

📚 Citations

If you use NeMo Gym in your research, please cite it using the following BibTeX entry:

@misc{nemo-gym,
  title = {NeMo Gym: An Open Source Library for Scaling Reinforcement Learning Environments for LLM},
  howpublished = {\url{https://github.com/NVIDIA-NeMo/Gym}},
  author={NVIDIA},
  year = {2025},
  note = {GitHub repository},
}