Built on Burn

Reinforcement learning that feels like Rust. Compose agents from swappable parts, catch wiring mistakes when you compile, and train at native speed with Burn on CPU, CUDA or Metal.

Get started

Backends

  • CUDA
  • Metal
  • ndarray

Environments

  • Classic control
  • Box2D
  • MuJoCo
  • Atari

Benchmarks

End-to-end Breakout PPO throughput.
1.6× Stable-Baselines3, 1.4× CleanRL.
40,960 transitions with matched preprocessing and training workloads.

Benchmark methodology
BREAKOUT / PPO TRANSITIONS / SEC ↑
ModuRL 6,000 /s
CleanRL 4,178 /s
Stable-Baselines3 3,737 /s

CUDA · Throughput only, not final reward or sample efficiency. Emulator versions differ. See the benches README for methodology and fairness controls.

Build an agent

Select an algorithm, and for PPO or A2C a network topology, to see an assembly example.

Algorithm
Network
your_agent.rs RUST
// CartPole · separate models · PPO
use burn::tensor::Device;
use modurl::prelude::*;
use modurl_gym::classic_control::cartpole::CartPoleV1;

let device = Device::flex().autodiff();
let env = CartPoleV1::builder().seed(7).build()?;
let env = VectorizedGymWrapper::from(env);
let mut env = DeviceMultiGymWrapper::new(env, Device::flex(), device.clone());
let observation_space = env.observation_space();
let action_space = env.action_space();
// actor_model, critic and optimizers: setup omitted; see the guide.
let policy = ProbabilisticPolicyModel::new(
    actor_model,
    CategoricalDistribution,
);

let networks = SeparatePPONetwork::builder()
    .actor_network(policy)
    .critic_network(critic)
    .actor_optimizer(actor_optimizer)
    .critic_optimizer(critic_optimizer)
    .actor_learning_rate(3e-4)
    .critic_learning_rate(3e-4)
    .build()?;

let mut agent = PPOAgent::builder()
    .observation_space(observation_space)
    .action_space(action_space)
    .network_info(networks)
    .batch_size(2_048)
    .mini_batch_size(64)
    .clip_range(ConstantSchedule::new(0.2))
    .training_horizon(100_000)
    .build()?;

agent.learn(&mut env, 100_000)?;

Algorithms

Implementations for on-policy and off-policy training.

Quick start

Train PPO on CartPole.

git clone https://github.com/ModuRL/ModuRL.git
cd ModuRL
cargo run --release -p examples --example ppo_cartpole

Requires a Rust toolchain with edition 2024 support. Read the getting started guide

ModuRL gear mascots

Start training in Rust.