OpenRLHF
MLOps & Training · Free + paid
Scalable RLHF framework supporting 70B+ PPO full tuning, iterative DPO, and LoRA
At a glance
- Made by
- OpenRLHF
- Korean
- None (English and a few others only)
- Available on
- CLI
- Open source
- Open source (the license name is not listed on the page)
- GitHub
- 10,083 stars · Apache-2.0
What is OpenRLHF?
OpenRLHF is an open-source RLHF framework that scales to models of 70B and above with a Ray + vLLM distributed architecture. It reduces the generation bottleneck with vLLM and trains HuggingFace checkpoints without conversion using DeepSpeed ZeRO-3. Algorithms such as PPO, REINFORCE++, GRPO, and RLOO can be switched with a single flag, and it supports single-turn rewards and multi-turn agent environments. It also supports SFT, reward model, and DPO training, as well as vision-language model (VLM) RLHF.
- Ray + vLLM distributed architecture (70B+ models)
- Switch between RL algorithms such as PPO, REINFORCE++, GRPO, and RLOO
- Hybrid engine (Actor, Critic, Reward, and vLLM coexist on the same GPU)
- Asynchronous training and partial rollouts
- Custom reward functions and multi-turn agent environments
- Vision-language model (VLM) RLHF
- SFT, reward model, DPO, LoRA and QLoRA, SLURM multi-node
Who uses OpenRLHF and for what
- Reinforcement fine-tuning with custom Python reward functions
Pricing
Open source + paid cloud-managed plans
Open-source version can be self-hosted for free, managed cloud services are paid based on usage/seats
Estimate based on AI knowledge - please verify on the official site
AI tools similar to OpenRLHF (alternatives)
OpenRLHF vs. alternatives at a glance
| AI | Pricing | Combos on the map | Connected tools | Free | Cheapest monthly | Korean |
|---|---|---|---|---|---|---|
| OpenRLHF | Free + paid | 0 | 3 | — | — | No |
| Axolotl | Free + paid | 0 | 3 | Yes | — | No |
| TRL | Free + paid | 0 | 3 | — | — | No |
| unsloth | Free + paid | 0 | 3 | Yes | — | No |
2 of these 4 tools have a free plan.
Combo and connection counts are based on the relationships recorded on AI Galaxy Map (2026-10-26); they are not a quality ranking.
OpenRLHF on AI Galaxy Map
- One of 45 AI tools in the MLOps & Training field.
- Connected to 3 other tools — #38 in its field.
Frequently asked questions
Can I use OpenRLHF for free?
Open source + paid cloud-managed plans. Prices can change, so check the official site before paying.
What AI can I use instead of OpenRLHF?
Tools that do the same job, such as Axolotl, TRL, unsloth, are connected to it on the map.
Can I use OpenRLHF in Korean?
None (English and a few others only)
Sources (official pages)
Checked on the official pages on 2026-10-14. Prices and features can change, so check the official site again before paying.