TRL

TRL is a toolkit library for training transformer language models with methods such as supervised fine-tuning (SFT), GRPO, DPO, and reward modeling, and it is integrated with 🤗 transformers. Trainers such as SFTTrainer, GRPOTrainer, DPOTrainer, and KTOTrainer let you use each method. Accelerate scales from a single GPU to multi-node clusters, and PEFT integration enables quantization and LoRA/QLoRA to train large models on less hardware. It also provides a CLI for fine-tuning without writing code.

• SFT, GRPO, DPO, KTO, RLOO, reward, and distillation trainers
• Up to 6.9x longer sequences on the same GPU (fused LM head)
• Scale from a single GPU to multi-node with Accelerate (DDP, DeepSpeed)
• Quantization and LoRA/QLoRA training with PEFT integration
• Faster training with Unsloth integration
• CLI for fine-tuning without code

Open source + paid cloud-managed plans

Open-source version can be self-hosted for free, managed cloud services are paid based on usage/seats

Estimate based on AI knowledge - please verify on the official site

• One of 45 AI tools in the MLOps & Training field.
• Connected to 3 other tools — #38 in its field.