LlamaGuard

Llama Guard is an LLM-based input and output safeguard model for human-AI conversation use cases. It has a safety risk taxonomy, and it classifies LLM prompts (prompt classification) as well as the LLM responses to those prompts (response classification). It is a Llama2-7b model instruction-tuned on a collected dataset, and the paper states that it matched or outperformed existing content moderation tools on benchmarks such as OpenAI Moderation Evaluation and ToxicChat. It produces multi-class classification and binary decision scores, and the classification categories can be adapted to the use case, or other taxonomies can be used via zero-shot or few-shot prompting. Meta states that it releases the model weights (as of the paper page, 2023-12-07).

• Prompt classification (input check)
• Response classification (output check)
• Safety risk taxonomy
• Multi-class classification and binary decision scores
• Customizing classification categories
• Using other taxonomies via zero-shot or few-shot prompting
• Model weights released

Open source + paid enterprise plans

Some open-source tools are free; enterprise security platforms are billed under separate contracts

Estimate based on AI knowledge - please verify on the official site

• One of 21 AI tools in the AI Security & Safety field.
• Connected to 6 other tools — #1 in its field.