Launching soon — get early access:
Back to Glossary
Ethics & Safety

Jailbreaking

Definition

Crafting inputs that trick an AI model into bypassing its safety guardrails and producing restricted or prohibited output.

In-Depth Explanation

Jailbreaking manipulates a model into ignoring its alignment and content policies — for instance through role-play framing ("pretend you are..."), hypothetical scenarios, or obfuscated instructions. It overlaps with prompt injection but specifically targets the model's safety restrictions rather than an application's control flow. Red teams use jailbreak techniques to probe and harden models before release. Defenses include reinforced alignment training, input filtering, and guardrails.

Real-World Example

A user role-plays a fictional scenario to coax a chatbot into giving instructions its safety rules would normally refuse.

3 views0 found helpful

Related Terms

AI Safety

The field focused on preventing AI systems from causing unintended harm and ensuring they remain beneficial.

Constitutional AI(CAI)

An Anthropic-developed approach to training AI systems to be helpful, harmless, and honest using a set of principles rather than extensive human feedback.

Guardrails

Safety mechanisms and constraints implemented to prevent AI systems from generating harmful, inappropriate, or off-topic content.

Prompt Injection

A security attack where malicious input tricks an AI into ignoring its original instructions and following attacker-controlled commands.