# Two failure modes help explain why safety-trained models can be jailbroken

> Wei, Haghtalab, and Steinhardt use competing objectives and mismatched generalization to design and test jailbreaks against several safety-trained language models.

Canonical: https://aischolar.cc/blog/jailbroken-training-failures

[← All articles](https://aischolar.cc/blog)

AISCHOLAR / BLOG

# Two failure modes help explain why safety-trained models can be jailbroken

Wei, Haghtalab, and Steinhardt use competing objectives and mismatched generalization to design and test jailbreaks against several safety-trained language models.

November 12, 2026

[Wei, Haghtalab, and Steinhardt](https://arxiv.org/abs/2307.02483v1) organize jailbreak design around two failures that can emerge from safety training: competing objectives and mismatched generalization. Their analysis leads to a test of 30 jailbreak methods against harmful-prompt datasets and several safety-trained systems, including GPT-4, Claude v1.3, and GPT-3.5 Turbo.

### Two routes from capability to failure

In the competing-objectives account, a model's ability to follow a request conflicts with its safety objective. Mismatched generalization describes safety training that has not carried over to a domain where the model retains relevant capabilities. The authors use these hypotheses to design attacks and find that vulnerabilities persist in the tested models despite red-teaming and safety training.

The paper frames itself as an early exploration of robustness. Its results concern specific model versions, harmful prompts, and attack designs; they give no deployment-rate estimate. Across these test conditions, the useful detail is which failure hypothesis guided an attack and which model response it elicited.

### Bibliography

-   Wei, A., Haghtalab, N., & Steinhardt, J. (2023). _Jailbroken: How Does LLM Safety Training Fail?_ arXiv:2307.02483v1. [https://arxiv.org/pdf/2307.02483](https://arxiv.org/pdf/2307.02483)

Original source: [https://arxiv.org/pdf/2307.02483](https://arxiv.org/pdf/2307.02483)

About AIScholar

## Study LLM behavior.

AIScholar is a research workspace for controlled, replicated studies of LLM behavior using single-turn text responses from OpenRouter-served models. The nine-stage workflow includes response coding, visualization, analysis and write-up; researchers validate measurements and interpret results.

[Explore AIScholar ↗](https://aischolar.cc/features)