Wei, Haghtalab, and Steinhardt organize jailbreak design around two failures that can emerge from safety training: competing objectives and mismatched generalization. Their analysis leads to a test of 30 jailbreak methods against harmful-prompt datasets and several safety-trained systems, including GPT-4, Claude v1.3, and GPT-3.5 Turbo.
Two routes from capability to failure
In the competing-objectives account, a model's ability to follow a request conflicts with its safety objective. Mismatched generalization describes safety training that has not carried over to a domain where the model retains relevant capabilities. The authors use these hypotheses to design attacks and find that vulnerabilities persist in the tested models despite red-teaming and safety training.
The paper frames itself as an early exploration of robustness. Its results concern specific model versions, harmful prompts, and attack designs; they give no deployment-rate estimate. Across these test conditions, the useful detail is which failure hypothesis guided an attack and which model response it elicited.
Bibliography
- Wei, A., Haghtalab, N., & Steinhardt, J. (2023). Jailbroken: How Does LLM Safety Training Fail? arXiv:2307.02483v1. https://arxiv.org/pdf/2307.02483