Jailbreak papers often report success rates from different prompts, scoring rules, and query budgets. Chao and colleagues built JailbreakBench to put those choices into a shared evaluation protocol.
What the benchmark releases
The package includes 100 harmful-behavior goals, an evolving repository of attack artifacts, a public leaderboard, and a framework specifying the threat model, system prompts, chat templates, and scoring. The authors evaluate several attacks and defenses on open- and closed-source models.
Attack results under a common protocol
In the paper's reported comparison, “Prompt with RS” reached 90% attack success on Llama-2 and 78% on GPT-4; the average query count was two for Vicuna and three for GPT-3.5. Defense results depended on the attack: Perplexity Filter worked against GCG, while the authors report that other tested defenses had different success across attack methods. The rates belong to these models and benchmark conditions.
The attacker is defined narrowly
The current benchmark focuses on text and excludes system-prompt modification and response prefilling. Its defense evaluation reuses attacks found against undefended models, a non-adaptive test that the authors describe as a simple case; an adaptive attacker could change the outcome. Each score therefore needs its threat model alongside it, especially when comparing defenses.
Bibliography
- Chao, P., Debenedetti, E., Robey, A., Andriushchenko, M., Croce, F., Sehwag, V., Dobriban, E., Flammarion, N., Pappas, G. J., Tramer, F., Hassani, H., & Wong, E. (2024). JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models. arXiv:2404.01318v5. https://arxiv.org/pdf/2404.01318