# A rule can have exceptions; models struggled to find them

> MoralExceptQA varies circumstances around three rules to test when human judgments permit breaking them.

Canonical: https://aischolar.cc/blog/moral-rule-exceptions

[← All articles](https://aischolar.cc/blog)

AISCHOLAR / BLOG

# A rule can have exceptions; models struggled to find them

MoralExceptQA varies circumstances around three rules to test when human judgments permit breaking them.

November 22, 2026

### What makes breaking a rule permissible?

A line-cutting rule sounds straightforward until someone urgently needs medicine or a bathroom cleaner needs to pass. Zhijing Jin and colleagues use such cases to test flexible rule application in [_When to Make Exceptions: Exploring Language Models as Accounts of Human Moral Judgment_](https://arxiv.org/pdf/2210.01478). MoralExceptQA covers three norms: no cutting in line, no interfering with another person’s property, and a new camp rule against cannonballing into a pool.

The authors built 148 vignettes by varying factors such as whether a rule’s purpose was undermined and who would benefit or be harmed by breaking it. They compared language-model predictions with human permissibility judgments and introduced MoralCoT, a multi-step prompt based on contractualist moral reasoning. MoralCoT achieved 64.47% F1, 6.2 points above the Delphi++ comparison. Several systems clustered toward strict rule-following or permissive exceptions.

The human judgments came from U.S. residents, and three norms cannot represent the range of moral rules. The authors also note the dataset was too small to hold out a development set for prompt tuning. Wider cultural samples and additional rules are the next test of whether the same reasoning structure tracks broader human judgments.

### Bibliography

-   Jin, Z., Levine, S., Gonzalez, F., Kamal, O., Sap, M., Sachan, M., Mihalcea, R., Tenenbaum, J., & Schölkopf, B. (2022). _When to Make Exceptions: Exploring Language Models as Accounts of Human Moral Judgment_. arXiv:2210.01478v3 \[cs.CL\]. [Full paper (PDF)](https://arxiv.org/pdf/2210.01478).

Original source: [https://arxiv.org/pdf/2210.01478](https://arxiv.org/pdf/2210.01478)

About AIScholar

## Study LLM behavior.

AIScholar is a research workspace for controlled, replicated studies of LLM behavior using single-turn text responses from OpenRouter-served models. The nine-stage workflow includes response coding, visualization, analysis and write-up; researchers validate measurements and interpret results.

[Explore AIScholar ↗](https://aischolar.cc/features)
