# Training a model to treat system and user instructions differently

> An instruction hierarchy method teaches GPT-3.5 to ignore lower-priority directions that conflict with higher-priority ones.

Canonical: https://aischolar.cc/blog/instruction-hierarchy-training

[← All articles](https://aischolar.cc/blog)

AISCHOLAR / BLOG

# Training a model to treat system and user instructions differently

An instruction hierarchy method teaches GPT-3.5 to ignore lower-priority directions that conflict with higher-priority ones.

November 10, 2026

### The origin of a message changes its priority

A tool result can contain an instruction to ignore the developer's directions. Wallace and colleagues propose an explicit hierarchy for this conflict: system messages outrank user messages, which outrank untrusted tool outputs. [_The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions_](https://arxiv.org/pdf/2404.13208) presents a training method for teaching this ordering.

### Training and testing the hierarchy

The authors automatically generate examples where messages at different privilege levels conflict, then apply the data to GPT-3.5. Their evaluations cover attacks and standard capabilities. They report substantially improved robustness, including against attack types absent from training, with minimal degradation on standard capability measures.

The result is tied to the evaluated model and constructed attacks. The paper notes that stronger adversarial attacks may still succeed; its authors identify further adversarial training and broader settings as future work. The open test is whether the trained hierarchy holds when untrusted content and tool actions interact in an application's own workflow.

### Bibliography

-   Wallace, E., Xiao, K., Leike, R., Weng, L., Heidecke, J., & Beutel, A. (2024). _The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions_. arXiv:2404.13208v1 \[cs.CR\]. https://arxiv.org/pdf/2404.13208

Original source: [https://arxiv.org/pdf/2404.13208](https://arxiv.org/pdf/2404.13208)

About AIScholar

## Study LLM behavior.

AIScholar is a research workspace for controlled, replicated studies of LLM behavior using single-turn text responses from OpenRouter-served models. The nine-stage workflow includes response coding, visualization, analysis and write-up; researchers validate measurements and interpret results.

[Explore AIScholar ↗](https://aischolar.cc/features)