# IFEval checks instructions that a program can verify

> A benchmark of roughly 500 prompts turns constraints such as word counts and required keywords into reproducible checks.

Canonical: https://aischolar.cc/blog/ifeval-verifiable-instructions

[← All articles](https://aischolar.cc/blog)

AISCHOLAR / BLOG

# IFEval checks instructions that a program can verify

A benchmark of roughly 500 prompts turns constraints such as word counts and required keywords into reproducible checks.

November 8, 2026

### From a natural-language request to a pass-or-fail check

Whether an answer sounds friendly can invite disagreement between reviewers; whether it contains a requested keyword three times is easier to verify. Zhou and colleagues built [Instruction-Following Evaluation for Large Language Models](https://arxiv.org/pdf/2311.07911), or IFEval, around this narrower class of verifiable instructions. They identify 25 types and create around 500 prompts with one or more constraints, including writing more than 400 words or mentioning a keyword at least three times. Programmatic rules check the resulting responses, and the paper reports benchmark results for two widely available LLMs.

IFEval makes these compliance checks reproducible, but its categories cover only the instructions that can be operationalized in this way. A response can satisfy a word count while missing a user's broader purpose; performance on IFEval should therefore be read as performance on its verifiable constraints. The benchmark's roughly 500 prompts provide the scope of that claim.

### Bibliography

-   Zhou, J., Lu, T., Mishra, S., Brahma, S., Basu, S., Luan, Y., Zhou, D., & Hou, L. (2023). _Instruction-Following Evaluation for Large Language Models_. arXiv:2311.07911v1 \[cs.CL\]. https://arxiv.org/pdf/2311.07911

Original source: [https://arxiv.org/pdf/2311.07911](https://arxiv.org/pdf/2311.07911)

About AIScholar

## Study LLM behavior.

AIScholar is a research workspace for controlled, replicated studies of LLM behavior using single-turn text responses from OpenRouter-served models. The nine-stage workflow includes response coding, visualization, analysis and write-up; researchers validate measurements and interpret results.

[Explore AIScholar ↗](https://aischolar.cc/features)