Prompt sensitivity in LLM studies
Prompt sensitivity is the degree to which measured model responses change across prompts intended to preserve the same meaning. It describes result variability under wording changes, not necessarily a change in the underlying model.
What "Prompt sensitivity" means
Researchers compare responses across alternative wording while checking that variants preserve the intended meaning. They can treat wording as a planned factor or a source of variation and report how outcome distributions differ.
A wording study could compare a neutral task in plain prose and bullet-list form after checking that both versions preserve the same meaning.
The result applies to the tested variants and models; variants that change meaning confound the wording comparison.
Related terms
Variance decomposition
Partitioning outcome variance into condition, model, prompt-variant, and residual components.Semantic uncertainty
Shannon entropy over meaning-clustered responses — response consistency as a measurable outcome.Method Specification Prompt (MSP)
A generated method summary that helps researchers inspect and report study procedures.
How AIScholar uses it
AIScholar supports researcher-defined prompt conditions and analysis of response differences; researchers remain responsible for confirming that compared variants preserve meaning.
See the method in action.
Explore an illustrative prompt-sensitivity study, then review execution costs and scope before planning your own.