Full factorial
Include every combination of factor levels when estimating the planned effects and interactions is important and the resulting number of cells is feasible.
Stages 1–2 · Hypotheses & Design
Design controlled experiments on LLM behavior
Specify discrete factors, levels, cells and replications before sending a model a prompt.
Start with a question and measurable outcome
Define a question about how an OpenRouter-served model responds under specified conditions. Choose an outcome that can be observed in the single-turn text response and decide how it will be measured before running trials.
Treat models as research subjects in an inference-only workflow: the platform sends prompts and records text responses; it does not train models or inspect their internals. Researchers define the question, justify the hypothesis, and decide whether the planned outcome measures it.
Build a design around the research question
Cross discrete factors when the research question calls for comparing combinations of conditions. Specify which prompt and model conditions are included, then choose an observable outcome and replication plan that fits the question.
Full-factorial, fractional-factorial, matched-pair, and custom designs are available. Consider the number of cells, budget, assumptions, and limits on inference when selecting among them.
Choose a design that matches the question
Fractional factorial
Use a supported subset to reduce cells only when its aliasing and confounding structure is acceptable for the research question.
Matched-pair or custom cells
Match conditions for focused comparisons or select particular cells. These choices narrow what can be estimated and should be justified in advance.
Specify outcomes and cells
Define what will be observed in each response, enumerate condition combinations, and review the resulting cell count.
Set replication targets
Choose repetitions in light of expected variability, analysis assumptions, and a defensible sample-size plan. Replicating a prompt does not create independent topics; no default can guarantee adequate power.
Review any wizard proposal
AI assistance can draft factors and levels, but you check the construct, wording, feasibility, and design before running.
Scope and next steps
See the prompt-sensitivity walkthrough, then use prompt construction. The factorial-design glossary and study-scope FAQ provide more context.
Ready to plan your own design? Start a study.
Study LLM behavior with a guided research workflow.
Design controlled text studies, collect model responses, and explore what changes across conditions. You decide what the evidence means.