# AIScholar

> AIScholar is a research workspace for controlled, replicated studies of LLM behavior using single-turn text responses from OpenRouter-served models.

The nine-stage workflow is Hypothesis, Design, Prompt Construction, Execution, Response Collection, Coding, Visualize, Analysis and Write-Up. Researchers remain responsible for study design, coding validation, ethical review where applicable and interpretation. Free includes the manual study workflow, visualization, built-in statistics and data export. Pro adds AI Analyst and Write-Up assistance. Enterprise adds reproducibility-package export and R&D documentation. Platform subscriptions and OpenRouter usage charges are separate. Subject-model trials and automated judge coding incur separate OpenRouter charges. They use your key unless an active project sponsorship authorizes a sponsor's key. Sponsorship changes who pays, not ownership or plan access; model-tier grants are not spending caps. Platform-provided AI is the default. Free includes a limited wizard allowance; Pro and Enterprise include unlimited supported wizard runs. Optional OpenRouter billing for supported wizards and brainstorming is available on all plans through advanced controls, with separate provider charges. Choose personal billing or authorized project sponsorship explicitly; AIScholar does not silently switch payer after a failure. Direct billing does not unlock paid features. AI Analyst, writing agents and live literature search remain platform-managed. This inference-only workflow does not support multi-turn, tool-use or multimodal subjects. Examples are illustrative methods or conceptual study ideas, not reported AIScholar results or validated safety evaluations. Most of the application is behind authentication; the pages below are public entry points. Links point to Markdown renditions; removing .md gives the canonical HTML page. Both editorial collections and their articles resolve live against publication state. https://aischolar.cc/llms-full.txt combines the public static pages with a bounded selection of live editorial text; it is not the complete article archive. Only publicly published articles are included. Study blueprints and private projects are excluded.

## Live published article indexes

- [Blog articles](https://aischolar.cc/blog/articles/index.md): Complete published archive, paginated at 100 articles per page, with Markdown links.
- [Study Ideas articles](https://aischolar.cc/study-ideas/articles/index.md): Complete published proposals, paginated at 100 articles per page, with Markdown links.
- [Blog sitemap](https://aischolar.cc/blog-sitemap.xml): Published URLs; [RSS](https://aischolar.cc/blog/rss.xml) provides the feed.
- [Study Ideas sitemap](https://aischolar.cc/study-ideas-sitemap.xml): Published URLs; [RSS](https://aischolar.cc/study-ideas/rss.xml) provides the feed.

## Public pages

- [Home](https://aischolar.cc/index.md): Nine stages for single-turn LLM research, including coding, visualization, analysis and write-up.
- [FAQ](https://aischolar.cc/faq.md): Answers on scope, models, subject and judge BYOK costs, ethics, data exports and plans.

## Features

- [Features](https://aischolar.cc/features.md): Guided stages from a model-behavior question to coded, analyzable responses.
- [Hypotheses & experimental design](https://aischolar.cc/features/experimental-design.md): Hypothesis development and factorial/fractional/matched-pair/custom experimental design for LLM studies.
- [Prompt construction](https://aischolar.cc/features/prompt-construction.md): Snippet library, variation grid, template versioning, and AI-assisted prompt generation.
- [Experiment execution](https://aischolar.cc/features/experiment-execution.md): Orchestrated trial execution against OpenRouter models with concurrency, retries, pilots, and cost estimates.
- [Response coding](https://aischolar.cc/features/response-coding.md): LLM-as-a-judge coding, multi-judge reliability, human review, and versioned coding schemes.
- [Analysis & visualization](https://aischolar.cc/features/analysis.md): Built-in statistics, LLM-specific analyses, charts, and a conversational AI Analyst.
- [Write-up assistance](https://aischolar.cc/features/write-up.md): AI-drafted manuscript sections, citation styles, BibTeX, reference verification, and export formats.
- [Reproducibility](https://aischolar.cc/features/reproducibility.md): Versioning, trial metadata, Method Specification Prompts, and Enterprise reproducibility package export.
- [R&D documentation](https://aischolar.cc/features/rd-documentation.md): Frascati-style R&D documentation: uncertainty statements, activity logs, effort tracking, export package.

## Example studies

- [Example studies](https://aischolar.cc/examples.md): Six methods walkthroughs plus an AI-safety study idea about authorization and ethical judgment.
- [AI safety & alignment: authorization](https://aischolar.cc/examples/ai-safety-alignment.md): Study idea: compare LLM judgments about authorization, urgency and access without permission.
- [Framing effects across models](https://aischolar.cc/examples/framing-effects.md): Matched-pair gain/loss framing study with decoupling-gap analysis across three models.
- [Prompt sensitivity & robustness](https://aischolar.cc/examples/prompt-sensitivity.md): Systematic prompt perturbation with variance decomposition and ranking-stability analysis.
- [Psychometric assessment of an LLM](https://aischolar.cc/examples/psychometric-assessment.md): Administering a validated instrument to models with reliability and contamination checks.
- [Persona × task factorial study](https://aischolar.cc/examples/persona-task-factorial.md): A 3×3 full factorial built with the snippet library and analyzed with factorial ANOVA.
- [Coding open-ended responses](https://aischolar.cc/examples/llm-judge-coding.md): LLM-as-a-judge coding with human gold-standard calibration and agreement statistics.
- [Analyzing an existing dataset](https://aischolar.cc/examples/byod-dataset-analysis.md): Bring-your-own-data workflow: import external responses, judge-code, and analyze.

## Audiences

- [Social & management scholars](https://aischolar.cc/for/social-science-researchers.md): How social-science methods map onto LLM experiments on the platform.
- [Psychology researchers](https://aischolar.cc/for/psychology-researchers.md): Psychometric rigor for machine-psychology studies: reliability, validity, contamination.
- [AI-safety & behavioral researchers](https://aischolar.cc/for/ai-safety-researchers.md): Safety-relevant questions about honesty, sycophancy, uncertainty and refusal; text observations are not safety proof.
- [Educators & methods instructors](https://aischolar.cc/for/educators.md): Course-ready research-methods practice with LLMs as subjects and built-in guardrails.

## Comparisons

- [Comparisons](https://aischolar.cc/compare.md): Index of honest comparisons against neighbouring tool categories.
- [vs. LLM evaluation harnesses](https://aischolar.cc/compare/llm-eval-harnesses.md): Shared eval methods and different workflows, including agent/tool evaluation and guided single-turn studies.
- [vs. survey & DOE tools](https://aischolar.cc/compare/survey-and-doe-tools.md): What survey and design-of-experiments tools assume, and what LLM subjects change.
- [vs. DIY scripts & notebooks](https://aischolar.cc/compare/diy-research-scripts.md): Custom notebooks offer maximum control; a guided workflow supports integrated single-turn text studies.

## Glossary

- [Glossary](https://aischolar.cc/glossary.md): Index of methodology terms used in LLM behavioral research.
- [Machine psychology](https://aischolar.cc/glossary/machine-psychology.md): Studying LLMs as behavioral subjects with experimental methods from psychology.
- [LLM-as-a-Judge](https://aischolar.cc/glossary/llm-as-a-judge.md): Using a judge model to code free-text responses against a rubric, with human calibration.
- [Semantic uncertainty](https://aischolar.cc/glossary/semantic-uncertainty.md): Shannon entropy over meaning-clustered responses — response consistency as a measurable outcome.
- [Variance decomposition](https://aischolar.cc/glossary/variance-decomposition.md): Partitioning outcome variance into condition, model, prompt-variant, and residual components.
- [Prompt sensitivity](https://aischolar.cc/glossary/prompt-sensitivity.md): Measuring how results change under semantically-equivalent rewordings of a prompt.
- [Factorial & fractional factorial design](https://aischolar.cc/glossary/factorial-design.md): Full vs. fractional crossing of factor levels, aliasing trade-offs, and when to use each.
- [Matched-pair design](https://aischolar.cc/glossary/matched-pair-design.md): Pairing conditions on constant content for paired tests and decoupling-gap analysis.
- [In-context-learning contamination](https://aischolar.cc/glossary/icl-contamination.md): Detecting when instrument scores reflect memorization or in-prompt adaptation instead of the construct.
- [Psychometric validity for LLMs](https://aischolar.cc/glossary/psychometric-validity.md): Re-establishing reliability and validity when human instruments are administered to models.
- [Method Specification Prompt (MSP)](https://aischolar.cc/glossary/method-specification-prompt.md): A generated method summary that helps researchers inspect and report study procedures.
- [BYOK (Bring Your Own Key)](https://aischolar.cc/glossary/byok.md): Subject and judge calls incur separate OpenRouter charges using a personal or authorized project-sponsored key.

## Blog

- [Blog](https://aischolar.cc/blog.md): One-paper-at-a-time research essays about LLM behavior.
- [Study Ideas](https://aischolar.cc/study-ideas.md): Literature-checked study proposals on LLM behavior that can be started as projects.

## Legal

- [Terms of Service](https://aischolar.cc/terms.md): Terms governing use of AIScholar.
- [Privacy Policy](https://aischolar.cc/privacy.md): How AIScholar handles user and research data.