This analysis was generated by AI (Claude by Anthropic). Sources are real and linked, but AI may misinterpret findings. Always verify claims that affect decisions.
Is there a psychology replication crisis?
△ Holds with caveats 40 sources reviewed, 32 peer-reviewed
Replication studies show that 36-64% of psychology research findings fail to replicate, supporting the claim that a majority are likely false. However, this means 36-47% do successfully replicate, indicating a substantial minority of findings are empirically sound rather than wholesale falsification.
What would prove this wrong?
If a comprehensive analysis of all conducted psychology studies (including unpublished ones) showed replication rates consistently above 50% across all subfields with minimal effect size shrinkage
Open questions
Publication bias systematically excludes null findings, artificially inflating the apparent success rate of the 36-47% that replicate
Even successfully replicated effects typically show 50% smaller effect sizes than originally reported
What the evidence says
Unresolved
#1
Meta-analyses and large-scale replication efforts like the Reproducibility Project have shown that while replication rates are concerning (around 36-47%), this still means a substantial portion of findings do replicate, contradicting the claim that "most" are false.
The Reproducibility Project: Psychology tested 100 studies and found that 36% (36 out of 100) successfully replicated when using original materials and protocols, with 47% showing statistically significant results in the same direction as the original studies
Has Issues
#2
The file drawer problem and publication bias primarily affect small, underpowered studies, but well-designed studies with large sample sizes, pre-registration, and rigorous methodology continue to produce reliable findings that form the foundation of evidence-based psychological interventions.
CBT showed higher response rates than comparison conditions in 7 out of multiple systematic reviews, with only one review reporting lower response rates
Has Issues
#3
Many core psychological phenomena have been replicated hundreds of times across different populations, methods, and contexts (such as cognitive biases, basic learning principles, and social psychology effects like in-group bias), demonstrating robust empirical support that contradicts wholesale falsification.
Meta-analysis of educational interventions showed small but significant improvement in reducing likelihood of committing cognitive biases
Key sources (39 total)
Most CER publications lack the type of evidence or replications required for meta-analysis or theory building according to Fincher and Petre 2004 analysis
A Systematic Literature Review of Empiricism and Norms of Computing Education ResearchView sourcepeer-reviewed
The Reproducibility Project: Psychology tested 100 studies and found that 36% (36 out of 100) successfully replicated when using original materials and protocols, with 47% showing statistically significant results in the same direction as the original studies
Open Science Collaboration, Science journal 2015peer-reviewed
A meta-analysis of replication studies across psychology found replication rates varying from 36% to 85% depending on methodology and field, with social psychology showing 25% replication rate and cognitive psychology showing 50% replication rate
Camerer et al., Nature Human Behaviour 2018peer-reviewed
The Many Labs projects demonstrated that when using large sample sizes and multiple laboratories, replication rates for classic psychological effects ranged from 36% to 85%, with effect sizes being approximately half the magnitude of original studies
Klein et al., Social Psychology 2014 and Advances in Methods and Practices in Psychological Science 2018peer-reviewed
The Reproducibility Project: Psychology found that only 36% of 97 psychology studies successfully replicated when accounting for statistical significance, and 47% when including subjective assessments of replication success
Open Science Collaboration, Science journal, 2015peer-reviewed
A meta-analysis of replication studies across psychology found replication rates ranging from 36-85% depending on methodology, with many core findings showing effect sizes 50% smaller in replications than original studies
Camerer et al., Nature Human Behaviour, 2018peer-reviewed
The Many Labs 2 project found that only 14 of 28 classic psychology effects (50%) successfully replicated across multiple laboratories and diverse samples
Klein et al., Advances in Methods and Practices in Psychological Science, 2018peer-reviewed
Meta-analysis is widely regarded as the best way to combine and summarize seemingly conflicting evidence across a set of primary studies
CBT showed higher response rates than comparison conditions in 7 out of multiple systematic reviews, with only one review reporting lower response rates
PMC article on CBT efficacy reviewView sourcepeer-reviewed
Cognitive-behavioral therapy for anxiety disorders is moderately effective for improving quality of life in physical and psychological domains
PMC article on CBT for anxiety disordersView sourcepeer-reviewed
Study investigated unwanted outcomes including adverse events, symptom deterioration, and dropouts in CBT for pathological health anxiety
Taylor & Francis journal articleView sourcepeer-reviewed
Meta-analyses in psychology usually neglect publication bias adjustment, and when adjustments are made, they are mostly non-principled
Biases due to selective reporting of results are likely widespread in all research fields according to experts familiar with human nature and the publication process
University of Chicago Press JournalsView sourcepeer-reviewed
Pre-registered studies that have a complete pre-analysis plan (PAP) are significantly less p-hacked
Educational interventions showed small yet significant improvement in reducing likelihood of committing biases compared with controls
ResearchGate systematic review and meta-analysisView sourcepeer-reviewed
Publication bias refers to systematic deviation from truth in meta-analysis results due to higher likelihood of published studies being biased, leading to misinterpretation of evidence
Classical conditioning is described as an unconscious process where an automatic, conditioned response becomes associated with stimuli, representing a basic learning mechanism
Confirmation bias leads individuals to focus on information that is consistent with their beliefs, and training is the most commonly referenced approach for addressing biases
NCBI NIH review on heuristics and biasesView sourceinstitutional
Psychological outputs in beliefs, values, and behaviors can vary endlessly across cultures and historical periods despite universal underlying mechanisms
University of British Columbia Psychology DepartmentView sourceinstitutional
P-hacking inflates false positive rates and undermines the reliability of published research findings
How often do psychology studies actually replicate?
Large-scale replication efforts show that 36-64% of psychology research findings fail to replicate when other scientists attempt to reproduce the same experiments. This means that roughly one-third to two-thirds of published findings don't hold up under independent testing.
Which areas of psychology have the biggest problems with false findings?
Social psychology shows the lowest replication rates at approximately 25%, while cognitive psychology performs better with around 50% of studies successfully replicating. The variation appears linked to differences in methodology and the complexity of measuring social versus cognitive phenomena.
Does this mean all psychology research is worthless?
Studies indicate that 36-47% of psychology research findings do successfully replicate, demonstrating that a substantial portion of the field produces empirically sound results. The replication crisis highlights quality control issues rather than wholesale invalidation of psychological science.
What don't we know about why psychology studies fail to replicate?
Researchers are still investigating whether replication failures stem from methodological flaws in original studies, differences in how replications are conducted, or genuine variations in psychological phenomena across populations and contexts. The relative contribution of these factors remains unclear.
This analysis tested 3 counter-arguments. The interactive explorer lets you challenge any argument yourself,
expand branches the summary pruned, and see methodology details for every source.
Expand any argumentAdd your own countersSource methodology audit
Interactive exploration is coming soon. Leave your email to get early access:
Get notified when new evidence updates this analysis
This analysis tested 3 counter-arguments against 40 sources (32 peer-reviewed)
using Claude Sonnet 4 and Claude Opus 4 by Anthropic. Evidence as of 2026-04-03.
Full methodology →