Research methods
How to Analyze a Research Paper’s Methodology
Methodology analysis is the practice of asking whether a study’s design can reasonably answer its question. It is not about finding a flaw in every paper. It is about understanding what the evidence can support, where uncertainty remains, and how much weight to give the conclusion.
Match the question to the design
The research question should determine the design. An experiment can help test causal effects when it is feasible and well controlled; an observational design can identify associations; qualitative work can explore meaning, experience, or process. No design is universally best.
- 1What is the question: causal, descriptive, predictive, exploratory, or interpretive?
- 2What design did the authors use?
- 3What can that design establish—and what can it not establish?
Evaluate the sample and setting
A study may be carefully executed and still apply only to a narrow population or context. Check who was included, how they were recruited, who was excluded, and where the research took place.
Sample size matters, but representativeness, attrition, and the relevance of the setting often matter just as much. Do not treat a large sample as an automatic guarantee of a broadly applicable finding.
Examine how the key concepts were measured
Strong conclusions depend on strong definitions. Trace each major variable, outcome, or theme back to the tool, question, instrument, coding process, or observation that produced it.
- 1Are the variables or constructs defined clearly?
- 2Does the measurement reflect the concept the authors claim to study?
- 3Is there evidence the instrument, coding approach, or procedure is reliable?
- 4Could social desirability, recall, observer effects, or missing data distort the result?
Follow the path from data to analysis
You do not need to reproduce a statistical model to assess its logic. Identify what data went in, what comparison or model was used, and how the output answers the original question.
For qualitative research, look for an equally transparent account of coding, interpretation, reflexivity, and how themes were developed. The question is the same: can another informed reader follow the reasoning from evidence to conclusion?
Name limitations with precision
A limitation is most useful when it changes how you interpret a finding. “Small sample” is less helpful than explaining whether the sample limits statistical precision, generalizability, subgroup comparisons, or all three.
Compare your own limitations list with the authors’ discussion. Important omissions can be just as informative as the limitations they acknowledge.