Department of Radiology
School of Medicine Queen's University
 

Critical Evaluation of a Published Paper

Critical Evaluation of a Published Paper

 

During the course of the semester we will be reading, presenting, “writing”, and critically evaluating journal articles (We will write our term papers as if we were writing a journal article, but with a modified Methods section and without the Results section). Below is a checklist that follows closely the format of a scientific report, conventionally divided into 6 sections. Please use the checklist as a reference for your a) term paper, b) peer review, AND c) as a guide when reading journal articles in general.

 

  1. Abstract

The abstract serves as a summary of the paper, presenting the purpose, scope, and major findings.  The title & abstract are often all that people will read, using this information to decide whether they want to continue. What did you do? What are the main results? What are your conclusions?

a) Is the abstract intelligible?  Does it…

b) …accurately describe the objectives & results of paper?

c) …include data not presented in the paper?

d) …include material that cannot be substantiated (conclusions unsupported by results)?

 

  1. Introduction

The introduction serves to logically present the background information/provide context for the study. 

What is the question (research/scientific hypothesis)? Why is it important (rationale & justification)? What are the alternative hypotheses & how do you test among them (statistical hypotheses)? Remember that a lot of studies begin by observing a pattern (correlation/association). Your research hypothesis states an explanation for this pattern (story), and a statistical hypothesis determines the generalities of the pattern. Hence, we test our predictions (statistical hypotheses) using statistical tests & use results of these tests to either support or refute the research hypothesis.

a) Did the authors indicate why the study was undertaken?

b) Was the background information adequate to understand the aims & objectives of  

     the study?

 

  1. Methods

The methods section should be a clear & succinctly stated, chronological description of what you did & how you did it.  Could someone else repeat the research with the information provided?  If the answer is “no,” your methods section is incomplete.

a) Were methods described in sufficient detail for others to repeat or extend the study?

b) Were adequate references cited if standard methods were used?

c) If methods were modified, were modifications described carefully?

d) Have the authors indicated why particular procedures were used, the potential problems of the methods used, & limitations of their methods?

e) Have the authors specified the statistical procedures used?

f) Are the statistical methods appropriate?

 

  1. Results

The results section is meant to highlight trends in the data (most often presented in figures and/or tables). Text should compliment the tables/figures, NOT repeat the information presented therein.

a) Are the results appropriate for the stated objectives?

b) Do the results make sense?

c) Do tables & figures clearly describe the data?

d) Have the appropriate statistical analyses been performed on the data?

 

  1. Discussion

Use this section to synthesize your results & to tie your results to the literature.

Do not repeat your results…relate them to other studies. What are the potential explanations for the results? Have other studies come to similar/different conclusions? How do you account for those discrepancies? WHAT is the take home message? WHAT is the “so what” about your work? Remember your scope of inference (don’t stretch your results too far). For example, if you sampled one species of lizard from a single county in Arizona, results do not apply to all lizards in North America.

a) Were the objectives of the study met? If not, do authors have an explanation as to

    why?

b) Were statistical hypotheses clearly supported or refuted?

c) Are results discussed in relation to similar studies?

d) Do authors indulge in needless speculation?

e) If results are statistically significant, are they also biologically significant?

f) Do authors adequately interpret their data & discuss the limitations of their study?

 

  1. References

a) Do authors cite appropriate papers for comments made?

b) Do authors cite their own publications needlessly?

Use the format of Journal of Physiology (http://jp.physoc.org/) for your list of references, as well as for the parenthetical notation throughout your paper. Try to find recent references (within the last 2 or 3 years) so that your analysis is up to date.