Critical appraisal is the process of carefully and systematically examining research to judge its trustworthiness, and its value and relevance in a particular context. It is an essential skill for evidence-based medicine because it allows people to find and use research evidence reliably and efficiently. All of us would like to enjoy the best possible health we can. To achieve this, we need reliable information about what might harm or help us when we make healthcare decisions.

Research involves gathering data, then collating and analysing it to produce meaningful information. However, not all research is good quality, and many studies are biased and their results untrue. This can lead us to draw false conclusions.

So, how can we tell whether a piece of research has been done properly and that the information it reports is reliable and trustworthy? How can we decide what to believe when research on the same topic comes to contradictory conclusions? This is where critical appraisal helps.

 

If healthcare professionals and patients are going to make the best decisions, they need to be able to:

  • Decide whether studies have been undertaken in a way that makes their findings reliable
  • Make sense of the results
  • Know what these results mean in the context of the decision they are making

Different types of question require different study designs. There are many sorts of questions that research can address.

  • Aetiology: what caused this illness?
  • Diagnosis: what does this test result mean in this patient?
  • Prognosis: what is likely to happen to this patient?
  • Harm: is having been exposed to this substance likely to do harm, and, if so, what?
  • Effectiveness: is this treatment likely to help patients with this illness?
  • Qualitative: what are the outcomes that are most important to patients with this condition?
Type of question Study design
Etiology and risk factors Cohort and case–control studies
Incidence and prevalence Cohort or cross-sectional
Harm Cohort and case–control studies
Prognosis Cohort/survival studies
Value for money Economic eval

uation (e.g. cost-effectiveness study or cost–benefit study)

Effectiveness Randomized controlled trial
Diagnosis Diagnostic test study (or randomized controlled trial)
Patient experience (e.g. of illness, treatment or service) Qualitative studies, e.g. questionnaires, focus groups, or interviews

 

Different questions require different study designs for critical appraisal; first, because you need to choose a paper with the right type of study design for the question that you are seeking to answer and, second, because different study designs are prone to different biases.

Thus, when critically appraising a piece of research it is important to first ask: did the researchers use the right sort of study design for their question? It is then necessary to check that the researchers tried to minimise the biases (that is, threats to internal validity) associated with any particular study design; these differ between studies.

The Critical Appraisal Skills Programme (CASP) aims to help people develop the skills they need to make sense of scientific evidence. CASP has produced simple critical appraisal checklists for the key study designs. These are not meant to replace considered thought and judgement when reading a paper but are for use as a guide and aide memoire.

All CASP checklists cover three main areas: validity, results and clinical relevance. The validity questions vary according to the type of study being appraised, and provide a method to check that the biases to which that particular study design is prone have been minimised.