PatientPlus articles are written by UK doctors and are based on research evidence, UK and European Guidelines. They are designed for health professionals to use, so you may find the language more technical than the condition leaflets.
Practising evidence-based medicine encourages clinicians to integrate valid and useful evidence with clinical expertise and each patient's unique features, enabling clinicians to apply evidence to the treatment of patients. There are five main steps to practising evidence-based medicine:
- Identify knowledge gaps and formulate a clear clinical question.
- Search the literature to identify relevant articles.
- Critically appraise the articles for quality and the usefulness of results; always question whether the available evidence is valid, important and applicable to the individual patient.
- Implement clinically useful findings into practice.
- Evaluate performance using audit.
Healthcare professionals must always apply their general medical knowledge and clinical judgement not only in assessing the importance of recommendations but also in applying the recommendations which may not be appropriate in all circumstances. The following questions should be asked when deciding on the applicability of evidence to patients:
- Is my patient so different from those in the study that results cannot be applied?
- Is the treatment feasible in my setting?
- What are my patient's likely benefits and harms from the therapy?
- How will my patient's values influence the decision?
Finding the evidence
- When looking for appropriate evidence:
- Search for available guidelines - eg, National Institute for Health and Care Excellence (NICE), Health Information Resources, professional bodies (eg, a relevant specialist site such as the Royal College of Obstetricians and Gynaecologists (RCOG)).
- If no guidelines are available, search for systematic reviews - eg, Cochrane database.
- If no systematic reviews are available, look for primary research - eg, PubMed.
- If no research is available, consider general internet searching (eg, Google), or discuss with a local specialist (at this level beware poor-quality information from the internet or individual personal bias from even the most respected specialist).
- The National Library for Health provides access to a range of medical search sites, including PubMed, Medline, EMBASSY, Bandolier, University of York's Centre for Review and Dissemination and the Cochrane database.
- National guidelines and guidance sites include NICE and the Scottish Intercollegiate Guidelines Network (SIGN). Guidance on many topics is also available at the website for NICE Clinical Knowledge Summaries (NICE CKS) - formerly 'PRODIGY'.
Add notes to any clinical page and create a reflective diary
Automatically track and log every page you have viewed
Print and export a summary to use in your appraisal
Critical appraisal of medical research
- The topic and conclusions: consider whether the message is important and believable, and whether it fits with existing knowledge and opinion (always look for other research, reviews and guidelines on the same topic).
- Consider whether there any obvious problems with the research and whether the research has been ethical.
- Consider whether the objectives are clear and the precise nature of the hypothesis being considered.
- Funding: drugs companies might seek to publish studies that show their product in a favorable light, but ignore negative studies.
- Conflict of interest: consider whether the authenticity of the research can be relied upon.
Type of study
In general, the hierarchy of studies for obtaining evidence is:
- Systematic reviews of randomised controlled trials (RCTs).
- Controlled observational studies - cohort and case control studies.
- Uncontrolled observational studies - case reports.
However, the hierarchy is dependent on the issue being researched. The Centre for Evidence-Based Medicine (CEBM) has recently published a table to identify the different levels of evidence for different types of questions (eg, prognosis, treatment benefits), including:
- For issues of therapy or treatment, the highest possible level of evidence is a systematic review or meta-analysis of RCTs or an individual RCT.
- For issues of prognosis, the highest possible level of evidence is a systemic review of inception cohort studies.
Expert opinion must not to be confused with personal experience (sometimes called eminence-based medicine). Expert opinion is the lowest level of acceptable evidence but, in the absence of research evidence, may be the best guide available.
- RCTs, especially those with double-blind placebo controls, are regarded as the gold standard of clinical research.
- These studies work very well for certain interventions - eg, drug trials, but it is much more difficult for other interventions, such as using sham acupuncture or sham manipulation as the control.
- Longitudinal or cohort studies:
- A group of people is followed over many years to ascertain how variables such as smoking habits, exercise, occupation and geography may affect outcome.
- Prospective studies are more highly rated than retrospective ones, although the former obviously take many years to perform. Retrospective studies are more likely to produce bias.
- The more data are pooled, the more valid the results but possibly the less relevant they become to individual patients. Meta-analysis can therefore be a useful tool but it has some important limitations.
- A meta-analysis takes perhaps 10 trials of 100 patients and to combine the results as if it were a trial of 1,000 patients.
- Although this technique rates highly, the methodology may not be identical in all studies and further errors may be caused by a bias to certain publications. A good meta-analysis should contain funnel plotting with cut and fill to assess the completeness of a publication.
- A large, well-conducted trial is, therefore, far more valuable than a meta-analysis.
- Selection of subjects is very important; some diseases are difficult to define - eg, irritable bowel syndrome, chronic fatigue syndrome, fibromyalgia. For many diseases there is huge variation in severity - eg, asthma. If subjects have been paid for taking part in the study, there may be possible bias.
- Questionnaires: assess the design of the questionnaires, whether they were piloted, whether the interviewers were properly trained and the interviews standardised.
- Recall bias may be important. The timing of the questionnaire may be important, especially for seasonal illness such as hay fever. Minor events may easily be forgotten.
- Setting and subjects:
- The study population should be clearly defined, as should whether the whole population or a subset has been studied. Consider whether the sample size seems big enough, whether the duration of the study was long enough for the outcome measure to occur and whether there is any possible selection bias - eg, only patients treated in hospital have been selected.
- Assess whether the control group was well matched and whether any exclusion criteria were valid.
- Consider the relevance of any patients who have dropped out of the study, the reasons for dropping out and the relevance for the results and conclusions of the research.
- Outcome measures: should be clearly defined, relevant to the objectives, reliable and reproducible, valid and consistent.
- Consider how convincing the results are, whether the statistics (eg, P value, confidence limits) are appropriate and impressive, and whether there are any possible alternative explanations for the results.
- Type of outcome: the results of a trial may be relatively simple to express in terms of numbers dying or surviving or may be much harder to quantify. The quality adjusted life years (QALY) index may be used for such parameters as pain, incontinence and disability.
- The results should be clearly and objectively presented in sufficient detail (eg, age or gender breakdown of results). Consider whether there was an adequate response rate in a questionnaire study (ideally above 70%) and whether the numbers in any study add up.
- Identify the rate of loss of follow-up during the study and how non-responders have been dealt with - eg, whether they have been considered as treatment failures or included separately in the analysis.
- Assess whether the results are clinically relevant and whether the conclusions are supported by the results of the research study.
- Check that the conclusions relate to the stated aims and objectives of a study and whether any generalisations made from a study carried out in one population have been applied inappropriately to a different type of population.
- Consider the possibility of any confounding variables - eg, age, social class, ethnicity, smoking, disease duration, comorbidity. Multiple regression analysis or strict matching of controls reduces this problem.
- Bias may have many forms - eg, observer bias such as non-blinding, trying to ensure a patient has drug rather than placebo, contamination where the intervention group passes on information to the control group in health education intervention studies.
- Annual and seasonal factors in the variation of disease may be important, especially for respiratory infections, rhinitis and asthma.
- The discussion should include whether the initial objectives have been met, whether the hypothesis has been proved or disproved, whether the data have been interpreted correctly and the conclusions justified.
- The discussion should include all the results of the study and not just those that have supported the initial hypothesis.
Hierarchical systems for levels of evidence used in recommendations and guidelines
- A variety of grading systems for evidence and recommendations is currently in use. The system used is usually defined at the beginning of any guidelines publication.
- The hierarchy of evidence and the recommendation gradings relate to the strength of the literature and not necessarily to clinical importance.
Grading of evidence
- Ia: systematic review or meta-analysis of RCTs.
- Ib: at least one RCT.
- IIa: at least one well-designed controlled study without randomisation.
- IIb: at least one well-designed quasi-experimental study, such as a cohort study.
- III: well-designed non-experimental descriptive studies, such as comparative studies, correlation studies, case-control studies and case series.
- IV: expert committee reports, opinions and/or clinical experience of respected authorities.
Grading of recommendations
- A: based on hierarchy I evidence.
- B: based on hierarchy II evidence or extrapolated from hierarchy I evidence.
- C: based on hierarchy II evidence or extrapolated from hierarchy I or II evidence.
- D: directly based on hierarchy IV evidence or extrapolated from hierarchy I, II or III evidence
A simpler system of ABC is recommended by the US Government Agency for Health Care Policy and Research (AHCPR):
- A: requires at least one RCT as part of the body of evidence.
- B: requires availability of well-conducted clinical studies but no RCTs in the body of evidence.
- C: requires evidence from expert committee reports or opinions and/or clinical experience of respected authorities. Indicates absence of directly applicable studies of good quality.
Guideline Recommendation and Evidence Grading (GREG)
In an attempt to improve the way recommendations and evidence statements are graded, the GREG grading system has been used:
- Evidence grade:
- I (High): the described effect is plausible, precisely quantified and not vulnerable to bias.
- II (Intermediate): the described effect is plausible but is not quantified precisely or may be vulnerable to bias.
- III (Low): concerns about plausibility or vulnerability to bias severely limit the value of the effect being described and quantified.
- Recommendation grade:
- A (Recommendation): there is robust evidence to recommend a pattern of care.
- B (Provisional recommendation): on balance of evidence, a pattern of care is recommended with caution.
- C (Consensus opinion): evidence being inadequate, a pattern of care is recommended by consensus.
Further reading & references
- Centre for Evidence-Based Medicine
- Journals and Databases; NICE Evidence Search
- National Institute for Health and Care Excellence (NICE)
- The Cochrane Collaboration
- Scottish Intercollegiate Guidelines Network (SIGN)
- Centre for Reviews and Dissemination; University of York
- Clinical Knowledge Summaries; NICE
- Song F, Parekh S, Hooper L, et al; Dissemination and publication of research findings: an updated review of related biases. Health Technol Assess. 2010 Feb;14(8):iii, ix-xi, 1-193. doi: 10.3310/hta14080.
- Straus SE, Sackett DL; Using research findings in clinical practice. BMJ. 1998 Aug 1;317(7154):339-42.
- Straus SE, Sackett DL; Applying evidence to the individual patient. Ann Oncol. 1999 Jan;10(1):29-32.
- Counsell C; Formulating questions and locating primary studies for inclusion in systematic reviews. Ann Intern Med. 1997 Sep 1;127(5):380-7.
- Levels of Evidence; Centre for Evidence-Based Medicine, June 2010
- Tonelli MR; The limits of evidence-based medicine. Respir Care. 2001 Dec;46(12):1435-40; discussion 1440-1.
- Sterne JA, Egger M, Smith GD; Systematic reviews in health care: Investigating and dealing with publication and other biases in meta-analysis. BMJ. 2001 Jul 14;323(7304):101-5.
- Johannesson M; QALYs, HYEs and individual preferences--a graphical illustration. Soc Sci Med. 1994 Dec;39(12):1623-32.
- Eccles M, Mason J; How to develop cost-conscious guidelines. Health Technol Assess. 2001;5(16):1-69.
- Burns PB, Rohrich RJ, Chung KC; The levels of evidence and their role in evidence-based medicine. Plast Reconstr Surg. 2011 Jul;128(1):305-10. doi: 10.1097/PRS.0b013e318219c171.
Disclaimer: This article is for information only and should not be used for the diagnosis or treatment of medical conditions. EMIS has used all reasonable care in compiling the information but make no warranty as to its accuracy. Consult a doctor or other health care professional for diagnosis and treatment of medical conditions. For details see our conditions.
Dr Colin Tidy
Dr Colin Tidy
Prof Cathy Jackson