Summarizing Evidence

The Problem: You've been asked to get the answer to a clinical question using the "best evidence." How do you do that? Where do you look? What should you read? How can you be sure one study is better than another? What if there are no RCTs?

Relax...This page can offer some hints and links to places to find information.

This is not easy work (what in medicine is?), but it can be fairly straightforward.

Step 1: Analyze your question.

• Is it a background question? (What are the treatments for...? What causes...? What is the approach to diagnosing and treating...?)

  • These questions aren't easily answered by original research.

    • You can look in a more evidence-based background source (like Up-To-Date, an evidence-based guideline, or a high-quality narrative review).

    • More importantly, go back and negotiate the question to something more specific (see below)

• Is it a more focused question about therapy, diagnosis, harm or prognosis? Read more about questions HERE and HERE.

  • use the PICO format

    • patients - which specific patients are we concerned with?

    • interventions - what is the intervention of interest?

    • comparison - are you comparing the intervention with placebo? another intervention? "traditional care"?

    • outcome - what outcomes are you most interested in? are they patient-oriented?

• Try to get an idea of what type of evidence will best answer the question.

  • If it's a therapy question, then a systematic review of randomized controlled trials on the specific population and therapy you're looking for are best. The quality of evidence decreases from there. (a group of RCTs, a single RCT, a cohort study, a case-control study, a case series). Also, if you have to extrapolate the findings to a different patient population or you have to compare therapies against each other when the studies only compared each against placebo, then some additional thought is involved.

  • If it's a diagnosis question, you have two options. The most useful evidence is a randomized trial of management using the diagnostic test compared with management without using that test. These studies are relatively uncommon. Instead, most diagnostic test studies are cohort or case-control studies of the test of interest against the reference standard diagnostic test.

  • If it's a harm question (Is cell phone use associated with brain tumors?), you're looking for either cohort studies (for common exposures) or case-control studies (for rare exposures).

  • If it's a prognosis question (For patients with diabetes, how likely is it that they'll get heart disease, and what factors increase that risk?). For these questions, you're generally looking for prospective or retrospective cohort studies, where the cohort is defined by the initial condition (diabetes) and the study looks forward to the outcome and the factors associated with it.

Step 2: Search for the evidence.

• Try Upstate Library's EBM Pyramid Page for systematic review databases as well as good evidence-based "secondary sources"

• Try PubMed's Clinical Queries Page to find filtered original research.

• Try the Librarians - they're trained specifically in this type of search.

Step 3: Choose the evidence.

• This step is a balance between what's relevant to your question and the strength of the evidence.

• Make sure you can identify the study designs.

• Examples

  • If you have a question on cholesterol therapy for women, and you have the best and biggest RCT on cholesterol therapy for men, that doesn't completely answer your question - you would need to keep looking for evidence about women. There may be a smaller, or less valid study on women, and you'll need to consider and acknowledge both pieces of evidence in your answer.

  • If you have a question comparing drug X versus drug Y, and the only studies you find study each drug against PLACEBO, then you need to consider if the studies use equivalent doses and are on similar populations in order to compare the results.

  • If you have a question that nobody's studied before, you'll have to construct an argument out of lesser evidence - "Should we give only blood screened for minor blood antigens to girls and women of childbearing age to prevent isoimmunization related disease when they get pregnant?" You will have to find evidence on: prevalence of isoimmunization from minor antigens, the effectiveness and cost of screening blood for these antigens, the chance that having a transfusion with unscreened blood will result in isoimmunization disease, and the potential impact (population attributable risk, e.g.) of instituting screening.

Step 4: Grade the individual pieces of evidence (the individual studies).

• For details, visit the GRADE website, particularly the FAQ and the list of Publications.

• Use worksheets to evaluate and grade individual pieces of evidence.

• It is best to use an Evidence Table to summarize all the studies. A simple table with any combination of the following headings would do:

  • Study identifier (first author, title, etc.)

  • Population (inclusion/exclusion criteria and sample size)

  • Intervention and comparator

  • Results (including some "user friendly statistics" such as Number Needed to Treat and Likelihood Ratios)

  • Comments concerning validity, applicability, other issues that would affect the study's value among the rest

Step 5: Build a Logical Summary

• Create an outline of your logic model (in words or pictures). For an example, look at the logic model for prostate cancer screening from the USPSTF.

• Write your summary using this outline. As you cite each piece of evidence, be sure to provide a succint notation of the strength of that piece of evidence (see the levels again).

• Be sure to account for outliers and inconsistencies in the evidence. Remember that consistency of the evidence is a crucial part of the overall Strength of Recommendation.

Step 6: Assign a Strength of Recommendation

• Strong recommendation

  • A strong recommendation is one for which guideline panel is confident that the desirable effects of an intervention outweigh its undesirable effects (strong recommendation for an intervention) or that the undesirable effects of an intervention outweigh its desirable effects (strong recommendation against an intervention).

  • A strong recommendation implies, that most or all individuals will be best served by the recommended course of action.

• Weak Recommendation

  • A weak recommendation is one for which the desirable effects probably outweigh the undesirable effects (weak recommendation for an intervention) or undesirable effects probably outweigh the desirable effects (weak recommendation against an intervention) but appreciable uncertainty exists.

  • A weak recommendation implies, that not all individuals will be best served by the recommended course of action. There is a need to consider more carefully than usual individual patient’s circumstances, preferences, and values.

Good Luck and Happy Summarizing!

Use the links in the sidebar to move on.