Critical Appraisal - Therapy Articles - Results

Introduction

Once you have declared the study valid enough to read, you must process and understand the results.

Descriptive analysis - information only about the sample you studied

Exploratory analysis - correlation - used to see which inferential and predictive analyses might be useful

Inferential analysis - estimating the quantity of risk, etc. AND the uncertainty about that estimate

Predictive analysis - (not causation) - simple models and lots of data

Causal analysis - changing a variable to change another variable

Mechanistic analysis - specific changes in variables that lead to specific outputs

There are three broad categories of results:

Descriptive Statistics

These are usually worth perusing to see who actually got into the trial, to look for baseline differences, and to see how bad the control groups fared.

You'll see both categorical (usually dichotomous) and continuous data here.

For continuous data, you'll see means with their standard deviations, or you'll see medians with interquartile ranges (which serve a similar function as standard deviations).

For categorical data, you'll may see confidence intervals around the percentages.

In both cases, you may see p-values looking at the differences between groups. Not all important differences are always statistically significant. For example, if you see a difference between, say, the smokers in the intervention group vs. the placebo group - think about how the ill effects of smoking might cause confounding of the results. You should look for "adjusted results" when you see interesting differences between groups in the Table 1 of a study.

Comparison or Inferential Statistics

These statistics relate the outcomes of one group (intervention) vs. the other (control).

The main concepts (and equations) to know here are to the right --->

Careful, these equations are for dichotomous outcomes only (disease present/absent, dead/alive, BP below 140 or not).

You may still see comparisons of continuous outcomes (means and medians) and may even see mean differences between groups. Think about each number you see in order to interpret it correctly.

Number Needed to Treat is a way to think about the effectiveness of a therapy as a clinician with a panel of patients. An example:

CER = 20%, EER = 10% (the outcome is a bad outcome)

then ARD = 10% or 0.10 - there is a 10% reduction in the bad outcome for the intervention group.

therefore, the NNT = 1/0.10 = 10. We'd need to treat 10 people before we prevent an additional bad outcome.

In EBM, we prefer to use the absolute statistics (ARD and NNT) because they give us a sense of how much an intervention matters. If a drug means the intervention group is half as likely to have the poor outcome as the control group, the "magnitude" or importance of that effect is related to the chance of that outcome to begin with. In the JUPITER study, there were dramatic hazard ratios (0.5 or so), but the overall of the events in the study was very low, results in a relatively high number needed to treat (@90 for 2 years to prevent the combined outcomes). That does not invalidate the study, but careful consideration is needed.

Equations courtesy of the awesome TeX equation editor

Graphical Display - Interpreting Figures

There are two commonly used figures in Therapy studies that it's useful to look at specifically. Most other figures are bar graphs and line graphs, but these two take some review.

Forest Plot

  • used in trials and individual studies to break down the results into subgroups (by intervention, confounders, stratifiers or outcome)

  • results for each group represented as point estimate (relative risk, odds ratio, absolute risk difference, mean difference etc) - the square or circle in the center of each line, and the "whiskers" or lines coming out of that square or circle (usually the confidence intervals, standard deviation or standard error). These individual lines are oriented around the line of no effect (0 for absolute statistics (ARR, mean difference) or 1 for relative statistics (RR, OR)). If the confidence interval line does not cross the line of no effect, then the result is statistically significant.

  • Here, the graph is missing a scale at the bottom, but will usually have one.

  • In this study, refecoxib significantly contributes to the risk of myocardial infarction, but none of the other NSAID medications do. Lumiracoxib in this study was associated with a slightly higher risk of stroke.

Kaplan-Meier Curve

  • also known as survival curve.

  • This is used to track what happens to people in a large trial, in which people enter and leave at different times, so have variable lengths of exposure to the intervention.

  • "censoring" is appropriately removing someone from the analysis once they have had the outcome - you can see censoring in the numbers on the bottom row.

  • Separation between the curves (intervention vs. control, e.g.) usually means that there is a difference between the groups, but this is not confirmed statistically without a p-value on the graph (lower left corner).

  • Read and understand the axes carefully - these can be oriented in a variety of ways - including starting with 0% in the left hand lower corner and moving diagonally upward - in that cases, the Y axis would be Number without the Outcome.

There's a video screencast about this section - Therapy Results Video