Exericse - Therapy Stats
This is a short tutorial/set of practice problems to help you brush up on the statistics for therapy studies.
Each page has a little info, and then a question or two.
You may want to use a piece of paper to write down your answers to the questions on each page - then you can compare them with the correct answers on the next page. We're not very complicated around here...
Several of my older patients complain about problems sleeping. We go through the usual reasons they might have trouble - depression, urinary problems, pain, anxiety, napping during the day, alcohol, caffeine and disorientation from mild dementia. Several of them have none of those problems - they just have trouble getting to sleep and/or staying asleep. There are many different ways to handle this, but I wonder particularly about one medication in particular: amitriptyline (a tricyclic antidepressant used in low doses for sleep).
So I did a study in my office of amitriptyline vs. placebo for insomnia in the elderly (not really...). There were a few outcomes I was interested in:
1. if we define moderate insomnia as sleeplessness every night with some impairment of function during the day, then how many subjects still have moderate insomnia while taking amitriptyline?
2. the subjects will rate the quality of their sleep as improved or not. how many patients' sleep will improve on amitryptiline?
3. the subjects will report the number of days in the study period that they have insomnia while on amitriptyline - do the subjects on amitriptyline have fewer nights of insomnia?
There are other interesting outcomes, but these will do for now.
Question: Which of these outcomes (1,2 or 3) is/are measured in continuous data (as opposed to categorical/dichotomous data)?
Well, only outcome 3 (number of days with insomnia) is measured in continuous data.
Outcomes 1 (persistent insomnia or not) and 2 (sleep improved or not) are dichotomous (categorical) outcomes.
Number of sleepless nights will be measured for each patient, then we'll take the average number of sleepless nights per patient by group, which may be decimal, and then we'll compare group averages...that's continuous data.
OK - OUTCOME 1 - persistent insomnia ("are you the still have insomnia on the medicine or not?")
I studied 100 elderly patients and got the following table of data regarding this outcome:
Persistent Inosmnia
No persistent insomnia
Amitriptyline
10
45
Placebo
30
15
The first thing you may notice is that the groups aren't equal (55 subjects in experimental (amitriptyline), 45 in control (placebo)). This happens in randomization - no biggie - just calculate the stats as they are.
So, what can we make of this data? First make RATES out of the two groups' data:
Experimental (amitriptyline) event rate = proportion with the outcome in the experimental group
Control (placebo) event rate =proportion with the outcome in the control group
Calculate these rates for this study and then:
Remember, we're calculating the rate of persistent insomnia - the "bad" outcome here. That's usually the way studies are designed - you measure the people that died in each group, or had a heart attack in each group. But there's nothing saying you cannot measure a good outcome (like, proportion of subjects stopping smoking). Just keep things straight when you're calculating the data.
OK, the experimental event rate (EER) is a/a+b or 10/55 = 0.18 or 18%
The control event rate (CER) is c/c+d or 30/45 = 0.67 or 67%
Hope you got that. If not, go back and review.
So what do we do now? It's useful to compare the groups to see if the group that had the intervention of interest does better than the placebo group. If everything else is equal between the groups (review the validity criteria for therapy studies), then we can say that the intervention helps. We can compare the groups with "relative" statistics or "absolute" ones. We'll do relative stats first, because we're saving the more meaningful stuff for last.
What's the relative risk (RR)?
What's the relative risk reduction (RRR)?
While you're thinking about how to answer these questions, consider that the other name for relative risk is "RATE ratio"...(hint, put the rates you just calculated into a ratio...;-)
How'd that go?
If the CER is 67% and the EER is 18%, and RR=EER/CER, then 0.18/0.67 = 0.27. Your patients are .27 times as likely to have persistent insomnia on amitriptyline than on placebo. That's good - persistent insomnia is a bad outcome and they are at less risk for it.
How much less risk? RRR=1-RR (1 represents the risk you'd have without amitriptyline) - 1-0.27 = .73. They're 73% less likely to have insomnia when they take amitriptyline.
Hmmm...73% less sounds like a LOT less, but it's hard to know for sure, because if you're only given that number, you don't know what the baseline risk is. You don't have a sense for how much risk is actually averted in an average group of elderly grandparents that you see everyday.
For that, we need absolute statistics. Absolute Risk Reduction and Number Needed to Treat. Speaking of those, let's calculate them.
ARR=?
NNT=?
Hmmm...
The ARR is the Difference between the two rates. The term "ARR" is slightly misleading - it should be Absolute Risk Difference (ARD) - we'll see why with the next outcome. For now, that name will work OK.
ARR = CER-EER = 0.67-0.18 = 0.49. You're 49% less likely to have persistent insomnia taking amitriptyline. Alert readers will notice that the statements about relative risk reduction and absolute risk reduction sound the same. They are - there are no standard ways of saying these things that will automatically differentiate absolute from relative risk reduction. You have to ask whoever's giving you these numbers whether they're relative or absolute.
Number needed to treat is a way of expressing absolute risk by getting a sense of how many patients we would have to treat in our practice before the intervention we're considering would make a difference in outcome for one of them (compared to not treating them).
OK, now how many patients need to be treated until we prevent an extra case of persistent insomnia ("will MY granny get some sleep?")?
NNT = 1/ARR = 1/0.49 ~ 2. We only need to treat 2 patients with amitriptyline to prevent a case of persistent insomnia...
Sounds pretty effective (at least these made-up numbers do). Let's look at the other outcomes...
OK, here's outcome 2. The number of elderly subjects who say that their sleep improved.
Big whoop, right? Should be the same as the previous example.
BUT!!!
The outcome, in this case, is a GOOD outcome. So relative risk is the risk of getting better, not staying sick. This risk hopefully INCREASES instead of reduces. You can get turned around with these things if you don't pay attention to this point.
The data:
Insomnia IMPROVED
Insomnia NOT improved
Amitriptyline
50
5
Placebo
15
30
See if you can calculate CER, EER, RR and RRR.
Sooooo....
Insomnia IMPROVED
Insomnia NOT improved
Amitriptyline
50
5
Placebo
15
30
CER = 15/45 = 0.33 = 33%
EER = 50/55 = 0.91 = 91%
RR = EER/CER = 0.91/0.33 = 2.75
(Gack!! Oh, right, this is the risk of IMPROVEMENT if you take the medicine...that's a good thing)
RRR = 1-RR = 1-2.75 = -1.75
(Double gack!!!! Well, now, think about it...it's a Negative reduction in risk, so that means it's actually a relative risk Increase, and since the outcome is improvement, that's OK. You're 1.75 times as likely to improve).
OK, now for the absolute statistics...
Calculate the ARR and NNT.
Did you click confidently?
I hope so.
If CER is 0.33 and EER is 0.91, then:
ARR = CER-EER = 0.33-0.91 = -0.58
(Less gack-ing I hope...Since it's a negative ARR, then it's really an ARI - absolute risk INCREASE, but don't worry about names, work the concept.)
You're 58% MORE likely to have improved sleep on the medication.
NNT = 1/ARR = 1.7 (or 2). For every 2 elderly insomniac Gam-Gams I treat with amitriptyline, 1 additional one has IMPROVED sleep. Yea!
OK - last outcome...then you can curl up the fetal position.
We also wanted to know how many days my elderly subjects had in each group. Hopefully, the number in the amitritpyline group was lower than the number in the placebo group. Let's say the study lasted for a month.
Please don't try to calculate RRR, ARR, NNT, XYZ, PDQ or any other statistic that's meant for dichotomous data here. Realize that you're just left with comparing averages (means or medians) from each group. At most, you'd be able to calculate the mean difference between the groups, but that doesn't get you much farther.
You'll have to settle for just knowing the differences. Your data for this might look like:
Amitriptyline
Placebo
Mean (SD) # of insomnia nights
4.2 +/- 1.0
10.1 +/- 1.1
Mean difference (SD)
5.9 +/- 0.5
p-value
p<0.01
You could say to your patients: "On average, people gain 6 more nights of sleep in a month on amitriptyline than on placebo." And there's a reasonable chance they'd understand that...
A few words about confidence intervals...
First - we didn't calculate them, because that's what statistical software and statisticians are for. We just have to know what they mean.
The reason we use confidence intervals is because whatever numbers we get for rates, risks and NNT's, they're only "point estimates" - one number we've gotten from the results in the SAMPLE (the study) that we want to use to describe the POPULATION (our patients). To see how confident we are about whether our SAMPLE result is what would really happen in the POPULATION, we use "confidence intervals". A 95% confidence interval means that if we repeated the study over and over and over again, 95% of the time, the point estimate would fall within that interval.
A relative risk of 0.6 from a study means that the people in the SAMPLE are 0.6 times as likely to get the outcome than those in the placebo group.
But if we want to use this for the POPULATION, we have to attach the confidence interval.
RR 0.6 (95% CI 0.4 to 0.8) - the data suggests that the true POPULATION value for the relative risk is between 0.4 and 0.8.
Because each end of that CI is less than 1, we call that a statistically significant difference because all likely values for the RR are less than 1.
BUT if: RR 0.6 (95% CI 0.1 to 1.1) - now the confidence interval includes the possibility of less risk (<1), the same risk (1), or more risk (>1) - so now we're not very confident at all that there is less risk, and so it's not "statistically significant".
Same concept with risk differences (ARR, RRR) - but because we're subtracting rates from each other in calculating these, we use zero as meaning the same risk - so the confidence interval for a statistically significant ARR will not cross zero.
ARR 35% (95% CI 15 to 55%).
A non-significant example: ARR 9% (95% CI -7 to 25%) (CI includes a negative ARR, which means ADDED risk, and also includes zero)
One more point. CIs can be applied to any point estimate. Even estimates of things that aren't comparisons. I can get an average LDL cholesterol in a group and put a confidence interval around it: average LDL 134 mg/dl (95% CI 114 to 154). There's nothing significant or insignificant about this average, it just is. It's a type of error-bound like a standard deviation in this case.
Ok...you're done.
Let me 'splain...No, there is too much, let me sum up...(distraction here)
make sure you can identify continuous vs. dichotomous data.
for continuous data:
means, standard deviations, mean differences
for dichotomous data:
make sure you check whether it's a positive outcome or negative outcome so your stats make sense to you.
CER, EER - rates in each group
RR, RRR - commonly used, but don't tell about the magnitude of change in risk
ARR, NNT - talk about magnitude of risk - a better story about how it'll work in practice.
confidence intervals
around non-comparative statistics (basic rates, sensitivity, specificity, etc.) - does not discuss significance, just precision of estimate
around differences or ratios between groups - tells both significance and precision.