Answering Clinical Questions with Systematic Reviews

Asking an unfocused question can lead us down a frustrating path when using the medical literature. Doing the work on the front end to narrow the question is invaluable. There are various forms of the "PICO" question - I'll use the most complete one I'm aware of, PICOTS, to work with this question.

• P - patients/population - in this case, we want patients with painful, acute, osteoporotic compression fractures (symptoms and acute/chronic are important)

• Intervention - calcitonin nasal spray

• Comparison - placebo, other analgesics (we can leave this a little open, as long as we keep track of the comparisons)

• Outcome - decreased pain, possibly improved function (outcomes that are important to the patient). This section is required to specify what "does it work?" means.

• Timeframe - (over what time to assess the outcome)

• Setting - outpatient and/or primary care

Much better...Now we have a "map" to get us started in our search and a filter to allow us to ignore references that don't address the question.

To find systematic reviews that can answer our question, we'll use three sources:

1. The Cochrane Library - a proprietary database, but one that most libraries have access to. It has Cochrane systematic reviews, links to CENTRAL (a database of all controlled trials included in Cochrane reviews) and links to the Database of Abstracts of Reviews of Effectiveness (DARE, a UK database of systematic reviews that was maintained until 2015).

2. TRIP Database - a "meta-search" engine that searches multiple databases at once and helps you filter out the best evidence.

3. PubMed Clinical Queries - an easy-to-use interface for MEDLINE that has a filter to help find systematic reviews.

In the Cochrane Library, I typed "calcitonin vertebral fracture" into the search box, selected the Cochrane Reviews tab and found about 18 reviews...none of which addressed vertebral fracture pain. There was a single study on treating pain from bony metastases - I made a mental note of this in case I didn't find anything else - it may still be possible to apply some "indirect" evidence in the absence of direct evidence.

On the TripDatabase page, I entered the same keywords ("calcitonin vertebral fracture") in the search box (without signing in). I then selected the "Systematic Reviews" filter on the right side of the page and found 27 results. Only three of these looked promising. They were all linked to the DARE. DARE provided structured abstracts so that readers could get the important details of the review quickly. I was able to exclude one review because it didn't appear to look at pain from fractures as an outcome. I was left with two others, but realized one was an update of the other (and apparently the lead author got married in the interim!):

Knopp JA, Diner BM, Blitz M, Lyritis GP, Rowe BH. Calcitonin for treating acute pain of osteoporotic vertebral compression fractures: a systematic review of randomised, controlled trials. Osteoporosis International 2005; 16(10): 1281-1290

Knopp-Sihota JA, Newburn-Cook CV, Homik J, Cummings GG, Voaklander D. Calcitonin for treating acute and chronic pain of recent and remote osteoporotic vertebral compression fractures: a systematic review and meta-analysis. Osteoporosis International 2012; 23(1): 17-38

At PubMed on the Clinical Queries page, I typed the same terms (calcitonin vertebral fractures) in search box. Pubmed's software converts this entry into a very nice set of inclusive search terms:

systematic[sb] AND (("calcitonin"[MeSH Terms] OR "calcitonin"[All Fields]) AND ("spinal fractures"[MeSH Terms] OR ("spinal"[All Fields] AND "fractures"[All Fields]) OR "spinal fractures"[All Fields] OR ("vertebral"[All Fields] AND "fractures"[All Fields]) OR "vertebral fractures"[All Fields]))

The search is run for Clinical Studies, Systematic Reviews and Medical Genetics. I clicked the "See All" link in the Systematic Reviews section of the results, and 15 reviews to look over. For longer lists, toggling back and forth between the "Recent" and "Best Match" sort orders might be helpful, but in this case a quick look yielded three reviews of interest.

Of these three, I chose to look at the latest version of the Knopp review. The Armeis review - when I looked at the full text - turned out to be an evidence-based guideline focused on low and middle income countries. While potentially useful, I decided to stick with my specific question here, and I'll file away this review to see if there are other treatments I should look into.

Now that we have our review, we need to do some quick validity checks to see that the review was done appropriately.

Remember, "validity" in a systematic review has two perspectives - "How well was the review itself done?" and "How valid are the studies that were found by the review?" It was reassurring to see the authors state that they used the Cochrane review methodology and the PRISMA reporting standards for systematic reviews. These are popular and well-accepted methodologies.

For this first question, we use the criteria found on these pages. In the Knopp review, we find:

• Search - The authors used a broad set of terms and MESH terms to search for studies in multiple international literature databases. They searched the grey literature and contacted authors of included reports. They did not limit the search based on language.

• Inclusion/Exclusion Criteria - The authors details the types of studies, interventions, patients and outcomes they used in detail. Experimental and observational studies were included. Older (>60) adults with acute or chronic pain from vertebral fractures were included. Any form of calcitonin administration was included. For outcomes, any way of assessing pain was included as well as use of other analgesics

• Validity Assessment - The authors used the Cochrane Risk of Bias methods to evaluate the validity of the included studies. This tool looks at: randomization methods, allocation concealment, blinding, withdrawals and completeness of outcome data, and selective outcome reporting.

• Heterogeneity Assessment - Heterogeneity was assessed in the meta-analyses by use of the I-squared statistics and the Chi-squared analysis - both accepted methods. The authors detailed plans for how they would look into reasons for heterogeneity - including sensitivity and subgroup analyses.

Thinking about the above criteria, we are comfortable that the review was done very well.

Now, the remainder of our assessment falls under "Results" - what did we get from the review?

Thirteen studies contributed data. Three only contributed to side-effects and withdrawal data. Five looked at acute pain and five at chronic (>3 months) pain. Because of how they presented data, the results from three studies could not be meta-analyzed. The quality of the studies "varied significantly" according to the authors and details were presented in a clear table.

For acute pain at rest and with movement, calcitonin provided statistically significant pain reduction of almost 3 cm on a 10 cm pain scale. This result alone is meaningless without knowing (as the authors note) that changes of about 1.3 cm have been found to be clinically significant for patients. However, there was significant heterogeneity in both of these analyses that the authors thought might be due to the route of administration differing between studies.

For chronic pain, there was less overall heterogeneity, but there was also no statistically significant effect. The authors note that the withdrawal rates were low in the studies, and there were no important safety concerns.

Reviewing this evidence, it seems like the best use of calcitonin would be in the setting of an acute painful vertebral fracture. Using the STEPS approach to evaluating a new medication:

• Safety - no important safety concerns were identified

• Tolerability - calcitonin is administered either IV, IM or by nasal spray. each of these have their costs to tolerability, but for patients that can overcome these concerns, it remains an option.

• Effectiveness - a statistically and clinically significant benefit in pain can be found for acute, painful vertebral fractures.

• Price - the generic nasal sprays of calcitonin can run into the $100 range per month, but widely-available discount programs can reduce that cost to the $30-40 range.

• Simplicity - calcitonin nasal spray is the only really convenient outpatient option, and is fairly simple at 1 spray in the nostril once a day. The instructions suggest that it is important to alternate nostrils daily, which could complicate the issue, slightly.

So, in summary, nasal calcitonin spray seems like a reasonable intervention for acute vertebral fractures from osteoporosis, and is affordable for those with are not financially challenged or who have insurance.