MAPS: Maitland-Australian homecontact us
find us on facebook find us on linkedin

RESEARCH COMMENTARY
Ejona (Ona) Jeblonski, DPT, COMT
Fellow in Training, MAPS Accredited Fellowship in Orthopedic Therapy

Chris R. Showalter, PT, COMT, OCS, FAAOMPT
MAPS Fellowship Program Director

Mobilization Increases Dorsiflexion in Chronic Ankle Instability (CAI) Patients


 
sign up for email updates from MAPS

 
The fragility of CPRs in predictive modeling…Are they really such robust clinical tools?

June 25, 2013

RESEARCH COMMENTARY

Chris Showalter PT, OCS, COMT, FAAOMPT


MAPS Clinical Director

Full Disclosure: This month’s Research Commentary discusses a peer- reviewed research article by Julie Schwind and co-authored by MAPS Research Consultant Chad Cook and MAPS Senior Faculty Members Ken Learman, Chris Showalter, and Bryan O’Halloran. The synopsis and link to JMMT are found below.

The fragility of CPRs in predictive modeling…

Are they really such robust clinical tools?

JMMT recently published a peer-reviewed article in the May 2013 issue, which is a secondary analysis of data from our previous RCT comparing thrust and non-thrust manipulation in patients with mechanical low back pain, entitled, “Early use of thrust manipulation versus non-thrust manipulation: A randomized clinical trialâ€? (see MAPS’ October 10, 2012 Research Commentary, “Mobilization and Manipulation are EQUALLY EFFECTIVE and produce the SAME OUTCOMES in Mechanical LBPâ€?).  

The Schwind study (JMMT, May 2013), “Different Minimally Important Clinical Difference (MCID) scores lead to different clinical prediction rules for the Oswestry disability index for the same sample of patientsâ€? examines the strength of predictive modeling in the face of different MCIDs and population baseline characteristics.   For clarification purposes, as MAPS faculty and consultants are listed co-authors with Julie Schwind on this study, all references below refer to the 2013 Schwind study as “ourâ€? study.

Minimally Clinical Important Difference (MCID) is the smallest measurable clinical difference as a result of an intervention.  MCIDs are patient derived scores that reflect changes in a clinical intervention that are meaningful for the patient.  In other words, it is a measure of the value of an intervention, as expressed by patient perception of change, rather than statistical changes in clinical measurements (which may not actually be meaningful to the patient).

MCID scores for outcome measures are frequently used in evidenced-based guides to gauge meaningful change.  There are numerous outcome instruments used for analyzing pain, disability, and dysfunction of the low back; perhaps the most common of these is the Oswestry Disability Index (ODI).

It is important to note that a single agreed-upon MCID score has not yet been established for the ODI.  What is also unknown is whether selected baseline variables will be universal predictors, regardless of MCID used for a particular outcome measure.

Our secondary analysis (Schwind, et. al, JMMT, May 2013) sought to determine the influence between MCID and predictive variables within a single patient population and outcome measure.  The results of this study show that different MCID’s used on the same outcome measure across the same patient population lead to notably different predictive models. 

Clinical prediction rules (CPRs) are forms of predictive models and include the low back pain manipulation CPR (Flynn 2002). CPRs have been assumed to be robust clinical tools that are usable in any given clinical situation.   

However, our findings suggest that this may not be the case.   

In addition, after modifying our analyses for baseline ODI scores, we found several different predictive models using different MCID interpretations. 

Our May 2013 study indicates that the value of any given predictive model, or CPR, is highly dependent upon:

1) the MCID cutpoints used in interpretation of “success� and

2) baseline characteristics of the population.


Below is a summary of the findings from the JMMT May 2013 Scwind study:

  1. Three variables were significantly associated with a 50% change in the ODI (meeting the CPR, younger age, diagnosis of lumbago and degenerative disease)
  2.  Four variables were significantly associated with a 30% change in the ODI (Lower baseline FABQ, shorter symptom duration, younger age, and diagnosis of lumbago and degenerative disease)
  3. Two variables were significantly associated with a 17 Point change in the ODI (Higher baseline ODI, and meeting the CPR)
  4. Two variables were significantly associated with both a 10 Point and 5 Point change in the ODI…(higher baseline ODI, and younger age)
  5. Three variables were significantly associated with a final ODI of ≤ 20% (lower baseline ODI, younger age, and meeting the CPR)
  6. Younger age was significant in five out of six models
  7. Higher ODI baseline score was significant in three out of six models
  8. Meeting the CPR was significant in three out of six models
  9. Diagnosis of lumbago and degenerative disease was significant in two of six models
  10. Lower baseline FABQ, Shorter symptom duration, and lower baseline ODI were       all significant in one of six models

Our study concludes that when different MCIDs are used, on the same outcome measure, across the same patient population, several notably different predictive models emerge. 

Thus, our study suggests the inherent fragility of predictive models.

Without a universally stable definition of “meaningful changeâ€?, it is likely that different MCID scores will result in very different predictive variables in different patient populations.  No variables in the study were determined to be universal prognostic indicators across all models, regardless of the baseline ODI measure.  It is highly likely that models that do not involve universal predictors are therefore not transferrable across different patient samples.  The same is likely true for models that use different MCIDs.

Our findings suggest that CPRs are likely to be markedly different depending on the definition of recovery.  For example, the CPR for lumbar manipulation, which used a 50% change of the ODI from baseline, would likely consist of different predictive variables if a 30% change from baseline was selected.  In addition, different baseline severity levels will likely interact with the MCID interpretations further varying a predictive model.  As such, it is strongly recommended that only universal predictors should be considered when creating predictive models, and at present this has been poorly studied and remains unknown.

Establishing universal prognostic indicators AND agreed MCIDs for common outcome measures (such as ODI) would be of enormous value in the development of truly robust predictive models such as CPRs

IMPORTANT TAKE-HOME MESSAGES  

  1. There is extreme variability between predictive models. (when using different MCIDs on the ODI within the same patient population)   

  2. This variability highlights the instability of predictive modeling (including CPRs)

  3. Predictive models are significantly affected by population baseline characteristics

  4. Predictive models are significantly affected by the MCID used

  5. The fragility of predictive modeling creates significant difficulty when attempting to apply CPRs to clinical practice.

  6. CPRs may not be reliable across all patient populations

  7. CPRs may not be as robust or clinically applicable as previously believed

     

Article Follows (with weblink to JMMT)

Cheers and Enjoy,

Chris R. Showalter PT, OCS, COMT, FAAOMPT 

© Chris R. Showalter and Maitland Australian Physiotherapy Seminars


Not to be reproduced, copied or retransmitted in any manner without author’s express written permission

Directing others to the MAPS website (www.ozpt.com) is permissible.


“Different minimally important clinical difference (MCID) scores lead to different clinical prediction rules for the Oswestry disability index for the same sample of patients�

Authors: Schwind, Julie; Learman, Kenneth; O’Halloran, Bryan; Showalter, Christopher; Cook, Chad

Source: Journal of Manual & Manipulative Therapy, Volume 21, Number 2, 2013, pp. 71-78(8)


Abstract:

Background: Minimal clinically important difference (MCID) scores for outcome measures are frequently used evidence-based guides to gage meaningful changes. There are numerous outcome instruments used for analyzing pain, disability, and dysfunction of the low back; perhaps the most common of these is the Oswestry disability index (ODI). A single agreed-upon MCID score for the ODI has yet to be established. What is also unknown is whether selected baseline variables will be universal predictors regardless of the MCID used for a particular outcome measure.

Objective: To explore the relationship between predictive models and the MCID cutpoint on the ODI.

Setting: Data was collected from 16 outpatient physical therapy clinics in 10 states.

Design: Secondary database analysis using backward stepwise deletion logistic regression of data from a randomized controlled trial (RCT) to create prognostic clinical prediction rules (CPR).

Participants and Interventions: One hundred and forty-nine patients (149) with low back pain (LBP) were enrolled in the RCT. All were treated with manual therapy, with a majority also receiving spine-strengthening exercises.

Results: The resultant predictive models were dependent upon the MCID used and baseline sample characteristics. All CPR were statistically significant (P < 001). All six MCID cutpoints used resulted in completely different significant predictor variables with no predictor significant across all models.

Limitations: The primary limitations include sub-optimal sample size and study design.

Conclusions: There is extreme variability among predictive models created using different MCIDs on the ODI within the same patient population. Our findings highlight the instability of predictive modeling, as these models are significantly affected by population baseline characteristics along with the MCID used. Clinicians must be aware of the fragility of CPR prior to applying each in clinical practice.

DOI: http://dx.doi.org/10.1179/2042618613Y.0000000028

By this author: Schwind, JulieLearman, KennethO’Halloran, BryanShowalter, ChristopherCook, Chad


Where to find the Articles  

1. http://www.manualtherapyjournal.com/article/S1356-689X(12)00189-0/abstract

Manual Therapy 2013;18(3):191-198

Early use of thrust manipulation versus non-thrust manipulation:

A randomized clinical trial

Cook C, Learman K, Showalter C, Kabbaz V, O’Halloran B.

 

2. http://www.ncbi.nlm.nih.gov/pubmed/12486357

Spine 2002;27(24):2835-43

A Clinical prediction rule for classifying patients with low back pain who demonstrate short-term improvement with spinal manipulation

Flynn T, Fritz J, Whitman J, Wainner R, Magel J, Rendeiro D, Butler B, Garber M, Allison S.

Back To Research Commentary
Find a COMT Course Schedule Course Descriptions Register for Courses Faculty
Looking for a COMT? This list of Certified Orthopedic Manual Therapists (or COMTs) includes all COMT graduates worldwide, as well as MAPS faculty. We suggest that you visit this page often to check course confirmation status and to find new courses which are added frequently. Our seminars incorporate the teaching of Geoff Maitland and others who have contributed to what has become known as the "Maitland-Australian Approach". Registering for classes is easy. Submit full payment with the registration form. Reservations are limited and are on a first come, first serve basis. Come meet our world-class faculty. Each of our faculty has experience in a wide variety of manual therapy practices and specialty areas.

 

HOME | ABOUT US | COURSES | RESEARCH | CLINICAL SITES | CONTACT US | SIGN UP FOR MAPS EMAIL UPDATES
© 2007 OZPT.com. All rights reserved.