INTRODUCTION
Scores such as the American Society of Anesthesiologists (ASA) Classification and the Charlson Comorbidity Index (CCI) are used as metrics to assess pre-operative health, morbidity and mortality risk, length of hospital stay, post-operative complications, and treatment cost (Lavelle, Cheney, and Lavelle 2015). Originally, the ASA classification was implemented to collect statistical data for an anesthesia episode, not as a surrogate for pre-operative risk (Dripps, Lamont, and Eckenhoff 1961). However, in 1963 it was accepted by the American Society of Anesthesiologists as an independent pre-operative risk predictor (Daabiss 2011), and in 2014 the current six category ASA classification was adopted (American Society of Anesthesiologists 2014). Similarly, the CCI was originally developed to predict one-year mortality risk in a single series of hospitalized patients (Charlson et al. 1987). The index was later adapted such that the requisite information could be acquired from patient medical records to predict mortality risk (Deyo, Cherkin, and Ciol 1992).
More recently, ASA and CCI scores have been applied as peri-operative risk predictors and determinants of healthcare utilization and cost. Worse scores have been associated with higher surgical site infection rates (Guo et al. 2016; Tan et al. 2013), higher rates of readmission following major trauma (Tran et al. 2017), higher complication rate and cost of care following spine surgery (Whitmore et al. 2014), worse post-operative outcomes after lumbar spinal fusion (Ondeck et al. 2018), longer length of stay following orthopaedic trauma (Lakomkin et al. 2017), and higher cost of care and mortality rates for hip fracture patients during the inpatient stay, at 30 days, and at one year (Johnson et al. 2015; Lau, Fang, and Leung 2016). Concurrently, patient self-report questionnaires have become more prevalent in recent decades for information gathering and monitoring across medical specialties (O’Neill et al. 2014). For instance, self-report questionnaires have shown more significance in the ability to predict disability and mortality in patients with rheumatoid arthritis than traditional measures such as laboratory values and radiographs (Pincus and Castrejon 2014), have been useful for psychological assessment of patients with tinnitus and hyperacusis (Aazh and Moore 2017), have been cited as a beneficial tool for monitoring cardiovascular conditions in patients with hypercholesterolemia (Englert et al. 2010), and have been employed in the outpatient orthopaedic surgery setting to obtain useful and accurate medical information (Boissonnault and Badke 2005).
Patient self-reported metrics have additionally been compared with these established ASA and CCI scores, with a fair amount of agreement in predictive value. In assessing one-year mortality, patient-self reporting had comparable accuracy to calculated scores generated from chart records (Chaudhry, Jin, and Meltzer 2005). In patients with acute coronary syndrome, comorbidity data acquired from patient reporting versus that obtained directly from medical records offered similar insight into quality of life and physical functional capacity, albeit with greater efficiency and lower cost when obtained via patient self-report (Olomu et al. 2012). Similarly, in hemodialysis patients the self-report data acquired from questionnaires was comparable to that obtained from calculated scoring metrics (Sridharan et al. 2014). Finally, in an emergency department study, patient-reported data exhibited comparable predictive ability to calculated scores in determining future utilization of medical services and functional decline (Susser, McCusker, and Belzile 2008). However, a word of caution regarding patient-reported data is that there is significant inconsistency as far as methodology. There are a multitude of differing questionnaires, time points of administrations, phrasing and ordering of questions, etc. This variability presents an additional challenge in assessing the generalizability of these study results to different patient populations, injuries and/or diseases, and healthcare systems (O’Neill et al. 2014). Patient-reported questionnaires that are standardized in their content, mandatory, and administered in an unselected fashion would allow a more level field upon which to measure and compare responses despite potentially differing demographics or pathology.
In 2007, the Centers for Medicare and Medicaid Services (CMS) instituted the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey as a mandated, national, and standardized tool to collect patient reported data after hospital discharge (Centers for Medicare and Medicaid Services 2016). Private third party vendors, such as Press Ganey, traditionally administer this survey on behalf of hospitals and often include further questions for a hospital’s internal quality improvement (Centers for Medicare and Medicaid Services 2017). These survey questions include patient self-reported demographics, educational level, perception of health status, and granular items relating to inpatient care and satisfaction for a variety of treatment areas. Given the standardization and mandatory administration, the self-reported health component of these surveys may hold promise as a surrogate or adjunct metric to evaluate overall health status, potential morbidity and/or mortality, and future healthcare utilization and cost. To our knowledge, no study to date has evaluated the relationship between HCAHPS and Press Ganey patient self-reported health questions and ASA and CCI scores. The present study assesses this relationship in a series of orthopaedic trauma patients to determine if patient-reported health data from government and private surveys correlates with traditionally accepted metrics of health status.
METHODS
This was a retrospective comparative study approved by our hospital’s institutional review board. Hospital medical records of all consecutive patients 18 years of age and older who were surgically treated for fractures at a Level 1 Trauma Center by fellowship-trained orthopaedic surgeons between January 1, 2014 to December 31, 2016 were retrospectively reviewed. Of these patients, those that were surgically treated for isolated fractures were included. The fracture categories were: 1) isolated upper extremity fracture, 2) isolated lower extremity fracture, and 3) isolated acetabulum or pelvis fracture. To obtain this information, each patient’s admitting history and physical note and/or orthopaedic surgery consultation note, operative report, and discharge summary were reviewed. Exclusion criteria included: patients under 18 years of age at the time of admission; patients with fractures that were not treated surgically; patients with polytrauma (ex: multiple fractures, fracture and abdominal organ injury, fracture and head injury, etc.); patients who were not admitted for at least one hospital day; patients suffering a postoperative complication requiring repeat operation; patients with a primary residence outside the state of Florida; or patients who did not respond to the post-discharge patient satisfaction survey. Acquiring this survey response information was necessary because prior to May 2015, it was not standard practice at our institution for all patients to be sent post-discharge surveys. As the study period overlapped this time, the information was important to accurately characterize the survey response rate for our study population. Additionally, this information decreased sampling bias, because without it one could not know if a lack of survey response was due to the patient not responding to a survey that was sent, or not actually receiving a survey in the first place (if it was before May 2015).
Additional patient demographic data were collected, including, age; sex; medical comorbidities; ASA classification (obtained from the pre-operative anesthesia physician note in the medical record); CCI and CCI 10-year estimated survival (calculated using the original, validated methodology (Charlson et al. 1987)); psychiatric comorbidities; smoking status; and substance abuse. Injury data were also collected, including, laterality; open versus closed injury; procedure performed; number of prior surgeries to the injured site; and hospital length of stay. A total of 2793 patients were initially reviewed. Of these, 124 had a residence outside the state of Florida, 208 were less than 18 years of age at the time of admission, and 1289 were not isolated fractures. Of the remaining 1172 (215 upper extremity, 885 lower extremity, 72 acetabulum/pelvis), 111 were duplicate entries, 340 were not sent a post-discharge satisfaction survey, and 586 were sent a survey but did not respond. This left 152 total patients that met all inclusion and exclusion criteria and were thus included in the study.
Press Ganey was the survey vendor that administered the post-discharge patient satisfaction survey used by our institution. The survey contains a total of 71 questions. The initial 32 questions comprise the government-mandated HCAHPS questions. The remaining 39 questions, titled “Additional Questions About Your Stay,” are included by Press Ganey to further evaluate patients’ hospital experience. Six questions out of the 32 total HCAHPS questions address self-reported patient demographics and health status. Of note, the HCAHPS component does not have a single “overall” or aggregate score. Rather, there are two scores referred to as “Global Rating Items,” which are titled “Rate Hospital 0-10” and “Recommend the hospital.” There is additionally a separate overall score for the “Pain Management” Domain. Of the total 39 additional Press Ganey questions, one addresses pain control as an aggregate score and another addresses the overall score for the Press Ganey section. Patient responses to these survey questions were obtained.
Patient responses to the above survey questions were compared with the ASA classification score and the CCI during the inpatient stay. Specifically, correlations were made between: 1) patient-reported health and ASA score, 2) patient-reported health and CCI, 3) patient-reported health and CCI 10-year estimated survival, and 4) demographic, comorbidity, injury, and surgery characteristics within the patient cohort. Statistical analysis included Spearman’s Rho for correlations, Wilcoxon Rank-Sum and Kruskal-Wallis for continuous variables, backwards stepwise regression and ordinal regression with bootstrapped confidence intervals for odds assessment.
RESULTS
A total of 152 patients with a mean age of 57 years were included and analyzed. Table 1 illustrates patient demographics, medical and psychiatric comorbidities, ASA and CCI data, and substance abuse characteristics. The isolated fractures were stratified into 29 upper extremity, 112 lower extremity, and 11 acetabular/pelvic. The majority were closed fractures, and the most common fixation methods were open reduction internal fixation (ORIF, 66.9%) and intramedullary nailing (IMN, 26.5%). Median hospital length of stay was 3 days (interquartile range [IQR] 2-5.5). Injury and surgery characteristics are depicted in Table 2.
Patient responses to the post-discharge HCAHPS and Press Ganey satisfaction survey questions specifically regarding self-identified demographics, health perception, education, and overall ratings for both survey components are presented in Table 3. Comparative data for patient demographics and comorbidities from the medical record versus the ASA, CCI, and CCI 10-year estimated survival are presented in Table 4. Strong correlations were initially found between patient-reported overall health (Q25) and ASA, CCI, and CCI 10-year estimated survival. Strong correlation was initially found between patient-reported overall emotional/mental health (Q26) and ASA, but not for CCI and CCI 10-year estimated survival. However, upon adjusting for patient age and sex, only the ASA score (OR 3.65, 95% CI 2.03-6.57; p<0.001) exhibited a strong correlation with patient-reported overall health ratings, as illustrated in Table 5.
CONCLUSION
The American Society of Anesthesiologists score and the Charlson Comorbidity Index are validated and broadly applied tools used across medical specialties to evaluate patients’ perioperative risk, potential for complications, post-surgical outcome, mortality at various time points, and healthcare cost and future resource utilization (Tran et al. 2017; Guo et al. 2016; Tan et al. 2013; Whitmore et al. 2014; Ondeck et al. 2018; Lakomkin et al. 2017; Johnson et al. 2015; Lau, Fang, and Leung 2016). Furthermore, patient self-reported health data have become more common in recent years, and demonstrate reliable predictive accuracy for ASA scores and CCI values in various studies (Chaudhry, Jin, and Meltzer 2005; Olomu et al. 2012; Sridharan et al. 2014; Susser, McCusker, and Belzile 2008). However, despite some promising data, the disparate methodology and lack of standardization with various patient-reported questionnaires remain challenges (O’Neill et al. 2014). Yet since the late 2000s, and particularly with the adoption of the Patient Protection and Affordable Care Act, patient-reported survey data have been collected en masse and used by individual institutions and the Centers for Medicare and Medicaid Services to establish care standards, rate hospitals and physicians, and guide compensation protocols (Centers for Medicare and Medicaid Services 2017, 2016). Relevant to the present study, these patient-reported surveys are government-mandated and thus uniformly administered by hospitals (or their approved third-party vendors), and are standardized in question format, ordering, and phrasing. They further include not only patient satisfaction data for the particular episode of care, but also questions about patient perception of their overall, mental, and emotional health. To date no prior study has evaluated the potential relationship between these standardized patient self-health perception questions and the more traditionally used and validated ASA and CCI metrics.
We initially observed strong correlations between patient-reported overall health and ASA score, CCI, and CCI 10-year estimated survival. We also observed strong correlations between patient-reported mental/emotional health and ASA, but not with CCI or CCI 10-year estimated survival, as seen in Table 4. Upon adjusting for patient age and sex (Table 5), only the ASA score remained strongly associated with patient-reported overall health (OR 3.65, 95% CI 2.03-6.57; p<0.001). Adjusting for age and sex is important in evaluating health literacy and patient self-reporting, as both have been described as potential confounding variables that can obfuscate results (Chesser et al. 2016; Clouston, Manganello, and Richards 2017). The agreement between ASA scores and patient-reported overall health is interesting for several reasons. ASA scores are readily available, as they are assigned to all patients pre-operatively, and can easily be collected in a prospective series or for a retrospective review. Unlike the CCI, the ASA score does not necessitate extensive chart review to gather multiple parameters to arrive at the index value of interest. Notably, comparative literature between ASA score and CCI has shown varying levels of agreement in predictive value; some studies have shown high agreement (Lavelle, Cheney, and Lavelle 2015), while others have found better predictive association with either ASA (Gronbeck et al. 2019) or CCI (Ulyett et al. 2017) for the particular research question.
Like the ASA score, patient-reported data from the government-mandated patient satisfaction surveys are readily available for direct analysis. Furthermore, the strong association with ASA scores observed in our patient cohort makes an initial argument that these patient-reported overall health ratings from government mandated surveys may hold promise as an adjunct or surrogate in studying healthcare utilization, patient outcomes, complication risk, etc. It may further allow providers and institutions to both identify patients at higher risk for negative sequelae and be proactive in mitigation.
However, this initial data must be qualified in light of the fact that this was a retrospective study of orthopaedic trauma patients. Although the findings are interesting in their potential applications, one cannot accurately comment on their relative generalizability to different patient demographics and/or pathologies. Further investigation of different patient and injury/disease cohorts may divulge new or even conflicting information as compared to the present study. Our included patients had a median age of 57 years and sustained only isolated orthopaedic injuries requiring operative intervention. Patients at the extremes of age, or those suffering from polytrauma or hospitalized secondary to chronic conditions, may not exhibit concordance between their self-health perception and ASA scores. Additionally, given that this was a retrospective study, the natural limitations of such research are present. We attempted to combat potential selection bias by reviewing all consecutive orthopaedic trauma patients during the desired study period and by adhering to strict predefined inclusion and exclusion criteria. Potentially confounding data in terms of differing or multiple injuries were addressed by limiting the investigative cohort to only isolated orthopaedic trauma patients.
Ultimately, larger prospective investigations are needed to further characterize the various permutations in patients and pathology, including how these factor into the relationship of patient-reported health perception and validated metrics like the ASA score. Gleaning this information will help elucidate whether patient-reported overall health ratings from standardized, government-mandated surveys are broadly useful, or are only relevant for specific patient cohorts or situations.
ACKNOWLEDGEMENTS
The authors would like to thank Barb Steverson for her tremendous assistance in organizing data and continued communication throughout the project.