Background
The importance of providing patient-centered care in orthopaedic trauma cannot be overstated. Orthopaedic surgeons have an obligation to value the healthcare experience of their patients and are being held accountable in the form of patient satisfaction assessment tools (Menendez et al. 2019; Peres-da-Silva et al. 2017; Maratt et al. 2015; Mistry et al. 2016). Patient-reported outcome measures (PROMs) are an example of this and are being used more frequently in orthopaedic trauma (Moore, Jayakumar, Laverty, et al. 2019). These surveys are sent directly to patients and interpretated independent of their associated medical conditions (Deshpande et al. 2011). Not only do PROMs give clinicians a better understanding of patient values and satisfaction, they also provide valuable information necessary for hospital reimbursement and help direct healthcare policy (Menendez et al. 2019; Menendez and Ring 2015; VanLare and Conway 2012; Sacks et al. 2015; Sheetz et al. 2014; Fenton 2012).
Although PROMs provide essential benchmarks for quality, they are not without limitation. The multiple-choice format of many PROM surveys limit patients’ range of response (Graham et al. 2015; Huppertz and Smith 2014; Malhotra et al. 2016). In addition, it is unclear if PROM scores alone provide an adequate depiction of a patients’ post-operative recovery course (Huppertz and Smith 2014). In order to provide patients additional response options, we built in free text comment areas within the standard PROM assessments for the past two years at our institution and analyzed patients unsolicited responses. This was done in an attempt to decrease survey burden by allowing patients to directly communicate with their orthopaedic trauma team about the topics they value most in their healthcare.
The goal of this study is to determine the frequency of supplementary comments on post-operative orthopedic trauma PROM surveys and to analyze their common themes. With this information, healthcare providers will have better insight into the underlying motives that lead to patient satisfaction or dissatisfaction (Abtahi et al. 2015; Menendez et al. 2015). We hypothesis that patients will have more comments related to their relationship with their orthopedic surgeon than their functional status in the post-operative period.
Materials and Methods
Institutional Review Board (IRB) approval was obtained for this study. We performed a retrospective review of electronic medical records and PROM assessments at a metropolitan Level I trauma center in the United States between January 2018 to April 2020. IRB was obtained to link our orthopaedic trauma registry and vendor-supplied PROM data which included free text responses. We included patients who completed a validated PROM survey, underwent operative intervention with the orthopaedic trauma services, were at least 18 years old, and had a minimum of 6 weeks post-operative follow up. Patients who did not sustain a traumatic injury or have an operative intervention were excluded. PROM surveys were sent via email or text, per patient preference, at six weeks, three months, six months, one year, and two years post-operatively. If the PROM was not completed within a reasonable time frame, an additional call was made to the patient. Overall, 86% of our surveys were obtained via emails, 9% were obtained via text messages, and 6% were obtained via phone. Because of the low portion of surveys completed by text messages and phone, we were not able to perform meaningful analysis for each method. We used a combination of general PROMs, such as the Patient-Reported Outcome Measurement Information System (PROMIS-10) (Hays et al. 2009), Single Assessment Numeric Evaluation (SANE) (O’Connor and Ring 2019), and the Short Form 12 (SF-12) (Ware, Kosinski, and Keller 1996), in addition to injury specific PROs including the Foot and Ankle Ability Measures (FAAM) (Martin et al. 2005), Disabilities of the Arm, Shoulder and Hand (DASH) (Hudak et al. 1996), Patient-Rated Wrist Evaluation (PRWE) (MacDermid 1996), American Shoulder and Elbow Surgeons (ASES) (Michener, McClure, and Sennett 2002), Hip disability and Osteoarthritis Outcome Score (HOOS) (Lyman, Lee, Franklin, Li, Mayman, et al. 2016), Knee injury and Osteoarthritis Outcome Score (KOOS) (Lyman, Lee, Franklin, Li, Cross, et al. 2016), and Majeed (Majeed 1989) (Table 1). Patients were given the opportunity to leave additional comments on four different questions if they felt pertinent information regarding their care was not adequately addressed by the standardized multiple-choice questionnaire. Lastly, patients were asked to rate their satisfaction with their overall care on a scale from 0-10 with 10 representing complete satisfaction.
A single reviewer individually interpreted every documented patient comment. A running list of common themes discussed by patients was constructed and presented to a board of reviewers. After deliberation, the decision was made to include four positive categories, four negative categories, and one neutral category. An example of our thought process can be extrapolated from the following comment, “Dr. Meyer is extremely kind and patient. He listens to my questions thoroughly and answers them thoroughly.” This specific comment was categorized as effective communication and empathy since this surgeon was complimented both on their communication skills in addition to their empathy. Once the comment review committee agreed upon the included categories, the reviewer categorized all comments accordingly. Controversial comments were marked and discussed by the committee.
The analysis of comments manifested into nine categories of data, including 1,2) positive and negative comments with regards to communication with the orthopaedic surgery trauma team, 3,4) positive and negative comments concerning functional outcome, 5) empathetic care expressed by staff members, 6) inadequate pain management, 7) suggestions for the survey itself, 8) appreciation for surgical care, and 9) neutral comments. Of note, a single free text comment could be placed into more than one category as described previously. The total number of comments made in each category was calculated and reported as a percentage of the total number of comment categories counted which exceeded the total number of comments. Patients were also identified as either commenting once or more than once throughout the potential two-year span.
The data was further subdivided into two major categories; those who completed a PROM and chose to leave a comment and those that completed a PROM and chose to not leave a comment. Demographic variables such as age and sex were determined for each cohort. The associated PROMIS-10 scores for each group were then analyzed using t-tests with p-values of <.05 to determine significance.
Results
In over two years of data collection, we observed an overall response rate of 38.5% on PROMs. Of the 3068 PROMs that were completed, 1112 (36%) surveys had additional comments from patients who had 742 procedures. Four hundred and forty-two commenters were female (60%) and 300 commenters were male (40%). The age range of our patients was 20 to 101 with an average age of 59.74 +/- 17.66 years old. This was compared to the group of non-commenters which consisted of 1803 total patients. Nine hundred and sixty-seven were female (54%) and 836 were male (46%). The average age in the non-commenting cohort was 59.26 +/- 21.06 (Table 2). There was no statistically significant difference between the PROMIS-10 scores of the commenters group and the non-commenters group. The number of patients included based on injury module were 181 ankle fractures, 123 hip fractures, 19 humerus/forearm fractures, 4 chest wall injuries, 25 foot fractures, 47 hardware removal, 94 tibial plateau/distal femur/patella fractures, 55 femur/tibial shaft fractures, 30 pelvis fractures, 19 scapular fractures, 112 shoulder/elbow fractures, and 33 distal radius fractures.
Overall, positive comments predominated at a rate of 57% with appreciation for the orthopaedic surgical team being the most common topic (34% of all comments). Other categorizes considered to be positive in nature were effective communication by orthopaedic surgical team members (9% of all comments), empathetic care (8% of all comments), and positive functional outcome (6% of all comments). The rate of negative comments was found to be 35% of included data. Themes assessed were poor communication with the orthopaedic surgical team members (16% of all comments), dissatisfactory functional outcome (8% of all comments), suggestions for survey improvement (6% of all comments) and inadequate pain management (4% of all comments). 8% of the comments contained in this study had a neutral connotation (Figure 1, 2). Following the dissemination of these findings in our department, we observed an increase of 2.5% in satisfaction rating in the immediate three-month period that followed.
Discussion
The high comment rate observed in the study supports the theory that a significant amount of data exists outside the current published PROM literature on patient experience and satisfaction. Interestingly, communication was a more frequented topic than physical function. The feedback we received from our staff was that our research provided meaningful insight into patient values as it translated into improved satisfaction in the care of trauma patients. Because of this, we recommend including open ended questions with free text areas on the standardized PROM questionnaire. This qualitative data can provide additional information on patients’ experience and, in turn, help providers improve upon post-operative care.
Other studies have asked this question and have also harvested unique data. Menendez et al (2019) analyzed comments following total shoulder arthroplasty and found 62% of the comments to be positive in nature, 32% negative, 5% mixed, and 1% neutral. Their study focused on the perioperative period and used the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) tool. Their comment topics revolved around inpatient care and included themes related to room condition, time management, communication, compassion, IV insertion, food, medication side effects, discharge instructions, and pain management. Likewise, Huppertz et al (2014) argued that the numerical scoring of the HCAHPS underestimated patients’ perception of their hospital care and cited a 20% additional comment rate. Fields outside of orthopaedics have also started to utilize comment analysis and Tanniru et al (2017) found nursing care, communication, and infrastructure to be the most commonly discussed topics related to patient satisfaction in the emergency department. In the spine setting, Donnally et al (2018) concluded that surgeons were more likely to receive favorable reviews on internet sites for factors related to functional outcome and surgeon likeability. Contrarily, negative reviews were commonly based on poor interactions with support staff, billing, and the office environment.
Though not directly consistent with our findings, previous literature, in combination with our study, suggests that the current analysis of patient-centered care may be insufficient to understanding patient values. A unique example of this possible misunderstanding found in our analysis was the mere 14% comment rate with regards to functional outcome. Orthopaedists closely monitor this metric in the outpatient setting and may put less emphasis on other important aspects of care such as gratitude, effective communication, and empathy. Perhaps more focus should be shifted toward the non-medical aspects of orthopaedic trauma care in the non-emergent environment. It is also worth recognizing the design of this study. Rather than asking patients to complete the survey as an inpatient, surveys were electronically issued to a personal email address or phone number. This allowed patients to complete the form without time constraints and in the comfort of their own home. We believe this permitted enhanced patient reflection and motivated patients to freely communicate about their experience.
Our study has several limitations. It represents a gross assessment of patient experiences in orthopaedic trauma at our institution and the results may not be generalizable. We collected comments from all postoperative trauma patients regardless of injury type and anatomical location. Hence, our cohort was heterogeneous and encompassed a variety of injury patterns, types, location, and severity. We utilized both general health and injury-specific-PROMs for the study. With that being said, the focus of this article was to investigate patient comments and not actual PROM score. Additionally, our data encompassed patient responses at various time points in the post-operative state. This likely influenced the topics most important to patients since the initial weeks following surgery are the most painful. Lastly, the categories we created were arbitrary, but created after a thorough review of all collected comments.
Future studies should focus on more homogenous populations to better understand patients’ perspective related to injury type and severity, PROM module, demographic cohort, and the post-operative time period. In addition, predictors of negative and positive comments should be investigated and association of comments and PROM scores can be determined.
Conclusion
This study should be viewed as a glimpse into the large subsect of unassessed data that exists in the analysis of patient-centered care. Given the wide array of results in personalized comment analysis in our study and others, the current understanding of value-based care may be subject to collection bias. To combat this, further attention should be given to unsolicited outcome data in the form of personalized comments. This information can be used to provide higher quality patient care and to promote better data collection techniques.
Abbreviations
Patient-reported outcome (PRO)
Patient-Reported Outcome Measurement Information System (PROMIS-10)
Foot and Ankle Ability Measures (FAAM)
Disabilities of the Arm, Shoulder and Hand (DASH)
Patient-Rated Wrist Evaluation (PRWE)
American Academy of Orthopaedic Surgeons self-assessment tool II (AAOS-II)
American Shoulder and Elbow Surgeons (ASES)
Hip disability and Osteoarthritis Outcome Score (HOOS)
Knee injury and Osteoarthritis Outcome Score (KOOS)
Ethics approval and consent to participate
All patients were consented and agreed to participate in this prospectively collected retrospective review.
Consent for publication
Not applicable.
Availability of data and materials
The datasets generated and analyzed during the current study are not publicly available due to the inclusion of personalized medical history but are available from the corresponding author on reasonable request.
Competing interests
The authors declare that they have no competing interests.
Funding
All funding for this research article came from the Department of Orthopaedic Surgery at Regions Hospital in St. Paul, MN.
Author contributions
TC performed background research on current published PRO literature, organized/reviewed data, and drafted the manuscript. SV designed the comment section on the PRO surveys and served as team leader/organizer/editor. AS managed data compilation and blinded reviewer to narrative responses on surveys. RH performed initial data collection and helped design the study. LS aided in editing and research design. PC provided guidance and editing throughout the project. MN was the corresponding author and oversaw all activities related to manuscript production.
Acknowledgements
Not applicable.