Introduction
The goal of orthopaedic residency is to train surgeons who function capably and responsibly within our current healthcare system. The Accreditation Council for Graduate Medical Education (ACGME) has outlined six core competencies for mastery by residents: 1) practice-based learning and improvement, 2) patient care and procedural skills, 3) professionalism, 4) medical knowledge, 5) interpersonal and communication skills, and 6) systems-based practice (“Program Requirements for Graduate Medical Education in Orthopaedic Surgery” 2019). Systems-based practice as described by the ACGME includes an understanding of the larger context and system of healthcare, financial considerations including value, cost-awareness, delivery and payment, and understanding of both external and internal administrative and regulatory requirements in the increasingly complex environments of medical practice.
In a nationwide survey of the 154 US orthopaedic residencies in 2001, only 14 residency programs self-reported placing emphasis on topics regarding business, healthcare economics, and systems-based practice within their curricula (Kiesau, Heim, and Parekh 2011; Stautberg Iii et al. 2018). While patient care, technical skills, and medical knowledge are the emphasis in the vast majority of orthopaedic residency training programs; systems-based practice is not commonly integrated into curricula (Crites and Schuster 2004). This deficit of knowledge means that as residents transition into practice they may not be well-equipped to establish a new practice, manage practice finances, or play an active role in quality improvement and cost awareness (Lusco, Martinez, and Polk 2005). With the increasing complexity of the healthcare system, entrepreneurial and business skills are essential to a successful career in orthopaedics. In most practice settings, physicians must interact with multiple stakeholders, and knowledge obtained during residency must prepare them to manage this increasingly complex healthcare environment (Ovadia et al. 2018; Gill and Schutt 2007).
Other medical specialties, particularly family medicine and internal medicine, have examined and begun formally integrating aspects of systems-based practice into resident education (David and Reich 2005; Peters et al. 2008; Crites and Schuster 2004). Currently there is little data documenting orthopaedic residents’ understanding of the systems-based practice core competency and how it is integrated into residency programs. The aim of our study was to assess orthopaedic residents’ understanding of systems-based practice and their interest in incorporating these principles into their curricula.
Methods
A survey was developed by orthopaedic faculty, research staff, and residency program directors in 2018 to evaluate resident demographics and ten domains of systems-based practice: personal finance, value of orthopaedics, practice dynamics, transition to practice, work life balance, research, professional development, leadership development, international medical work, and law. Each of the 16 questions was formatted as a modified Likert scale, rated from 0 to 10, with 10 indicating maximum interest/understanding and 0 implying none. Resident responses of 8 and above were considered strong interest/understanding. The survey was pretested on a resident focus group at a single institution to confirm completion rate, completion time, and best survey software.
Following survey development and testing, 10 orthopaedic residency programs were contacted, and 9 agreed to participate. The survey was disseminated to residents of these 9 programs via email using Qualtrics software (Provo, Utah).
Residents were categorized into subgroups based on years of training: junior residents (years 1-3) and senior residents (years 4-5). Comparative testing between institutions and years of training was performed using nonparametric chi-squared analysis. Statistical analysis was performed using SAS 9.4 (SAS Institute Cary, NC). IRB exemption was received in 2018 prior to survey development.
Results
Of the 275 residents who received the survey at nine different residency programs, 151 completed it for an overall response rate of 55%. Of residents that responded, junior residents comprised 61.6% and senior residents 38.4% (see Table 1)
Across all institutions a significantly larger proportion (91.4% vs 8.6%) reported poor understanding of the systems-based practice competency when residents were asked to rate their understanding of what this competency entails. Self-rated understanding of systems-based practice was statistically identical (p=1.00) between junior and senior residents; 91.4% of both groups indicated low comprehension of the systems-based practice competency.
Personal finance
There was no significant difference seen between junior and senior residents’ levels of interest in integration of RVUs, salary, and investments (p= 0.65) into curricula, with 51.7% overall reporting strong interest. Likewise, there was no significant difference between junior and senior residents’ levels of interest in integration of debt management and insurance (p=0.09) into curricula, with 76.2% reporting strong interest (Table 2).
Value of orthopaedics
There was no significant difference between junior and senior residents’ levels of interest in integration of value concepts including reimbursement, health insurance, contracts, and cost awareness (p=0.55), with the majority of respondents (82.1%) expressing strong interest in incorporating the topic into residency curricula (Table 2).
Practice dynamics
No significant difference between junior and senior residents was found in interest in understanding of team member roles (p = 0.89) or delegating tasks to improve efficiency (p=0.72), with the majority of respondents indicating strong interest in incorporating these topics into curricula (51% and 60.3%, respectively) (Table 2).
Practice models
No statistical difference was seen between resident years regarding interest in a fellowship forum that would encompass types of practice opportunities, reimbursement structure, anticipated changes and job market (p= 0.30). A large majority (122, 79.2%) believed that a fellowship forum would be helpful in navigating these topics (Table 2).
Transition to Practice
Senior residents showed significantly lower interest versus junior residents (p= 0.01) in incorporating topics of job searching, contract negotiations, contract law, and practice type overviews into residency curricula, although nearly three fourths of respondents overall (111, 73.5%) indicated that transition to practice should be better addressed during residency (Table 2).
Work life balance
No significant difference was found between junior and senior residents in topics of time management, work life balance, and avoiding burnout (p=0.54), with 48.1% of respondents indicating strong interest in incorporating this domain into curriculum (Table 2).
Research
No significant difference was found in interest in incorporating the research domain into curricula between junior and senior residents (p=0.54). Just 30.5% of overall respondents showed strong interest in topics including how to be a successful researcher, integration of research into a community-based practice, funding, and building a successful research team (Table 2).
Professional development
Regarding professional development in areas of industry relations, social media, and creating a referral base, senior residents showed significantly lower interest compared to junior residents (p= 0.04). However, no difference was found in topics of learning new skills, continued education, and orthopaedic society involvement (p= 0.60) (Table 2).
Leadership development
Senior residents showed no significantly higher level of interest in learning about leadership development compared to junior residents in topics of how to manage people and have tough conversations (p=0.58), build effective teams in the workplace and maximize staff productivity (p=0.09) or be an effective educator (p=0.97); approximately half of all respondents expressed strong interest in learning more about these topics during residency education: 43.7%, 57%, and 45%, respectively (Table 2).
International medical work
No significant difference in interest was seen between junior and senior residents regarding international medicine (p= 0.33). This aspect of systems-based practice accounted for the lowest interest seen by residents, with less than a third (30.5%) showing strong desire for integration of organization, funding, and logistics of international medical work into curricula (Table 2).
Law
No significant differences were found between junior and senior residents in interest in learning about handling and avoiding lawsuits (p=0.95), and two-thirds of respondents (67.5%) expressed strong interest in these topics being addressed in residency training (Table 2).
Discussion
Limited data exists on the implementation of systems-based practice education in orthopaedic residency. The aim of this study was to assess resident understanding and the level of interest that orthopaedic residents have in more formal integration of these topics into curricula. We divided respondees into junior and senior residents in order to compare comprehension and interest between residents earlier and further along in training. It was anticipated that senior residents (years 4-5) would have higher comprehension of systems-based practice.
The results of this study showed there is a large educational gap among orthopaedic residencies regarding this competency throughout the US. Across all institutions a significantly larger proportion (91.4% vs 8.6%; p=0.04) reported poor understanding of systems-based practice and no single institution showed decreased interest for this competency. With the exception of professional development and transition to practice (p = 0.04, p= 0.01), junior residents did not express higher interest than senior residents in incorporating systems-based practice topics into curricula, while reported understanding of systems-based practice competency was identical between junior and senior residents (p=1.00). Furthermore, residents have a resoundingly strong interest in integration of a systems-based practice curriculum according to the topics addressed by our survey. Domains of highest interest included personal finance (76.2%), value (82.1%), and practice models (80.1%)
Results of this study align with previous reports both in orthopaedics and in other specialties on the integration of systems-based practice into residency curricula. When general surgery program directors were asked about their residents’ training in areas of practice management and business, 87% agreed or strongly agreed that residents should be trained in these areas, yet 70% believed their current trainees were inadequately trained (Lusco, Martinez, and Polk 2005). Another study of 160 plastic surgery residents reported that 43.5% had virtually no business training in residency (Ovadia et al. 2018). A recent survey of orthopaedic residents and educators indicated systems-based practice is not consistently taught and is formally assessed even less. Of those that responded that it was being taught, 33% indicated it was only addressed through clinical observation (Roberts et al. 2012).
Family medicine and internal medicine have analyzed ways to successfully integrate formal education of this core competency into a residency curriculum. Family medicine residents at Wright State University demonstrated improvement in understanding, assessed through pre and post-test knowledge, after practice management lectures were integrated into their curriculum over a year (Crites and Schuster 2004). David and Reich developed a systems-based practice curriculum for internal medicine residents formatted as monthly workshops. They showed increased resident understanding and comfort level with topics evaluated through Likert scale questions and subjective assessment before and after integration (David and Reich 2005). In another study, Peters designed a course supported by web-based curriculum for internal medicine residents that increased residents’ competency of systems-based practice. Competency was assessed by a knowledge-based test administered before and after the course was administered (Peters et al. 2008). These other residency programs have illustrated that through a formal curriculum it is possible to improve resident knowledge and understanding of systems-based practice.
However, barriers remain in determining how to practically and effectively prepare residents for a transition to practice. Time and resources are major constraints that limit many institutions’ ability to address systems-based practice. Limited resident availability makes monthly workshops or weekly curriculum difficult to fit into residents’ already overloaded schedules (Lusco, Martinez, and Polk 2005). Additionally, few faculty are qualified to teach these topics at a given institution in the form of courses or workshops (Roberts et al. 2012; Lusco, Martinez, and Polk 2005). Another major obstacle lies in issues of alignment and incentive. While systems-based practice is a requirement for accreditation, the language used for accreditation is general, making it difficult to enforce. For example, statements such as, “Residents must demonstrate an awareness of and responsiveness to the larger context and system of health care”, and “considerations of value, cost awareness, delivery and payment” are notably vague. “Practice Management” questions were not included on the Orthopaedic In-Training Examination (OITE) until 2017, when they comprised 6 of 271 total questions (American Academy of Orthopaedic Surgeons 2017); in 2018, they comprised 6 of 269 questions (American Academy of Orthopaedic Surgeons 2018), and in 2019 they comprised 3 of 259 (American Academy ofOrthopaedic Surgeons 2019). Systems-based practice is unlikely to increase in priority for residency programs when only 1-2% of OITE questions relate to the topic. As healthcare evolves, the need for education in leadership, business, healthcare policy, and the economics of medicine will be increasingly vital.
This is the largest survey to date of orthopaedic residents at multiple institutions assessing comprehension and interest in further education of the systems-based practice competency, and the first to assess differences in comprehension and interest between junior and senior orthopaedic residents. Strengths of this study include the large cohort of respondents, with 151 orthopaedic residents who completed the entire survey, giving a fairly large sample size. Weaknesses of the study include those inherent to all survey studies including a low response rate of 55%; our study results may not be a true reflection of overall orthopaedic resident comprehension and interest of systems-based practice competency across the US. Additionally, given the cross-sectional nature of the survey, this study cannot infer that students are not learning systems-based practice throughout training, but can only report that students further along in training had no increased comprehension compared with students earlier in training at the time point we assessed. Finally, without demographic or detailed information about resident respondents’ individual exposure and experiences, multivariate analysis was not possible. Future research should measure interest and understanding of systems-based practice amongst orthopaedic residents longitudinally, evaluate factors leading to improved baseline understanding and investigate best practices for incorporating these competencies into curricula to prepare residents for the business of orthopaedics.
Conclusion
This survey indicated that orthopaedic residents are interested in the integration of systems-based practice into residency. Responses revealed institutions across the country have a gap in education regarding these topics, with senior residents showing no higher comprehension or lower interest compared with junior residents. We propose that systems-based competencies be formally integrated into residency curricula to better prepare residents for the transition to practice.
Funding
This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.