INTRODUCTION
Matching into orthopaedic surgery residency in the United States (US) has become increasingly difficult over the past several years (“National Resident Matching Program, Results and Data: 2018 Main Residency Match®” 2018). The average number of applications received per residency position increased from 54.1 in 2008 to 85.7 in 2018, but the number of US orthopaedic residency programs only slightly increased from 160 to 171 during the same period (Trikha et al. 2020). As a result, one in four applicants now go unmatched (Trikha et al. 2020). Applicants have responded by increasing the number of applications sent to programs from 46.5 in 2008 to 74.9 in 2018, further raising the level of competition (Trikha et al. 2020). Orthopaedic surgery applicants are increasingly scrutinized by program directors as they attempt to differentiate themselves among the competitive applicant pool.
Clerkship grades and evaluations are one such differentiator; however, these grades are inherently susceptible to preceptor bias and there is significant variability among medical school grading systems (i.e. pass-fail vs norm-referenced grading) (Takayama et al. 2006). Previous studies have identified areas of potential bias towards factors unrelated to student performance. Lee et al. found that older students were more likely to report worse grades in neurology and internal medicine and minority students were more likely to report worse grades across all clerkships (Lee et al. 2007). Additionally, previous studies have demonstrated inaccurate self-prediction of clerkship grades, suggesting that they may be somewhat ambiguous and subjective (Edwards et al. 2003; Weiss et al. 2005). While clinical grades are supposed to incorporate factors beyond aptitude in pre-clinical coursework, this criteria is rarely clearly defined, making performance extremely subjective. In fact, one study showed that the top and bottom 25% of students at one institution varied significantly in their standardized test scores, but did not vary in their clinical performances (Silver and Hodgson 1997). Recognizing the subjective nature of clerkship evaluations, medical students pursuing a career in orthopaedic surgery may choose not to disclose their career choice out of fear of bias and the influence this may have on their grade.
As the residency application process becomes more competitive, it is important to identify areas of potential bias towards medical students to improve clerkship performance evaluation. The purpose of this study was to identify factors that may lead to grading bias, including career choice disclosure, among medical students interested in a career in orthopaedic surgery. We hypothesized that medical students interested in orthopaedic surgery have a perception that they will receive lower clerkship grades if they choose to disclose a career interest in orthopaedics during that clerkship. Furthermore, students who chose to disclose an interest in orthopaedic surgery may receive worse grades compared to those that did not disclose their interest.
METHODS
Prior to study initiation institution review board approval was obtained at all participating institutions. An anonymous electronic survey was distributed to current 3rd and 4th year medical students as well as postgraduate year 1 (PGY1) and PGY2 orthopaedic residents at four different US medical schools in January 2021. Institutional review board approval was obtained at all participating institutions prior to initiation of the study. Surveys were distributed to students who demonstrated an interest in orthopaedic surgery as a career choice. Participants were identified through orthopaedic club or interest group involvement, orthopaedic surgery grand rounds, orthopaedic surgery clinical rotations including acting internships or sub-internships, and recently matched orthopaedic residents. The survey was distributed via email using the Qualtrics XM (Provo, UT) survey platform.
Medical students and junior residents were asked if they disclosed their personal interest in a career in orthopaedic surgery to preceptors during core clinical rotations including internal medicine, general surgery, psychiatry, neurology, obstetrics and gynecology (OB/Gyn), emergency medicine, family medicine, and pediatrics. Participants were asked to elaborate on why they chose or did not choose to disclose this information during the aforementioned clerkships. Those participants who did share their interest in orthopaedic surgery were asked their subjective opinion on the impact this information may have had on their final clerkship grade. Demographic variables including age, sex, race, ethnicity, and geographical location data were collected to identify other potential areas of bias. Survey participation was voluntary and anonymous. 157 responses were received. 8 responses were eliminated because the respondent did not have an interest in orthopaedic surgery, leaving 149 responses included in the final analysis.
Mean and frequency were calculated for each variable. Percentage of students achieving honors among each clerkship were compared using Fisher’s exact test across all independent variables. A p-value of <0.05 was considered statistically significant. Quotes were extracted from subject responses as to why they chose or did not choose to disclose to qualitatively highlight student impression of the grading process.
RESULTS
Demographics
Out of 80 respondents that chose to provide demographic information, the majority were 26–32 years old (55, 68.8%), identified as male (54, 67.5%), and were White/Caucasian (58, 72.5%). Most respondents were 3rd (63, 42%) and 4th (76, 52%) year students and went to school in the northeast (41, 51.9%) (Table 1).
The overall disclosure rate among clerkships was 60.1% (90/149). Females were more likely to disclose in surgery clerkships (24/25, 96.0%) compared to males (41/53, 77.3%) (p=0.039). This was also true for neurology (22/24, 91.6% female vs 34/51, 66.7% male) (P=0.02), OB/GYN (22/24, 91.6% female vs 35/50 male, 70.0%) (p=0.038), psychiatry (22/25, 88.0% female vs 34/51, 66.7% male) (P=0.047), and emergency medicine (22/23, 95.7% female vs 32/42, 76.2% male) (P=0.045).
There was variability in response rate due to respondents completing clerkships at different times in their training. The highest disclosure rate was during the surgery clerkship (87/143, 64.4%) and the lowest was in psychiatry (77/132, 58.3%). Of those that disclosed, the most respondents felt it negatively impacted their grade in emergency medicine (4/34, 11.8%) and the least felt it negatively impacted their grade in neurology (2/46, 4.3%) (Table 2).
The most respondents achieved honors in family medicine (60.3% honors, 23.8% high-pass, and 15.1% pass) and the least achieved honors in emergency medicine (28.8% honors, 23.7% high-pass, and 38.8% pass). No respondent failed a clerkship (Table 3).
Predictors of Grade
Internal medicine: Respondents that disclosed an intention to apply to orthopaedic surgery received a better grade than those that did not. Of those students and residents that disclosed a career interest in orthopaedic surgery, 63.3% (38/60) achieved honors, whereas 37.0% (17/46) of those that did not disclose achieved honors (p=0.011). No other variable was associated with the final grade in internal medicine.
Surgery: There was a significant association between sex and grade. 76.0% (38/50) of men achieved a grade of honors, whereas 45.8% (11/24) of women achieved a grade of honors (p=0.017). Race also had an impact on grade. 70.6% (48/68) of those identifying as White/Caucasian, Black/African American, Asian/Pacific Islander, Indian/Native American, and Hispanic achieved honors, whereas 16.7% (1/6) of those identifying as a race other than one of these achieved honors (p=0.015). The geographic location of the respondent’s medica school also had an effect on grade. 77.5% (31/40) of those in the northeast achieved honors compared to 52.9% (18/34) in the rest of the country (p=0.030).
OB/Gyn: Respondents that disclosed their intention to apply to orthopaedic surgery received a better grade than those that did not. Of those that disclosed a career interest in orthopaedic surgery, 48.3% (28/58) achieved honors, whereas 25.6% (11/43) of those that did not disclose achieved honors (p=0.024). No other variable was associated with the final grade in OB/Gyn.
Pediatrics: Race had an impact on grade. 73.8% (45/61) of those identifying as White/Caucasian, Black/African American, Asian/Pacific Islander, Indian/Native American, or Hispanic achieved honors, whereas 20.0% (1/5) of those identifying as a race other than one of these races achieved honors (p=0.027).
No variables were associated with clerkship performance in neurology, family medicine, psychiatry, or emergency medicine.
Respondents provided explanations for their interest in orthopaedic surgery during clerkships. Select responses are available for review in Table 4.
DISCUSSION
This study found that, from a diverse cohort of medical students interested in orthopaedic surgery, approximately half of the students chose to disclose their interest in orthopaedics during core clerkships, with 5–11% of students feeling that if they did disclose it had an impact on their clerkship grade. Contrary to this belief among some students, it appears that those that disclosed an interest in orthopaedic surgery received a higher grade than those that did not. Furthermore, most students were able to achieve honors in all clerkships, with the exception of OB/Gyn and emergency medicine, in which approximately a third of students achieved honors. We also found that sex and race may impact a student’s final grade in some core clerkships.
The association between willingness to disclose an interest in orthopaedic surgery and higher clerkship performance in internal medicine and OB/Gyn is unclear. It is possible that students that chose to disclose their interest in orthopaedics felt more confident doing so because that had received high grades previously. Confidence may play a role in performance. Lee et al. surveyed 2395 medical students across the US to determine demographic variables, perceived quality of clerkship experience, assertiveness, and grade received (Lee et al. 2007). They found an association between “assertiveness” and high clerkship performance, except for in internal medicine. Students that had not received high grades in clerkships, or were less confident with their clinical skillset, may have chosen to avoid other potential sources of bias that could impact their grade and thus chose not to disclose.
As orthopaedics becomes more competitive, achieving higher clerkship grades becomes more important to differentiate oneself from other applicants (“National Resident Matching Program, Results and Data: 2018 Main Residency Match®” 2018; Trikha et al. 2020). The majority of students interested in orthopaedic surgery in this sample were able to achieve honors in most clerkships, with the exception of OB/Gyn and emergency medicine. Reasons for this variable performance are unclear. While there are areas of overlap, the skillset required for performance in OB/Gyn and emergency medicine are largely different from that of orthopaedics. This does not explain why students in orthopaedics were able to achieve higher grades in other seemingly unrelated specialties such as psychiatry and pediatrics. It is possible that certain clerkships may have more rigorous criteria for achieving honors, which can vary by institution and department. We did not survey students that were not pursuing orthopaedics, so determining an average grade in a specific clerkship among all students was not possible.
We found an association between female sex as well as race defined as “other” and worse clerkship performance in surgery. This could be due to bias towards females and racial minorities. Female sex representation in surgical fields has historically been lacking (Pories et al. 2019). However, recent efforts to increase female representation has led to an increase in the proportion of females in surgical fields, including within orthopaedics (Day, Lage, and Ahn 2010). Poon et al. analyzed data from the American Association of Medical Colleges (AAMC) and found that the percentage of female orthopaedic surgery residents increased from 10.9% in 2006 to 14.4% in 2015 (Poon et al. 2019). Improved representation of women in orthopaedic surgery may lead to decrease bias against female residents and faculty. Lee et al. surveyed 2,395 medical students and found that underrepresented minorities were more likely to report worse grades across all clerkships and those identifying as Asian/Pacific Islander were more likely to report worse grades in OB/GYN, pediatrics, and neurology (Lee et al. 2007). Identifying areas of bias towards factors unrelated to clerkship performance may allow for more objective evaluations in the future.
We found that 5–11% of students believed disclosing their interest in orthopaedic surgery impacted their grade on certain clerkships; however, we did not find this to be the case. Other studies have found students to be poor predictors of their own clerkship grades (Weiss et al. 2005; Silver and Hodgson 1997). One possible explanation for this is students may believe that factors outside of clerkship performance, such as career choice disclosure, will impact their grade. This can lead a student to falsely predict clerkship grade. Open and objective conversations are important in such scenarios where students feel bias towards them to assist in understanding of their grades.
There are several limitations to this study. First, variability in performance can be affected by the number of clerkships completed. As students progress through clerkships, they may learn how to perform better, which may improve their grade. This was not accounted for in this study, but we also did not observe this trend. For example, emergency medicine is often completed later in medical school, but students performed worse in emergency medicine. Conversely, students may also put forth less effort on rotations that are completed in fourth year compared to their third year of medical school because they have less bearing on residency match. Second, responses are subject to recall bias and thus may not accurately reflect clerkship performance. For example, residents that already matched into orthopaedics may have less negative feelings towards receiving a lower grade than students that have not yet matched. Third, students interested in orthopaedics are highly motivated and thus represent an inherently biased sample. Willingness to participate in a survey study captures highly motivated respondents even among those interested in orthopaedic surgery. This can shift our sample to reflect higher performing students. Fourth, other confounding variables and differences among institutions in the grading process may have altered results. Schools may weigh standardized tests higher than clerkship evaluations, which are seemingly unaffected by bias. We did find that most institutions used an honors, pass, fail system which helped standardize results. There are also different cultures among institutions regarding the student clerkship experience, which can shift grading curves. Our large sample size can help mitigate some of these effects. We also included multiple institutions in different geographic locations throughout the United States to offset this effect. Even so, results may not be generalizable to all medical schools and higher- powered studies are needed to more accurately predict a national trend.
CONCLUSION
Disclosure of an interest in orthopaedic surgery as a career during third year clerkships is relatively low. Some students believe disclosing an interest in orthopaedics may negatively impact their grade, but the opposite may be true. Other potential areas of bias that may impact grades include sex and race. Students should be aware of these findings, as it can decrease stress regarding being honest about career intentions. Conversely, clerkship preceptors can use this data to improve the grading process for students. More research is needed to improve the clerkship evaluation process given the increasingly competitive nature of the orthopaedic surgery residency application process.
Acknowledgements
None
Funding
This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.