_by_orthopaedic_tr.tiff)
INTRODUCTION
Trauma is one of nine orthopaedic subspecialties available for fellowship training and according to the Orthopaedic Trauma Association (OTA), fellowship match rates have ranged from 60% to 86% between the years 2010 and 2021 (Sandhu et al. 2020; “Orthopaedic Trauma Fellowship Match Report,” n.d.). While there are many components of the orthopaedic trauma fellowship application, recent studies have consistently identified letters of recommendation (LoR) as one of the most important factors in both the selection of an applicant for an interview and for final placement in the rank order list for the orthopaedic fellowship match (Sandhu et al. 2020; Shaath et al. 2021; Baweja et al. 2017; Grabowski and Walker 2013).
From 2015 to 2021, the number of applicants for orthopaedic trauma fellowships has increased from 71 applicants to 123 applicants, however the number of offered positions have only increased from 78 to 92. This growth of the applicant pool has resulted in a drop in the match rate from 86% to 60%, making the role of supportive LoR even more important (Sandhu et al. 2020; “Orthopaedic Trauma Fellowship Match Report,” n.d.; Shaath et al. 2021; Ruddell, Eltorai, DePasse, et al. 2018). Supportive words from a trusted colleague can be more indicative of the quality of an applicant superseding any number of publications or standardized test scores. With greater than 90% of orthopaedic surgery graduates pursuing additional fellowship training after residency, the quality of LoR plays a critical role in determining the likelihood of an applicant matching into their desired subspecialty and program (Daniels and DiGiovanni 2014; American Medical Association, n.d.).
To date, there have been no investigations into what specific quality or performance metrics orthopaedic faculty use to deem applicants as ‘unqualified’ for fellowship training and how this subsequently affects these applicants’ LoR. The purpose of this study is to: (1) identify and rank quality/performance metrics that may characterize an applicant as “unqualified” for orthopaedic trauma fellowship training, (2) gauge how these quality/performance metrics affect orthopaedic trauma fellowship program director’s willingness to write an applicant a LoR, and (3) evaluate how orthopaedic trauma fellowship program directors respond to requests for LoRs from applicants who are characterized as “unqualified.”
METHODS
A complete list of orthopaedic trauma fellowship programs and fellowship director e-mail addresses were obtained from the OTA’s Fellowship Directory web page. A five-question, web-based, anonymous survey was developed to identify the factors orthopaedic trauma fellowship PDs prioritize before drafting LoR for prospective applicants. The survey was distributed to all 66 fellowship directors via email by the senior author (BC), using Qualtrics Experience Management platform (Seattle, Washington) (Figure 1).
The survey’s first question (multiple choice) aimed to identify the state in which each respondent’s fellowship program resided. The second question (free response) gauged how fellowship PDs would respond to an applicant requesting a LoR, who they characterize as unqualified for orthopaedic trauma fellowship training. Respondents were then asked (multiple choice) whether they would (1) not write the applicant a letter, (2) write the applicant an underwhelming letter, or (3) write the applicant a supportive letter. The qualitative, free-response items from the second question in the survey were organized according to each PDs’ response to question three. For PDs who would not write a LoR, free response items were categorized to determine whether the applicant would receive justification for the refusal, be redirected to another faculty member for a LoR, or be encouraged to pursue an alternative subspecialty. For PDs who would write an underwhelming LoR, free response items were categorized to determine whether applicants would be redirected to another faculty member for a letter, be encouraged to pursue an alternative subspecialty, or be made aware of the lapses in performance that would be reflected in the contents of the letter.
For the fourth question, PDs were presented with a list of 10 factors that may deem an applicant unqualified for orthopaedic trauma fellowship training. Respondents were asked to rank these factors in order of increasing importance (1 = the most important and 10 = the least important). The factors included, 1) lack of personal responsibility, 2) lack of professionalism, 3) poor clinical skills, 4) poor feedback from other faculty orthopaedic surgeons, 5) poor surgical skills, 6) poor clinical team performance, 7) poor patient rapport, 8) low orthopaedic in-Training Examination (OITE) scores, 9) lack of published research in specialty/subspecialty, and 10) low total number of publications. The final question (free response) provided PDs with the opportunity to submit any additional quality or performance metrics they consider when deeming an applicant “unqualified” for orthopaedic trauma fellowship training. To avoid bias, the order of all multiple choice and factors listed for ranking were randomized for each survey.
Statistical analysis was done using a combination of the Qualtrics Experience Management platform and Microsoft Excel (Microsoft; Redmond, WA). For the fourth question, the number of times a given variable was chosen as the “most important” (#1 ranking) was counted. Additionally, a weighted score for each factor was also generated in accordance with the methodology employed by Baweja et al. using the following scale: 5 points for each time a factor was ranked first, 4 points for each second-place rank, 3 points for each third-place rank, 2 points for each fourth-place rank, and 1 point for each fifth-place rank (Baweja et al. 2017). Cumulative scores were then aggregated for each factor and the factors were ranked by their weighted scores.
RESULTS
At the time of this study, 66 OTA-accredited orthopaedic trauma fellowship programs were present in the United States, and the PDs were identified. Of the 66 fellowship PDs surveyed, 42 (64%) responded, and all respondents answered each question. The distribution of responses encompassed every major region in the United States, with the most completed surveys coming from California (n = 6) and Ohio (n = 5). (Figure 2) When grouped based on geographic region, the west yielded the highest response rate (82%), then the southeast (64%), then the northeast (58%,), and last, the Midwest (52%) (Figure 3).
When faced with a request for a LoR from an “unqualified” applicant, 22 responding PDs suggested they would not write a LoR for the applicant (52%), 19 PDs indicated they would write an underwhelming letter (45%) and one PD stated they would write a supportive letter (2%) (Figure 4). Of the 22 PDs who would not write an LoR: 11 (50%) would clearly refuse to provide a LoR, seven (32%) disclosed that they would have a conversation with the applicant to justify their refusal, and four (18%) would attempt to redirect the applicant to another faculty member or support the pursuit of an alternative subspecialty (Figure 5a). For the 19 PDs who would write an underwhelming LoR: 11 (58%) would redirect applicants to another faculty member for a letter or encouraged to pursue an alternative subspecialty, four (21%) would communicate the lapses in performance that would be reflected in the contents of the LoR and four (21%) would agree to write a LoR (Figure 5b). A single PD communicated that they would write a supportive LoR that “would be honest; however as supportive as possible.”
Lack of personal responsibility was considered the most important factor (weighted score = 159) when characterizing an applicant as “unqualified” for orthopaedic trauma fellowship training. Lack of professionalism was considered the second-most important factor (weighted score, 155), followed by the poor clinical skills (weighted score, 89), poor feedback from other faculty orthopaedic surgeons (weighted score, 63), poor surgical skills (weighted score, 58), poor clinical team performance (weighted score, 54), poor patient rapport (weighted score, 36), low Orthopaedic In-Training Examination scores (weighted score, 11), lack of published research in specialty/subspecialty (weighted score, 5), and low total number of publications (weighted score, 0) (Figure 6).
Additionally, lack of professionalism (38.1%) and lack of personal responsibility (30.95%) were commonly ranked as the most important factor considered by PDs when characterizing an applicant as “unqualified” (Figure 7). Finally, when offered the opportunity to mention other important factors to consider, 11 PDs (26.2%) specifically named “questionable ethics/dishonesty” and three PDs (7.1%) cited whether the applicant had been placed on “probation or held back a year by the residency program.”
_by_orthopaedic_tr.tiff)
DISCUSSION
Our study identified lack of personal responsibility and professionalism as the most important factors valued by PDs regarding an applicant for an orthopaedic trauma fellowship. A previous survey of academic orthopaedists consistently associated these terms with qualities including integrity, trustworthiness, reliability, and accountability (Rowley et al. 2000). These qualities may be considered unteachable and thus unable to be further remedied during fellowship training, unlike an applicant’s poor clinical or surgical skills. The three lowest rated factors were those associated with low research output and low Orthopaedic In-Training Examination scores. These findings are consistent with other studies among orthopaedic subspecialities and emphasize a shift in importance towards factors associated with interpersonal abilities, rather than research output or academic accolades (Baweja et al. 2017; Benzon et al. 2015; Egro et al. 2017; Chun, Preciado, Brown, et al. 2014; Nies et al. 2014). Given these low ratings, it is our opinion that orthopaedic trauma fellowship PDs do not feel that standardized, multiple-choice examinations or publications are predictive of success within the field of orthopaedic trauma.
Mentoring plays a critical role in the career development of orthopaedic residents as they navigate from training to clinical practice. A key component of effective mentorships involves providing targeted, constructive feedback to challenge mentees and motivate them to improve (Mulcahey et al. 2018). In this study, 88% of responding PDs would in some capacity communicate their apprehensions to an “unqualified” applicant pursuing fellowship by refusing to write a LoR, redirecting the applicant to other orthopaedic faculty/subspecialties, or providing candid feedback directly to the candidate. Given these findings, we advise that residents seek feedback from trauma faculty and build upon constructive criticism early in residency to better address any perceived deficiencies prior to requesting a LoR.
LoRs provide fellowship PDs invaluable information about an applicant’s work ethic, dedication, integrity, and leadership potential, all of which are critically important for orthopaedic trauma surgeons. Thus, performance during residency training is of primary importance to fellowship directors and LoR from trusted orthopaedic trauma colleagues act as clear evidence of an applicants’ long-term track record. The importance given to high quality LoRs for applicant selection has been corroborated across multiple medical subspecialties and has been repeatedly identified among the top three criteria when ranking fellowship candidates (Sandhu et al. 2020; Baweja et al. 2017; Benzon et al. 2015; Egro et al. 2017; Chun, Preciado, Brown, et al. 2014; Nies et al. 2014; Meyer and Dewan 2010).
This study has several noteworthy limitations. First, the data obtained for this study was from a voluntary questionnaire with limitations that commensurate with other survey-based studies. Our findings were drawn from a small sample pool of 66 orthopaedic trauma fellowship directors and although a 64% response rate is sufficient, the absence of even a few responses leaves out a significant fraction of the target population. It is also important to note that due to the limitations in sample size, our survey is not powered to assess the significance of our results. Additionally, the survey only listed 10 factors to be ranked and was not comprehensive; however, PDs were provided the opportunity to write in additional factors not originally included.
CONCLUSION
Orthopaedic trauma fellowship PDs specifically identified lack of personal responsibility and professionalism as the most important factors when deeming an applicant ‘unqualified’ for fellowship training. Prior to requesting LoR, applicants should prioritize personal responsibility and make every effort to display professionalism when building relationships with orthopaedic trauma faculty.
Funding Statement
This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.