Introduction
It is necessary for surgeons, even new surgeons, to be able to dictate detailed operative notes. Comprehensive notes are critical for accurately describing a patient’s diagnosis, treatment, and future care, as well as being necessary for both the medicolegal aspect of patient care and billing. In a 2004 study, Flynn and Allen evaluated the role of operative notes as billing documents in a multispecialty surgical clinic (Flynn and Allen 2004). The authors found that the most important document for justifying reimbursement of surgical services was the operative note, and that surgical residents should be instructed in the details that need to be included in an operative report for generating proper Current Procedural Terminology (CPT) codes for billing. Smyth et al found that, “inadequate resident operative reports have been shown to result in incorrect coding in up to 28% cases” and that, “these deficiencies were shown to have the potential to reduce reimbursement by an average of $18,200” (Smyth et al. 2020).
A 2004 survey of Obstetrics and Gynecology (OBGYN) residency Program Directors (PDs) found that only 23% of residency programs provide formal teaching regarding operative dictation (OD) training and education, while 83% of PDs thought it was an important skill to teach (Eichholz et al. 2004). In the same study, OBGYN residents at the University of Iowa were provided with a 30-minute formal teaching session regarding OD. Their notes were scored before and after the training using a 0–20-point scoring system developed by the authors. The mean score received by all residents increased from 9.06 to 18.56 following the formal teaching session (P<.001), suggesting that formal teaching of OD could significantly improve the quality of the residents’ operative note dictation (Eichholz et al. 2004).
Despite being of great relevance to medical education and practice, there is a paucity of literature assessing the teaching of OD in surgical residency programs or the quality of residents’ OD. The purpose of this systematic review was to analyze the existing literature to determine if and how surgical residents are taught to dictate operative notes. The secondary aim was to compile perceptions regarding the teaching and quality of resident OD and ways in which it might be improved.
Methods
A systematic review was performed following PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. Electronic databases PubMed, CINAHL, Web of Science, and EMBASE were searched with the following terms: dictating operative notes, dictating operative notes AND orthopaedic surgery, operative notes AND dictation, teaching operative dictation, operative notes AND surgical residency, operative notes AND residency training, operative notes AND fellowship, operative notes AND general surgery, operative notes AND plastic surgery, operative notes AND vascular surgery, operative notes AND thoracic surgery, operative notes AND ENT surgery. The searches were performed in January and February of 2023.
Studies were included if they met the following criteria: discussed how to teach the skill of dictating operative notes, were published between 2010 to 2023, published in the English language, were performed in the United States, and were published in peer-reviewed journals. Publication types that were not peer-reviewed were excluded, such as: conference abstracts, dissertations, study protocols, and commentaries. The references from the selected articles were manually searched for further studies related to the effective teaching of operative note dictation by surgical residents.
Results
The search strategy yielded 782 studies, 9 of which met the inclusion criteria and were included in the systematic review. The 9 included studies were subdivided into 3 categories: 1) perceptions of resident OD and/or its teaching, 2) quality of resident OD, 3) interventions for improving OD. Only 2 (22%) studies evaluated perceptions, 5 (56%) assessed OD quality, and 3 (33%) included an intervention. One study examined both perceptions of resident OD and teaching and also employed an intervention; it was thus included in both categories.
Perceptions of current resident operative dictation and/or operative dictation teaching
Two studies focused on the perception of resident OD and/or teaching of OD. The first was a 2013 survey of PDs from 10 Accreditation Council for Graduate Medical Education (ACGME) accredited surgical specialties (colon and rectal surgery, neurological surgery, obstetrics/ gynecology, ophthalmology, orthopaedic surgery, otolaryngology, plastic surgery, surgery, thoracic surgery, and urology) (Melton et al., n.d.). The survey was distributed to 1,096 surgical residency programs and responses were received from 441 PDs (40% response rate). Of the 441 respondents, 295 (67%) reported that they considered the teaching of OD to be a priority, yet only 76 (17%) reported providing OD instruction (Melton et al., n.d.).
In 2019, Smyth et al surveyed orthopaedic surgery residency PDs and also assessed resident dictation before and after an educational intervention (Smyth et al. 2020). The survey was distributed to 154 orthopaedic surgery residency PDs of which 54 responded (35% response rate). Twenty-seven of 54 (50%) respondents believed that poor OD could impact communication between team members and lead to problems with postoperative care, yet 30 of 54 (56%) respondents did not consider OD training one of their program’s priorities. Forty-five of 54 (83%) respondents indicated that their program provides no formal training to residents on dictating operative reports and 35 of 53 (65%) reported that they do not intend to change their current operative report education (Smyth et al. 2020).
Assessment of the quality of resident operative report dictation
Five studies included in this systematic review assessed the quality of resident OD. Key takeaways from each article are summarized in Table 1. Gur et al compared the quality and degree of completeness of traditional dictated operative reports with that of computerized, synoptic operative reports (Gur, Gur, and Recabaren 2012). Synoptic reports make use of discrete data fields with specific formatting for each field, which standardizes the collection, storage, and retrieval of data (Smyth et al. 2020). Dictated reports were completed by attending surgeons and synoptic reports were completed by residents for the same procedure. Synoptic reports were found to be significantly more complete than dictated reports with regard to certain categories of information, such as demographic, preoperative, and procedural information. The overall completeness of synoptic reports (94.7%) was also found to be greater than dictated reports (66%) (P = .001). When comparing the synoptic reports of junior vs. senior residents, no statistically significant differences were reported (Gur, Gur, and Recabaren 2012).
The remaining 4 studies assessed the quality and degree of completeness of operative reports by comparing notes dictated by residents of varying post-graduate year (PGY), and in some cases comparing resident notes to attending physician notes. Zwintscher et al found only 2 out of 999 (0.2%) resident dictated operative reports included all the information deemed important, which were categorized into variables that were patient-centric, physician-centric, and Joint-Commission/Medicare-required documentation (Zwintscher, Johnson, Martin, et al. 2012). The authors found that dictated reports from PGY-1 and PGY-2 resident years were significantly less complete compared to upper-level residents and staff, but that no statistical difference in completeness existed between PGY-1 and PGY-2 or between PGY-3, PGY-4, and PGY-5 (Zwintscher, Johnson, Martin, et al. 2012).
Wheeler et al performed a chart review of patients who had undergone endoscopic sinus surgery and compared the narrative dictations of these procedures by group, where groups were either attending physicians, senior residents (PGY-3 and above), or junior residents (PGY-1 and 2) (Wheeler et al. 2019). Quality indicators (QIs) for narrative reports were developed and inclusion percentages of QIs for each physician level were measured. Attendings had the highest rate of inclusion of QIs (89%) followed by junior residents (87%), then senior residents (80%). Notably, at the study site, junior residents followed an informal template when learning how to dictate reports. Efficiency of dictation was also measured as QI inclusion per total word count and was highest for attendings followed by senior residents, then junior residents (Wheeler et al. 2019).
Similarly, Perez et al assessed the quality of obstetrics/gynecology (OBGYN) resident dictation by comparing resident dictations to a standardized audit form of the procedure completed by attending physicians for the same procedure (Perez, Babbar, and Vricella 2021). Of 100 operative reports reviewed, 72 (72%) contained either major or minor documentation errors. A major error was defined as, “one with potential direct impact on perioperative care or future obstetric or gynecologic medical decision-making” while a minor error was defined as, “one without direct anticipated impact on perioperative care or medical decision-making.” PGY-level was associated with the presence of major operative note error, with PGY-1 residents having the greatest number of major errors, followed by PGY-2 then PGY-3-4 residents grouped together. Minor operative note errors, however, had the inverse relationship, with PGY-3-4 residents having the greatest number of minor errors followed by PGY-2, then PGY-1 residents (P = 0.50; not statistically significant) (Perez, Babbar, and Vricella 2021).
Finally, Porterfield J.R. Jr. et al conducted a retrospective chart review for patients who underwent incisional hernia repair and compared the completeness of these operative notes by PGY-level (Porterfield et al. 2011). A “minimally acceptable” dictation was defined as, “a report, including the hernia size, hernia location, number of defects, and repair type.” Notes missing any of these data points were categorized as “not acceptable.” In the study, PGY-1 residents had the highest rate of acceptable operative reports (n=58, 42%) among all groups; as compared to PGY-1s, the PGY-3s, PGY-4s, and PGY-5s were significantly more likely to dictate an “unacceptable operative” report. PGY-1s also had a higher rate of “acceptable” dictations than attending physicians, but this did not reach statistical significance (Porterfield et al. 2011).
Interventions directed at improving the quality of resident OD
Three studies utilized educational interventions to determine if formal education surrounding resident OD improved the overall quality (Table 2). Smyth et al implemented an education session conducted by a faculty member for orthopaedic surgery residents (Smyth et al. 2020). Using an author-developed operative report assessment tool (ORAT), 100 operative reports before the education session and an additional 100 reports after the education session were evaluated. The mean overall pre-education assessment score was 24.2 +/- 3.5, while the mean overall score of the post-education operative reports was 31.8 +/- 2.1, a 22% increase on the 35-point ORAT scale (p < 0.01) (Smyth et al. 2020).
Zoghbi et al measured the time it took for general surgery interns to complete 7 electronic medical record (EMR) tasks before and after implementing their intervention, which was watching “how-to” videos for each task (Zoghbi, Caskey, Dumon, et al. 2018). The intervention resulted in a statistically significant decrease in the time spent performing all 7 EMR tasks. Two EMR tasks were relevant to our review: (1) time spent writing a day-of-surgery H&P update note decreased from 54 sec ± 34 to 34 sec ± 17 (p <0.01), and (2) time spent writing a brief operative note decreased from 52 sec ± 27 to 33 sec ± 11 (p<0.01). According to intern survey responses, the intervention also resulted in increased confidence among interns in performing 5 of the 7 EMR tasks, including an increased confidence in writing a brief operative note from 3.7 ± 1.2 to 4.6 ± 0.7 (P = 0.05). Notably, there was no significant change in confidence for writing a day-of-surgery H&P update note (Zoghbi, Caskey, Dumon, et al. 2018).
Finally, Hyde et al implemented a lecture for residents provided by a faculty expert highlighting the role of dictation in surgical practice and the pertinent aspects of dictation (Hyde, Biderman, and Nelson 2018). Each resident also received a pocket-sized card that could be used as a template to aid in dictation. Two hundred eighty charts from before the intervention were compared to an additional 140 charts created after the educational intervention. Dictations were scored out of 19 points, with either 0 or 1 assigned to 19 variables according to their absence or presence, respectively. In other words, a whole number score was assigned to each report that equaled the number of criteria it included. The overall resident pre-intervention score was 16.28; the post-intervention score was 17.37 (P = 0.001). The scores of junior residents improved more after the intervention than did the scores of senior residents (Hyde, Biderman, and Nelson 2018).
Discussion
The purpose of this review was to analyze the existing literature to determine if and how surgical residents are taught to dictate operative notes. The secondary aim was to compile perceptions regarding the teaching and quality of resident operative dictation and ways in which it might be improved. In one survey of surgical residency PDs, 44% (68 of 154 respondents) considered teaching OD a priority. In a separate survey, 67% (295 of 441 respondents) of PDs considered teaching OD a priority despite only 17% (76 of 441 respondents) offering formal education (Melton et al., n.d.; Smyth et al. 2020). OB/GYN programs were more likely to offer OD training (OB/GYN 35% vs. surgery 18%, neurosurgery 16%, ophthalmology 14%, orthopaedic surgery 14%; p < 0.05 each) (Melton et al., n.d.). Computerized, templated synoptic reports were more complete than traditional dictated reports (94.7% vs. 66%; P = .001) (Gur, Gur, and Recabaren 2012). Completeness of resident OD did not always correlate directly with level of training (PGY-level) (Perez, Babbar, and Vricella 2021; Porterfield et al. 2011; Wheeler et al. 2019; Zwintscher, Johnson, Martin, et al. 2012). Educational interventions such as lectures and video-based instruction improved completeness of OD by up to 22% (Hyde, Biderman, and Nelson 2018; Smyth et al. 2020; Zoghbi, Caskey, Dumon, et al. 2018).
In 2013, Melton et al surveyed PDs in all surgical specialties. Of survey respondents, 67% (n = 295) viewed formal teaching of OD as a priority (Melton et al., n.d.). Yet in a 2020 study of only orthopaedic surgery residency programs, that number dropped to 44% (n = 68) (Smyth et al. 2020). It is likely that orthopaedic surgery PDs tend to find OD education less important than PDs of other surgical specialties. This is supported by statistics from the 2013 study that noted that OBGYN programs were more likely than any other surgical specialty to offer formal OD teaching (OBGYN 35% vs orthopaedics 14%) (Melton et al., n.d.). It is also possible, however, that formal OD teaching has simply become less important to orthopaedic surgery PDs during the time between the two surveys (i.e., between 2013 and 2020).
Regarding quality of resident OD, Gur et al suggested that implementation of synoptic reporting may improve the completeness of not only resident operative notes but also notes generated by attending physicians (Gur, Gur, and Recabaren 2012). That said, synoptic reporting is a controversial topic. Although becoming slightly more common with the increased use of EMRs (Gur, Gur, and Recabaren 2012; Smyth et al. 2020), in a national survey of surgical residency PDs, most respondents had negative views regarding synoptic reporting as a teaching modality for OD (Melton et al., n.d.). Seventy (16%) PDs provided comments about synoptic reporting, of which 49 (70%) were negative. Comments included a perception that synoptic reports were of lesser educational value, might not capture important information, might fail in less formulated circumstances, and might result in poor quality documentation. In the same survey, positive aspects of synoptic reporting noted by PDs included improved operative note quality and completeness, time savings, potential benefit with resident education, and usefulness for documenting common/simple procedures (Melton et al., n.d.). Given these mixed opinions, we think it is unlikely that synoptic reporting will replace traditional dictation as the primary mode for generating operative notes in U.S. surgical residency programs.
Another area lacking consensus is the relationship between quality of resident OD and increasing PGY-level. We hypothesized that the quality of resident OD would improve as residents became more experienced and progressed further in their education. This hypothesis was supported by findings from Zwintscher et al, which found that dictated reports from PGY-1 & PGY-2 were significantly less complete compared to upper-level residents and staff (Zwintscher, Johnson, Martin, et al. 2012). It was also supported by the portion of the Wheeler et al study that evaluated efficiency of the OD, where attending physicians were able to include more QIs while using fewer total overall words than were senior residents; junior residents had the least efficient dictations (Wheeler et al. 2019). Further, PGY-level was directly correlated with the presence of major operative note error in an OBGYN residency program, with PGY-1 having the greatest number of major errors, followed by PGY-2 then PGY-3-4 grouped together (Perez, Babbar, and Vricella 2021). This data would suggest that, in the absence of formal teaching of OD to residents, junior residents create less complete, lower quality OD than do senior residents and attendings.
Despite the above information, there was also data that contradicted our hypothesis. Minor operative note errors, for example, had an inverse relationship with OBGYN PGY-level (i.e., PGY-3-4 having the greatest number of minor errors followed by PGY-2, then PGY-1 level) (Perez, Babbar, and Vricella 2021). A minor error was defined as “one without direct anticipated impact on perioperative care or medical decision-making.” Additionally, although efficiency of dictation followed the pattern we expected, Wheeler et al also found that junior residents included more QIs (87%) than their senior resident counterparts (80%) (Wheeler et al. 2019). Perhaps most strikingly, Porterfield J.R. Jr. et al found that PGY-1s had the highest rate of “acceptable” operative reports (42%) for incisional hernia repairs compared to both senior residents and attendings (Porterfield et al. 2011). These findings suggest that informal teaching of OD is not sufficient to generate high quality operative reports for physicians in any level of training, and that formal education may benefit physicians at all levels.
PDs of surgical residency programs perceive several barriers to formal education of OD including factors related to billing, reimbursement, and perceived quality issues with resident dictations; time considerations; perceptions that OD teaching is not necessary; and a lack of established teaching pedagogy for OD teaching (Melton et al., n.d.). If formal OD teaching were to be implemented, PDs described formal lectures, OD teaching aids, and one-on-one teaching sessions as possible OD instruction-modality categories.
At least three interventions for formal OD teaching have been implemented between 2010-2023. Smyth et al and Hyde et al implemented formal lectures/education sessions while Zoghbi et al implemented of a series of “how-to” videos that could be watched on residents’ mobile phones to aid them in completing various EMR tasks (Hyde, Biderman, and Nelson 2018; Smyth et al. 2020; Zoghbi, Caskey, Dumon, et al. 2018). All three interventions were shown to improve the quality of resident OD and were reportedly not overly burdensome to implement. These results contradict perceptions that formal OD teaching would be too cumbersome, costly, or time consuming to make a regular part of the surgical training curriculum for residents. Given that inadequate resident operative reports have been shown to result in incorrect coding in up to 28% cases, leading to potential reimbursement losses of an average of $18,2002, implementation of low-cost formal OD training for residents is likely to generate cost-savings for teaching hospitals in the long-run.
This study had several limitations. First, there is a scarcity of literature looking at the topic of resident OD. Given a similar 2014 systematic review by Dumitra et al looking at this topic, we initially sought to limit our search to the years 2014-2023 (Dumitra et al. 2015). However, due to limited availability of relevant, peer-reviewed texts, we chose to expand our search further back in time to 2010-2023. Given the small “n” of studies, it is difficult to extrapolate findings across all U.S. surgical residency programs. Second, though we worked to create comprehensive search terms in various databases, it is entirely possible that relevant studies may have been missed, which would make this review’s conclusions incomplete. Heterogeneity of the selected studies in terms of populations, methodologies, and outcomes also made it challenging to pool data and draw meaningful conclusions. Outside of methodology, it should be mentioned that some residency programs do not allow residents to dictate operative reports and that the ACGME does not recognize the operative report dictation as a portion of resident evaluation.
Based off the studies reviewed, we recommend that orthopaedic surgery residency programs implement interventions for teaching residents to dictate operative notes. Specifically, we suggest a brief faculty-taught lesson focusing on teaching residents to categorize what information is salient and necessary for inclusion vs what is extraneous. These categories should be addressed from both a patient care and a billing perspective. A quick reference guide may also be provided to residents, and resident operative dictation should periodically be assessed for accuracy and completeness. Video training sessions have also proven helpful, but we feel that generation, recording, and dissemination of video content requires more front-end effort for residency programs and thus is less likely to be implemented.
With increased adoption of the EMR, synoptic reporting, and smart phrase tools, we think residents will have received less exposure to dictation of notes in any field, including surgical fields, thus compounding the importance of formal training in residency programs that continue to utilize operative dictation, such as orthopaedic surgery.
Conclusion
Despite the crucial role of OD in patient care, medicolegal aspects, and billing, this review underscores a lack of routine and formal instruction of this skill in U.S. surgical residency programs. Limited studies suggest potential improvement through formal educational interventions. Future research should delve into the costs and barriers hindering the integration of formal dictation training, shedding light on why PDs have not universally adopted such initiatives.