Background
When the coronavirus disease 2019 (COVID-19) pandemic ravaged the globe in 2020, clinics and hospitals were quickly overwhelmed by critically ill, infectious patients and medical education came to a screeching halt in view of concerns for student safety (Research Education INnovation in Surgery (REINS) initiative COVID-19 group et al. 2021; TMS Collaborative et al. 2021). While medical schools quickly pivoted to distance learning and other remote options in efforts to provide quality education for their students, the unfortunate and significant detrimental effect on surgical and procedural learning was unmistakable (DEDEILIA et al. 2020; Foong et al. 2020). For medical students in Singapore, pandemic restrictions translated to halting, shortening, or imposing significant restrictions on hospital rotations and hands-on practice for procedural skills or observing surgeries (DEDEILIA et al. 2020; Liang, Ooi, and Wang 2020).
As a direct consequence of pandemic restrictions, virtual reality and surgical simulators received unprecedented attention as an attractive alternative for providing interactive, hands-on training for surgical education (DEDEILIA et al. 2020; Ehrlich, McKenney, and Elkbuli 2020; Sommer et al. 2021). One such simulator is Touch Surgery, a mobile surgical simulation application available on iOS and Android for free (Nehme, Bahsoun, and Chow 2016; “Touch Surgery,” n.d.). Endorsed by the Royal College of Surgeons of Edinburgh (“RCSEd Endorses Touch Surgery App | RCSEd,” n.d.) and created through the collaboration of experts from prominent academic medical centers such as Stanford, Duke, Harvard, Johns Hopkins University, and Imperial College London (Uzoho, Doganay, and Watson 2015), the Touch Surgery application utilizes cognitive task analysis (CTA) in conjunction with anatomically accurate animations to provide an interactive, immersive virtual reality cognitive simulation and rehearsal tool (Paro, Luan, and Lee 2017; Sugand, Mawkin, and Gupte 2015; Wingfield et al. 2014). The platform has since been acquired by Medtronic’s Surgical Robotics business in 2020 (“Medtech Conglomerate Medtronic Purchases Digital Surgery in Bid to Expand Their Surgical Robotics Arm” 2020).
For medical students barred from entering operating theatres or emergency departments, the app provides an engaging means of learning from world class experts about surgical decision making and rehearsing the steps to performing surgery in a comfortable, low-risk environment without fear of harming real patients. Being able to pause and replay modules, test their own knowledge, and access modules from anywhere with a mobile device further enables learners to learn comfortably at their own pace.
With the rise of virtual reality and surgical simulators, the opportunity arises for surgical education to move from the traditional opportunistic, random learning model to a structured, targeted curriculum with deliberate practice. Especially with growing concerns over training hour restrictions and patient safety issues, the pressure is on for surgical education to equip trainees with both the skills and surgical decision making to produce safe, effective surgeons at the end of their training (Ahmed et al. 2014). Touch Surgery and other surgical simulators may be the solution to addressing this gap in surgical education. In fact, previous studies have demonstrated that specific Touch Surgery modules in intramedullary nailing and carpal tunnel releases have construct, face, and content validation and in certain instances, are superior to traditional learning methods in instructing novices on how to perform procedures (Paro, Luan, and Lee 2017; Sugand, Mawkin, and Gupte 2015; Wingfield et al. 2014). We sought to identify if the knee arthroscopy module in Touch Surgery could be used as an educational tool for novices in both theoretical and technical knowledge acquisition.
The primary objective of this study was to identify the training effect of Touch Surgery’s knee arthroscopy module in novices using validated objective multiple choice question testing administered before the first and after three consecutive attempts. Secondary objectives consisted of qualitative feedback on CTA simulators for surgical education.
Materials & Methods
Touch Surgery application
The Touch Surgery application is a clinically validated, immersive, and interactive virtual surgical cognitive-task simulator that is available for free on mobile smart devices such as iOS and Android. Simulations are authored by qualified surgeons across the world, further developed at Touch Surgery Labs, and feature anatomically accurate illustrations or real patient videos. Simulations on the app feature step-by-step tutorials to complete common surgical procedures across a wide range of surgical disciplines, including general surgery, obstetrics & gynaecology, orthopaedics, plastic surgery, otorhinolaryngology, ophthalmology, and others. Test modules require users to complete manual steps (e.g. positioning of the patient, insertion of a scope, examining the lateral gutter). Surgical decision-making is assessed with single best answer questions out of four options at critical steps of the procedure. Participants were given the Touch Surgery app preloaded on a mini-iPad with the knee arthroscopy module already installed.
The knee arthroscopy simulation (Figure 1) was further divided into the following sections:
(i) patient preparation
(ii) arthroscope insertion
(iii) assessment of knee joint
(iv) arthroscopic probe insertion
(v) inspection of medial knee joint
(vi) inspection of anterior cruciate ligament
(vii) inspection of lateral knee joint
(viii) closure
Study participants
Study approval was obtained from our academic medical center’s Institutional Review Board. All graduate medical students enrolled in the school’s integrated Musculoskeletal medicine clerkship - consisting of rotations through orthopaedic, hand, and plastic surgery, as well as rheumatology - were eligible to enroll in the study from 2019-2020. Prior to the start of the clerkship, an email detailing the study and requesting for volunteers were sent to students. Enrolling in the study was on a purely voluntary basis and participation in the study had no impact on clerkship grade. Students were also given the option of withdrawing from the study at any time. Inclusion criteria included graduate medical students, over the age of 18, and naive to use of both the Touch Surgery application and the knee arthroscopy procedure. Exclusion criteria included having previously used the Touch Surgery application or observing and/or assisting in a knee arthroscopy prior to study participation.
Data collection
Participants who enrolled in the study were given a questionnaire consisting of a multiple-choice questions (MCQ) test assessing both surgical theory and surgical technique on knee arthroscopy and a qualitative survey on attitudes towards use of mobile medical apps in surgical education. The MCQ test assessed the primary learning objectives of the knee arthroscopy simulation. No MCQs were repeated from the MCQ given within the app. The MCQ test was validated by five orthopaedic surgeons selected as the expert cohort and who could perform knee arthroscopy independently. The survey consisted of questions presented either on a 7-point Likert scale or a multiple-choice selection and focused on assessing existing behaviors regarding surgical education, mobile medical apps, and their perception on the convenience and efficacy of the mobile medical apps in surgical education. After completing at least three attempts on the knee arthroscopy simulation, the participants were asked to answer the questionnaire again. See Supplementary Material 1 for MCQ test questionnaire.
Statistical analysis
The Statistical Package for the Social Sciences (SPSS)® version 21 (IBM Corp., Armonk, New York) was used for the analyses performed in this study. Descriptive statistics including mean, mode, median, and standard deviations were calculated for MCQ answers before and after simulator use. A 2-tailed paired student t-test was used to assess for statistically significant changes between pre- and post-study MCQ scores. A result was considered statistically significant when a two-tailed p-value < 0.05.
Results
Demographics
A total of 13 participants were initially recruited to the study without any dropouts. The mean age of the student group was 27.8 years of age with 69% females. All students had at least a bachelor’s degree, all of which were in the science, technology, engineering, and mathematics (STEM) fields. The mean completed years of medical school studies was 3 years.
Baseline Studying Behaviors
Table 1 shows the most common materials and methods employed by students for knowledge acquisition during medical school. Table 2 shows the most common usages for mobile applications during medical school. Of note, mobile applications were not as popular as classical knowledge acquisition modalities such as lectures and videos and none of the students reported using mobile applications as simulators.
Knowledge Acquisition
Median simulation performance scores demonstrated significant improvements with each attempt across all participants within the study (p < 0.05). Mean scores for pre- and post-simulator use demonstrated statistically significant improvements for technical and overall knowledge acquisition (p < 0.05). While mean scores for pre- and post-simulator use demonstrated improvements for theoretical knowledge acquisition, the difference was not found to be statistically significant (p = 0.052) (Figures 2 & 3).
_Theory,_(B)_Technique_domains_and_(C)_Total_sco)
Attitudes Towards Surgical Simulators
After usage of the Touch Surgery simulators, students were significantly more likely to have had a positive experience with mobile medical apps, feel they were easy to use and were a useful adjunct to medical education (p < 0.05). On average, the students agreed with the following statements:
-
“The Touch Surgery app was easy to use.”
-
“The Touch Surgery app was fun to use.”
-
“Compared to other study material, I find Touch Surgery to be more interactive.”
Discussion
This is the first study that evaluated the effectiveness of CTA simulators like Touch Surgery in improving surgical knowledge acquisition for medical students, since most of the earlier studies have usually focused on experts including senior trainees and consultants. Our results demonstrate that these simulators lead to a significant improvement in technical and overall knowledge acquisition in this cohort. There was also an improvement in theoretical knowledge, although not as significant. This is expected since these simulations likely focus more on the technical aspects of the surgery, which is also likely more difficult to acquire. This is similarly seen in other studies assessing the training effect of Touch surgery. For instance, Sugand et al (Sugand, Mawkin, and Gupte 2016) demonstrated substantial training effect via logarithmic learning curves that there was a strong positive correlation between the number of attempts using and scoring on the app procedural modules, with scoring variability and distribution being reduced and more predictable per attempt. It has been argued that CTA-based training might be less effective when applied to the medical student population compared to expert surgical candidates (Bathalon et al. 2005; Kowalewski et al. 2017; Sugand, Mawkin, and Gupte 2015). In context, it has been suggested medical students should first have some experience within surgery and the operating theatre to attain those complex learning points as provided by the CTA- based training, based on theory of constructivism proposed by Piaget. Our study results demonstrate that there is still efficacy for CTA training in medical students. We recommend that they might benefit even further with at least some prior exposure to observing and assisting cases in the operating theatre, and providing relevant structured foundational learning material beforehand, to equip them with some baseline knowledge.
Mobile surgical simulators offer a useful adjunct to traditional learning methods in surgical education. From our qualitative feedback, the Touch Surgery platform was widely regarded as an easy, fun to use, engaging way of learning based amongst study participants, which are in line with acceptability validity similarly demonstrated in other studies13,14. The interactive modules provide a more engaging medium than the more traditional and passive forms of education (e.g. lecture notes, medical textbooks, journal articles, videos etc). They also enable spaced and repetitive learning outside time in the theatre, that is low cost, accessible, and safe for both students and patients.
There are some limitations to our study. First, our results are based on a small sample size over a short period with the lack of a control group. There is no longitudinal follow up to evaluate if there is continued retention of this knowledge. Second, within this group of medical students itself , there might be inherent bias since these students recruited on a voluntary basis might be more interested in surgical subspecialties with higher overall inherent abilities, and thus might perform better on the simulators than the general cohort. The exclusion of orthopaedic residents from the test group also means that the results of these novices may not be transferable to orthopaedic residents. Third, the assessment of theoretical and technical knowledge acquisition was limited to MCQs. This is not designed to evaluate whether skills were acquired or improved in theatre performance. Third, these simulations lack transferability to surgical errors and complications. There is an inability to make a mistake in a surgical step and continue along that path to see the outcome of the error. This has been recognized by former CEO and co-founder Jean Nehme of Touch Surgery app, who also shared future plans to address this by building an adaptive behavior engine based around predictive decision-tree analysis, coupled with a real-time 3D graphics system.11
Further research should be conducted beyond this pilot study to assess how to better incorporate CTA simulators in surgical education. This could include larger and more definitive cohorts with appropriate control groups – including a broader profile of medical students, and also analyzing and comparing against expert candidates such as orthopaedic residents. Coming out of post-covid era with greater access to hospital facilities, it would be worthwhile to explore whether a combination of CTA and surgical hands-on experience might outweigh just CTA alone.
More longitudinal studies would be useful to evaluate learning retention and performance over time. More robust evaluation of training effectiveness might also be achieved by considering newer modalities and outcomes to measure, including specific steps, accuracy versus precision, time for procedure, translation to actual procedural skills etc.
This would be useful to identify more specific benefits that these simulations can potentially offer, beyond our current understanding that they help technical – but less so for theoretical- knowledge acquisition. For expert candidates, it is also worth exploring if providing a more structured study protocol – such as starting with a stipulated reading or lecture material, followed by simulation, then actual surgery – might further enhance their learning outcomes. Finally, as technological solutions continue to evolve, evaluating and comparing the impact of newer tools including virtual reality platforms (e.g. PrecisionOS, Osso VR), web-based multimedia tools or other mobile app based simulations on surgical education would also be an exciting area to explore.
Conclusion
Cognitive task simulations including Touch Surgery provide an effective means of the technical knowledge acquisition for novices such as medical students and offer a useful training adjunct for surgical education.
Author Contribution
ACC Chou, PPY Chen and S Compton were responsible for the study conceptualization and design. JSD Cheng was responsible for the data collection and data analysis. FDJ Ng was responsible for the data analysis and manuscript preparation under the supervision of ACC Chou.
Funding statement
There is no funding source to report for this study.
Competing Interest Statement
The authors have no competing interest to declare in this study.
Ethics approval statement
The study is approved by NUS Institutional Review Board (Reference number: S-17-275).