Introduction
With over 190,000 marketed medical devices and a “dozen new or modified devices” being approved every day (Weiss and Mohr 2018), the pace of advancements in medical device innovation reinforces the dependent relationship Surgeons have with Surgical Device Representatives (SDRs). The American Medical Association, American College of Surgeons, and the Association for PeriOperative Registered Nurses all acknowledge the critical role SDRs have in the safe delivery of patient care, but are clear that the presence of the SDR in the operating room does not negate the need for the Surgeon and/or the Surgical team to receive training and be able to demonstrate competency in the application of the surgical device or technology, prior to the procedure (“AORN Position Statement on the Role of the Health Care Industry Representative in Perioperative and Invasive Procedure Settings” 2014; Association AM, n.d.; Surgeons ACo 2016). Consequently, there is a considerable emphasis on the technical competence of the SDR, in order to be able to successfully transfer the necessary knowledge and skills to the Surgeon and Surgical team (Bedard 2021), in a manner that is timely, comprehensive, and does not create additional burden to the staff or work environment. As the pace of surgical innovation increases, focusing on the optimization of training for SDRs will positively impact patient care by accelerating the safe use of surgical devices and technologies.
Current SDR training practices for new devices vary by company but typically are provided as in-service training with a mix of didactic and hands-on learning using simulators, including synthetic models and/or cadaveric laboratories. A return demonstration of competence may also be required for training completion. Producing, using, and maintaining training resources can be costly expenditures with variable levels of published evidence supporting the obtainment of the intended skill acquisition and retention (Nousiainen et al. 2016; Zendejas et al. 2013). Recently, Immersive Virtual Reality (IVR) has demonstrated itself to be a disruptive technology for surgical education, due to its improved learning efficiency and ability to quantitatively measure learning outcomes, in both technical and cognitive skill domains for surgical trainees (Lohre, Bois, Pollock, et al. 2020; Lohre, Morrey, and Goel 2020; Lohre, Wang, et al. 2020; Lohre et al. 2021a; Mao et al. 2021). Current offerings have been developed to leverage the interactive and realistic experience of IVR, maximizing evidence-based cognitive and behavioral-science learning concepts. These include concepts of deliberate practice, productive failure, and experiential learning cycles that replicate real-world training while mitigating the potential for patient harm.
As an emerging technology, researchers are yet to evaluate SDR’s perception of the suitability for using IVR for new device training. The primary objective of this study therefore is to determine how inclusion of IVR in annual training is received by SDR’s, and to examine learning efficiency through the application of virtual performance metrics.
Methods
Study Design
This is a cross-sectional study of n=286 SDRs attending an annual sales meeting for a single company. The goal of the annual sales meeting is to train SDRs on new and updated technologies within their product portfolio. Participants completing the “Revision Knee System” training were included in the study. The “Revision Knee System” includes new hardware and associated instrumentation to perform a revision total knee arthroplasty (TKA).
Intervention
Traditional Training
All participants received traditional new-device training consisting of a one-hour didactic lesson from an expert orthopedic surgeon hired by the device manufacturing company, reinforced with an audio-visual presentation demonstrating the functioning of the Revision Knee System. Over the next two days participants self-selected into one of four smaller groups to complete hands-on training. This consisted of a 30-minute semi-structured group discussion led by a moderator, followed by an opportunity to physically interact with the device. At the completion of the traditional new-device training, participants moved onto an IVR training session.
Immersive Virtual Reality Training
The IVR training session was led by the device manufacturing company’s digital leader. Each participant was provided a VR headset (Meta, Menlo Park, CA) pre-loaded with the company’s proprietary Revision Knee System digital experience (PrecisionOS, Vancouver Canada). This digital experience includes nine revision TKA scenarios of increasing complexity. Once a user chooses a scenario, the system provides a realistic, scaled, and immersive digital operating room complete with patient set up. The system divides the overall revision TKA procedure into key steps which the user completes independently, or with digital performance assists. The user progresses through the module using instrumentation and implants created from real-world, manufacturer specific, computer assisted drawing (CAD) files. Feedback is provided for relevant iterative steps as well as a summative Precision ScoreTM. The Precision ScoreTM is a proprietary and validated assessment score correlating with potential adverse patient outcomes (Lohre, Bois, Pollock, et al. 2020; Lohre, Leveille, and Goel 2021; Feeley et al. 2021). Users can use their individualized feedback and scores to repeat the modules and track their progress over time, allowing for both cognitive and technical learning in a deliberate practice learning model that follows Kolb’s learning theory of education.
Participants were provided general instructions on how to turn on and operate the VR device headset as well as a basic orientation to the Revision Knee System digital experience. Once participants had obtained a basic familiarity with the device, they were instructed to first complete the tibia and then the femur portions of the training modules. A single module was evaluated in this study, though eight additional are available with varying degrees of femoral or tibial bone loss necessitating the use of augments. The system provided formative feedback immediately upon a critical error as well as summative feedback (Precision ScoreTM) at the conclusion of the modules. Once complete, each participant was asked to complete a survey of their experience.
Data Collection
Immersive Virtual Reality Performance
Performance metrics were tracked for each participant, including the number of module completions, the duration of use, and outcomes of each key step in the Revision Knee System training module. Key steps of the revision arthroplasty module were considered as part of both tibial and femoral preparation and implantation (Table 1). Soft tissue injuries to the medial (MCL) or lateral (LCL) collateral ligaments and/or popliteal artery were also detected and measured. A virtual assist mode is present for new users that provides guidance for completing the module and performing correct surgical sequences and tasks. Module completion metrics were collected with the virtual assist in the on and off mode. A summary aggregate performance score is also provided for each participant (Precision ScoreTM).
SDR Experience Survey
A third-party designed a short, 8-question survey designed to determine the perceptions of SDRs on the application of IVR for new-device training (ROI Canada, Ottawa). Question categories included relevance, importance and practicality of use, desirability of use, ability to train, and longitudinal use and recommendation to colleagues. A non-validated question set was used to qualify surgical device representative opinions on the use of IVR as a validated metric has not been previously produced. Questions were a mix of Likert-scale and binary yes/no questions (Table 2).
Statistical Analysis
For quantitative data derived by the Revision Knee System digital experience, values are presented as mean and standard deviation, and medians and range when appropriate. All data included in analysis were tested for normality prior to analysis. Correlation coefficients were produced to compare IVR performance metric data using Pearson’s r or Spearman’s ρ.
Results
Immersive Virtual Reality Performance
A total of n=286 participants completed n=1048 repetitions of the Revision Knee System digital experience =3.7/participant). The mean training time per individual repetition, completing both tibial and femoral IVR components, was approximately 10 minutes =9.4±4.6 min). Cumulative training time accounting for each individual’s multiple repetitions was close to 20 minutes =18.3±8.5 min).
Tibial IVR Module Performance
A total of n=617 repetitions of the tibial component of the Revision Knee System digital experience were completed by participants. The assist mode was active during n=311 repetitions, with the assist mode off for the other n=306 repetitions. The average number of tibial module repetitions was 2 =2.2), with each repetition taking on average 6 minutes =6.0±3.1 min). The total time spent on tibial training, considering multiple repetitions was approximately 13 minutes =13.3±6.9 min).
Of the key steps in the tibial sequence, tibial preparation, performing the revision tibial cut, and tibial component sizing and positioning, accounted for the majority of procedural time =4.3min). As expected with novice learners there was a high rate of critical soft tissue injuries (74%, n=458 occurrences). The most affected tissues were the medial and lateral collateral ligaments (MCL=196, LCL=185). Critical soft tissue injury to the popliteal artery was less common (n=77). The Precision ScoreTM was most strongly correlated to critical soft tissue injuries including MCL (r=-0.32), LCL (r=-0.62), and popliteal artery (r=-0.47) and amount of bony resection (r=-0.39) (Table 3; Figure 1). The total amount of bony resection was also correlated to LCL (r=0.19) and arterial (r=0.12) injuries. Injuries and resection amount incorporate technical skill and cognitive understanding into performance, and the Precision ScoreTM accounts for these learning domains.
Femoral IVR Module Performance
There were 30% less repetitions of the femoral portion of the Revision Knee System digital experience, with participants completing n=431 repetitions. Of these, the assist mode was active with n=258 repetitions and inactive for n=173 repetitions. The average session time was approximately 3 minutes =3.3±3.3 min). The average number of femoral module repetitions was slightly lower than the tibial module =1.5), with each repetition taking on average 5 minutes =5.0±5.1 min). The largest amount of time was spent on determining correct femoral sizing and positioning =2.6min), followed by femoral preparations =0.8min). Compared to the tibial module there were considerably less critical tissue injuries with n=38 tibial and n=75 arterial injuries (n=113/571, 20%). An inverse relationship exists between the Precision ScoreTM and the occurrence of popliteal arterial injury (r=-0.76), or tibial injury (r=-0.10). Baseplate orientation (r=0.18) was also seen to contribute to the overall Precision ScoreTM. These parameters incorporate steps of both technical and cognitive skill domains.
SDR Experience Survey
Of the n=286 SDR’s that participated in the company’s Knee Revision System training, n=245 completed the survey questionnaire with a response rate of 86%. Participants overwhelmingly 99% (244/245) viewed the use of IVR for new-device training to be “Very Relevant” to their role. The level of knowledge and skills obtained using IVR was rated by 87% (213/245) of participants to be “High” or “Very High” and was viewed by 86% (210/245) as “Very Likely” to contribute to their future career success. The large majority, 89% (218/245) of all participants found the IVR system to be practical to use, with nearly all of the participants, 99% (243/245) responding that they believe the use of IVR could “Significantly” or “Moderately” increase sales volume. In considering the translation of new-device training to SDR’s current role, the majority of participants, 99% (243/245) stated that they intended to use the acquired knowledge and skills directly in their job. When comparing new-device training using IVR vs traditional learning approaches, nearly all participants preferred IVR training, 91% (223/245) and would recommend the use of IVR training to others 99% (243/245) (Table 2)
Less than 1% (0.8% 2/245) of all participants responded that they did not believe IVR would influence sales volumes and would not be important for their future career success. (Table 2)
Discussion
This study reveals that SDRs receiving training on a new Revision Knee System strongly prefer immersive virtual reality (IVR) over traditional education practices. The universal acceptance and preference for IVR over traditional didactic-based learning is due to IVRs alignment with Experiential Learning Theory - a well-established educational theory first proposed by Kolb (Kolb 1984). Experiential learning theorists view learning as a process of knowledge obtainment through the transformation of experience (Kolb 1984). Learning-by-doing generates knowledge through four stages:
-
Having a concrete experience (Experiencing)
-
Reflecting on the experience and gaining insights into performance (Reflecting)
-
Connecting insights to an existing knowledge base and understanding (Conceptualizing)
-
Formulating new hypotheses and developing new plans for testing (Experimenting)
This learning strategy has been previously proposed to occur in IVR (Lohre et al. 2021b; Lohre and Goel 2019). In this study, participants experienced the new Revision Knee System in an environment that allowed for critical errors. Participants immediately received formative and summative feedback on their performance, allowing them to reflect and place into context how their actions led to the outcomes they experienced. Participants used this feedback to construct new plans of action and retook the module in an effort to master the Revision Knee System content. Although the traditional learning approach did provide participants an opportunity to physically interact with the new-device, removing the environmental context provides limited insight into the new-device’s application in an actual procedure.
The ability to self-pace and infinitely repeat learning loops enables SDRs to master new-device technologies rapidly with IVR, versus traditional learning models which Kolb see’s as antithetical to learning as they focus on the accumulation and storing of facts or habits representing behavioral responses to specific stimulus conditions (Kolb 1984). The benefits of experiential learning are demonstrated with participants being able to practice a complex revision TKA multiple times in the span of minutes, while standing in a conference hall. A previous IVR review highlighted key steps in the revision knee arthroplasty module comprising both technical and cognitive performance domains (Lohre, Goh, Parvizi, et al. 2021). Cognitive domains are those that involve decision making such as amount of revision bony resection, trialing and sizing, keel alignment, and component rotation while technical include canal reaming by receiving haptic feedback as to the appropriate size. The Precision ScoreTM was correlated to performance of key steps during revision TKA including both cognitive and technical skill domains, as well as performance errors. In our study, providing a Precision ScoreTM allows for tracking of user performance over time. The use of IVR increases the speed of knowledge and skill acquisition and reduces the overall training time for SDRs. Improving training efficiency will reduce the time away from their role and can ultimately accelerate the safe transfer of surgical devices and technologies to Surgeons and Surgical teams.
The use of IVR to improve learning efficiency is beginning to be recognized in medicine. For example, IVR has been shown to outperform other learning modalities for teaching cardiac anatomy (Zinchenko et al. 2020), neuroanatomy (Schloss et al. 2021), and ultrasound (Hu et al. 2020), where a spatial understanding of the phenomenon is needed. But IVR in and of itself is insufficient without a high level of simulation fidelity providing the sensory and behavior cues that learners can use to create presence and immersion in VR, leading to learning efficiencies (Våpenstad et al. 2017). Current Immersive VR offerings such as the PrecisionOS Revision Knee System leverage the computing power of current VR hardware capabilities to provide a realistic and interactive digital environment. Previous studies have shown that these modules are interpreted as realistic by surgeons, including the appearance and interactive ability of digital surgical instruments (Lohre, Bois, Pollock, et al. 2020; Lohre, Bois, Athwal, et al. 2020). The learning capacity of VR is likely contingent on the immersive factors and sensory interaction with both hardware and software. In this sense, with 100% of participants recommending the Knee Revision System digital experience training to their colleagues, the module developed for new-device training was of a fidelity that enabled participants to be immersed and engaged learners.
These results are similar to previous studies reporting favorable responses on realism, learning capacity, and enjoyability by both surgeons and surgical trainees (Lohre, Bois, Pollock, et al. 2020; Lohre, Bois, Athwal, et al. 2020). Previous randomized controlled trials (RCTs) of senior orthopaedic trainees have shown improvements in learning efficiency using IVR when compared to technical instructional documents/journal articles and videos by 570% and 387%, respectively, as well as improved measurable skills, including improved instrument handling and overall combined validated metric scores using IVR (Lohre, Bois, Pollock, et al. 2020; Lohre, Bois, Athwal, et al. 2020). A reliable digital equivalent to real world performance is the Precision ScoreTM. In these previous studies, measurable real-world performance was correlated strongly to the Precision ScoreTM. Recently, a case report illustrated that deliberate practice training using IVR resulted in improved performance and outcomes for a patient receiving revision percutaneous pinning of a slipped capital femoral epiphysis (Lohre, Leveille, and Goel 2021). A senior orthopaedic surgical trainee utilized the system four consecutive times prior to the procedure to understand anatomy, fixation, and hardware techniques. The Precision ScoreTM increased from 70% to 93% with a 232% reduction in IVR training time and 840% reduction in virtual fluoroscopic images over the training period. The procedure itself placed two screws perpendicular to the physis using 7.6x less cumulative radiation dose than the index procedure (Lohre, Leveille, and Goel 2021). Our current study provides further evidence that the Precision ScoreTM weights virtual performance based on patient outcomes to reduce error. IVR has been shown to improve both cognitive and technical skill, resulting in reduced error and this principle also applies to the SDR population.
The role of the SDR is to provide technical hardware solutions to the surgeon and additional members of the surgical team. Though SDR technical skills and procedural understanding are different to that of the surgeon, knowledge of key surgical steps provides a “what comes next” frame of knowledge. Anticipatory knowledge of upcoming key steps and how to mitigate hardware issues in those steps provides significant increases in efficiency and thus overall value to the surgical team. The results of this study provide corroborating evidence to the efficacy of IVR training for not only surgeons, but other key members of the surgical team including SDRs. This is due to the nature of IVR and how the immersive experience complements learning.
The IVR system designed for this study embeds principles of deliberate practice, active learning, and productive failure, to hasten the transition from novice-to-expert in skill acquisition. Critical errors for SDRs in practice are different than those of surgeons. These may include unavailability of back-up instrumentation, errors in assembly instructions, or not anticipating next events clearly. These all contribute to OR inefficiency. The ability of the SDR to perform the surgery in IVR with immediate metric feedback provided by the Precision ScoreTM is a novel method of supporting SDRs to understand the pace and attention of surgeons.
Our study showed 196 MCL injuries produced by the training SDRs. Understanding that the revision tibial cut is an important differentiating step in the procedure and that anticipating the need for constrained knee options, provides significant surgical time reductions and added value to the operating team. With repeated use, the SDR can reduce these errors by experimenting in IVR to develop a more perfect understanding of implant positioning and surgical steps. Progress towards this goal could be reflected using a trackable cumulative performance metric, such as the Precision ScoreTM adopted in this study. Similarly, if there is bone loss on the tibia or femur, the SDR can interpret this as having available and ready the assembled augments. This is possible in IVR as these modules are produced using manufacturer specific computer assisted drawing (CAD) files, providing for a realistic appearance and behavior in virtual space that in-turn increase the level of immersion and learning efficiency. The Precision ScoreTM is correlated to critical errors such as vessel, ligamentous injury, or bone resection volume and using this score to improve overtime aids in understanding of critical component steps in the procedure.
Our findings of near universal acceptance of IVR as a training modality provides an important new insight into the willingness of SDRs for alternative educational formats. With an acceptance and willingness to adopt work-from-home strategies stemming from the Covid-19 crisis, a shift to dedicated IVR training would not only be cost-effective for the company but also signal the adoption of new employee-centric work policies.
Highly effective SDR’s and the widespread use of IVR technologies, produces traditional (costs, learning efficiency), and when carried into the operating room, non-traditional returns. These may include time savings for multiple staff and surgeons, improved training frequency and utilization, trackable proficiency and knowledge data, increased operating room involvement and surgical/hospital interactions, improvement in team performance with more knowledgeable staff, and improved patient related outcomes. The ability for SDRs to effectively train highly specialized teams in an efficient and effective manner using IVR, provides added value to all health care stakeholders and accelerates the safe use of surgical devices and technologies.
This study has two key limitations. The study design did not attempt to control training exposure between traditional and IVR training. All participants received traditional training and then self-selected to one of four IVR training sessions. To parse out the effect of IVR versus traditional training on knowledge acquisition, a randomized control trial is needed. As such we make no claims to the level of knowledge or skill acquisition obtained in this study. Characterizing these are the focus of future studies. Second, the survey instrument used in this study was purposely brief. To explore the attitudes and beliefs of SDRs towards the use of IVR technology for training, further qualitative studies are needed.
Conclusion
Sales Device Representatives universally prefer Immersive Virtual Reality for new-device training over traditional training practices. Participants completed n=1048 repetitions of a Revision Knee System, resulting in a high rate of critical soft tissue injuries captured by the IVR training system. IVR best aligns with Experiential Learning Theory, produces efficient learning, and is cost effective. This is the first study of its kind to define the attitudes and relevance of experiential IVR using the PrecisionOS system in SDR’s.