INTRODUCTION
Coronavirus disease 2019 (Covid-19) was declared a global pandemic by the World Health Organization (WHO) on 11 March 2020 (World Health Organization 2020). To date, there have been 664,873,023 confirmed cases and 6,724,248 reported deaths (World Health Organization 2023). As of the 24th of January 2023, there have been a total of 13,156,047,747 vaccine doses delivered (World Health Organization 2023). In the months and years following the onset of the pandemic, its effects have significantly disrupted patient health and healthcare systems in the US and across the globe (Haileamlak 2021; Poudel et al. 2021; J. Xiong et al. 2020; Li, Wang, and Wang 2021; Youn et al. 2022). These effects have also significantly affected the practice of orthopaedics, serving as the impetus for a vast amount of research and publications (Liu et al. 2021).
While COVID-19 precipitated a proliferation of scientific production in orthopaedics, many of the initial publications consisted of editorials, letters to the editors, and expert opinions (D’Ambrosi 2020). However, researchers have since conducted extensive clinical research, including the effects of the pandemic on incidences of trauma, major complications, and morbidity (Egol et al. 2020; Bram et al. 2020). These articles also delineated the impact COVID-19 had upon healthcare providers, patients, and the general practice of orthopaedics and have since been disseminated as published works to serve as guidance (Zagra et al. 2020; Hampton et al. 2020; Wong and Cheung 2020).
The reach of this aforementioned guidance and clinical evidence can be measured though bibliometric analysis (Soytas et al. 2021; Familiari et al. 2021; Mercurio et al. 2022). For example, it can gauge the quantity of citations and social media attention a research output has received, indicating what the clinical research field and the public deems most notable (Ellegaard and Wallin 2015; Robinson et al. 2021). Given the vast amount of research performed on COVID-19 and orthopaedic patients within the context of rapidly evolving information during the early stages of the pandemic, it is important to retrospectively evaluate which articles gained the most attention and thus provided the most clinical utility. Presently, no published literature has been made to examine the breadth of work which has been published on COVID-19 and orthopaedic patients. Consequently, the aim of the current study is to: 1) assess the top-50 most influential articles pertaining to COVID-19 and orthopaedic surgery through determining the most cited articles as well as exploring Altmetric characteristics; 2) determine the locations and journal characteristics pertaining to the most influential articles; and 3) collate lessons learned within the most influential articles pertaining to the COVID-19 pandemic. Though the global pandemic is over, and an event exactly like it may never happen again, it is important to understand how the resilience of the healthcare system adapted during a worldwide time of crisis so that we may determine what, if any, durable evidence has emerged in the literature. Studying the most highly cited publications on this topic may also help demystify the overwhelming wave of research on this topic in the years immediately following the 2020 pandemic.
MATERIALS AND METHODS
Data collection
The Clarivate Web of Science Core Collection was systematically used to gather data on COVID-19 and orthopaedic research from January 9 until January 22, 2023. The terms ‘COVID-19 AND orthopaedic surgery’ and ‘COVID-19 AND orthopedic surgery’ were used to search this database, returning 388 articles published since 2020.
Article selection
The abstracts of the 388 articles were independently reviewed and categorized by LL, KR, JC, BC, AB, and each article was reviewed by a second author to ensure accuracy. 200 articles were queried due to the belief they contain the top 50 most cited articles after duplicates were removed (Figure 1). Expert opinion, systematic reviews, and review articles were excluded from this analysis, yielding only original studies. Systematic reviews with meta-analyses were included if each article within the article met the inclusion criteria of this study. Article title, journal name, type of study, country of publication, study information, and number of citations for each article were collected and reported using descriptive statistics. There were no restrictions based on the language in which the article was written. The most influential articles were determined to be the top-50 articles with the greatest number of citations meeting our inclusion criteria of original articles. Journal impact factors were determined using the Clarivate Web of Science Journal Citation Reports database.
Altmetrics data collection
Altmetrics, or alternative metrics, is a tool to discover online activity surrounding research, which may supplement traditional citations-based bibliometric data. Altmetrics data is accumulated more quickly than citations, which allows for a measure of immediate global research interest and engagement following publication (Altmetric 2023). The bibliometric data collected for each article through the Altmetric Explorer Database includes an Altmetric Attention Score (AAS), title, journal title, publication date, OA status, number of dimensions citations, number of Mendeley readers, and media mentions, including: news, blog, policy, patent, peer review, Twitter, Facebook, Reddit, Wikipedia, video, and syllabi mentions. The AAS is derived from an algorithm to represent a weighted count of online attention directed towards each article (Altmetric 2023). Further, AAS has been demonstrated to correlate with citation count for articles in previous literature (Collins et al. 2021). Thus, The Altmetric Explorer database was searched for the top-50 articles found during article selection to determine research engagement in addition to traditional citation-based metrics. Of the 50 articles in this analysis, Altmetrics data was reported for 35 articles. As Altmetrics data is not available for each individual article in this analysis, it was not the primary database used to determine the most influential articles published during this period.
Statistical analysis
Descriptive statistics were calculated including mean ± standard deviation (SD) or N (%) where appropriate. Microsoft Excel was used to store data and to perform statistical analysis. Keywords were queried via PubMed and Clarivate, and quantities were analyzed using Microsoft Excel version 16.70, King County, Washington USA.
RESULTS
Characteristics of the 50 most cited articles
Our bibliometric analysis of the most influential manuscripts concluded the top-50 articles (Egol et al. 2020; Bram et al. 2020; Zagra et al. 2020; Hampton et al. 2020; Wong and Cheung 2020; Guo et al. 2020; Cipollaro et al. 2020; Catellani et al. 2020; LeBrun et al. 2020; Scott et al. 2020; Liebensteiner et al. 2020; Chaudhry, Nadeem, and Mundi 2020; Hernigou et al. 2020; Clement et al. 2020; Gruskay et al. 2020; Thaler et al. 2020; Sherman et al. 2020; Rizzi et al. 2020; Haffer et al. 2020; Luceri et al. 2020; Megaloikonomos et al. 2020; Benazzo et al. 2020; Siow et al. 2020; An et al. 2020; D.-G. Chang et al. 2020; J. Chang et al. 2020; Luengo-Alonso et al. 2020; Morelli et al. 2020; Giorgi et al. 2020; Thakrar et al. 2020; Lubbe et al. 2020; Greenhalgh et al. 2020; Dupley et al. 2020; MacDonald et al. 2020; G. Xiong et al. 2021; Baxter et al. 2020; Karayiannis et al. 2020; Meng et al. 2020; Chui, Thakrar, and Shankar 2020; Konda et al. 2020; Mitkovic et al. 2020; Yamada et al. 2020; Staunton et al. 2021; Probert et al. 2020; Andreata et al. 2020; Mackay et al. 2020; Zahra et al. 2020; Lazizi et al. 2020; Wignall et al. 2021; Lakhani et al. 2020) were published between April 23, 2020 and July 1, 2021 and were collectively cited 1,875 times. The mean number of total citations per article was 38 (95% confidence interval (CI) (Sherman et al. 2020; Thakrar et al. 2020)), ranging from a minimum of 13 to a maximum of 111 citations per article (Table 1). The most influential articles experienced the greatest increase in citations in 2021 with 915 total citations and an average of 18 citations per article, compared to 299 total citations in 2020 and 659 total citations in 2022 (Figure 3). All 50 study designs contained multiple human subjects, and there was a median sample size of 310 individuals in each study.
Additionally, we analyzed the keywords used in the top 50 articles, correcting for modifications of words or exact synonyms, for example, “COVID-19” and “coronavirus” (Table 4). All articles had a total of 115 keywords or phrases, and we found that the three most frequently used words were: “Coronavirus” (60), “Trauma” (23), “Fracture” (18). The most frequent key phrases, consisting of two or more words, were: “Hip Fracture” (6), “Orthopaedic surgery” (4), “Orthopaedic trauma” (4). 10 articles reported no keywords or key phrases for this analysis.
Altmetrics characteristics
The Altmetric Explorer database was utilized to gauge the social media impact of the 50 articles identified in this study. This assessment is advantageous as traditional citations can take many years to accumulate while an article’s AAS indicates the immediate impact of literature. Of the total 50 articles in this analysis, 35 articles were associated with an Altmetric Attention Score (AAS) and are reported alongside citation rankings in Table 1. The AAS scores ranged from 317 to 0, with a mean of 20.2 ± 54.1. These AAS scores have different weight corresponding with different media representations for each article (Altmetric 2023). These scores corresponded with news mentions ranging from 0 to 44 per article, with a mean of 2.02 ± 7.89; Twitter mentions ranging from 0 to 105, with a mean of 10.49 ± 18.73; Facebook mentions ranging from 0 to 2, with a mean of 0.26 ± 0.56; and a number of Mendeley readers ranging from 25 to 475, with a mean of 140.43 ± 96.65.
Journal characteristics
The most popular journals for COVID-19 and orthopaedic research were Bone & Joint Open and International Orthopaedics each with 10 of the top-50 manuscripts (Table 6). All 50 articles were published in English. The top three countries included England (14), the United States (11), Italy (10) (Table 5). The top affiliations were IRCCS Istituto Ortopedico Galeazzi (5), and the University of London (4) (Table 1). The highest number of articles were published in September 2020 and August 2020 with 15 and 14 articles, respectively (Figure 2). A downward trend was observed in the number of publications per month between September 2020 and July 2021 (Figure 2).
Clinically Relevant Findings of the Most Cited Articles
Though the most cited articles were very heterogeneous, they touched on a few similar themes which emerged in the field of orthopaedic surgery during the COVID-19 pandemic. As healthcare workers adapted to the circumstances ahead of them, numerous articles were published documenting the lessons learned during this period. The most cited articles highlighted the change in resource utilization, growth of telemedicine, development of infection protocols, healthcare workers well-being, and patient-centered care within the field of orthopaedic surgery (Table 1).
Resource utilization was an important part of understanding change amidst a global pandemic and was emphasized in many of the top-50 manuscripts. The most cited article by Bram et al. described how the closure of schools, sports, and playgrounds precipitated by COVID-19 was followed by a significant decrease in pediatric orthopaedic trauma cases, which in turn led to the reallocation of resources to clinical areas with higher patient volume, increased emphasis on basic home safety precautions, and the simplification of pediatric care with a greater reliance on telemedicine and removable splinting (Bram et al. 2020). Another study by Wong, et. al. utilized the perspective of the initial Wuhan surge in COVID cases to describe modifications made to orthopaedic staffing and resource allocation, stating that despite a decrease in overall patient volume, demand for orthopaedic services remained high (Wong and Cheung 2020).
As infections rose and more stay-at-home orders were in place, telemedicine was increasingly utilized in the field of orthopaedic surgery. Some of the most cited articles explored the relationship between satisfaction levels between telemedicine and in-person visits, and another found that telemedicine would be beneficial to both providers and patients (Chaudhry, Nadeem, and Mundi 2020; Rizzi et al. 2020). Telemedicine played a pivotal role in the delivery of healthcare during the pandemic and allowed orthopaedic surgeons to see patients virtually for rehabilitation monitoring, post-operative evaluations, and initial assessments. The success of telemedicine in the field of orthopaedics during the pandemic signified its potential for long-term utilization into patient care in the post-pandemic period.
Alongside resource allocation and telemedicine utilization, many of the most influential articles contained information regarding infection protocols for COVID-19. Guo et al. described the risk of COVID-19 infection for orthopaedic surgeons and provided guidance on necessary precautions to minimize risk factors (Guo et al. 2020). Specific recommendations for orthopaedic surgeons included using appropriate PPE (including an N-95 respirator); staying updated on training regarding infection prevention and resource conservation; considering the minimization, postponement, or cancellation of elective procedures; following U.S. CDC guidelines; avoiding contact with family members at home following suspected exposure to infected persons; and reducing fatigue, which could decrease the body’s immune response. In addition, the fifth most-influential study, a meta-analysis by Cipollaro et al., discussed musculoskeletal symptoms associated with COVID-19 infection, specifically identifying near ubiquitous myalgia, arthralgia, and fatigue (Cipollaro et al. 2020). The authors suggested these symptoms be incorporated in early detection of infection and can be used to approximate the extent of infection on the human body if effectively paired with laboratory findings (Cipollaro et al. 2020). In another study by Chang, et. al., the authors emphasized that even in the post-pandemic period when routine orthopaedic services resume, orthopaedic surgeons should continue to prioritize safety and emphasize risk for COVID-19 infection (J. Chang et al. 2020). The infection protocols enacted during the pandemic were used to ensure the safety of patients and healthcare workers during COVID-19 and remains an important safety measure even in the post-pandemic period.
Certain articles additionally touch on the well-being of healthcare workers during the pandemic period within the field of orthopaedic surgery. The pandemic placed immense stress on orthopaedic surgeons, and mental health concerns were evident during this period. With regards to residency training, Megaloikonomos et. al. found that orthopaedic trainees expected that the reduction in workload and educational opportunities during the pandemic would negatively impact their training (Megaloikonomos et al. 2020). Other articles consequentially discussed the impact the pandemic had on orthopaedic residency training and hoped for a fostering work environment in the future (An et al. 2020; D.-G. Chang et al. 2020).
The final lesson obtained in these articles discussed the importance of patient-centered care during the COVID-19 pandemic. In a prospective cohort study by Egol et al., the authors described how the pandemic influenced hip fracture care in New York City and noted an increase in morbidity and mortality in these patients (Egol et al. 2020). As such, they recommended that physicians treating patients with confirmed or suspected COVID-19 infection should counsel their family of the significantly increased risk of complications and death following hip fracture. Patient-centered care became even more valuable during the pandemic as orthopaedic surgeons were required to engage in transparent discussions with patients and their families regarding risks and alternatives to surgery when priorities had to be changed due to COVID-19.
DISCUSSION
As it has been over three years since the World Health Organization declared COVID-19 a public health emergency of international concern, numerous articles have been published related to COVID-19 and orthopaedic surgery (“Determination That a Public Health Emergency Exists” 2020). As the pandemic was a turbulent time of uncertainty with rapidly evolving knowledge, it is useful to perform bibliometric citation analyses for the identification and mapping of the rapid paced changes which occurred (Osareh 1996). The number of citations of an article is frequently used as a measure of influence, as the increased number of citations indicate that a contribution was significant (Callaham 2002). Studying the citations of a paper allows for tracking of the dissemination of the information they contain and how it has been applied toward training, future investigations, and changing clinical practice (Arshi et al. 2016).
Reflecting on the literature published in the field of orthopaedic surgery provides a direct snapshot of how the field responded to changes from COVID-19. From the top 50 most cited papers during that time, there was an average of 38 citations per paper with a minimum of 13 and maximum of 111. The greatest number of citations occurred in 2021 versus 2020, with an average increase of 18 citations per paper. When considering citations over time, these results can be compared to the top-100 most cited papers between 1944 and 2014 across all orthopaedic surgery research. Lum, et. al. analyzed notable trends in the top-100 articles and found the average number of growth of citations for each article of 85 across a 5-year period (range 0-535), and the 6-month average growth of citations was 5 per article (range 0-45) (Lum et al. 2020). Within only one to two years of publication, the number of citations in 2021 for orthopaedic related COVID-19 literature outpaces the rate of citation for the 100 most cited papers across all orthopaedic surgery literature, when the average citation rate is extrapolated for one year over the rates for 5 years (17 per year) and 6 months (10 per year). These findings point to the demand for literature addressing how COVID-19 affected the field of orthopaedic surgery (Tahamtan, Safipour Afshar, and Ahamdzadeh 2016). The two journals with the most articles (10) in the top-50 most impactful publications related to COVID-19 and orthopaedic surgery were Bone & Joint Open with an impact factor of 5.082, and International Orthopaedics with an impact factor of 3.479. Bone & Joint Open was ranked third and International Orthopaedics was ranked seventh in highest impact factor among all 16 journals included in our analysis. In consideration for the lower impact factor ranking of International Orthopaedics yet high number of impactful publications, an initial explanation could be that COVID-19 was a pandemic that increased demand for academic contribution globally. However, several other journals on our list included international journals without as many impactful publications. With respect to the individual quality of each publication, the evidence level of III had the most articles (30), which may be due to the limited time frame during COVID-19 and lack of prior research (Table 3). Similarly, when looking at the studies performed, most studies (27) were retrospective or prospective cohort studies, which allow us to make comparisons between pre- and post-COVID-19 groups (Table 4).
The countries with the most publications in the top 50 were England (14), the USA (11), and Italy (10). As Italy is the only non-English speaking country, their larger proportion of publications is likely due to Italy being the first European nation affected by COVID-19. As Italy was the first country in Europe that was severely impacted by COVID-19, they were unprepared for the ultimate serious consequences of the pandemic and likely faced a high demand for emerging literature on COVID-19 (Indolfi and Spaccarotella 2020).
As the most frequently used keywords in this analysis were: “Coronavirus” (60), “Trauma” (23), and “Fracture” (18) the high frequency of these words was hypothesized to correlate with the most common topics (Table 4). Thus, during COVID-19, the clinical literature reflected that the field was focused on how the trauma service was changed, including admissions, patient flow, and clinical decision making. Treatment for traumatic injury is often emergent and indicates the need for surgery. In particular, the word “surgery” appeared frequently in titles and was complicated during the pandemic for multiple reasons, including uncertain pathology of COVID-19, personnel shortages, new prophylactic measures, and elective versus emergency surgery (Kibbe 2020). A large component of orthopaedic trauma is fracture care, which likely explains its increased prevalence in the literature. Timely surgical repair of fractures is essential and can be lifesaving, which was a challenge for orthopaedic surgeons to overcome as COVID-19 evolved (Pietri and Lucarini 2007). Compared with other specialties, telehealth is a popular alternative for treatment, but it is not as often seen in the most influential clinical literature for orthopaedic surgery which may be due to the urgency of orthopaedic trauma injuries mandating them as a higher priority for the field (Soytas et al. 2021).
Correspondingly, data from the Altmetrics Data Explorer provided useful information regarding the online activity surrounding COVID-19 and orthopaedics research. The AAS scores ranged from 317 to 0, with a mean of 20.2 per article. These scores correspond with a study conducted by Lynch et al. in which the researchers found the average AAS value to be 4.8 +/- 20.7 for non-open access articles and 6.8 +/- 32.1 for open access articles in a study of 5245 lumbar spine articles in 2022 (Lynch et al. 2022). Additionally, Grillo et al. analyzed Altmetric attention scores amongst the top 50 articles involving oral health during the COVID-19 pandemic and found a mean AAS was 65.56, which ranged from 27 to 2149 (Grillo, Lopes, and Teixeira 2022). Consequentially, the AAS analysis of articles involving COVID-19 and orthopaedic surgery indicates that the top-50 most cited articles addressing the effects of COVID-19 on orthopaedics garnered a larger social media impact than traditional open access articles involving orthopaedic surgery, such as lumbar spine surgery, but had less online impact than articles involving oral health during the pandemic. This is likely due to large public interest of aerosols and oral infectious susceptibility to COVID-19.
Of the top-50 most cited articles published regarding COVID-19 and orthopaedic surgery, many articles contained meaningful lessons pertaining to the practice of orthopaedics during the pandemic. This included resource utilization, expanding telemedicine, developing infection protocols, insuring healthcare workers well-being, and increasingly applying patient-centered care. The initial stages of the COVID-19 pandemic were a time of uncertainty for the field of medicine, and future research was necessary to direct changes in surgical practice. The COVID-19 pandemic resulted in fewer patients being seen for orthopaedic injuries and increased a lack of confidence in the future of elective orthopaedic procedures (Table 1). Alongside a decreased patient population, the relationship between patients infected with COVID-19 and postoperative outcomes had yet to be established. The necessity for more research in the field of orthopaedic surgery during the pandemic was fulfilled on a global scale in the months and years following the start of the pandemic. The top-50 most influential articles were published within a 16-month period from April, 2020 to July, 2021 and served to guide clinical practice during the pandemic and beyond. Despite fewer patients with primary orthopaedic complaints, the top articles explained how orthopaedic surgeons remained essential for patients experiencing life-threatening and traumatic injuries, and precautions emerged to prevent patient and provider infection with COVID-19 due to its effects on postoperative outcomes and unknown long-term effects.
The lessons learned from the top-50 articles remains impactful in the post-pandemic period. The utilization of resources during the pandemic was critical in managing orthopaedic trauma and patients with orthopaedic injuries who were infected with COVID-19 to ensure the best care during this time of need, and information learned from these studies can be used to guide the optimization of resources if future crises were to arise. As more patients continue to be diagnosed with COVID-19 in 2023, research involving the utilization of resources when caring for these patients remains imperative. Telemedicine protocols additionally discussed in the top-50 articles are likely to continue to play a role in improving access to orthopaedic care through reducing unnecessary in-person visits and increase healthcare delivery to patients who may not be able to attend every orthopaedic visit in-person. Moreover, infection protocols developed during COVID-19 will remain essential in preventing infectious diseases. In the post-pandemic period, orthopaedic resident education and well-being has been heavily impacted by COVID-19, and educational experiences have been adapted to improve learning opportunities. Additionally, the emphasis COVID-19 placed on patient-centered care will continue to persist to increase patient engagement with their treatment and healthcare decisions. The top-50 articles written during the COVID-19 pandemic pertaining to orthopaedic surgery thus serve as a valuable guide for a more robust, efficient, and patient-centered healthcare system today.
LIMITATIONS
There are few limitations to this study with respect to the modes of analysis used. Though citation number was used to determine the impact of a particular study, citation number is not always the best method to determine significance. In using citation number as a primary metric of influence, the highest ranked articles are subject to citation biases including popularity of publishing journal, author self-citation, and the generation of citations through increased exposure (Familiari et al. 2021; Lefaivre, Shadgan, and O’Brien 2011; Kuhn 1962). Furthermore, articles are not always assessed and valued independently of their scientific rigor prior to each instance of citation (Familiari et al. 2021; Kuhn 1962). Many of the top-50 articles deemed significant in this analysis come from journals which may be regarded as having lower impact or significance as indicated by platforms such as Scimago and Google Scholar. Though the citation index was high for many of these papers, this number may be artificially inflated and not reflect a truly impactful research article during the COVID-19 pandemic. Though self-citation may cause falsely elevated citation numbers, in some cases self-citation may be appropriate when building upon research findings, particularly during the COVID-19 pandemic when little research was known about how it would impact orthopaedic practice and patient care. Citation number was nonetheless chosen for this analysis for it provides a useful metric to determine the most influential articles regarding a particular topic within an academic setting as they provide a context to evaluate rapidly evolving information, and most journals report citation number over other metrics used to assess article viewership and impact.
As the authors only utilized one database for this analysis, the Clarivate Web of Science, there may be limitations regarding the selection of articles. The Web of Science has been shown to prioritize English-language content and regional content, which may result in exclusion of valuable published research in other languages and countries from less widely recognized sources (Pranckutė 2021). Though the authors utilized only one database in this study, the Web of Science remains one of the leading databases containing bibliometric data and metrics and has taken effort in recent years to expand their coverage of articles in non-English languages and additional regions (Pranckutė 2021).
Articles excluded in this study were those of narrative reviews, opinion pieces, method demonstrations, theoretical models, and literature reviews in favor of articles contributing results through the study of patient data. The influence of gray text, including website articles, textbooks, non-journal articles (Familiari et al. 2021; Cassar Gheiti et al. 2012; Lefaivre, Shadgan, and O’Brien 2011; Seglen 1997), and mass media coverage of the COVID-19 pandemic on the selected list of articles is unknown. These articles were excluded to keep articles which provided foundational changes to the field of orthopaedic surgery during the COVID-19 pandemic and exist as foundational research today.
CONCLUSION
Since the beginning of the COVID-19 pandemic in early 2020, 388 peer-reviewed articles have been published related to COVID-19 and orthopaedic surgery. This study systematically compiled and analyzed the top-50 most influential articles. Common topics included trauma and fracture care, and the majority of evidence is level III. The most clinically relevant findings relate to the demand for orthopaedic surgery during the COVID-19 pandemic and recommendations to limit infection to improve patient outcomes with a patient-centered view of healthcare. The authors hope this bibliometric analysis of the 50 most influential papers assists researchers in uncovering impactful areas for additional research and highlights the advances research made during the pandemic in shaping the current practice of orthopaedic surgery today.