1. INTRODUCTION
The rapidly expanding usage of the OpenAI platform, ChatGPT, brings substantial concerns to the medical community in regard to the integrity and validity of information produced (Májovský et al. 2023; Salih 2024). ChatGPT is a language learning model that was designed to generate human responses from extensive databases of published information (Dergaa et al. 2023). These AI platforms have many applications in healthcare such as performing scholarly searches, exercising clinical judgment, and writing and editing manuscripts (Jeyaraman et al. 2023; Vander Griend et al. 2024). ChatGPT and other AI models are increasingly utilized across healthcare settings for quick and convenient generation of information from a database of previously published sources (Jeyaraman et al. 2023). ChatGPT has even been demonstrated to reach passing thresholds on United States Medical Licensing Examinations (USMLE) (T. H. Kung et al. 2023). The potential for AI to be a valuable asset in medical education and research is abundantly clear, but the accuracy and reliability should be thoroughly assessed before it is considered a valid resource (Jeyaraman et al. 2023; T. H. Kung et al. 2023).
While ChatGPT provides revolutionary access to knowledge and information for medical research, it is necessary to investigate the accuracy of the information provided to ensure the integrity of research in healthcare (Bhattacharyya et al. 2023; Fayed et al. 2023). ChatGPT has been shown to have some drawbacks, including the generation of text/information which seems plausible but deviates from fact which are referred to as hallucinations (Preiksaitis and Rose 2023; Sharun et al. 2023). The possibility of hallucinations poses a threat to the advancement of scientific research as well as to patient care due to its inaccuracies (Bhattacharyya et al. 2023; Fayed et al. 2023; Jeyaraman et al. 2023). The use of AI continues to expand in the field of orthopaedics (Myers et al. 2020). A 2018 systematic review by Cabitza et al asserts nearly a tenfold increase in the use of AI in Orthopaedics since 2010.(Cabitza et al. 2018) As with the use of any technology in healthcare, the responsibility falls on the operator to ensure ethical use and application of information provided by AI platforms such as ChatGPT (Myers et al. 2020).ChatGPT continues to become widely utilized in the field of orthopaedic surgery due to its efficiency and ability to produce easily digestible information (Myers et al. 2020). Some researchers in the field use ChatGPT for its ability to generate sources and citations for manuscripts and research presentations (Moritz et al. 2023; Preiksaitis and Rose 2023). One study found up to 75% of ChatGPT users in the healthcare field use the platform for research, making this platform specifically pertinent to study in the context of generating research citations (Ozkan et al. 2025). However, the legitimacy of the sources referenced, and accuracy of information provided by ChatGPT has been questioned (King and chatGPT 2023). Therefore, an investigation into the reliability of these sources and generated information from ChatGPT is necessary to ensure ethical research practices in the field of orthopaedic surgery. Therefore, the purpose of this study was to evaluate the validity of ChatGPT as a resource to assist orthopaedic researchers in their exploration of published literature and credible sources.
2. METHODS
Inquiry Breakdown
The hip, knee, and shoulder were chosen to investigate as they are commonly injured joints seen in orthopaedic sports medicine. The specific conditions were chosen based on the most common topics for sports medicine on OrthoBullets (www.orthobullets.com). For the hip, the three conditions chosen were labral tears, femoroacetabular impingement, and snapping hip syndrome. For the knee, anterior cruciate ligament (ACL) tears, meniscal tears, and medial patellofemoral ligament (MPFL) tears. For the shoulder, rotator cuff tears, bankart lesions, and adhesive capsulitis were chosen.
ChatGPT Search
ChatGPT-3.5, an OpenAI chatbot (www.chat.openai.com), is an open-source software which is freely available to the public and was launched in November 2022. ChatGPT-4.0 was released in March 2023 as an updated version to ChatGPT-3.5, however, requires a monthly subscription. AI producers claim ChatGPT-4.0 to be more advanced, making fewer errors and capable of more complex tasks tested with academic benchmarks such as the bar exam, and also reduce the incidence of hallucinations (https://openai.com/index/gpt-4-research/). Both ChatGPT-3.5 and 4.0 were queried in this study to evaluate the possible differences between versions. Due to the evolving nature of AI chatbots, the platforms were probed as quickly as possible by 3 authors over the course of June 2024. The following prompt was input into ChatGPT: “Write an outline for an orthopaedic surgery research presentation on ACL tears, include at least 10 references.” The outline itself was disregarded and the 10 references provided were recorded. This process was then repeated, but the term “ACL tears” was replaced by each subsequent condition (example: rotator cuff tears, meniscus tears, bankart lesions, etc.). The subsequent references were recorded for all queried conditions. These prompts were input in ChatGPT-3.5 and 4.0 for their respective conditions and the references were recorded which resulted in a total of 180 references (n=180).
Reference Categorization
Each recorded reference was then searched in PUBMED, Google, and Google Scholar to evaluate the source’s legitimacy. If the reference was found, the following information was recorded: the journal in which the article was published, the journal’s impact factor, author list, and publication year. The mentioned references were then identified, verified, and cross-referenced against the ChatGPT provided citation (respectively, for each version). References were categorized as follows: “Does not exist”, “improperly cited”, or “properly cited”. References that did not exist did not return a published study in any of the three databases. References that were categorized as “improperly cited” returned legitimate studies when the ChatGPT-provided citation was searched in at least one of the databases, but the ChatGPT-provided citation inaccurately cited the article title, authors, year, or journal. The authors’ categorization did not mark any citation as “improperly cited” based on grammar, style or spelling. Content such as journal name, title, year published, and correct authors were considered “improperly cited” if one or more of these elements did not match the actual manuscript citation. In contrast, references that were categorized as “properly cited” returned legitimate studies and the ChatGPT-provided citation accurately cited the article title, authors, journal, and year. Figure 1 shows a flowchart of the methodology used in this study.
3. RESULTS
Of the 180 references provided, 58/180 (32.3%) references did not exist, 36/180 (20%) were improperly cited, and 86/180 (47.8%) were properly cited. For the ChatGPT 3.5 searches, 29/90 (32.2%) did not exist, 18/90 (20%) were improperly cited, and 43/90 (47.8%) were properly cited. While the ChatGPT 4.0 searches produced different citations, it had the exact same breakdown of properly, improperly cited, and did not exist as ChatGPT 3.5. (Figure 2).
Knee
For ACL tears, 5/20 (25%) did not exist, 6/20 (30%) were improperly cited, and 9/20 (45%) were properly cited. For meniscus tears, 3/20 did not exist (15%), 4/20 were improperly cited (20%), and 13/20 (65%) were properly cited. For MPFL tears, 8/20 (40%) did not exist, 6/20 (30%) were improperly cited, and 6/20 (30%) were properly cited. (Figure 3).
Shoulder
For rotator cuff tears, 2/20 (10%) did not exist, 2/20 (10%) were improperly cited, and 16/20 (80%) were properly cited. For adhesive capsulitis, 5/20 (25%) did not exist, 4/20 (20%) were improperly cited, and 11/20 (55%) were properly cited. For Bankart lesions, 4/20 (20%) did not exist, 8/20 (40%) were improperly cited, and 8/20 (40%) were properly cited. (Figure 4)
Hip
For labral tears, 11/20 (55%) did not exist, 4/20 were improperly cited (20%), and 5/20 (25%) were properly cited. For snapping hip syndrome, 15/20 (75%) did not exist and 5/20 (25%) were properly cited. For femoroacetabular impingement, 5/20 (25%) did not exist, 2/20 (10%) were improperly cited, and 13/20 (65%) were properly cited. (Figure 5)
Properly Cited Sources
In total, 42 journals were properly cited. American Journal of Sports Medicine (ASM) was referenced 15 times (35.7%). Journal of Bone and Joint Surgery (JBJS) was referenced 12 times (28.6%). Arthroscopy and Clinical Orthopaedics and Related Research (CORR) were both referenced 8 times (19%) and Journal of Shoulder and Elbow Surgery (JSES) was referenced 6 times (14.2%).
The average impact factor of all the journals cited was 9.31. The average impact factor for ChatGPT 3.5 cited journals was 7.53 and the average impact factor for ChatGPT 4.0 cited journals was 10.9. The highest impact factor journal cited was the New England Journal of Medicine (NEJM) with an impact factor of 158.5. The lowest impact factor journal cited was the Instructional Course Lectures with an impact factor of 0.7. There was no difference in reference accuracy with increasing journal impact factor. Meaning across all impact factors of the journals cited, the likelihood of producing a hallucination was the same.
4. DISCUSSION
To our knowledge, this study is one of the first of its kind in the field of orthopaedic surgery, in providing objective evidence that citations generated by AI tools such as ChatGPT may be inaccurate. While this study was specifically done within the field of orthopaedic surgery, ChatGPT-generated inaccuracies have been studied across the medical field and within scientific writing (Athaluri et al. 2023; Bhattacharyya et al. 2023; Ghanem et al. 2024). Overall, this study provides value to the emerging and intersecting fields of AI and scientific research by understanding that AI tools such as ChatGPT are not foolproof. Therefore, individuals should remain diligent in checking their citations and ensuring their overall accuracy because without it, the risk of unintentional use of falsified citations increases. However, most importantly, the cornerstone of research and providing accurate credit to existing ideas falter and may ultimately lead to the troubling trend of misrepresentation and falsification.
Primarily, our results demonstrated that 52.3% of the references generated by ChatGPT either did not exist or were improperly cited. This result is particularly concerning given the growing widespread use of ChatGPT in the medical field and by students in general (Ganjavi et al. 2024). A recent review study examining the use of ChatGPT within medical research identified the following areas as being influenced by AI: refining medical reports, creating literature reviews on healthcare topics, and aiding with data analysis (Ruksakulpiwat et al. 2023). Additionally, Aydin and Karaarslan in 2022, asked ChatGPT to produce a literature review on a particular healthcare concept using literature that was selected and specifically input by the authors (Aydın and Karaarslan 2022). The portions of the review written by the authors had low levels of plagiarism compared to the portions written by ChatGPT which had high levels of plagiarism, as measured by the Ithenticate tool (Aydın and Karaarslan 2022). Thus, demonstrating while ChatGPT can be useful, it should also be used with caution and should not be used without human involvement given the high chance of plagiarism. Our results show that while 47.8% of the generated references were correct, the fact that 52.3% were incorrect should give individuals a warning they should not rely solely on ChatGPT. It further demonstrates the importance of human involvement within the research process and highlights the need to double-check information provided by ChatGPT.
More recently, ChatGPT has gained significant popularity in the orthopaedic research community. Many papers, in a plethora of subspecialties including total joint arthroplasty (TJA), sports medicine, spine, and hand, have investigated its reliability to answer specialty-focused questions versus the standard web-search tool Google (Croen et al. 2024; Dubin et al. 2023; Megalla et al. 2024; Nian et al. 2024; Tharakan et al. 2024). Overall, these studies found that ChatGPT, while not completely similar to Google, did provide satisfactory responses to most specialty-focused questions. Another study evaluated ChatGPT’s ability to perform on the Orthopaedic In-Training Examination (OITE) (J. E. Kung et al. 2023). They investigated both ChatGPT 3.5 and 4.0, and input text-only questions from OITEs 2020-2022. The authors found ChatGPT 3.5 answered 54.3% correct which was similar to the percent correct of a postgraduate year 1 (PGY-1) resident, and ChatGPT 4.0 answered 73.6% correct which was the average percent correct of a PGY-5. The authors also noted that ChatGPT 4.0’s percentage correct on each year’s OITE corresponded to a passing score for the American Board of Orthopaedic Surgery Part I Examination (ABOS) (J. E. Kung et al. 2023). ChatGPT’s introduction has provided a new dimension to orthopaedic research and the number of ChatGPT-related papers in orthopaedics will likely continue to rise.
Literature specific to the use of ChatGPT in scientific writing has identified its useful capabilities and potential drawbacks (Dave et al. 2023; Huang and Tan 2023). Huang and Tan argue that its pros include saving time, developing outlines, improvements in writing style, and assisting those who do not speak English or for whom English is not their first language (Huang and Tan 2023). In contrast, the list of potential drawbacks includes overreliance on the tool, inability to understand complex healthcare topics, cost, and inaccurate information (Huang and Tan 2023). Notably, they suggested that ChatGPT can assist in citation and referencing as they wrote “ChatGPT can assist scientists in accurately citing and reference their sources by generating the appropriate citation format and suggesting related articles to cite” (Huang and Tan 2023). While ChatGPT may be helpful in creating a citation if the input includes all relevant information and the desired citation style (APA, AMA, etc.), our study provides objective evidence that ChatGPT is not reliable for suggesting related articles to cite or providing accurate references.
Despite the inaccurate results demonstrated by our study, there were also positive results. First, the fact that 47.8 % of ChatGPT-generated references were accurate shows optimism and great potential for AI tools such as ChatGPT moving forward. Furthermore, this accuracy was achieved with a rather vague inputted command of “Write an outline for an orthopaedic surgery research presentation on ACL tears, include at least 10 references”. One could argue had the inputted command been made more specific, the accuracy of these generated references may have increased. For example, had the command included text such as “Write an outline…include at least 10 PubMed indexed references”, the results may have been slightly different. Moreover, our study is not without limitations. As mentioned above, the original inputted command was not as descriptive as it could be, and tools like ChatGPT, which are trained on pattern recognition, likely benefit from overly specific rather than vague commands. More specific input criteria into the chatbot such as suggesting including only high impact journals may have yielded more accurate results. Future studies could help determine exactly what inputs would improve results.
Additionally, further research is necessary to determine whether the prompt itself, including factors such as question length or word choice, or user skill influenced the rate of erroneous citations.
5. CONCLUSION
Conducting research is a time-consuming process that relies on the accurate use of references and citations so past work is properly acknowledged. With the increasing popularity of AI tools such as ChatGPT that provide lots of information in a significantly shorter amount of time, one may wonder about the accuracy of its generated references. Our study demonstrated that only 47.8% of the ChatGPT-generated references on nine common orthopaedic conditions were accurate, which meant that the other 52.3% were either incorrect or did not exist. These results highlight that while ChatGPT can be useful in assisting in the research process, they underscore the need for human involvement in double-checking the accuracy of the provided information. Overall, there is great potential for the use of AI and with time and constant refinement these tools will likely improve, but the need for human involvement will never be diminished.
Fundings
Not applicable
Conflicts of Interest
Not applicable
Availability of data and materials
Not applicable
Ethics Approval
Not applicable
Author Contributions
Authors Mumtaz, Stevens, and Connors contributed significantly to the data collection; Authors Hahn, Stevens Mumtaz and Connors contributed significantly to the drafting of the work; Authors Hahn, Grace, Corvi and Megalla contributed significantly to revision of the work as well as interpretation of stats and conception of the idea for the study; Authors Partan and Coyner contributed significantly for revision of the final work as well as approval to be published. All authors agree that they contributed significantly to the project as a team and are accountable for accuracy and integrity of all parts of the work.