Introduction
Traditional manuscript publishing employs a reader-pay model to access journal content. Conversely, open access (OpAcc) publication involves free-of-charge access to peer-reviewed online articles for readers, where authors typically bear the cost of publishing (Laakso et al. 2011). The four main OpAcc publishing models include hybrid, open, closed license, and open repository, grouped in a tier system based on article fees (Table 1). Gold journals (open journals), where the cost of publishing is borne by the author, involve free-of-charge access for readers and allow unrestricted access and sharing of articles through a creative commons license (Jackson and Richardson 2014). Hybrid journal articles are an intermediate form of OpAcc and the reader-paid model, where authors pay a fee to make their articles freely accessible to readers within an otherwise traditional journal (Kim and Atteraya 2023; Zhang et al. 2022). Bronze journals (closed license) provide free access for readers to journal content but often do not provide an open license for sharing and reuse (Zhang et al. 2022). Since publishers retain the copyright, they can choose to revoke free access to journal articles at any time. Lastly, green journals (open repository) offer free-of-charge access to articles for readers and allow authors to deposit manuscripts into a repository such as Zenodo or institutional archives at no cost (Björk et al. 2014). However, the version of the manuscript that is deposited may be subject to changes by the publishing company.
Multiple studies have recognized the steadily growing number of publications in OpAcc journals as compared to traditional journals over the past two decades (Buchanan et al. 2024; Van Noorden 2013; Laakso and Björk 2012; Piwowar, Priem, and Orr 2019; Root et al. 2024; Cueto et al. 2023). Contributing to this change is the increased citation rate for OpAcc articles compared to those in non-OpAcc journals (Cueto et al. 2023; Piwowar, Priem, Larivière, et al. 2018). As the volume of scientific literature continues to grow, researchers seek to quantify the impact of an article in its respective field, commonly done by assessing the citation rate (Vucovich, Blaine Baker, and Smith 2008). When evaluating the relationship between OpAcc publishing and research impact in total knee arthroplasty (TKA) literature, one study found an associated increase in social media attention but no statistically significant effect on citations (Cueto et al. 2023). Moreover, another study evaluated the relationship between OpAcc publication type and resulting citations, readership, and social media attention metrics in TKA literature. The findings indicated that hybrid journals were more cost-effective than open journals across all evaluated metrics (Buchanan et al. 2024). Such research elucidates the exact benefits of OpAcc publication in TKA literature for scholars looking to publish in the field. In a 2023 study examining recent 5-year trends in TKA and total hip arthroplasty (THA) literature, Poursalehian et. al (Poursalehian, Javadzade, and Mortazavia 2023; Poursalehian et al. 2023) found fewer THA publications and citations compared to TKA literature. Given the disparities in publication volume and citation rates, findings from TKA research may not necessarily translate to the field of THA. Another difference in these fields’ literature is that THA literature has yet to explore the associated cost and readership between the four major OpAcc modalities.
Researchers seeking to publish in the field of THA need to consider these distinctions to help determine if the investment in OpAcc publication results in a meaningful difference in readership and citations. Thus, this study aimed to identify which OpAcc types within THA literature exhibited: the highest mean citations per article; the highest mean readership per article; the highest mean attention per article; and the greatest cost-adjusted values for citations, readership, and attention among these modalities.
Methods
Inclusion and Exclusion Criteria
To collect peer-reviewed THA literature, a PubMed search was performed for articles published from May 2016-May 2023. This search used the terms (((“Hip”) AND (“Arthroplasty” OR “Replacement” OR “Implant”)) NOT (“Hemi” OR “Hemiarthroplasty” OR “Partial”)). Any works other than peer-reviewed published journal articles were excluded, including data sets, news articles, preprints, books, and clinical trials without published outcome data.
Database
The Altmetric database was used to collect the following data from each article: title, journal, publication date, OpAcc group, Altmetric Attention Score (attention), Mendeley Readership Score (readership), Dimensions citations (citations), and mentions through various media modalities: news sources, Wikipedia, blogs, X, Facebook, Google+, Reddit, YouTube, policies, peer review platforms, patents, syllabi, and Stack Exchange. The weighted attention score quantifies the previously noted media mentions (Altmetric 2021). Mendeley tracks the number of times an article is added to researchers’ libraries as readership (Bonasio 2014). This serves as a proxy for how many researchers have read each article with plans to use the article in their research. Number of citations were quantified using Altmetric dimensions citations which utilizes sources like PubMed, CrossRef, and data directly from publishers to measure mentions in research related contexts like pre-print proofs, publications, clinical trials, grant applications, and conference proceedings (“Why Did We Build Dimensions,” n.d.; Altmetric 2020).
Articles were grouped using the Altmetric Explorer advanced search tool as outlined above and redefined here: a publication made available for free with an open license in a subscription journal (hybrid journal), an article in a completely OpAcc journal (open journal), a copy of a manuscript posted in an OpAcc repository (open repository), and a document made available by a publisher without an open license (closed license). Articles published in the top seven orthopaedic journals—The American Journal of Sports Medicine, Arthroscopy: The Journal of Arthroscopic & Related Surgery, The Journal of Bone & Joint Surgery, Knee Surgery, Sports Traumatology, Arthroscopy, The Journal of Arthroplasty, The Bone & Joint Journal, and Clinical Orthopaedics & Related Research—were grouped for regression analysis to account for the influence of these journals. This was based on Google Scholar’s H5-index on May 21, 2023, when the database search was performed, which quantifies the cumulative citations of a journal in the previous five years to measure the journal’s impact (Google, n.d.). Although the top 10 orthopaedic journals were initially considered for this regression analysis, the European Spine Journal, Spine, and the Journal of Shoulder and Elbow were excluded since they do not publish in THA literature. This subgroup analysis was conducted to evaluate how distinct open access modalities perform within high-impact journals, where baseline visibility and dissemination may differ from the broader THA literature.
Statistical Analysis
Adjustments were made for the time-based increase in attention, citations, readership, and media mentions via negative binomial regression. Each article’s mentions were normalized by days since publication. Two-sample t-tests were used to calculate means, standard deviations, and 95% confidence intervals. All statistical analyses were performed using Stata/SE 17.0 (StataCorp LLC, College Station, TX, USA) and a significance of P<.05. Each variable was analyzed for intragroup similarity using Tukey’s test (α=.05). Tukey’s test compares all of a cohort’s means at once to test for statistical differences between the group as a whole (Haynes 2013).
Adjusting for Cost
The OpAcc article types with no fee (Closed license and Open repository) were excluded from cost analysis. To estimate average cost per article, the mean fee per OpAcc type was established based on those for open journal ($2097 United States Dollars [USD]) and hybrid journal articles ($2727 USD) in a report from Björk and Solomon from 2014 evaluating various funding strategies in helping achieve an effective market for article fees in OpAcc publishing (Björk and Solomon 2014).
The mean citations, attention, and readership were divided by the mean fee for each OpAcc type in a cost ratio per $1000 (i.e., citations/$1000, attention/$1000, readership/$1000). To estimate average cost per article in the top 7 orthopaedic journals, the mean fee of $3,624 was calculated based on the article processing charges listed on each journal’s official website at the time of data collection. The mean citations, attention, readership scores were then divided by the mean fee in a cost ratio per $1000. All costs were reported in USD as above.
Results
Article Grouping
The initial PubMed search yielded 9405 publications. Of these, 44.8% (4217) were OpAcc. Further grouping into OpAcc categories showed that 2315 articles were open journal (54.9% of THA OpAcc literature, 24.6% of all THA literature), 490 were hybrid journal (14.0% THA, 6.26% overall), 669 were closed license (15.9% THA, 7.11% overall), and 743 were open repository (17.6% THA, 7.90% overall).
OpAcc Type Analysis
Closed license articles had the highest mean citations while open journal articles had the lowest mean citations (12.04±15.16 and 7.61±10.66, respectively). Open repository articles had the second highest mean citations while hybrid journal articles ranked third (11.48±14.80 and 9.67±12.97, respectively). (Table 2)
Open repository articles had the highest mean readership value while open journal articles had the lowest mean readership (31.74±29.32 and 23.54±25.07, respectively). Closed license articles had the second highest mean readership while hybrid journal articles ranked third (28.98±27.11 and 27.03±27.48, respectively). (Table 2).
Hybrid journal articles had the highest mean attention value while open journal articles had the lowest mean attention (8.11±16.05 and 4.27±9.24, respectively). Open repository articles had the second highest mean attention while closed license articles ranked third (7.70±15.01 and 7.07±13.78, respectively). (Table 2)
Tukey’s test showed variance between groups for citations, attention, and readership (P<0.001). Closed license averaged greater citations than hybrid journal articles (P=.009) (Table 3). Hybrid journal, closed license, and open repository articles averaged greater citations, attention, and readership compared to open journal articles (P≤.045).
Cost Analysis
Compared to hybrid journals, open journal articles had higher mean citations/$1000 (3.629 vs 3.544) and readership/$1000 (11.226 vs 9.910) (Table 4). Hybrid journal articles had a higher mean attention/$1000 than open journal articles (2.973 vs 2.038).
Top 7 Analysis
Among articles published via OpAcc modalities in the top 7 orthopaedic surgery journals, open repository articles achieved the highest mean tested values in all evaluated metrics, including citations, readership, and attention (Table 5). Each OpAcc modality exhibited similar citations, attention, and readership values apart from a significantly higher readership in open repository compared to closed license articles (33.98 vs 27.05, P=.030) (Table 6). In the cost analysis of all hybrid journal articles as compared to hybrid journal articles from the top 7 orthopaedic journals, overall hybrid journal articles had the highest mean attention/$1000 and readership/$1000, while hybrid articles from the top 7 ortho hybrid journal had the highest mean citations/$1000 of the paid OpAcc modalities (Table 7) (“The American Journal of Sports Medicine,” n.d.; “The Journal of Arthroplasty,” n.d.; “The Journal of Bone and Joint Surgery,” n.d.; “Knee Surgery, Sports Traumatology, Arthroscopy,” n.d.; “Clinical Orthopaedics & Related Research,” n.d.; “The Bone and Joint Journal,” n.d.; “Arthroscopy: The Journal of Arthroscopic & Related Surgery,” n.d.).
Discussion
While OpAcc publishing has gained widespread popularity over the past two decades, the value of paying for associated fees was previously under investigated in THA literature (Piwowar, Priem, Larivière, et al. 2018; Laakso and Björk 2016; Siler et al. 2018). Researchers seeking to publish in THA literature may benefit from understanding how different OpAcc modalities, with varying levels of article processing charges, affect the reach of their work. The findings from this study highlight the varied impact of these modalities on the dissemination of THA research based on attention, readership, and citations. Open journal articles showed the lowest average metrics across all the OpAcc types assessed while hybrid journal articles garnered the highest attention value, open repository articles had the highest readers, and closed license articles showed the highest citations. When comparing paid (open and hybrid journal) vs unpaid OpAcc types (closed license and open repository), unpaid modalities had higher readers and citations. Among paid modalities, open journals had the best cost per reader and citation, while hybrid journal articles were more cost-effective for increasing attention. Among the top 7 orthopaedic journals, open repository articles achieved the highest average values across all evaluated metrics. The superior metrics of open repository articles within top journals may result from the combined effect of higher journal impact factors and unrestricted access, which together broaden dissemination. In contrast, open repository articles in the overall cohort come from journals with more variable impact, potentially leading to less consistent influence despite being freely accessible.When comparing all hybrid articles to those specifically from the top 7 orthopaedic journals, hybrid articles from the top 7 orthopaedic journals had a lower cost per citation but a higher cost per reader and attention. This differing effect may reflect variation in journal audiences, where hybrid articles in high-impact journals are perhaps more likely to be cited due to their academic reach but may attract comparatively lower levels of media attention and casual readership than hybrid articles in the broader literature.
Closed license and open repository articles, involving free-of-charge publishing for authors, both demonstrated higher mean readership and citation scores compared to paid OpAcc modalities, open and hybrid journals (Table 2). The possible factors contributing to this distinction have previously been explored in other contexts. In a study examining the prevalence and impact of OpAcc articles throughout scholarly literature, Piwowar et al (Piwowar, Priem, Larivière, et al. 2018) found a steadily decreasing citation rate over time for open journal articles, a paid modality, compared to other OpAcc types. This trend may reflect findings from Siler et al (Siler et al. 2018) stating that authors working at lower-ranked institutions were more likely to utilize free-to-publish modalities. This may lead to less funded but significant studies being published through nonpaid routes, increasing the citation rate of these free-to-publish modalities. Also, in a study evaluating the evolution of OpAcc between 2008 and 2013, Archambault et al (Archambault et al. 2013) found that the younger and smaller nature of open journals adversely affected their citation rate, as authors may have preferred to read and cite from more established journals. An additional factor potentially influencing the high readership and citation values of unpaid OpAcc modalities is self-archiving within open repositories. Self-archiving involves the submission of a peer-reviewed journal article that was previously published via another route to an open repository (Björk et al. 2014). In a study evaluating the impact of OpAcc in the context of global health research, Smith et al (Smith et al. 2017) showed that self-archiving allowed the published work to be viewed and cited by a broader audience, augmenting the open repository citation rate. Moreover, in a study evaluating the citation advantage of OpAcc articles, Eysenbach (Eysenbach 2006) noted that prominent authors were often affiliated with leading institutions that were more likely to have an institutional repository, and that authors may have chosen to self-archive their best work, increasing the citation rate of this free-to-publish modality. Collectively, these studies outline several factors that may enhance citation and readership rates for free-to-publish OpAcc types. Considering the increased values in these analyzed metrics, further research investigating how these modalities affect scholarly dissemination throughout scientific literature is warranted.
Within paid OpAcc modalities, open journal articles exhibited higher cost-adjusted citations and readership compared to hybrid journal articles (Table 4). This finding may be attributed to the reduced cost prohibition in open journals for authors seeking to publish in scientific literature. High quality research, which may garner more citations and readers, may be published as open journal articles due to a lower article fee (Van Der Heyden and Van Veen 2018). Conversely, hybrid journal articles exhibited the highest mean attention value across all assessed modalities (Table 2). There are several potential reasons that could explain hybrid articles’ increased attention. More prestigious journals tend to utilize subscription modalities, including hybrid journals (Buchanan et al. 2024; Smith et al. 2017). Well-established institutions also tend to pursue prestige, be well-funded, and be more inclined to cover the cost of publishing in journals with higher article fees (Siler et al. 2018). This may explain hybrid journals’ increased attention values as articles by researchers from well-known institutions may garner more media attention. It is evident that authors seeking to attract significant media attention and reach an audience beyond the scientific community may find it beneficial to invest in the higher article fees associated with hybrid journals. Nonetheless, this highlights broader equity concerns, as authors from less-resourced institutions may face systemic barriers to publishing in prestigious hybrid journals, which often serve as gateways to academic recognition and career advancement.
Based on the previously discussed advantages of hybrid articles, authors in THA literature prioritizing prestige may consider hybrid publishing in a top 7 orthopaedic journal (Cueto et al. 2023). Nonetheless, our results show that the mean article fee for these top hybrid journals is markedly elevated at $3,624 compared to the mean article fee for all hybrid journals at $2,727 (Table 7). This high article fee may be attributed to the significant prestige associated with publishing in top-tier journals and the concentrated market of major publishers (Smith et al. 2017). This limited competition among major publishers decreases incentives to lower article fees (Larivière, Haustein, and Mongeon 2015). In our cost analysis, the top 7 orthopaedic journals’ hybrid articles were more cost-effective for increasing citation rates with decreased cost-effectiveness for attention and readership compared to all hybrid journal articles. Non-top 7 hybrid journal articles garnering more attention and readership but fewer citations likely signifies that these articles may have less impact in academic research but generate widespread interest overall, which may include laypeople and those reading without intention for citing in future research such as clinicians without research pursuits. Our results help authors make informed decisions in identifying a hybrid journal for publication, depending on whether they want to maximize attention, citation, or readership, while also considering the cost involved.
Certain potential limitations for this study should be noted. Citations, readership, and attention are interrelated metrics. Articles with significant media attention may be increasingly read or cited. Similarly, journal articles cited more frequently may show increased social media attention (Shrivastava and Mahajan 2016). Since an open repository can technically include both self-archiving or primary publication, an important concern to address is the possible inflation of scores by counting statistics that should belong to other publication methods, which could lead to overestimation of resultant citations, readership, and attention for open repository articles. Altmetric explorer helps mitigate this by defaulting to classifying articles based on their original published format with an earlier publication date (Piwowar, Priem, Larivière, et al. 2018). Altmetric explorer data should also be interpreted with caution, as the amount of attention a publication garners does not always correlate with its quality (Fraumann 2018). Social media attention is not categorized by the type of attention in the Altmetric Attention Score, resulting in a higher value even if the attention is negative (Root et al. 2024). For instance, a widely criticized article with inaccurate data or controversial statements may garner significant media attention, which would only produce an additive effect on the attention metric. Additionally, certain “hot topic” issues in orthopeadics, as described by Root et al (Root et al. 2024), may show increased social media attention, but not significantly impact future research. It is also important to recognize that while Altmetric utilizes PubMed, an extensive and widely used repository of scientific publications, there may be articles that were excluded because they were only accessible on other databases (Gusenbauer and Haddaway 2020). The publication year restrictions we utilized for Altmetric data starting from 2016 were chosen conservatively based on when all seven THA journals began offering OpAcc publication and the date of data collection (Cueto et al. 2023). Moreover, exclusion of journals such as Spine and the European Spine Journal, which have increasingly been publishing literature on hip-spine syndrome, may have led to the omission of relevant total hip arthroplasty articles, potentially limiting the comprehensiveness and generalizability of our findings. Finally, the estimated article processing charges for hybrid journals were based on a 2014 analysis by Solomon and Björk (Björk and Solomon 2014), which may not reflect current pricing. More recent data published in 2022 indicated that hybrid APCs have increased by 3.6% that year, with a steady annual growth over prior years as well (“News & Views: Open Access Charges – Continued Consolidation and Increases” 2022).
Closed license and open repository OpAcc exhibited the highest average readership and citations, respectively, making them excellent choices at no cost to authors. Among paid modalities, hybrid journal articles are the most expensive OpAcc type followed by open journal articles as the second most costly option. In THA literature, authors prioritizing citations should consider publishing in closed license journals. For those seeking to maximize attention, hybrid journals are the optimal choice, while those prioritizing readership should consider publishing in an open repository. When evaluating paid modalities while considering the cost involved, hybrid journals are the modality of choice for authors prioritizing attention, while open journals are the most cost-effective option for authors seeking to maximize citations and readership.