Introduction
Many medical societies hold annual meetings which showcase the latest in clinical practice, techniques, and research while offering medical professionals new educational opportunities (Kearney et al. 2019). The Arthroscopy Association of North America (AANA) and the American Orthopaedic Society for Sports Medicine (AOSSM) are widely regarded as two of the most prominent societies within orthopaedic sports medicine, each hosting their own annual meeting (Kinsella et al. 2015; Frank et al. 2017; Baweja, Kraeutler, and McCarty 2018).
Literature presented at these meetings is disseminated as presented abstracts or through lectures by selected content speakers. As lectures have the potential to influence clinical decision-making, an active role in recent literature may demonstrate that the speaker is well-versed in recent technological innovations. Previous studies have evaluated the future publication rates of presented abstracts at sports medicine annual meetings, but no study was identified that evaluated the publication history or associated academic metrics of their speakers (Kinsella et al. 2015; Frank et al. 2017; Baweja, Kraeutler, and McCarty 2018; Lehman et al. 2017). Hyer et al attempted to objectively measure the quality of selected speakers by evaluating speakers’ previous peer-reviewed publications on their presentation topic at foot and ankle society meetings. In their review, the majority of selected content speakers at prominent foot and ankle society meetings were not previously published on their presented topic (Hyer et al. 2022). While prior literature has suggested the use of topic-specific publications as a metric for gauging speaker quality, meeting planning committees likely consider a number of factors such as prominence in the field, prior high-quality lectures, society affiliation, and employment at academic centers.
In order to determine whether AANA and AOSSM rely on publication history and associated academic metrics during content speaker selection for the 2016-2019 annual meetings, the authors sought to answer the following questions: 1) What percentage of content speakers at sports medicine annual meetings have previous peer-reviewed publications on their presented topic?; 2) What are the publication-associated academic metrics of selected content speakers?; 3) Which content categories lack speakers with previous peer-reviewed publications on their presented topic?
Methods
The programs of 2016-2019 AANA and AOSSM annual meetings were accessed through the organization’s website or by contacting the organization when programs were not publicly available. Institutional Review Board (IRB) approval was not required for this review. Three authors independently screened each meeting program to identify selected content speakers. Only speakers who held a Doctor of Medicine (MD) or Doctor of Osteopathic Medicine (DO) degree were included. Content speakers were included if their speaker-assigned lectures were listed under the “Feature Lecture,” “Symposium,” “General Session,” and “Concurrent Session” subheadings in the final programs. Oral abstract and poster presentations, debates, instructional course lectures, industry or society-sponsored presentations, cadaveric workshops, medical-legal discourses, and business content were excluded to focus on those lectures given by invited speakers.
Prior peer-reviewed publications by selected content speakers were identified by searching the authors’ first and last names in the Scopus database (Amsterdam, Netherlands). For an identified manuscript to be included, the date of publication had to precede or include the year of the respective annual meeting. Conference papers, data papers, editorials, erratum, letters, notes, retracted articles, and short surveys were excluded. Abstracts of publications were assessed for relevance to the specific topic, and their full-length texts were subsequently reviewed to confirm relevance. Full-length manuscripts that were specific to the presented topic were included. For example, if the title of the lecture was “Meniscal Root Repair”, only publications pertaining to meniscal root repair were included. If the title was broader, such as “PCL,” then all publications pertaining to the posterior cruciate ligament were included. Authorship order of previous publications was not considered, and any authorship listing was considered a previous publication. Each author’s Scopus-listed Hirsch index (H-index), a metric that measures an author’s overall impact and productivity, specific to the year of the annual meeting, was recorded as listed on Scopus. It is defined as the number of publications with a citation number ≥ h . (Hirsch 2007, 2005; “Welcome to Scopus Preview,” n.d.) Hirsch states that after 20 years of scientific activity, an H-index of 20 is successful, 40 is outstanding, and 60 is “truly unique.” (Hirsch 2005) A weighted citation count (WCC) was calculated for each topic-specific publication by dividing the number of citations by the number of years since publication (Hyer et al. 2022). A WCC was used to assess the academic impacts of each prior publication. While manuscripts published at two different time points may have the same number of citations, the publication that is more recent would have a higher WCC, and thus an increased academic impact.
Each society designated lectures in their respective programs by content subcategories, but these subcategories were not aligned between societies. To maintain consistency, a total of nine content categories were created for which each lecture was assigned by the reviewers: biologics, elbow, foot and ankle, hand and wrist, hip, knee, nonoperative sports medicine, pediatrics, and shoulder. The nonoperative sports medicine category included content regarding injury prevention, performance enhancement, concussion management, and non-musculoskeletal topics. Lectures were assigned to the category most representative of their topic, with disputes determined by a third reviewer when necessary. The categories were analyzed for the average number of speakers, H-index, number of previous topic-specific publications, and WCC.
The number of lectures, H-index, number of previous topic-specific publications, and WCC for each society and topic were analyzed using means and proportions. Comparisons were performed with two-sample T-tests for continuous variables and two-sample Z-tests for proportions for categorical variables, with p-values less than 0.05 indicating statistical significance.
Results
A total of 432 lectures met inclusion criteria and were eligible for final analysis. Overall, 371 lectures (86%) were given by speakers who previously published on their presented topic, while only 61 lectures (14%) were given by speakers who were not previously published on their presented topic (p<0.001). The majority of selected content speakers had between 1-10 prior topic-specific publications (58%), followed by those with more than 20 previous topic-specific publications (11%).
The most commonly presented lecture categories were knee (30%, 129/432), shoulder (30%, 128/432), and nonoperative sports medicine (16%, 71/432). Hand and wrist was the least frequently presented topic (0.7%, 3/432). Nonoperative sports medicine had the highest percentage (34%, 24/71) of lectures delivered by speakers without prior topic-specific publications, followed by foot and ankle (13%, 2/15) and biologics (13%, 2/16).
The average H-index of speakers was 35.5. Speakers presenting on the hip held the highest average H-index (41.7). The average number of prior topic-specific publications by selected content speakers was 9.6. Speakers presenting on the elbow had the most prior topic-specific publications (12.2). The average WCC of speakers was 4.8. Those presenting on biologics had the highest average WCC (8.6). (TABLE 1)
Among AANA and AOSSM speakers, nonoperative sports medicine was the only category with more than 20% of speakers having no prior topic-specific publications (29% and 35%, respectively). AOSSM speakers had a greater average H-index than speakers at AANA (37.3 v. 32.5, p=0.027). Otherwise, there was no difference between meetings in the proportion of speakers without prior topic-specific publications (16% v. 13%, p=0.412), the total average number of prior topic-specific publications (9.4 v. 9.7, p=0.882), or the total average WCC (5.3 v. 4.5, p=0.146). (TABLE 2) (TABLE 3)
Discussion
Annual meetings have the potential to impact clinical practice, and thus it is important to ensure that speakers are active in current literature. While it may be assumed that orthopaedic societies value prior publication when selecting content speakers for their annual meetings, recent literature has suggested that other factors may play a larger role in speaker selection (Hyer et al. 2022). Using the presence of prior topic-specific publication as an objective measurement of speaker quality, Hyer et al found that roughly 71% of guest speakers at prominent foot and ankle society annual meetings were not previously published on their presented topic (Hyer et al. 2022). The previous publications and associated academic metrics of selected content speakers at orthopaedic sports medicine meetings had not been previously assessed.
Our analysis demonstrates that the majority of selected content speakers at orthopaedic sports medicine society meetings have prior peer-reviewed publications on their topic of presentation. Each conference had more than 80% of speakers with prior topic-specific publications. In contrast to the previous review of foot and ankle society meetings, both sports medicine societies demonstrated a much smaller proportion of selected content speakers without prior topic-specific publications (Hyer et al. 2022). However, publication history cannot be considered a reliable metric of speaker quality, as the specific criteria utilized by individual societies for speaker selection may or may not include publication history.
Categories with the fewest number of speakers without previous topic-specific publications included pediatrics (0%), hand and wrist (0%), and elbow (8%). However, these categories also constituted the lowest number of total speakers. While these categories are presented infrequently at the sports medicine society meetings, they are more likely to be presented by speakers with prior topic-specific publications. Conversely, the categories with the highest proportion of speakers with no prior topic-specific publications were nonoperative sports medicine (34%), foot and ankle (13%), and biologics (13%). Despite foot and ankle lectures having a relatively high proportion of speakers without prior publication, this percentage of speakers with a publication history was greater than previously reported on foot and ankle society meetings (Hyer et al. 2022). However, the majority of speakers in these content categories exhibited prior topic-specific publications with comparable average H-indexes and WCC to other categories. To ensure adequate incorporation of recent innovations within nonoperative sports medicine, foot and ankle, and biologics, greater consideration of the speaker’s previous publications in these content areas may improve future annual meetings.
There were no differences between the AANA and AOSSM annual meetings in regard to the proportion of speakers without previous topic-specific publications, the total average number of prior topic-specific publications per speaker, or the total average WCC. As previously suggested, Hirsch et al. reports that an H-index of 20 is considered successful (Hirsch 2005). Both AANA and AOSSM speakers had an average H-index over 30, suggesting that speakers were highly successful in academic research. While AANA and AOSSM did differ in H-index, both conferences exhibited superior H-indexes, number of prior topic-specific publications, and WCC compared to the findings reported by Hyer et al (2022). These findings suggest that AANA and AOSSM program committees may emphasize publication history and academic metrics more than their foot and ankle counterparts.
Disparities in publication-specific metrics between society meetings may be attributed to the selection criteria of each individual meeting committee. While previous research may be one factor that is considered when selecting speakers for national meetings, other factors that may influence committee decisions include prominence in the field, prior high-quality lectures, societal affiliation, or employment at academic centers. Lectures by prominent orthopaedic surgeons and speakers with a history of high-quality presentations may drive meeting attendance and thus are likely to be valued by the meeting committees. Society affiliation can also exert influence, as societies may be more apt to extend invitations to individuals who have displayed committed engagement with the society. Additionally, physicians affiliated with academic centers possess valuable experience that may contribute to the quality of lectures, including their active engagement in research and involvement in the teaching of medical trainees. While we may speculate on what influences committee decisions, each society likely has its own unique formula for speaker selection based on years of experience in delivering high-quality meeting programs.
This study is not without limitations. Categories were created by the authors and thus subjectivity may exist in classification. Although a detailed review of programs was performed by two authors, there may have been previous publications that were not properly identified or incorrectly included in the analysis. To minimize this limitation, a third reviewer was queried for disagreements between categorization of the other two reviewers. Due to the exclusion of all speakers without an MD or DO degree, it is possible that published speakers without these degrees were omitted. Our methods did not account for the possibility of ongoing research that may be known to members of the program planning committee. To account for emerging research for which the speaker may be discussing, authors included all manuscripts published within the same year as the annual meeting. While most of the categories were well represented, three categories (hand and wrist, elbow, pediatrics) had less than 15 speakers, and thus the data may be underpowered to detect statistical significance. Although this review assessed the academic metrics of selected content speakers, authors acknowledge that a variety of other factors likely contribute to speaker selection. Individual program planning committees may value clinical practice experience and those with organization affiliations. These factors, as well as the surgeon’s practice setting (academic versus private), may influence selection but were not evaluated in this study. While authors acknowledge the tremendous importance of racial and gender equality within content speakers, these factors were not evaluated as authors would have had to assume gender and ethnicity, as this information is not often readily available.
Conclusion
AOSSM and AANA annual meetings largely include selected content speakers with prior peer-reviewed literature on their presentation topic. While prior publications and associated academic metrics may be a factor in speaker selection, each society likely has its own individual selection criteria for delivering high-quality programs.