Health Technology Assessments as an Alternative to Clinical Practice Guidelines

Introduction

Sackett first defined evidence-based medicine (EBM) 30 years ago (Sackett et al. 1996). Evidence is stratified into five levels of evidence (I-V) and four categories (therapeutic, prognostic, diagnostic, and economic and decision analyses) (Wright et al. 2003). Professional societies, such as the American Academy of Orthopaedic Surgeons (AAOS), use Grading of Recommendations Assessment, Development and Evaluation (GRADE) methodology to develop clinical practice guidelines (CPGs) regarding treatment recommendations (BMJ, n.d.). Randomized controlled trials (RCTs) are initially considered to be high-quality therapeutic evidence and are potentially downgraded for biases. Observational studies are initially considered to be low-quality evidence. The AAOS CPG development process requires high- or moderate-quality evidence to make a recommendation. PICO questions (population, intervention, comparison, and outcomes) with insufficient evidence are included as an appendix, but are not included in the CPG recommendations. CPGs developed with GRADE methodology make binary recommendations: effective or not effective. No recommendations are made for treatments that lack high- or moderate-quality therapeutic evidence and clinicians must navigate pre-authorization requirements without CPG recommendations or indications/conditions for treatment.

Insurers must determine whether or not to authorize coverage of technologies/treatments with or without moderate- or high-quality therapeutic evidence. Insurers typically use health technology assessment (HTA) reports to determine treatment coverage. With or without moderate- or high-quality therapeutic evidence for a specific technology, HTAs include observational studies or non-randomized studies of interventions (NRSIs) to determine which technologies will be covered and under what conditions. In addition to evaluating the effectiveness of a specific technology, HTA reports also consider harms and safety, differential effectiveness, and cost-effectiveness. The consideration of harms/safety, differential effectiveness, and cost-effectiveness make HTAs more robust technology reviews.

The Washington State Legislature created the Health Technology Assessment program within their Health Care Authority (HCA) (Landaas et al. 2016; Rothman et al. 2018; Ahn et al. 2020). To determine treatment coverage and conditions, the HCA utilizes the Health Technology Clinical Committee (HTCC), which “is composed of community health care practitioners; makes coverage decisions that apply to state purchased health care programs; and uses the HTA reports and other information to determine the conditions for coverage.” (Washington State Health Care Authority 2025) HTAs are developed by independent organizations contracted to complete the HTA report for each technology. Four key questions are reviewed for each technology: (1) effectiveness, (2) harms and safety, (3) differential effectiveness, and (4) cost-effectiveness. Coverage and conditions decisions are made during an open public meeting. Public comments/presentations are allowed. The contractor presents the HTA report and answers questions from the HTCC. Following an open discussion, the HTCC votes to cover the technology unconditionally, cover with conditions, or not cover the technology. If the vote is to cover with conditions, the clinical conditions required for coverage are specified.

Because observational studies are considered low-quality evidence, CPG recommendations are effectively limited to RCTs. An alternative method is needed to assess specific technologies. The purpose of this review is twofold:

1. To analyze all musculoskeletal-related HTA reports and coverage decisions by the Washington State HCA HTCC and determine the number of effectiveness studies, highest effectiveness strength of evidence, and coverage and conditions decisions since inception of the HTCC; and

2. To compare the HTCC decisions with AAOS clinical practice guideline recommendations for congruence.

Methods

Ninety-two HTA reports and HTCC coverage and conditions decisions from the Washington State HCA were reviewed from inception (first HTA report May 18, 2007) through January 31, 2025. The HTA reports are all available on-line (https://www.hca.wa.gov/about-hca/programs-and-initiatives/health-technology-assessment/health-technology-reviews). HTA reports related to musculoskeletal technologies (treatments) were reviewed and data abstracted from individual HTA reports by all three authors. Abstracted data included report date, topic, subtopic (if relevant), comparisons, number of RCTs for each decision/comparison, number of NRSIs for each decision/comparison, highest effectiveness quality of evidence (QoE) for each decision/comparison, and coverage decision. The quality of evidence was reported as high-quality evidence – 3, moderate-quality evidence – 2, low-quality evidence – 1, very low-quality evidence – 0. Early HTA reports included published meta-analyses and/or systematic reviews. Numbers of meta-analyses for each decision/comparison and numbers of systematic reviews for each decision/comparison were abstracted if used.

Descriptive statistics were used to identify the percentages of musculoskeletal HTA reports, highest strength of evidence included in each coverage decision, and frequency of coverage decisions (not covered, covered with conditions, or unconditional coverage) made by the HTCC.

When possible, HTCC coverage decisions were compared to AAOS CPG recommendations to assess the congruence between HTCC coverage decisions and AAOS CPG recommendations. Congruence was defined as agreement between the clinical practice guideline recommendation and HTCC coverage decision: AAOS recommends using a technology/treatment and HTCC voted to cover conditionally or unconditionally, or AAOS recommends against using a technology/treatment and HTCC voted not to cover.

Results

Between May 18, 2007 and December 31, 2024, 92 HTA evidence reports were developed. Thirty-eight percent of the HTA evidence reports (35/92) reports addressed musculoskeletal technologies. The 35 HTA evidence reports resulted in 61 coverage decisions (Table 1). Of the 35 HTA topics, 63% (22/35) made a single coverage decision, 11% (4/35) made two coverage decisions, 14% (5/35) made three coverage decisions, and 11% (4/35) made four coverage decisions. A PRISMA flow diagram delineates the HTA reports (Figure 1). In 2011, the HTCC switched from reporting only the majority decision for coverage to reporting the individual votes for coverage. Of the 47 coverage decisions after the change in reporting, 60% (28/47) of the coverage decisions were unanimous (Table 1).

Figure 1.PRISMA flow diagram.

Seven of the topics had two or three HTA reports and coverage decisions (Figure 1). Vertebroplasty, kyphoplasty, and sacroplasty had two reports (HTA reports 1 and 27). Knee chondral defects had two reports (HTA reports 2 and 25). Spinal cord stimulation had two reports (HTA reports 3 and 29). Hyaluronic acid and platelet-rich plasma had three reports (HTA reports 4, 15, and 21). Sacroiliac joint fusion had two reports (HTA reports 5 and 8). Femoracetabular impingement had two reports (HTA reports 7 and 26). Artificial disc replacement had two reports (HTA reports 13 and 32).

Thirty-six of the 61 (59%) reviewed technologies were not covered; 25 of the 61 (41%) reviewed technologies were covered with conditions; no technology was covered unconditionally (Table 2). Only 8% (2/25) of the technologies covered with conditions had high-quality effectiveness evidence supporting the coverage decision (Table 2). The percentage of technologies with moderate-quality, low-quality, and very low-quality effectiveness evidence supporting the coverage decision were 32% (8/25), 40% (10/25), and 12% (3/25), respectively (Table 2). Two technologies (8%) were covered with conditions based on external systematic reviews (Table 2). Seven of the first 15 HTA reports included meta-analyses and/or systematic reviews not performed by the contractor and effectiveness strength of evidence was not explicitly reported (Table 1).

Each technology applied to one or more joints (knee, hip, shoulder, elbow) and/or one or more subspecialties (spine, foot/ankle, hand, trauma). The most common subspecialties or joints that were included in the coverage decisions were spine (19/61, 31%), knee (12/61, 20%), and hip (7/61, 11%) (Table 3).

Eleven technologies had applicable AAOS CPG recommendations (Table 4). Six of the AAOS CPG recommendations were congruent with HTCC coverage decisions: (1) not using hyaluronic acid for knee osteoarthritis, (2) not using hyaluronic acid for hip osteoarthritis, (3) not routinely using hyaluronic acid for knee osteoarthritis, (4) use of computer navigation for total knee arthroplasty, (5) use of unicompartmental knee arthroplasty, and (6) not using arthroscopic debridement for knee osteoarthritis. Of the five non-congruent decisions, four had limited evidence to support the recommendation: (1) platelet-rich plasma for knee osteoarthritis, (2) denervation for knee osteoarthritis, (3) shock wave therapy for knee osteoarthritis, and (4) transcutaneous electrical nerve stimulation (TENS) for knee osteoarthritis. One recommendation regarding the use of platelet-rich plasma for knee osteoarthritis was inconclusive in the Treatment of Osteoarthritis of the Knee, Second Edition (Jevsevar 2013).

Discussion

Sackett’s initial definition of EBM included the best available clinical evidence and physician experience (Sackett et al. 1996). Decisions are not based solely on RCTs. Non-randomized studies of interventions (NRSIs) are included in the HTA reports when NRSIs are the best available clinical evidence. Observational studies, case-control studies, and case series are included when they are the best available therapeutic evidence (Table 1). Additionally, observational studies, case-control studies, and case series report surgeons’ collective experiences and are used to assess safety and harms (Key Question #2). Prospective, observational cohort studies with greater than 80% follow-up are level I prognostic evidence (Wright et al. 2003). GRADE methodology is used to grade the quality of all evidence, but decisions are not restricted to only moderate- and high-quality evidence. Over half of the covered technologies were approved with low-quality (40%) and very low-quality (12%) therapeutic evidence (Table 2). The methodology used by the Washington State HCA HTCC provides an alternative approach to developing evidence-based recommendations for musculoskeletal technologies.

The GRADE methodology makes binary recommendations; a treatment is either effective or not effective. The HTCC approach allows coverage with conditions. By specifying necessary clinical conditions, a more nuanced interpretation of the evidence is possible. The AAOS Management of Osteoarthritis of the Knee CPG (Brophy and Fillingham 2022) recommends: “Arthroscopic partial meniscectomy can be used for the treatment of meniscal tears in patients with concomitant mild-to-moderate osteoarthritis who have failed physical therapy and other nonsurgical treatments.” This recommendation was criticized because RCTs found no difference in outcomes comparing physical therapy/exercise to arthroscopic meniscectomy as the primary outcome (Leopold 2022; Arakgi 2022; Leopold 2023). However, two of the RCTs (Herrlin et al. 2013; Katz et al. 2013) found that approximately one-third of patients improved with meniscectomy after failed physical therapy/exercise (Brown et al. 2023). This recommendation is consistent with the HTCC methodology. By specifying clinical conditions: (1) mild-to-moderate knee osteoarthritis and (2) failed physical therapy and other nonsurgical treatments, this recommendation avoided the usual binary recommendation choices and provided a more appropriate interpretation of the RCT outcomes.

Six of the HTCC coverage decisions are congruent with AAOS CPG recommendations. Although hyaluronic acid (HA) injection recommendations for knee osteoarthritis (OA) are congruent with HTCC coverage decisions, the AAOS recommendations changed from not recommending HA injections (Jevsevar 2013) to not recommending HA injections “for routine use in the treatment of symptomatic osteoarthritis of the knee.” (Brophy and Fillingham 2022) The HTCC coverage decisions switched from covering HA injections for knee OA with conditions (November 15, 2013) to not covering HA injections (July 21, 2023). These differing recommendation/decision switches can be explained by the HTCC’s inclusion of cost-effectiveness analyses. The rationale in the AAOS Management of Osteoarthritis of the Knee (Non-Arthroplasty), Third Edition (Brophy and Fillingham 2022) notes the number needed to treat (NNT) for HA injections for knee OA was 17 (American Academy of Orthopaedic Surgeons 2021). This means that 17 patients would need to receive HA injections before one patient would have an additional positive response compared to placebo injections. Consequently, the cost of 17 HA injections for one additional positive outcome is not cost-effective.

Five of the HTCC coverage decisions were not congruent with AAOS CPG recommendations. HTCC voted to not cover four technologies for knee OA: platelet-rich plasma (PRP) injections, nerve ablation, shock wave therapy, and TENS. All four of the recommendations for these technologies had “limited” strength of evidence supporting the recommendation (Brophy and Fillingham 2022). The recommendation for PRP injections in the Treatment of Osteoarthritis of the Knee, Second edition was ‘inconclusive." (Jevsevar 2013) The limited evidence supporting the effectiveness of these technologies does not justify the costs so the technologies are not cost-effective. The HTCC decision is consistent with the Centers for Medicare and Medicaid Services’ (CMS) decision to not cover PRP injections.

AAOS does not have a clinical practice guideline for hip arthroscopy. HTCC has reviewed femoroacetabular impingement (FAI) twice (September 16, 2011 and November 22, 2019) and both times the HTCC has voted to not cover hip arthroscopy for FAI. This decision is based on the assessment that hip arthroscopy is not significantly better than physical therapy in managing FAI pain. The focus of the HTA report should be hip preservation, not FAI. With a focus on FAI, the RCTs comparing the effectiveness of hip arthroscopy to physical therapy are usually limited to 1-2 year follow-up. With a focus on hip preservation, the primary outcomes become delaying or preventing the development of secondary hip arthritis (Ganz et al. 2003; Siebenrock et al. 2004; Beck et al. 2004; Tanzer and Noiseux 2004; Ganz et al. 2008; Tannast et al. 2008; Clohisy et al. 2010), conversion to total hip arthroplasty (THA) (Nepple et al. 2023; Fenn et al. 2024), and long-term (10 year) patient-reported outcome measures (PROMs) (Danilkowicz et al. 2025). For hip preservation, long-term observational studies will provide prognostic evidence (Wright et al. 2003) (not therapeutic evidence) regarding hip preservation, THA conversion, and long-term PROMs. Physical therapy cannot modify the long-term effects of mechanical impingement.

The HTA report on hip resurfacing utilized observational data to assess the safety profile of hip resurfacing. Registry data from three national registries (Australian Orthopaedic Association National Joint Replacement Registry – 2012 report; Swedish Hip Arthroplasty Register – 2011 report; and National Joint Registry for England and Wales – 2012 report) demonstrated higher revision rates for hip resurfacing compared to conventional THA. Epidemiologic studies demonstrated increased metal ion concentrations in blood. By incorporating observational data into the HTA report, the HTCC decided to not cover hip resurfacing due to harms and safety concerns. The AAOS Management of Osteoarthritis of the Hip evidence-based CPG (Rees and Barba 2020) does not make any recommendations regarding hip resurfacing because observational studies are automatically downgraded to low-quality evidence.

The major limitation of these analyses is that AAOS does not develop CPGs for spine conditions. Spine technologies had the highest percentage of coverage decisions that applied to a subspecialty or joint (19/61; 31%; Table 3). This means many of the coverage decisions could not be assessed for congruence with AAOS CPG recommendations. The congruence analyses are limited to coverage decisions for knee osteoarthritis, knee arthroplasty, and hip osteoarthritis.

Conclusions

Clinical practice guidelines developed with GRADE methodology provide binary recommendations that a given technology is effective or not effective. GRADE methodology automatically downgrades observational studies to low-quality evidence. These same observational studies provide the best prognostic evidence (Wright et al. 2003) that is needed to allow shared decision-making with individual patients. The process used by the Washington State Health Care Authority Health Technology Clinical Committee could complement CPGs by providing more nuanced, condition-specific guidance. Using the best available clinical evidence and collective physician experience described by Sackett (Sackett et al. 1996) couples coverage decisions with appropriate clinical conditions. The inclusion of safety and harms, differential effectiveness, and cost-effectiveness provides a robust assessment of technologies. Standardized coverage conditions would simplify pre-authorization.