Low back pain (LBP) is the leading cause of disability worldwide with an increasing social and economic burden.1 Suboptimal care results in a large economic burden and an increased risk of transition from acute to chronic LBP.1 The World Health Organization defines clinical practice guidelines (CPGs) as a document containing “systematically developed evidence-based statements that assist providers, patients, policy makers and other stakeholders to make informed decisions on health care and public health policy.”2 Dissemination of the recommendations provided by CPGs are encouraged to increase use of effective care. Furthermore, recommendations provided by CPGs are used to inform research and health funding allocation decisions. The National Institute of Clinical Excellence (NICE) LBP CPG has been graded as high quality while including a broad scope of treatment options ranging from invasive to non-surgical treatments.3,4

While the quality and reporting of the NICE LBP CPG is considered to be high,3 little is discussed about the quality of evidence underpinning its recommendations – particularly on the impact of trial design, such as the comparators used in the trials that inform the CPG. Testing a treatment against a suboptimal comparator may produce inaccurate results by overestimating the treatment effect, this is known as comparator bias.5 The use of studies with comparator bias could result in misleading recommendations in CPGs, which ultimately impacts clinical care, health funding allocation decisions, and research.

To our knowledge, there has been no analysis assessing the quality of comparators in trials included in CPGs for LBP. Therefore, the aim of this study was to categorize the comparator used in trials included in the NICE LBP CPG, determine the proportion of trials that used a suboptimal comparator, and assess the relationship between using a suboptimal comparator and the recommendation made by the NICE LBP CPG.

We conducted a cross-sectional analysis of trials that informed recommendations in the NICE LBP CPG.6 We chose the NICE CPG as it has been consistently graded as high-quality and includes a broad range of treatment from invasive (e.g., fusion surgery, disc replacement, etc.) to non-invasive (e.g., pharmacological, manual therapy, exercise, etc.) interventions.3 We reported this study per the STROBE guidelines.7

We developed a standardized data extraction form which was independently tested and piloted by two authors (GB and BC) on a random sample of 205 (50%) trials followed by discussion between the two authors to get familiar with the coding framework. One author (GB) extracted data from the remaining trials (n = 203) using the agreed coding framework. We extracted the following data from trials: title, author, journal, publication year, treatment, comparator (e.g., exercise, manual therapy, NSAIDs, paracetamol, opioids, etc.), comparator effectiveness (effective, ineffective, or unknown effectiveness), type of comparator (type of treatment, no treatment, minimal treatment, usual care, or placebo) and whether the treatment was recommended for or against by NICE. Examples of minimal treatment include booklets/advice or treatments defined as ‘minimal treatment’ by trial authors themselves.

We used evidence from systematic reviews, preferentially Cochrane reviews, published in the past 10 years to classify comparators as effective, ineffective, or unknown effectiveness. We chose Cochrane reviews as our primary benchmark for evidence quality as their quality is typically higher than non-Cochrane reviews.8 We classified comparators in the included trials as optimal or suboptimal. Optimal comparators were treatments for which there was evidence of effectiveness or placebo comparators. Suboptimal comparators were treatments known to be ineffective, those with unknown effectiveness, or when participants in the trials received no or minimal treatment. We classified comparators including multiple treatments as optimal if at least one component of the intervention was effective as judged by both authors extracting the data. If a comparator was named usual care, we classified the comparator by the actual treatments delivered when study reporting was sufficient. The unit of analysis of this study is the number of control groups, not trials, as many trials had more than one control group.

We described the proportion of suboptimal comparators in the overall sample and stratified by type of treatment (non-surgical, pharmacological, or invasive). We also investigated whether treatments compared to suboptimal comparators were more likely to be recommended for or against by the NICE CPG using logistic regression. The recommendation made by NICE (for/against) for each treatment was the independent variable. Output from the logistic regression was reported as odds ratio (95% CI). The analysis was conducted in Stata version 17.0 (StataCorp LLC, College Station, TX).

The NICE LBP CPG included 424 unique trials to inform their treatment recommendations, and we were able to retrieve full-texts from 408 trials and 580 comparators There were 284 (69.6%) non-surgical, 80 (19.6%) invasive, and 44 (10.8%) pharmacological trials. Overall, 179 (30.9%) comparators were classified as suboptimal. The largest proportion of suboptimal comparators were in invasive trials with 33 (32.4%) suboptimal comparators, followed by 134 (32.3%) in non-surgical, and 12 (19.0%) in pharmacological trials (Table 1). There were 106 (28.0%) suboptimal comparators used in trials of treatments recommended by NICE. Overall, trials that had a suboptimal comparator were less likely to have their treatment recommended (odds ratio 0.68, 95% CI: 0.47, 0.98).

We found that almost a third of comparators used in trials that informed the NICE LBP CPG are suboptimal. Trials of invasive treatments most often used suboptimal comparators. Interestingly, using suboptimal comparators reduced the likelihood of the treatment being recommended for use by NICE. To the best of our knowledge, this is the first study assessing comparator quality in trials testing treatments for LBP, and more broadly, musculoskeletal conditions. A 2019 study found that 17% of trials of oncological drugs approved by the United States Food and Drug Administration used suboptimal comparators.9 This is similar to the proportion of suboptimal comparators in pharmacological trials in our study (19.0%). However, we found a greater proportion of suboptimal comparators overall (30.9%). This may be because they assessed trials approved by the United States Food and Drug Administration, which does not include trials denied due to quality.

Strengths of this study include using a large sample size of trials (n = 408) with various comparators (n = 580) and using a systematic data extraction form to assess the quality of the comparators. Using the NICE LBP CPG, which is high-quality and comprehensive, to investigate the proportion of LBP trials that use a suboptimal comparator can also be considered a strength.3 A limitation of our study is that by only investigating the body of evidence that informs the NICE LBP CPG, we have missed trials not included in the NICE LBP CPG including those published after the 2016 NICE LBP CPG publication. Evaluating all trials of treatments for LBP would not be feasible due to the large number of published trials. However, the trials that we did assess are part of a distinct group of studies that directly shaped the recommendations of a high-quality CPG used globally.

The proportion of trials in LBP using suboptimal comparators is concerning as they may overestimate treatment effects, and negatively influence CPG recommendations, funding allocation decisions, and clinical practice.5 Our findings show that a substantial proportion of trials underpinning the recommendations in favor of treatments that are now considered first-line care use suboptimal comparators. Trials with suboptimal comparators do not contribute to advancing care and waste resources. Future trials must use optimal comparators commonly used in clinical practice that are appropriately methodologically designed for the specific research question.10