Sedentary behaviour is an emerging field of research, and as with all emerging fields there has been much excitement about early findings. As the field grows, it is important that it is closely scrutinised for potential issues of bias. Systematic reviews and meta-analyses are ways of synthesising published research literature. However, issues such as the non-publication of negative findings (publication bias), the publication of some outcomes and not others (selective reporting), the reporting of same outcomes multiple times, and the clarity of outcomes reported in separate papers arising from the same study (duplicate publication bias), affect the trustworthiness of the original publications making reviews more challenging and therefore review findings1 and can lead to the overestimation of effects.2

In an emerging research field, it is also common to see a large number of outcomes being assessed in each trial as new mechanisms of action are hypothesised and exploratory outcomes considered. However, as the number of outcomes assessed per individual participant increases, so too does the risk of Type I error – that is finding statistically significant results that are spurious.3 Clinical trial registries are designed to guard against some of these issues, and most ethics committees, research governance bodies, and medical journals now require all clinical trials to be prospectively registered prior to participant recruitment. While the Cochrane Collaboration requires all reviews to consider issues of publication bias and selective reporting by comparing published findings with trial registry records, this is not widespread across all published systematic reviews, including those in sedentary behaviour research.4

In the world of exercise physiology research, there are two broad fields; physical activity research, focused on the moderate to vigorous end of the exercise intensity spectrum and sedentary behaviour research, focused on the effects of prolonged sitting. While the former is a well-established field with a long history of publication, the latter is in its relative infancy. Much of the physical activity research pre-dates the requirement to register clinical trials. However, the newer field of sedentary behaviour research, especially the literature investigating sedentary behaviour interruption with short bouts of physical activity, provides a unique opportunity to carefully examine how information found in publications are consistent with trial registration information, so as to determine the rigour of reporting of published work.

In this paper, to maintain a reasonably narrow scope, we chose to assess biases by focusing on experimental trials investigating the effect of interrupting prolonged periods of sitting time with frequent, short bouts, of physical activity or standing. In the past five years, this field of research has rapidly expanded with an increasing number of primary trials and systematic reviews being published each year. This is important, given the increasing use of these findings to inform intervention and guideline development.

Therefore, the aim of this study was to examine issues in trial registration, which may suggest publication bias, selective reporting, and risk of Type I error in studies investigating the effect of frequent activity interruptions to prolonged sitting. The specific research questions were in experimental trials investigating the effect of interrupting prolonged periods of sitting time with frequent, short bouts, of physical activity or standing;

• 1.

  How many published trials have been registered, and of those registered, have all completed trials been published?

• 2.

  Are duplicate publications from the same trial easily identifiable?

• 3.

  Are all outcomes recorded in trial registries published?

• 4.

  How many different outcomes have been measured in each trial?

Data extraction was performed independently by two reviewers and cross-checked for accuracy. Trial registry number, participant characteristics, and measured outcomes for each trial, recorded separately for registered and non-registered trials, were extracted from the included studies. Information on the number of outcome measures collected, number of outcome measures registered, number of outcome measures ‘registered and reported’, number of outcome measures ‘registered but not reported’, number of outcome measures ‘reported but not registered’, and reported outcomes that were not registered but for which significant results were found in each trial were recorded. Narrative analysis using descriptive statistics was undertaken to describe the nature of the studies and to answer the research questions. Bias definitions are provided alongside the research questions and detailed here.

Potential publication bias was examined by reporting how many published trials had been registered, and of those registered, were all completed trials been published. Any potential duplicate publication bias was observed by carefully examining papers by the same or similar author lists for substantial similarities between participant inclusion criteria, sample sizes, and methods. Possible selective outcome reporting was examined by matching outcomes reported in trial registries to those published. Lastly, any possible risk of Type I error was identified by observing how many different outcomes had been measured in each trial.

Out of 7645 title and abstracts, 69 were eligible for full text screening. Out of the full text papers reviewed, 32 trials reported in 45 papers were included (Fig. 1). The sample sizes per trial ranged from 9 to 70 (Table 1), with a combined total sample of n=606 (289 females, 48%). The mean age of included participants ranged from 21 to 69 years, and trials included a variety of population groups including people who were healthy (n=16), overweight (n=13), had type 2 diabetes (n=2), or post-stroke (n=1). All papers were published between 2012 and 2018, with 36 (80%) published in the last 5 years.

Figure 1.

PRISMA flowchart of the review.

(0.29MB).

How many published trials are registered, and of those registered, have all completed trials been published?

Only 16 (50%) of published trials were registered (Table 1) in registries including the ANZTR (n=86–23), ClinicalTrials.gov (n=724–30), and ISRCTN (n=131). The remaining 16 (50%) trials (published in 19 papers) were not registered.32–50 All registered trials were completed and published (Fig. 2).

Figure 2.

Flowchart of the included studies.

(0.34MB).

How many different outcomes have been measured in each trial?

A total of 46 different outcomes (out of 53 reported outcome measures, some similar outcome measures were collated. e.g.: high density lipids, low density lipids and total cholesterol were combined as postprandial lipids) were reported across all trials (including registered and unregistered trials), the majority (31 [67%]) being collected in ≤ 2 trials (Fig. 3). The number of outcomes collected per trial ranged from 1 to 12 (mean 5 [SD 3]). The most commonly measured outcomes were postprandial glucose (26 [81%]), postprandial insulin (21 [66%]), and triglycerides (13 [41%]) (Table 1). Across all the registered trials, results from 32 additional outcomes not mentioned in trial registry records were published. Of these additional outcomes 22 (69%) reported positive, significant between-condition differences (Table 1).

Figure 3.

Frequency of different outcomes measured in included trials.

(0.37MB).

We conducted systematic searches of databases and trial registries to investigate the issues in trial registration in experimental trials investigating the effect of interrupting prolonged periods of sitting time with frequent, short bouts, of physical activity or standing. We found direct evidence of issues in trial registration which may lead to publication bias. All completed registered trials were published. However, our finding that only half of all published trials were registered creates uncertainty concerning the number of other unregistered trials which may also remain unpublished. Similarly, while there was no direct evidence of selective reporting (results from all registered outcomes have been published), 70% of registered trials published results from additional outcomes that were not registered. The search revealed that more than half of the published, non-registered outcomes reported statistically significant positive findings. However, this may be because some reported outcomes reflect the use of new technologies or techniques that may not have been present at the time of registration, particularly for some earlier studies that used metabolomics data. It is important to note that we chose to focus on the field of experimental sedentary behaviour research, given most papers were published after trial registration became common practice. It is possible that similar issues exist in other fields of research, including those with a longer publication history such as physical activity research.

More than half of the trials were not registered, making it difficult to thoroughly assess the degree of publication bias in the literature. This is an issue not limited to the field of sedentary behaviour research. Several previous reviews of randomised controlled trials published in medical journals in the past decade have found between 25 and 50% of trials were not registered, even those published in high impact factor journals.51,52 When trials are not registered, it is impossible to know what additional outcomes were measured but not reported. Selective reporting of the collected outcomes could be influenced by the significance of the results,53,54 leading to potentially misleading evidence.1 We found that the majority of non-registered outcomes for which findings were published had significant, positive findings, lending weight to this concern. Many authors favour submitting their work with positive results for publication and many editors prefer to publish significant results.55,56 To mitigate against this risk, it is good practice that effect sizes and confidence intervals for all outcomes are reported, regardless of the significance of the findings. Journal editors can assist in addressing this issue by giving equal consideration to papers reporting non-significant findings, particularly where trials are adequately powered to show an effect. Moreover, conflict of interests such as connections with related industries may be another potential contributing factor to publication bias.

Finally, the large number of different outcomes measured across trials warrants greater attention in future trials. While this is perhaps not surprising, given the field of the physiology of sedentary behaviour is still relatively young, and much of the research remains exploratory in nature, care must be taken to guard against potentially spurious findings. For future trials we recommend careful selection of outcomes, with a clear biological rationale for each and a single, pre-registered primary outcome with sufficient statistical power, as recommended by the CONSORT statement. Consideration should be given to adjusting analyses for multiple comparisons where this is appropriate.3 The reporting of any additional outcomes, particularly those with insufficient statistical power or that were not prospectively registered, should clearly identify that the analyses were exploratory in nature. Additionally, reaching consensus on a core outcome dataset for sedentary behaviour research, as has been done in other fields57 would be a useful step forward. We also suggest researchers conduct two separate analyses for registered and non-registered trials in future systematic reviews.52 Future reviews may also consider investigating which factors (such as study design, methodological quality, and impact factor of the journals) were associated with registering or not registering trials. Furthermore, in future reviews it would be interesting to investigate if outcomes have been upgraded (i.e., from secondary outcome in the register to primary outcome in the published trial), or downgraded, and if these modifications were made in light of the results of the trial. This was not possible to do in our review as many registered trials did not clearly state a single primary outcome variable. It is important that future trials pre-register a clearly defined single primary outcome measure.6,7,11

The strengths of this study included a comprehensive search strategy, including all relevant databases and trial registries. Moreover, the independent data extraction between two authors adds further rigour to our work. Limitations include excluding trials not published in English, excluding conference abstracts and grey literature. While we included a consultation phase, it is possible that some recent papers may still have been missed. By only searching the five most commonly used trial registries, it is possible that some trial registrations may have been missed. However, no published trials reported registration numbers from registries other than those we searched, making this unlikely. We also acknowledge that the search strategy was developed to address the research objectives of the previous published scoping review, therefore we did not include searches of unpublished studies. While we are confident that we have included all relevant literature at the time period of search, further studies may have been published since. In addition, this review was designed to provide a broad overview of evidence on issues in trial registration which may lead to publication bias, selective reporting, and risk of Type I error in experimental studies investigating the effect of frequent activity interruptions to prolonged sitting time and therefore did not report on the magnitude or direction of the findings.

We found a lack of widespread trial registration, which may lead to publication bias and selective reporting among experimental trials investigating the effect of interrupting prolonged periods of sitting time with frequent, short bouts, of physical activity. To further strengthen confidence in this field, we recommend that all trials are prospectively registered, trial registration numbers are clearly stated in all publications along with explicit reference to other papers reporting results from the same dataset, trials are adequately powered for a primary outcome and exploratory analyses of secondary outcomes are clearly identified as such, and finally that the development of a core outcome dataset for the field is prioritised.

The authors declare no conflicts of interest.