Sedentary behaviour research is a relatively new field, much of which has emerged since the widespread acceptance of clinical trial registration. The aim of this study was to investigate the trial registration and related issues in studies investigating the effect of frequent activity interruptions to prolonged sitting-time.
MethodsSecondary analysis of a scoping review including systematic searches of databases and trial registries. We included experimental studies investigating the effects of frequent activity interruptions to prolonged sitting-time.
ResultsWe identified 32 trials published in 45 papers. Only 16 (50%) trials were registered, with all 16 trials being completed and published. Of the unregistered trials, we identified three (19%) for which similarities in the sample size and participant demographics across papers was suggestive of duplicate publication. Identification of potential duplicate publications was difficult for the remaining 13 (81%). Results from 53 (76%) of the 70 registered outcomes were published, but 11 (69%) registered trials reported results from additional outcomes not prospectively registered. A total of 46 different outcomes (out of 53 reported outcome measures, similar measures were collated) were reported across all trials, 31 (67%) of which were collected in ≤2 trials.
ConclusionsWe found direct evidence of trial registration issues in experimental trials of breaking up sitting-time. The lack of prospective registration of all trials, and the large number of outcomes measured per trial are key considerations for future research in this field. These issues are unlikely to be confined to the field of sedentary behaviour research.
Sedentary behaviour is an emerging field of research, and as with all emerging fields there has been much excitement about early findings. As the field grows, it is important that it is closely scrutinised for potential issues of bias. Systematic reviews and meta-analyses are ways of synthesising published research literature. However, issues such as the non-publication of negative findings (publication bias), the publication of some outcomes and not others (selective reporting), the reporting of same outcomes multiple times, and the clarity of outcomes reported in separate papers arising from the same study (duplicate publication bias), affect the trustworthiness of the original publications making reviews more challenging and therefore review findings1 and can lead to the overestimation of effects.2
In an emerging research field, it is also common to see a large number of outcomes being assessed in each trial as new mechanisms of action are hypothesised and exploratory outcomes considered. However, as the number of outcomes assessed per individual participant increases, so too does the risk of Type I error – that is finding statistically significant results that are spurious.3 Clinical trial registries are designed to guard against some of these issues, and most ethics committees, research governance bodies, and medical journals now require all clinical trials to be prospectively registered prior to participant recruitment. While the Cochrane Collaboration requires all reviews to consider issues of publication bias and selective reporting by comparing published findings with trial registry records, this is not widespread across all published systematic reviews, including those in sedentary behaviour research.4
In the world of exercise physiology research, there are two broad fields; physical activity research, focused on the moderate to vigorous end of the exercise intensity spectrum and sedentary behaviour research, focused on the effects of prolonged sitting. While the former is a well-established field with a long history of publication, the latter is in its relative infancy. Much of the physical activity research pre-dates the requirement to register clinical trials. However, the newer field of sedentary behaviour research, especially the literature investigating sedentary behaviour interruption with short bouts of physical activity, provides a unique opportunity to carefully examine how information found in publications are consistent with trial registration information, so as to determine the rigour of reporting of published work.
In this paper, to maintain a reasonably narrow scope, we chose to assess biases by focusing on experimental trials investigating the effect of interrupting prolonged periods of sitting time with frequent, short bouts, of physical activity or standing. In the past five years, this field of research has rapidly expanded with an increasing number of primary trials and systematic reviews being published each year. This is important, given the increasing use of these findings to inform intervention and guideline development.
Therefore, the aim of this study was to examine issues in trial registration, which may suggest publication bias, selective reporting, and risk of Type I error in studies investigating the effect of frequent activity interruptions to prolonged sitting. The specific research questions were in experimental trials investigating the effect of interrupting prolonged periods of sitting time with frequent, short bouts, of physical activity or standing;
- 1.
How many published trials have been registered, and of those registered, have all completed trials been published?
- 2.
Are duplicate publications from the same trial easily identifiable?
- 3.
Are all outcomes recorded in trial registries published?
- 4.
How many different outcomes have been measured in each trial?
This paper is an extension of a scoping review5 with systematic searches that aimed to review the evidence for the effect of interrupting prolonged sitting with frequent bouts of physical activity or standing on first or recurrent stroke risk factors. During the initial scoping review, standard guidelines were followed such as independent title and abstract, and full text screening, with discrepancies resolved by a third member of the research team.5 Five databases including Medline, Embase, Allied and Complementary Medicine, Cumulative Index to Nursing and Allied Health Literature (CINAHL), and the Cochrane Library were searched from the inception to February 2018 (full search strategy and methodology previously published5). Experimental trials with adult participants (age ≥18 years), investigating the effect of interrupting prolonged periods of sitting time compared with supervised interventions of frequent, short bouts, of physical activity or standing published in English were included in the review. Only studies that included a control condition of uninterrupted prolonged sitting were included. While the scoping review included only studies that reported at least one outcome measure related to stroke risk factors, in this study we did not exclude studies based on outcome measures.
Identification of trial registrationWe used a two-staged approach to search for registered trials. We first scrutinised all included articles for reference to a trial registry identification number. Secondly, we searched trial registries for trials meeting the same inclusion criteria. We searched the five most commonly used registers for trials published in English, including the Australian and New Zealand Clinical Trial Registry (ANZCTR), ClinicalTrials.gov, International Standard Randomised Controlled Trial Number (ISRCTN) registry, World Health Organisation International Clinical Trials Registry Platform, and the European Union Clinical Trials Register. Key terms used were a combination of “sedentary” OR “sitting” AND “break” or “interrupt” or “activity”. Where possible, results were filtered to include only trials that were “completed”, “interventional”, and/or conducted with “adult/elderly” participants.
Data extractionData extraction was performed independently by two reviewers and cross-checked for accuracy. Trial registry number, participant characteristics, and measured outcomes for each trial, recorded separately for registered and non-registered trials, were extracted from the included studies. Information on the number of outcome measures collected, number of outcome measures registered, number of outcome measures ‘registered and reported’, number of outcome measures ‘registered but not reported’, number of outcome measures ‘reported but not registered’, and reported outcomes that were not registered but for which significant results were found in each trial were recorded. Narrative analysis using descriptive statistics was undertaken to describe the nature of the studies and to answer the research questions. Bias definitions are provided alongside the research questions and detailed here.
Potential publication bias was examined by reporting how many published trials had been registered, and of those registered, were all completed trials been published. Any potential duplicate publication bias was observed by carefully examining papers by the same or similar author lists for substantial similarities between participant inclusion criteria, sample sizes, and methods. Possible selective outcome reporting was examined by matching outcomes reported in trial registries to those published. Lastly, any possible risk of Type I error was identified by observing how many different outcomes had been measured in each trial.
ResultsOut of 7645 title and abstracts, 69 were eligible for full text screening. Out of the full text papers reviewed, 32 trials reported in 45 papers were included (Fig. 1). The sample sizes per trial ranged from 9 to 70 (Table 1), with a combined total sample of n=606 (289 females, 48%). The mean age of included participants ranged from 21 to 69 years, and trials included a variety of population groups including people who were healthy (n=16), overweight (n=13), had type 2 diabetes (n=2), or post-stroke (n=1). All papers were published between 2012 and 2018, with 36 (80%) published in the last 5 years.
Summary of trials and outcomes.
Trial registry no. | Papers | n (trial) | Participant characteristics | Outcomes registered (published outcomes are bolded) | Outcomes reported not registered (outcomes with significant positive between condition differences are italicised) |
---|---|---|---|---|---|
ACTRN12613000576729 | Dempsey et al. (2015)7Dempsey et al. (2016)8Dempsey et al. (2017)6Grace et al. (2017)11Dempsey et al. (2018)23 | 24 | Type 2 diabetic overweight/obese and inactive adults 10 females (41.7%) 62±6 years | Postprandial glucoseContinuous glucose measurementPostprandial insulinC-peptide analysisBlood pressureFatigue | TriglyceridesOther cholesterolsNorepinephrine/NoradrenalineRespiratory exchange ratioTotal energy expenditureHeart rate |
ACTRN12615001189516 | English et al. (2018)22English et al. (2018)10 | 19 | Post-stroke people (between 3 months and 10 years) ambulant with minimal assistance and not taking diabetic medication other than metformin9 females (47.4%)68.2±10.2 years | Postprandial glucoseInsulin controlBlood pressureFibrinogenFeasibility (adherence to protocol including fatigue) | Nil |
ACTRN12609000656235 | Larsen et al. (2014)15Latouche et al. (2013)16Dunstan et al. (2012)9Howard et al. (2013)13 | 19 | Overweight/obese adults8 females (42.1%)53.8±4.9 years | Postprandial glucose levelsPostprandial insulin levelsPostprandial free fatty acid levels | Hemostatic parameters (fibrinogen, platelet count, activated partial thromboplastin time, Von Willebrand factor antigen)Haematological parameters (volume, Hct, Hb, RBC, WBC, PC, MPV)Blood pressureHeart rateGene expression |
ACTRN12614000624684 | Homer et al. (2017)12Mete et al. (2018)17 | 36 | Healthy, normal weight adults25 females (69.4%)25.4±3.9 years | Postprandial glucosePostprandial insulinPostprandial triglyceridePostprandial NEFASubstrate utilisationEnergy expenditureEnergy intakeAppetite | None |
ACTRN12611000632998 | Thorp et al. (2014)19Thorp et al. (2014)20 | 23 | Overweight/obese sedentary office workers6 females (26.1%)48.2±8 years | Postprandial glucosePostprandial insulinPostprandial triglyceridePostprandial free fatty acid levels | FatigueMusculoskeletal discomfortWork productivityWorkstation acceptability |
ACTRN12610000657022 | Larsen et al. (2015)14 | 19 | Overweight/obese, sedentary adults8 females (42.1%)56.7±6.5 years | Postprandial glucosePostprandial insulinPostprandial triglyceridesPostprandial free fatty acid | None |
ACTRN12610000953033 | Peddie et al. (2013)18 | 70 | Healthy, normal weight adults working in a predominately sedentary occupation42 females (60.0%)25.9±5.3 years | Postprandial glucosePostprandial insulinPostprandial triglycerideBlood pressureSubstrate utilisation | Heart rate |
ACTRN12613000137796 | Wennberg et al. (2016)21 | 19 | Overweight/obese adults9 females (47.7%)59.7±8.1 years | Continuous glucose measurementCortisolCatecholaminesExecutive functionMemoryBrain-derived neurotrophic factor, BDNFInterleukin-6, IL-6 | InsulinBlood pressureFatigueHeart rate |
ISRCTN48132950 | Brocklebank et al. (2017)31 | 17 | Healthy office workers in a predominately sedentary occupation9 females (52.9%)52.4±5.1 years | Postprandial glucosePre-prandial interstitial glucose | None |
NCT02215603 | Benatti et al. (2017)24 | 14 | Healthy, physically inactive males14 males (100%)30.1±8.8 years | Postprandial glucoseContinuous glucose measurementPostprandial insulinPostprandial lipidsPostprandial cytokine | C-peptideTotal energy expenditure |
NCT02717377 | Bergouignan et al. (2016)25 | 30 | Healthy sedentary adults21 females (70.0%)30±5.6 years | Energy and mood levelCognitive function | Appetite ratingsEpinephrine/AdrenalineNorepinephrine/NoradrenalineDopamineCortisolHeart rate |
NCT00945165 | Dijk et al. (2013)26 | 20 | Type 2 diabetic males20 males (100%)64±1 years | HyperglycemiaMean blood glucose | Heart rate |
NCT02135172 | Henson et al. (2016)27 | 22 | Postmenopausal, overweight/obese dysglycemic women22 females (100%)66.6±4.7 years | Postprandial glucosePostprandial insulinPostprandial triglyceridesLipoprotein lipase activity | NEFA levels |
NCT02743286 | Kerr et al. (2017)28 | 10 | Postmenopausal, overweight/obese and sedentary women10 females (100%)66±9 years | Change in glycemic regulationChange in endothelial functionChange in mitochondrial metabolitesStudy feasibility | Blood pressure |
NCT02493309 | McCarthy et al. (2017)29 | 34 | Healthy, non-obese adults working in a predominately sedentary occupation18 females (52.9%)40±9 years | Postprandial glucosePostprandial insulinPostprandial triglyceridesBlood pressure | None |
NCT02909894 | McCarthy et al. (2017)30 | 13 | Obese and inactive adults at risk of type 2 diabetes7 females (53.8%)66±6 years | Postprandial glucosePostprandial insulinBlood pressureCognitive function | Mood |
Holmstrup et al. (2013)41 | 11 | Obese, young adults with impaired fasting glucose3 females (27.2%)25.2±1.3 years | N/A | Postprandial glucosePostprandial insulinC-peptidePeptide YY tAUCHunger and satiety | |
Holmstrup et al. (2014)42 | |||||
Masaki et al. (2015)44 | 15 | Healthy, normo-lipidaemic men15 males (100%)26.80±2 years | N/A | Postprandial glucosePostprandial insulinNEFA levelsOther cholesterolsLPL protein/ApolipproteinOxidative stress markers3-OHB | |
Miyashita et al. (2013)46 | |||||
Mullane et al. (2017)47 | 9 | Overweight/obese and physically inactive (pre-hypertensive or with impaired fasting glucose)7 females (77.8%)30±15 years | N/A | Sleep efficiencyHeart rateCognitive performance | |
Zeigler et al. (2016)50 | |||||
Altenburg et al. (2013)32 | 11 | Healthy young adults6 females (54.5%)21.4 (19.5–23.1) years | N/A | Postprandial glucoseC-peptideTriglyceridesTotal cholesterolHDL cholesterolLDL cholesterolMuscle activity | |
Bailey and Locke (2015)34 | 10 | Healthy, non-obese adults3 females (30.0%)24.0±3 years | N/A | Postprandial glucoseTriglyceridesTotal cholesterolHDL cholesterolBlood pressure | |
Bailey et al. (2016)33 | 13 | Healthy, inactive, sedentary adults7 females (53.8%)26.6±8.5 years | N/A | Postprandial glucosePostprandial insulinPeptide YY tAUCAcylated ghrelin tAUCAppetite ratingsEnergy intake | |
Barone Gibbs et al. (2017)35 | 25 | Overweight/obese adults with pre to stage 1 hypertension9 females (36.0%)42±12 years | N/A | Blood pressurePulsed wave velocityHeart rate | |
Bhammar et al. (2017)36 | 10 | Overweight/obese and physically inactive adults5 females (50.0%)32±5 years | N/A | Postprandial glucoseBlood pressure | |
Carter and Gladwell (2017)37 | 10 | Healthy adults4 females (40.0%)27.3±8.1 years | N/A | Blood pressureEndothelial functionShear rate | |
Engeroff et al. (2017)38 | 18 | Healthy young premenopausal women18 females (100%)25.6±2.6 years | N/A | Total cholesterolHDL cholesterolLDL cholesterol | |
Hansen et al. (2016)39 | 14 | Healthy young, normal weight and recreationally active adults8 females (57.1%)22 (20 – 23) years | N/A | Postprandial glucose | |
Hawari et al. (2016)40 | 10 | Overweight/obese, normoglycaemic males10 males (100%)33±13 years | N/A | Postprandial glucosePostprandial insulinTriglyceridesSubstrate oxidationTotal energy expenditure | |
Kim et al. (2014)43 | 9 | Healthy young recreationally active males9 males (100%)24±4 years | N/A | Postprandial glucosePostprandial insulinTriglyceridesOther cholesterolsSubstrate oxidationRespiratory exchange ratio | |
Miyashita et al. (2016)45 | 15 | Postmenopausal women15 females (100%)68.8±3.2 years | N/A | Postprandial glucosePostprandial insulinTriglyceridesNEFA levelsRating of perceived exertionHeart rate3-OHB | |
Pulsford et al. (2017)48 | 25 | Healthy, inactive, weight stable males25 males (100%)40.2±12.2 years | N/A | Postprandial glucoseMatsuda indexPostprandial insulin | |
Thosar et al. (2015)49 | 12 | Healthy inactive young non-obese males12 males (100%)24.2±4.2 years | N/A | Flow mediated dilationShear rate |
Abbreviations: ACTRN, Australian New Zealand clinical trials registry; Hb, haemoglobin; Hct, haematocrit; HDL, high-density lipoproteins; ISRCTN, international standard randomised controlled trials number; LDL, low-density lipoproteins; LPL, lipoprotein lipase; MPV, mean platelet volume; N/A, not applicable; NCT, national clinical trial; NEFA, non-esterified fatty acids; PC, platelet count; RBC, red blood cell count; tAUC, total area under the curve WBC, white blood cell count; 3-OHB, 3-hydroxybutyra.
Only 16 (50%) of published trials were registered (Table 1) in registries including the ANZTR (n=86–23), ClinicalTrials.gov (n=724–30), and ISRCTN (n=131). The remaining 16 (50%) trials (published in 19 papers) were not registered.32–50 All registered trials were completed and published (Fig. 2).
Are duplicate publications from the same trial easily identifiable?Outcomes from the 16 registered trials were published in 26 papers. In all of these papers either a trial registration number was quoted or the authors clearly identified that the paper was reporting additional outcomes from a primary study. Identifying duplicate publications from the unregistered trials was more difficult. For three trials,41,42,44,46,47,50 similarities in the sample size and participant demographics across papers was suggestive of duplicate publication. We were unable to definitively determine how many of the remaining 13 unregistered trials were duplicate versus singular publications (Fig. 2).
Are all outcomes recorded in trial registries published?Of the registered trials, a total of 70 outcomes were registered, and of these, results from 53 (76%) were published. Eleven (69%) registered trials had at least one outcome measure that was registered but the results not published, and 11 (69%) of the trials included results from additional outcomes that were not prospectively registered (Fig. 2). Published studies rarely specified whether the outcomes reported were primary or secondary measures. As 50% of trials were not registered, the degree to which outcome reporting bias is present (e.g. due to upgrading secondary outcomes to primary outcomes and vice versa) could not be established.
How many different outcomes have been measured in each trial?A total of 46 different outcomes (out of 53 reported outcome measures, some similar outcome measures were collated. e.g.: high density lipids, low density lipids and total cholesterol were combined as postprandial lipids) were reported across all trials (including registered and unregistered trials), the majority (31 [67%]) being collected in ≤ 2 trials (Fig. 3). The number of outcomes collected per trial ranged from 1 to 12 (mean 5 [SD 3]). The most commonly measured outcomes were postprandial glucose (26 [81%]), postprandial insulin (21 [66%]), and triglycerides (13 [41%]) (Table 1). Across all the registered trials, results from 32 additional outcomes not mentioned in trial registry records were published. Of these additional outcomes 22 (69%) reported positive, significant between-condition differences (Table 1).
DiscussionWe conducted systematic searches of databases and trial registries to investigate the issues in trial registration in experimental trials investigating the effect of interrupting prolonged periods of sitting time with frequent, short bouts, of physical activity or standing. We found direct evidence of issues in trial registration which may lead to publication bias. All completed registered trials were published. However, our finding that only half of all published trials were registered creates uncertainty concerning the number of other unregistered trials which may also remain unpublished. Similarly, while there was no direct evidence of selective reporting (results from all registered outcomes have been published), 70% of registered trials published results from additional outcomes that were not registered. The search revealed that more than half of the published, non-registered outcomes reported statistically significant positive findings. However, this may be because some reported outcomes reflect the use of new technologies or techniques that may not have been present at the time of registration, particularly for some earlier studies that used metabolomics data. It is important to note that we chose to focus on the field of experimental sedentary behaviour research, given most papers were published after trial registration became common practice. It is possible that similar issues exist in other fields of research, including those with a longer publication history such as physical activity research.
More than half of the trials were not registered, making it difficult to thoroughly assess the degree of publication bias in the literature. This is an issue not limited to the field of sedentary behaviour research. Several previous reviews of randomised controlled trials published in medical journals in the past decade have found between 25 and 50% of trials were not registered, even those published in high impact factor journals.51,52 When trials are not registered, it is impossible to know what additional outcomes were measured but not reported. Selective reporting of the collected outcomes could be influenced by the significance of the results,53,54 leading to potentially misleading evidence.1 We found that the majority of non-registered outcomes for which findings were published had significant, positive findings, lending weight to this concern. Many authors favour submitting their work with positive results for publication and many editors prefer to publish significant results.55,56 To mitigate against this risk, it is good practice that effect sizes and confidence intervals for all outcomes are reported, regardless of the significance of the findings. Journal editors can assist in addressing this issue by giving equal consideration to papers reporting non-significant findings, particularly where trials are adequately powered to show an effect. Moreover, conflict of interests such as connections with related industries may be another potential contributing factor to publication bias.
Finally, the large number of different outcomes measured across trials warrants greater attention in future trials. While this is perhaps not surprising, given the field of the physiology of sedentary behaviour is still relatively young, and much of the research remains exploratory in nature, care must be taken to guard against potentially spurious findings. For future trials we recommend careful selection of outcomes, with a clear biological rationale for each and a single, pre-registered primary outcome with sufficient statistical power, as recommended by the CONSORT statement. Consideration should be given to adjusting analyses for multiple comparisons where this is appropriate.3 The reporting of any additional outcomes, particularly those with insufficient statistical power or that were not prospectively registered, should clearly identify that the analyses were exploratory in nature. Additionally, reaching consensus on a core outcome dataset for sedentary behaviour research, as has been done in other fields57 would be a useful step forward. We also suggest researchers conduct two separate analyses for registered and non-registered trials in future systematic reviews.52 Future reviews may also consider investigating which factors (such as study design, methodological quality, and impact factor of the journals) were associated with registering or not registering trials. Furthermore, in future reviews it would be interesting to investigate if outcomes have been upgraded (i.e., from secondary outcome in the register to primary outcome in the published trial), or downgraded, and if these modifications were made in light of the results of the trial. This was not possible to do in our review as many registered trials did not clearly state a single primary outcome variable. It is important that future trials pre-register a clearly defined single primary outcome measure.6,7,11
The strengths of this study included a comprehensive search strategy, including all relevant databases and trial registries. Moreover, the independent data extraction between two authors adds further rigour to our work. Limitations include excluding trials not published in English, excluding conference abstracts and grey literature. While we included a consultation phase, it is possible that some recent papers may still have been missed. By only searching the five most commonly used trial registries, it is possible that some trial registrations may have been missed. However, no published trials reported registration numbers from registries other than those we searched, making this unlikely. We also acknowledge that the search strategy was developed to address the research objectives of the previous published scoping review, therefore we did not include searches of unpublished studies. While we are confident that we have included all relevant literature at the time period of search, further studies may have been published since. In addition, this review was designed to provide a broad overview of evidence on issues in trial registration which may lead to publication bias, selective reporting, and risk of Type I error in experimental studies investigating the effect of frequent activity interruptions to prolonged sitting time and therefore did not report on the magnitude or direction of the findings.
ConclusionWe found a lack of widespread trial registration, which may lead to publication bias and selective reporting among experimental trials investigating the effect of interrupting prolonged periods of sitting time with frequent, short bouts, of physical activity. To further strengthen confidence in this field, we recommend that all trials are prospectively registered, trial registration numbers are clearly stated in all publications along with explicit reference to other papers reporting results from the same dataset, trials are adequately powered for a primary outcome and exploratory analyses of secondary outcomes are clearly identified as such, and finally that the development of a core outcome dataset for the field is prioritised.
Conflicts of interestThe authors declare no conflicts of interest.
This paper presents independent research funded by the National Institute for Health Research (NIHR) under its Programme Grants for Applied Research Programme (Development and evaluation of strategies to reduce sedentary behaviour in patients after stroke and improve outcomes, Reference number RP-PG-0615-20019). The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health and Social Care, UK.
Associate Professor English was supported by a National Heart Foundation Future Leaders Fellowship (#101177). Professor Dunstan is supported by a National Health and Medical Research Council of Australia (NHMRC) Senior Research Fellowship (# 1078360), an NHMRC Centre for Research Excellence Grant (# 1057608) and by the Victorian Government's Operational Infrastructure Support Program. Dr Janssen was supported by a New South Wales Department of Health Early Career Researcher Fellowship.