Two decades ago, more than 200 randomized controlled trials (RCTs) of neurological physical therapy interventions in adults and pediatric populations were identified from 1958 to 2000, with half rated moderate to high quality on the Physiotherapy Evidence Database (PEDro) scale. Does the current panorama remain the same?
ObjectiveTo investigate the changes in volume and quality of RCT of neurological physical therapy in adults and paediatrics indexed in the PEDro database between 1958 to 2021. In addition, to investigate if there is a relationship between journal impact factor and methodological quality.
MethodsAll RCTs of neurological physical therapy in adults and pediatrics indexed in PEDro between 1958–2021 were included. Descriptive statistics described trial volume, quality, and trends over the years. Spearman's rho correlation test assessed the association between methodological quality and journal ranking (Journal Impact Factor (JIF) and Journal Citation Indication (JCI)).
ResultsA total of 6291 RCTs of neurological physical therapy in adults and pediatrics were indexed in PEDro between 1958–2021, with a mean PEDro scale score of 5.3/10 (SD 1.6). The quality of RCTs improved over time, with a mean score of 5.7/10 (SD 1.4) for RCTs published between 2018–2021, compared to 1/10 between 1962–1965. A weak and significant relationship was found between methodological quality and JIF (r=0.153; p<0.001) and JCI (r=0.146; p<0.001).
ConclusionThere is a large and growing volume of RCTs on neurological physical therapy in adults and pediatrics indexed in PEDro, with increasing quality over time, though still moderate. Journal ranking should not be used for selecting high quality RCTs.
Evidence-based practice (EBP) is the application of the scientific evidence while considering patient values and clinical experience concerning the best treatment.1 This type of practice enhances professional respect and confidence as well as the patient safety and effectiveness of the treatment.2 Randomized controlled trials (RCTs) are a type of scientific study that analyzes the interventions’ efficacy/effectiveness and safety3 and provide crucial evidence to inform the physical therapist's decision-making process.
The Physiotherapy Evidence Database (PEDro)44is a free of cost database created to increase the access of clinical research and to evaluate their methodological quality.1 It aims to offer the most appropriate evidence for the clinical question chosen by the physical therapists in an objective and timely manner, thus being extremely relevant for the implementation of EBP. The database indexes only systematic reviews, RCTs, and guidelines, with methodological quality of RCTs assessed according to the PEDro Scale.4 To be indexed in PEDro, RCTs must compare at least two interventions or an intervention with a control group; and investigate an intervention that is part of physical otherapy practice or that is likely to become in the future.5 Studies must also be performed exclusively on human subjects who are a representative sample of the respective population, and also involve random allocation.
In 2013, Elkins et al.6 stated that the number of trials in PEDro doubled every 3.5 years and Maher et al.7 also found that there has been an exponential increase in the volume of RCTs in all physical therapy subdisciplines since PEDro's establishment. It is important to investigate if this increase in volume has been followed by improvements in quality, because low trial quality is an additional barrier for clinicians to use EBP as poor-quality trials are more likely to offer biased estimates than higher-quality ones. Assessing methodological quality requires appropriate evaluation and measures. Thus, it cannot be considered similar to the journal's impact factor, which is an average of journal citations over the last two years.9
The volume of RCTs of neurological physical therapy interventions was investigated two decades ago10 and it was found that there were 238 RCTs in the area of neurological physical therapy, with 54 % of these having moderate to high quality according to the PEDro scale. Considering the exponential growth of physical therapy RCTs in the last years, there is a need to re-investigate the volume of neurological physical therapy RCTs and their methodological quality.
Therefore, the research questions of this study were:
- 1.
What is the volume, quality, and changes in volume and quality over time of RCTs indexed in the Physiotherapy Evidence Database (PEDro) between 1958 to 2021?
- 2.
What is the relationship between journal ranking and methodological quality of the RCTs?
A scoping review of RCTs of neurological physical therapy interventions in adults and paediatrics populations indexed in PEDro was conducted following the extension of Scoping Reviews Checklist from Preferred Reporting Items for Systematic Reviews and Meta-Analyses and the scoping review guideline from Joanna Briggs Institute.11,12 A scoping design was selected given our aim to describe the volume and quality and identify knowledge gaps of neurological physical therapy.13 The study design was developed by the authors and the search strategy was developed together with a PEDro member. A request was sent to PEDro, via e-mail, to receive the data directly from the database. Thus, the search was conducted by a PEDro member and sent to the authors. The senior author on this publication is a member of the PEDro committee and facilitated the process.
Data extractionAll studies of the method “RCTs” from the subdiscipline “neurology” were selected. There was no year or language restriction. The subdiscipline “neurology” includes paediatrics and adult trials that involve “lesions of the central and peripheral nervous systems excluding those whose primary presentation is pain or paresthesia”.10 The extracted data for all included trials were: title, authors, year of publication, DOI, PubMed ID, Registration number, abstract, language, therapy, problem, body part, journal name, PEDro total score, and PEDro individual item scores. The data were sent to the authors in an Excel spreadsheet.
We used the Journal Impact Factor (JIF) and the Journal Citation Indication (JCI) to quantify journal ranking. The first one (JIF) is a “journal-level metric calculated from data indexed in the Web of Science Core Collection” .14 The second one (JCI) is a ranking with “the average Category Normalized Citation Impact (CNCI) of citable items (trials & reviews) published by a journal over a recent three-year period”.14 The 2022 JIF and JCI was downloaded from the Web of Science (Journal Citation Report). Two authors cleaned and analysed the data.
PEDro scaleThe methodological quality of the RCTs was evaluated with the PEDro scale, which was created according to a list developed by the Delphi technique.15 The scale includes 11 items: (1) Eligibility Criteria; (2) Random allocation; (3) Concealed allocation; (4) Baseline comparability; (5) Blind subjects; (6) Blind therapists; (7) Blind assessors; (8) Adequate follow-up; (9) Intention-to-treat analysis; (10) Between-group comparisons; (11) Point estimates and variability.16 Item 1 refers to external validity and it is not considered in the final score.15,16 Items two to nine are related to internal validity and items 10 and 11 are associated with statistics and they are used to calculate the total PEDro score which varies from 1–10.
There is no consensus on how to classify the methodological quality of trials according to the PEDro scale. However, a previous study classified the scores as: low methodological quality if the score is less than four points; moderate if the score is four or five points; good if it is six to eight; and excellent if it is nine or ten points.16 PEDro scale does not guarantee the validity of the study conclusion or the evidence that the exposed treatment is useful.15,16
Data analysisDescriptive statistics were conducted to explore the volume and the quality of the trials. Trends over time were investigated by analyzing the volume and quality (total PEDro score and individual items) of RCTs according to year publication divided into 4-year intervals. We used Spearman's rho correlation to investigate the association between the journal ranking (JIF and JCI) to the trial quality (PEDro total score) to answer the second question. The cut-off values of the correlation coefficient were: r >0.60: strong; 0.30–0.59: moderate; <0.29: weak.17
ResultsVolumeA total of 6291 RCTs in the area of neurological physical therapy in adults and paediatrics were indexed in PEDro between 1958–2021. A total of 1507 trials (23.9 %) were published between 2018–2021, which was the year with the highest number of publications. The period of 1962–1965 and 1966–1969 were the periods with the least number of publications, with 1 trial in each (Figs. 1 and 2).
From the 6291 indexed trials, the mean score on the PEDro Scale was 5.3/10 (SD1.5). The period with the highest PEDro score was 2018–2021, with 5.7/10 (SD1.4), and 1962–1965 was the period with the lowest score, with 1/10 (Fig. 3). There were 2786 (44.2 %) RCTs with a score between 4/10 - 5/10 and were considered to have moderate methodological quality. A total of 2763 (43.9 %) RCTs had a score equal to or higher than 6/10 and had a good methodological quality (Fig. 4). The percentage of RCTs meeting each item of the PEDro scale is shown in Fig. 5. Since 1958, most RCTs used random allocation (6048 trials - 96.1 %), between-group statistical comparisons (5904 trials - 93.8 %), point measures and variability data (5797 trials - 92.1 %), and groups similar at baseline (5021 trials - 79.8 %). The items that were not commonly fulfilled were blinding of therapist (70 trials - 1.1 %), blinding of subject (258 trials - 4.73 %), intention-to-treat analysis (1523 trials - 24.5 %), and concealed allocation (1862 trials - 29.5 %) (Fig. 5).
A significant and weak relationship was observed between the total PEDro score and both the 2022 JIF (r=0.153; p<0.001) and the 2022 JCI (r=0.146; p<0.001).
DiscussionThe present study presents data regarding the volume and quality of neurological physical therapy RCTs indexed to PEDro between 1958–2021. A total of 6291 neurological RCTs were indexed. The first study investigating the volume of neurological trials indexed to PEDro was conducted by Moseley et al.10 who found that until 2000, there were 238 RCTs in the subdiscipline of neurology. Other study investigating the volume of trials in PEDro,18 found a growth in the number of neurological trials which reached 1837 in 2011. The present study findings confirm the exponential growth of neurological RCTs indexed to PEDro over the years,19 showing an increase of 2643 % in the volume of trials (6053 trials) from the 2000s10 and 342.4 % (4454) from 2011.18 Moseley et al.1 found that the amount of PEDro indexed trials are doubling in every 6 years. Many reasons can justify the increase in neurological trials. One of them is the decrease of the stigma from functional neurological disorders, which leads clinicians to be more direct and open about their patient's diagnosis, and the individual to be more hopeful, helpful, and less frustrated about their condition.20 The increase in scientific evidence demonstrating the benefits and effects of rehabilitation methods may contribute to the increased interest in research in the neurological area. The successful evidences encourages clinicians to adopt evidence-based practices not only to guide their interventions but also to justify the underlying rationale for physical therapy.2,10,21 The increase of trials that offered good evidence, the expansion of its access and the advancement of specific technological tools for assessment and diagnosis are other causes that contributed to the development of the area.10,21
The mean PEDro Scale score found by the present study was 5.3 (SD1.5), and this might be considered of moderate quality.16 This value is slightly higher than the one found by Moseley et al18 which states a mean of 5.1 (SD1.5) for RCTs of all subdisciplines, and of 5.0 (SD1.6) for RCT for neurological physical therapy interventions. A similar result was also found for RCTs on musculoskeletal subdiscipline, which is the area with the highest number of trials in PEDro, where a mean score of 5.2 (SD1.6) was found.22,23 In the previous analysis of RCT of neurological physical therapy published in 2000,10 54 % of the RCTs were of moderate to high quality. The present study found that overall the quality of trials has improved since then with 69 % of neurological RCTs indexed to PEDro showing moderate to high quality. This indicates that neurological trials have a slightly higher methodological quality compared to other subdisciplines, with a minimal difference but showing a trend of continuous improvement based on previous studies.9,18
The present study demonstrated that the most satisfied criteria on the PEDro Scale were random allocation (96.1 %) and between-group statistical comparisons (93.8 %). Conversely, blind therapists (1.1 %) and blind subjects (9.5 %) were the least satisfied criteria, highlighting the practical challenges in implementing blinding in neurological physical therapy RTCs. Neurological physical therapy interventions often involve active participation and hands-on therapy, making it difficult, if not impractical, to achieve complete blinding of therapists and participants. This challenge is widely recognized in the literature, as noted by Moseley et al. ,24 who reported similar findings regarding the low feasibility of blinding therapists in physical therapy trials. Moseley et al. ,24 stated that the non-blindness of the subjects and therapists for most of the physical therapy RCTs happened because it involves physical activity, exercise, rehabilitation, and education. It is also declared that the items eligibility criteria, baseline comparability, between-group comparisons, point estimates and variability, concealed allocation and intention to treat analysis are criteria that can be satisfied by all trials, concluding that a score of 8 on PEDro scale could be reached for all physical therapy RCTs.23 Despite the fact that it is possible to perform intention-to-treat analyses and conceal allocation in all trials, in the present study these items were followed by less than 30 % of the RCTs, representing a huge room for improvement. The implementation of concealed allocation to prevent selection bias is often not adequately performed, undermining the comparability of treatment groups at the study outset.23 The adherence to intention-to-treat analysis poses another challenge, especially when there is loss to follow-up or incomplete protocol adherence. It is also possible to blind assessors in neurological trials where the outcome is not self-reported. All trials in the future should aim to fulfil these items and clearly report them in the manuscript.
The findings of the present study highlight that although many trials have been published, there is a small number of trials with fulfilling at least 8 items (9.4 % scored 8 or more). This suggests that while there is a significant volume of research, many studies still face challenges in meeting the highest standards of methodological rigor. A similar problem has been found in the physical activity field in an analysis of trials indexed in PEDro8 and more broadly.25 Improving these methodological aspects is crucial for robust, evidence-based treatments in clinical practice.24 Specific recommendations have been described elsewhere.8
There was a significant and weak relationship between methodological quality and the JIF and JCI scores of the journal where they were published, with the conclusion that the PEDro score is not related to the journal's impact factor.A previous study by Costa et al.26 examined the core journals of physical therapy, their JIF, and their RCTs PEDro scores and it was stated that there is no correlation between the methodological quality and the JIF.25 Pinheiro et al.8 had similar results to the present study, confirming that there is a small correlation between JIF and PEDro score of physical activity intervention trials indexed in PEDro.8 These two studies reinforce the present study findings of not relating the PEDro score to the JIF. Thus, RCTs with low methodological quality negatively impact the evidence of the trial, signaling researchers and clinicians to be alert to the RCTs conclusions and if they are endorsed by the article’s presented data.27
Comparing our findings with systematic reviews and methodological studies conducted in other databases provides insights into the challenges and improvements in methodological rigor across different research domains. Vinkers et al.28 examined over 176,000 RCTs in all medical disciplines published between 1966 and 2018, revealing a positive trend in bias reduction over time, particularly in journals with higher impact factors.28 Similarly, Nascimento et al.29 analyzed systematic reviews of low back pain and found that although journals with higher impact factors often endorse PRISMA recommendations, there was no significant association with the methodological quality of the reviews themselves, which frequently exhibited critically low quality standards.29 This underscores the ongoing need for rigorous reporting and methodological standards in systematic reviews beyond mere endorsement by high-impact journals.
The International Classification of Functioning, Disability and Health (ICF) is a tool that aims to evaluate the individual with a biopsychosocial model, to classify the functioning and/or disability of the person, while considering the contextual factors.30,31 Although PEDro does not classify the trials according to the ICF domains, this tool is extremally important to EBP. In the PEDro database it is possible to search trials according to the body part and problem, which has some correspondence with the ICF domains. Future studies could map the RCTs of neurological physical therapy interventions according to the ICF domains.
This study has some limitations. This paper is specific to RCTs about neurological physical therapy that are indexed to PEDro and cannot be extrapolated to other subdisciplines. Further research should study the volume and methodological quality of other subdisciplines of physical therapy. In addition, other PEDro codifications, such as body part should be more explored in new studies as well as the ICF use. As PEDro is a database that only indexes RCTs, systematic reviews, and guidelines, further studies should also investigate other databases to evaluate other study types that are also relevant to the literature. The trials with low scores on the PEDro Scale demonstrate flaws in the literature and open an opportunity for new investigations, showing that there are topics that need more studies to improve the interventions offered to society.
ConclusionThis study is an update of the previous report by Moseley et al.10 and reveals that the volume and methodological quality of neurological adult and paediatrics trials indexed to PEDro are increasing through the years. Moreover, it presents the correlation between the methodological quality and the journal impact factor, guiding clinicians and researchers not to choose trials based only on the journal's reputation because only a weak relationship was found.