Physical therapists obtain information from a variety of sources. The sources may influence their believability and use in clinical practice.
ObjectivesIn this hypothesis-based study, we queried physical therapists (PTs) on the believability of evidence across six musculoskeletal treatment domains and analyzed variables that predicted the strength of beliefs.
MethodsThis international survey included six different language portals and used a snowball dispensation strategy. PTs who were credentialed, licensed, or who practiced in the field, were queried on the believability of six treatment domains (i.e., exercise, manual therapy, psychologically-informed practice, sports/occupational performance, thermal/electrical agents, and pain science/patient education) and potential predictors of believability (i.e., social media use, years of practice, time and access to literature, specialization, confidence in reviewing literature and attributions of the researcher).
ResultsIn total, 1098 PTs from 36 countries completed the survey. PTs had strong beliefs in what they read or hear about exercise, sports/occupational performance, pain science/patient education, and psychologically-informed interventions. There was only moderate believability regarding manual therapy treatment and weak believability associated with thermal/electrical agents. Multiple linear regression analyses revealed that the most robust predictor to outcome relationships included time and access to literature and believability of pain science/patient education, years of clinical practice and believability of psychologically informed practice, and believability of thermal/electrical agents.
ConclusionAn important takeaway from this study is that believability was influenced by several factors (primarily by years of practice, attributions of the researcher, and time and access to literature) and appeared to vary across treatment domains.
Historically, journal articles and search engines such as PubMed were the most frequently used sources of evidence-based information for physical therapists (PTs).1 However, PTs have been progressively inundated with information on treating patients with musculoskeletal disorders. This information has many forms, such as journal publications, social media, clinical practice guidelines, free training videos on the internet, podcasts, online and live continuing educational courses, conferences, and peer-to-peer interactions. Even well-informed, motivated clinicians have difficulty navigating the extreme amount of information (i.e., >1.275 million biomedical papers are published each year).2 Because most physical therapy-related musculoskeletal treatments have similar effects and are “in the gray zone,”3 disseminators of evidence are looking at creative ways to support their intervention preferences.4
Recent studies have shown that predatory publications have eroded the believability (trustworthiness) of information in the health professions secondary to an increasing numbers of retractions, replication failures, biased results from corporate-sponsored studies, researcher bias, spin, method inadequacy, omission, or withholding of contradictory results, dropping of unsupported hypotheses,5-9 and an emphasis on publishing studies that demonstrate “incredibility more so than credibility”.10 In addition, randomized controlled trials are hampered by p-hacking,11,12 changing research questions from the primary outcome,13 or modifying the analysis after initial findings fail to support the researchers' beliefs13 through statistically significant results.14 By definition, believability is the quality of being credible, convincing, trustworthy, or realistic.15 For clinicians, believability occurs when they find both the information and the source are credible and trustworthy. A lack of believability might be one of the many reasons there is a 17-year delay in implementing research into practice.16
Musculoskeletal interventions used by PTs include several domains, such as therapeutic exercise, manual therapy, sports/occupational performance, psychologically-informed treatment, pain science/patient education, and thermal/electrical agents (modalities). Over the last 50 years, thousands of studies have explored each domain's treatment effectiveness. Unfortunately, the findings are mixed and characterized by inconsistencies in the fidelity of interventions within treatment domains.17-19 Mixed findings and inconsistencies in fidelity can lead to a distrust of reported findings and, ultimately, a potential lack of believability in the information provided. Fidelity and believability issues may be why 50% to 70% of PTs use interventions of unknown value.20,21
We used an international survey that queried PTs on the believability of musculoskeletal treatment domains (i.e., exercise, manual therapy, psychologically-informed practice, sports/occupational performance, thermal/electrical agents, and pain science/patient education). Our survey was available in English, French, Spanish, Portuguese, German, and Italian. We used a hypothesis-based approach for this study,22 designed to investigate a pre-set series of assumptions before data collection and analyses. Hypothesis-based approaches use specific provisional statements that are designed to measure the relationships between two variables. Hypothesis-based approaches identify the variables of interest, the expected strength of association of the variables, and function as a substitute for traditional aims or objectives in a study.22 For this study, we focused on answering three primary directional hypotheses:
Hypothesis One: Treatments involving manual therapy and sports/occupational performance would demonstrate moderate believability scores, whereas exercise, psychologically informed practice, thermal/electrical agents, and pain science/patient education would demonstrate strong believability scores.
Hypothesis Two: All predictive variables would exhibit bivariate relationships with our treatment domains. Specifically, we hypothesized that social media engagement, confidence in reviewing the literature, attributions of the researcher, and time and access to literature would be positively associated with stronger believability. Conversely, we hypothesized that advanced certificate or masters-level training and years of clinical practice would exhibit negative associations with treatment believability.
Hypothesis Three: During multivariable modeling, higher social media engagement and attributions of the researcher would be positively associated with stronger believability across the majority of treatment domains and would yield the highest effect measures of all predictors.
MethodsThe Consensus-Based Checklist for Reporting of Survey Studies (CROSS) guidelines23 were used to design and report this study. The study was approved by Duke University's institutional review board (Pro00109794).
The study design was an international, cross-sectional survey that was implemented through social media platforms (e.g., Twitter, Facebook, Instagram) using snowball sampling. It was accessible from November 5th 2021 to December 13th 2021 (38 days). Reminders were sent weekly to maximize the survey visualization. The study targeted licensed, qualified, or registered PTs worldwide who also had access to the social media platforms that were used to identify respondents. Only PTs were included in the summary statistics.
We created a novel questionnaire that had 27 questions and was divided into three sections. The first section of the survey was designed to query individuals' believability of the information they read/hear on the six domains of physical therapist musculoskeletal treatment interventions. The six domains included: a) therapeutic exercise; b) manual therapy-based treatment; c) psychologically-informed practice (e.g., cognitive behavioral therapy); d) sports/occupational performance; e) thermal/electrical agents (modalities); and f) pain science/patient education. The six domains were decided upon by the consensus of the authorship group and were confirmed for appropriateness by eight external research PTs.
The second section included several questions related to the beliefs and experiences of PTs who completed the survey. Some of the sections' items were modified from pre-existing tools,24-26 whereas others were novel. The goal of the second section was to query their social media engagement, time dedicated to reading research and access, the PTs' confidence in reading research, and the role of the researcher attributions (the researcher's reputation). Finally, the third section was designed to capture demographic data regarding the respondent.
Study investigators included PTs from the United States, Brazil, France, Germany, and Chile. A majority of investigators are bilingual, whereas two are multilingual. All investigators' native tongues were represented in this study. The investigators used dual back-translation from the English consent and survey document (all cases involved at least two people). All those who participated in back-translation were PTs and native to the languages translated and had the material context. Each back-translator works in a translation role (from English to their language), in continuing education courses, journal editorship, or university education. A majority were also educators with PhD training.
We pre-tested the face and content validity of the survey by giving it to 30 PTs of varying clinical backgrounds in which English (N = 11), French (N = 4), Portuguese (N = 3), Spanish (N = 4), German (N = 5), and Italian (N = 3) was the primary language. For pre-testing, the PTs were instructed to indicate the clarity of the survey and whether the information addressed their thoughts associated with believability (content validation). We edited the survey to reflect the clinicians' suggestions and had six distinct surveys in English, French, Portuguese, Spanish, German, and Italian that represented the same underlying context, but were modified for each country's unique cross-cultural distinctions. The 30 PTs who provided input were not part of the final survey; their role was to contribute to the face and content validity of the survey.
We captured three primary types of data for our outcomes of interest: 1) outcomes, 2) predictors, and 3) descriptive. Outcomes' data included the six treatment domains. For each of the six domains, respondents provided a believability score that ranged between 1 (complete disbelief) and 5 (complete belief), reflecting what PTs' read or heard.
Predictor variables included those for the second and third sections of the survey. Those pertinent to the study included social media engagement, time dedicated to reading research, confidence in reading research, researcher attributions (reputation), specialization training, and years of clinical practice. In addition, several variables were summated to create the single targeted predictor variable in many cases. Finally, additional descriptor variables included sex, highest clinical degree, highest academic degree, area of practice, continent and country of practice, and specific specializations (e.g., sports, manual therapy, pain science, etc.).
We powered our study for hypothesis three, which would require multiple variable regression analyses. A sample of at least 98 individuals providing 80% power to detect an effect size of 0.15, allowing for six predictors and an alpha level of 5% was required.27 Because of variability in electronic survey sampling (e.g., incomplete surveys, etc.), we targeted a much larger sample because the survey was available for completion across multiple languages and involved snowball sampling methods.
Qualtrics served as our survey platform. We set a benchmark that at least 60% of the survey must be completed before considering its use in the secondary objective. Missing data were managed using Listwise deletion, which instructs the statistical software to skip the missing variable and omit it from the analysis.28
We analyzed means, standard deviations, proportions, and percentages to describe our survey respondent population. For our first hypothesis, we reported the mean and standard deviation of the believability scores (1 to 5, a score of 1 representing disbelief, and a score of 5 representing complete belief). Based on consensus selection across the authorship group, we categorized strong believability of the evidence as scores from 3.75 to 5.0. A moderate believability of the evidence included scores from 2.75 to 3.74, whereas a weak believability of the evidence included scores from 0.0 to 2.74.
We ran univariate linear regression analyses for each treatment domain using our predictor variables for the secondary hypothesis. We reported unstandardized beta coefficients, standardized Beta coefficients, t-statistics, and p values for each analysis. Standardized beta coefficients were calculated so that the variances of dependent and independent variables equal 1.29 Standardized coefficients are unitless (but transferable across analyses) and refer to how many standard deviations a dependent variable will change per standard deviation increase in the predictor variable. Higher values from zero reflect a greater effect size. T statistics are calculated by dividing the beta coefficient by its standard error.29 It is a form of measure of precision in which larger values reflect a stronger relationship (and effect) between the predictor and the outcome variable.30 If the standardized coefficient and t statistic are negative, the association has an inverse relationship.30
A multiple linear regression analysis was completed to answer our third hypothesis. After checking for violations of collinearity, relationships that demonstrated a p-value of <0.15 were included in a multiple linear regression analysis (with a reverse stepwise regression) to determine what combination of beliefs and experiences of the PTs were related to the believability score for each domain. For each analysis, p values of <0.05 were considered statistically significant.
ResultsInitially, 1429 individuals consented to participate in the survey. Of those, 325 (28%) did not complete the requisite 60% of the survey. Of the 28% who did not complete 60% of the survey, nearly every survey left the believability questions blank. An additional six reported they were not PTs and were removed from the dataset. This left 1098 (77% of the original sample) completed (>60%) surveys for analyses. Respondents were from 36 different countries across six continents (Fig. 1). A majority were from Europe (67.5%) and South America (22.4%), practiced in musculoskeletal settings (89.9%), and identified as male (70.1%). Certificate or Master's level training was reported in a low of 17.7% (psychologically-informed practice) to a high of 36.8% (masters of musculoskeletal medicine) of the sample (Table 1).
Respondent characteristics (N = 1098).
Hypothesis One: The highest mean levels of believability were for the treatments associated with exercise (4.34/5.0), followed by pain science/patient education (4.18/5.0), and then psychologically informed practice (4.03/5.0) (Table 2). These findings confirmed our first hypothesis that exercise, psychologically-informed practice, and pain science/patient education would exhibit strong believability. Findings did not confirm our hypothesis regarding thermal/electrical agents, which scored weak believability. Overall, manual therapy believability was moderate, which supported our hypothesis; however, sports/occupational performance exhibited strong believability, albeit barely (we projected only moderate believability).
Believability of evidence associated with physical therapy interventions.
Hypotheses Two: Table 3 outlines our six predictor variables' bivariate linear regression associations with the six practice domains, respectively. As we hypothesized, higher social media engagement levels were positively associated with believability. This was present in four of the six treatment domains (exercise, psychologically-informed treatment, sports/occupational performance, and pain science/patient education). Time and access to the literature were positively associated with two domains (manual therapy and thermal/electrical agents), which confirmed our hypothesis. They were negatively associated with one domain (pain science/patient education), which refuted our hypothesis. Confidence in reviewing the literature was negatively related to the believability of pain science/patient education, which refuted our hypothesis. As hypothesized, years of clinical practice and certificate or Master's training were negatively associated with believability for most domains.
Univariate relationships between respondent characteristics and believability of evidence.
Area of Belief | Unstandardized Beta Coefficient | Standardized Beta Coefficient | T-score | P value | |
---|---|---|---|---|---|
Certificate or Master's Training in Dedicated Areas | |||||
Exercise treatment | −0.087 | .047 | −1.83 | .06 | |
Manual therapy treatment | .064 | .034 | 1.14 | .27 | |
Psychologically-informed treatment* | −0.108 | .049 | −2.20 | .03 | |
Sports/occupational performance | .029 | .017 | .550 | .58 | |
Thermal and electrical agents | .012 | .005 | .154 | .88 | |
Pain science and education* | −0.186 | −0.105 | −3.49 | <0.01 | |
Self-Reported Social Media Engagement Levels | |||||
Exercise treatment* | .063 | .016 | 4.09 | <0.01 | |
Manual therapy treatment | .026 | .019 | 1.39 | .16 | |
Psychologically-informed treatment* | .045 | .085 | 2.78 | <0.01 | |
Sports/occupational performance* | .048 | .085 | 2.79 | <0.01 | |
Thermal and electrical agents | .029 | .035 | 1.14 | .25 | |
Pain science and education* | .055 | .097 | 3.16 | <0.01 | |
Years of Clinical Practice since Graduation | |||||
Exercise treatment* | −0.046 | −0.101 | −3.29 | <0.01 | |
Manual therapy treatment | .017 | .030 | 0.99 | .32 | |
Psychologically-informed treatment* | −0.065 | −0.138 | −4.53 | <0.01 | |
Sports/occupational performance | −0.004 | −0.008 | −0.258 | .79 | |
Thermal and electrical agents* | .095 | .127 | 4.15 | <0.01 | |
Pain science and education* | −0.081 | −0.158 | −5.22 | <0.01 | |
Time and Access to Literature | |||||
Exercise treatment | −0.010 | −0.012 | −0.387 | .69 | |
Manual therapy treatment* | .069 | .072 | 2.34 | .02 | |
Psychologically-informed treatment | −0.024 | −0.030 | −0.964 | .34 | |
Sports/occupational performance | .073 | .038 | 1.23 | .22 | |
Thermal and electrical agents* | .156 | .118 | 3.89 | <0.01 | |
Pain science and education⁎⁎ | −0.064 | −0.071 | −2.32 | .02 | |
Attributions of the Researcher | |||||
Exercise treatment | .123 | .082 | 2.67 | <0.01 | |
Manual therapy treatment | .149 | .082 | 2.67 | <0.01 | |
Psychologically-informed treatment | .092 | .060 | 1.94 | .05 | |
Sports/occupational performance | .116 | .070 | 2.29 | .02 | |
Thermal and electrical agents | .135 | .055 | 1.78 | .07 | |
Pain science and education | .089 | .053 | 1.72 | .08 | |
Confidence in Reviewing the Literature | |||||
Exercise treatment | −0.020 | −0.025 | −0.829 | .41 | |
Manual therapy treatment | .025 | .026 | .857 | .39 | |
Psychologically-informed treatment | −0.045 | −0.056 | −1.83 | .06 | |
Sports/occupational performance | .020 | .023 | .758 | .45 | |
Thermal and electrical agents | .078 | .060 | 1.96 | .05 | |
Pain science and education⁎⁎ | −0.087 | −0.099 | −3.24 | <0.01 |
Hypothesis Three: For the six different multiple linear regression analyses, there were three variables that were consistent predictors of the believability outcomes: 1) years of clinical practice, 2) attributions of the researcher, and 3) time and access to the literature (Table 4). Although we hypothesized that the researcher's social media engagement and the attributions of the researchers would increase the believability of research, this was not supported by our findings. The strongest predictors in the models were the relationship of time and access to literature and believability of pain science/patient education (t statistic=−4.73), and years of clinical practice and the believability of psychologically-informed practice (t statistic=−4.53) and believability of thermal/electrical agents (t statistic=4.34). Social media engagement was included in only one model (exercise), whereas attributions of the researcher were positively associated in three models (believability for exercise, manual therapy, and sports performance).
Multiple variable relationships between respondent characteristics and believability of evidence.
We explored the believability of selected treatment domains that are common in the profession of physical therapy. We opted to query PTs worldwide because PTs generally have similar scopes of practice and use common sources of educational materials, journals, and social media share sites. We used a hypothesis-based approach to query a priori assumptions22 to ensure that the variables were selected for good reasons and contributed to the overall research question.22 We were correct on several hypotheses but found selected instances where the opposite finding occurred. Our results are worth discussing further.
Hypothesis One: Overall, there was strong believability across most treatment domains. Our findings suggest that the respondents believed what they read or heard regarding exercise, sports/occupational performance, pain science/patient education, and psychologically-informed interventions. This strong level of belief occurs despite challenges in the quality of the literature used to summarize treatment effectiveness. For example, more than 87% of systematic reviews exploring musculoskeletal physical therapy interventions have “critically low” quality on the AMSTAR 2, with one-half of the reviews “spinning” the results and one-third generating conclusions based on predominantly low-quality clinical trials.31 It has also been identified that 58% of the randomized clinical trials included within systematic reviews exploring musculoskeletal physical therapy interventions have unknown external validity and include randomized clinical trials with internal validity scores rated as low quality (< 6/10) on the PEDro scale.31
These limitations notwithstanding, exercise for musculoskeletal conditions is well supported in the literature for treatment of a variety of diagnoses, such as low back pain,32 frozen shoulder,33 and neuropathic pain.34 Evidence supports the use of pain science/patient education and psychologically-informed practice.35-37 Still, these interventions exhibit slightly smaller effect sizes within the literature35-37 (as compared to exercise), and might be more effective when combined with exercise.38 Sports/occupational performance gaps in the literature are quite pronounced,39-41 which is why we hypothesized only moderate believability.
Our study's believability around manual therapy was moderate (mean ± SD: 3.53 ± 0.77). This may be a reflection of the negativity involving manual therapy on social media,42 as manual therapy treatments are supported by systematic reviews,43–46 and independent clinical studies,47–54 and are embedded in several clinical practice guidelines.55–59 Believability around thermal/electrical agents was weak (2.69 ± 1.07) and was the lowest of the six treatment domains. At present, literature supporting thermal/electrical agents as a stand-alone intervention is limited,60 although evidence in preclinical work suggests disease-modifying mechanisms.61 We are uncertain why respondents appear not to believe this published evidence (or what they hear on social media); perhaps they see value in using thermal/electrical agents, which was not queried in our study.
Hypothesis Two: Unsurprisingly (because of our sampling methods), 60.4% of individuals indicated that they routinely used social media to improve their physical therapy knowledge. Social media can effectively disseminate information because the necessary data are not restricted behind a firewall.4 It is important to recognize that there is a high risk of misinformation on social media, and further exploration of its potential for promoting less than credible and untrustworthy results is needed.62 Risks of misinformation are especially problematic when social media influencers have a poor background in research and/or are biased in their opinions or support for a specific intervention.4
We found a positive relationship between the researcher's attributions (the researcher's reputation of record) and the believability of the evidence in a treatment domain. In fact, 75.5% of respondents reported that the attributions of the researcher influenced their believability of the information. Researchers prefer to cite other researchers with strong reputations.63 Just a single widely cited high-impact article is very likely to cause what the group calls “a reputation boost”.63 In contrast, the Matthew effect, which is a phenomenon that well-known scientists often get more credit for their work even if it is very similar to that of an unknown colleague,64 can negatively influence unknown researchers who develop similarly designed studies. This suggests that well-known researchers can influence clinical practice by their suggestions and biases, especially if combined with an active social media presence.
Physical therapists pursue advanced training for several reasons, including adapting to an expanded role of a first-contact provider, widening one's skill set to address a broadening scope of practice, and curiosity.65 A past study has shown that fellowship-trained PTs were more likely to achieve greater treatment effect sizes than PTs without residency or fellowship training.66 However, the benefit of years of experience is less convincing within the literature as it does not seem to improve outcomes.67,68 We found that increased years in practice and certificate and Master's level training were inversely associated with the believability of the literature for pain science/patient education, exercise, psychologically-informed treatment, and thermal/electrical agents. We hypothesized that increased clinical experience would be related to a lower likelihood of believing what is read or heard because clinicians would likely develop a defined pattern over time. Variations in literature may not fit the pattern. Future studies should examine this further.
Hypothesis Three: In our multiple linear regression models, three predictors were consistently retained: years of clinical practice, attributions of the researcher, and time and access to literature. Years of clinical practice was discussed previously, was negatively associated with believability, and was retained in four of the six models (exercise, psychologically-informed practice, thermal/electrical agents, and pain science/patient education). Attributions of the researcher was also previously discussed and was retained in three of the six models (exercise, manual therapy, sports/occupational performance). We find it interesting that two treatment domains (manual therapy and sports/occupational performance) have historically involved strong advocates with charismatic influences on treatment approaches (more so than published evidence), which suggests they still influence practitioners. Time and access to the literature were positively associated with the believability of manual therapy and thermal/electrical agents but negatively associated with pain science/patient education. The strongest effect size was the negative association between time and access to literature and believability of pain science/patient education. Recent reviews suggest that there is value in pain science/patient education for chronic non-specific spinal pain,69 migraine and overlapping pain conditions,70 and chronic musculoskeletal pain when combined with exercise.71 Potentially, because pain science/patient education is complex and involves dedicated study, the negative believability reflects the lack of training in this area.
Limitations: There are several limitations to this study. Using social media to identify candidates for the survey is a form of selection bias, which should have increased the importance of social media as a predictor within the study; it did not. The sample was Eurocentric, with most respondents completing the survey in French, English, or German. A majority of respondents were also male, which does not reflect the sex characteristics of the profession but does reflect activity on social media. There is a risk of order bias because the questions were not alternated in the survey. To consolidate the survey, we combined sports and occupational performance and pain science education and patient education, and one could argue that these factors are independent of one another. Although significant efforts were made for face, content, and cross-cultural validity, there is a risk that our novel survey may lack validity. Finally, in our survey, we did not measure the source of believability (e.g., Twitter, manuscript, podcast, etc.). We did not attempt to measure if the respondents believed the literature supported or condemned a dedicated treatment approach.
ConclusionAlthough most PT respondents believed the evidence they read or heard about exercise, sports/occupational performance, pain science/patient education, and psychologically-informed interventions, there was only moderate believability regarding manual therapy treatment and weak believability associated with thermal/electrical agents. Several factors influence the believability of the information, the most compelling involved years of clinical practice (which had a negative relationship toward believability), attributions of the researcher (which was positively associated with believability), and time and access to the literature (which had mixed associations).
The authors would like to thank all the participants of the international survey.