Rasch analysis resulted in an improved Norwegian version of the Pain Attitudes and Beliefs Scale(PABS)

2.1 Design and participants

Data for our analysis of the PABS were collected from Norwegian physiotherapists responding to a cross-sectional web-based survey, as described in detail elsewhere [34]. Briefly, 3849 Norwegian physiotherapists were invited by e-mail to fill out the 36 items of the PABS, together with demographic and professional data. Written information was provided regarding the purpose of our study. Consent of responders was assumed if they completed the questionnaire. The study was accepted by the Norwegian Centre for Research Data (project nr. 28806). Responses from 921 therapists were obtained (response rate 24.8%). Therapists who had not been involved in back pain management for the last 6 months were excluded (n = 147). The remaining 774 participants filled out the PABS questionnaire, whereof 679 provided valid PABS scores.

The PABS was initially developed by adapting items from four questionnaires, as well as items developed by the researchers. The instrument aims to discriminate between biomedical and biopsychosocial treatment orientations of physiotherapists in LBP management [17]. Responders indicate on a six-point Likert scale (1 = totally disagree, 6 = totally agree) their endorsement on each statement. Treatment orientation is measured on two subscales, one labelled “biomedical” (10 items), and the other “biopsychosocial” (9 items). Subscale scores are calculated by a simple summation of the responses to the subscale items. Higher scores on a subscale indicate a stronger treatment orientation.

2.2 Data management and analysis

In order to assess whether discarded items from the original 36-item pool may contribute to measurement when added to the original subscales, all 36 PABS items were subjected to exploratory factor analysis (EFA) for assignment to either a biomedical or a biopsychosocial item set. The purpose of this EFA was not a validation of dimensionality, but rather to get an overview of item clustering and to identify a candidate set of items for Rasch analysis. Following principal component analysis with oblimin rotation of the total sample (n = 667), 20 items loaded onto the first component (ten items in addition to the original ten biomedical items), whereas 16 items loaded on the second component (seven items in addition to the original nine biopsychosocial items, see the supplementary material of this paper). Subsequently, four item sets were extracted corresponding to the original 10-items biomedical subscale (BMS-10), the original 9-items biopsychosocial subscale (BPSS-9), an extended 20-items biomedical item set (BMIS-20) and an extended 16-items biopsychosocial item set (BPSIS-16). The four item sets were separately entered into the RUMM2030 software package [44] and examined for item/scale performance, person measurement and scale-to-sample targeting.

The frequency of responses, including missing data, for each item was assessed. Gender and age were entered as person factors. Participants were categorized in five age groups: 20–30 years, 31–40 years, 41–50 years, 51–60 years and over 60 years of age. Significant likelihood ratio tests (p < 0.0001) suggested that the partial credit Rasch model was the most appropriate to use for analysis.

2.3 Rasch analysis

The Rasch measurement model presumes that any person should always have a greater probability of receiving a higher score on an “easier” item than on a more “difficult” item and that persons with a high level of the trait should have a higher probability of receiving a higher score on any items compared to persons with lower levels [45]. Essentially, Rasch analysis involves a series of tests to see if the data meet the presumptions of the Rasch model, which besides the above mentioned probabilistic relationship between items (stochastic ordering), include local independency and unidimensionality [37]. When Rasch requirements are fulfilled, the scale’s ordinal, raw scores may be transformed into linear, interval level measures and parametric calculations such as means and differences can be applied [36, 38, 41, 43, 46].

Overall fit of data to the model was checked with item-trait interaction statistics. A non-significant chi squared probability value (Bonferroni adjusted) indicates that the hierarchical ordering of the items is consistent across all levels of the trait and does not compromise the required property of invariance. The fit of individual items and persons was assessed by inspecting their fit residuals. This examines the difference between their observed and expected logit values. Potential misfit was considered if fit residuals were beyond standardized +±2.5 (99% CI), or if an item showed a significant chi squared probability value. High negative fit residuals are normally interpreted to indicate redundancy of an item, whereas high positive fit residuals indicate misfit. The mean of all fit residuals across all items of the scale and across all person estimates was examined and should be close to 0 with a standard deviation (SD) close to 1, preferably <1.40.

The assumption of local response independency is violated when responses of persons to an item not just depend on their trait level, but on their responses to other test items [47]. Local dependency artificially inflates reliability and results in spurious multidimensionality. Local dependent items were identified through a residual correlation matrix between all items. Two items were considered to be dependent if the residual correlation between them was more than 0.20 above the average residual correlation of all items [48].

Scores on individual items should only be summed if the scale measures one single latent construct. Violation of this assumption of unidimensionality is a potential source of misfit. To test the assumption of unidimensionality, independent t-tests were carried out on an individual basis. A t-test was done for each person, comparing the ability estimates derived from two subsets of the most diverting items [49].The number of significant t-tests in the sample determined the degree of unidimensionality of the scale. Significant multidimensionality was noted to be present if more than 5% of these t-tests were significant or if the lower confidence interval of the observed proportion fell below the 0.05 value in a binominal test of averages [50].

Differential Item Functioning (DIF) by gender and age was tested. DIF is identified when subgroups of respondents (e.g. males and females) with the same level of the trait, respond to items differently, thus violating the requirement of invariance.

The Person Separation Index (PSI) provides an indication of how reliably the (sub)scale is able to discriminate between person locations and is equivalent to Cronbach’s alpha for internal consistency. A PSI >0.70 was considered desirable for group-level comparisons and taken as evidence of sufficient discriminative ability [51].

Targeting of persons and items. Scale-to-sample targeting was made by comparing person locations from the sample with item locations of the scale. Their means should be close to 0 logits with a SD close to 1. A person-item threshold distribution histogram informs about the suitability of the sample for evaluating the scale and the suitability of the scale for measuring the sample.

Response Thresholds Ordering. An increase in response option in items, represented by their transition point between categories (thresholds), should reflect a logical progression of the underlying trait. If this does not occur, response thresholds are disordered. This may be the case when responders cannot reliably distinguish between the presented categories. Ordered response thresholds are a prerequisite to obtain reliable parameter estimates and necessary rescoring by collapsing categories was done before any further scale improvements were attempted [41, 52].

For scale refinement purposes, the Rasch analysis was progressed in two alternative ways [53]. In the original two subscales, resolution A attempted to account for the misfit that had been highlighted. Resolution A sought to maintain as many original scale items as possible by making the appropriate amendments to account for response dependency and DIF. Where amendments could not be made to account for the source of misfit, individual items were removed from the item set. For local dependency, the dependent items were grouped into “testlets”, meaning that the total raw score derived from the items did not change, but the dependent relationship between the items had been eliminated [54, 55].

In resolution B, misfitting items were removed iteratively to obtain a pure set of items which satisfied all fit parameters. Then, the removed items were individually reintroduced back to see whether or not the original source of misfit was still apparent. If the source of misfit was still present, then the item would again be removed. Resolution B sought to find a set of items, free from any form of significant individual or collective misfit, which act together to form a unidimensional scale [56].

As chi-squared statistics for almost all tests of model fit tend to become significant and will indicate misfit when sample size increases [57], we validated our results by creating two randomized split half samples and then repeat our analyses.

3.1 Original biomedical subscale (BMS-10)

Initial analysis of the BMS-10 showed misfit to the Rasch model and slight response threshold disordering of item 24. Individual item fit revealed evidence of four problematic items displaying fit parameters outside the normally expected and accepted range.

Misfitting items were typically located at the extremes of the continuum. Summery fit statistics are presented in Table 2. Following rescoring of item 24, all items displayed ordered categories. At this stage, response dependency was apparent between item 10, 25, 20 and 22. There was no evidence of DIF by age or gender. Individual item locations on the logit scale, sources of item misfit at this stage and the rescored response code are summarized in Table 3. A hierarchy of item endorsement (ranging from most to least endorsed) across the sample was apparent. Items concerning the belief that tissue damage/structural deficits are important issues in LBP were located at the more “difficult” side of the item hierarchy, characterizing high levels of biomedical attitude when endorsed. Items concerning spinal vigilance and recommendations to adapt activity to pain seemed to cluster at the “easier” side of the hierarchy, characterizing lower levels of the trait (Table 3). The scale-to-sample targeting seemed adequate (Fig. 1). The mean person location (–0.66 logits, SD 0.71, range –3.15 to 1.80) indicated that the subscale was targeted at somewhat higher levels of biomedical treatment orientation possessed by responders in this sample. The item thresholds (range –2.63 to 3.69) spread over a broad range of the construct and exceeded the spread of person measures. A limited spread of the trait in the population could be read from the ten class interval locations ranging from –1.93 to +0.60 logits, thus covering less than three logits.

Fig. 1

Upper graph: Person-item threshold distribution of the 10-items biomedical subscale (BMS-10). The distribution of persons (upper plot) and items (lower plot) are compared on the same logit scale. Lower graph: Person-item threshold distribution of the refined, unidimensional 7-items biomedical solution. Item thresholds on the negative end of the continuum are more likely to be endorsed (easy items), whereas items at the positive end are less likely to be endorsed (difficult items).

3.3 Inclusion of initially discarded items into the biomedical subscale

The original biomedical subscale (BMS-10) was supplemented with 10 items distracted from the original 36-items pool. The resulting item set (BMIS-20) failed to meet Rasch model expectations (Table 2) and eight items displayed disordered thresholds. Following rescoring all items displayed ordered thresholds. At this stage, extensive response dependency was apparent between ten items, whereas four items displayed DIF by gender. Sources of individual item misfit are summarized in the supplementary material of this paper. Scale-to-person targeting was comparable to the original BMS-10, as indicated by its mean person location (–0.88 logits, SD 0.61).

Misfitting and biased items were removed iteratively for scale refinement. The result was a resolution B corresponding to the biomedical 6-items core item set (items 10,14, 23, 24, 30, 31) supplemented with re-introduced item 4 (Reduction of daily physical exertion is a significant factor in treating back pain). This solution was strictly unidimensional and free from any form of misfit, whereas the PSI increased close to recommended values (PSI = 0.69). Reasons for removal of items are listed in Table 4.

3.4 Original biopsychosocial subscale (BPSS-9)

Initial analysis of the BPSS-9 revealed misfit between the data and the model (Table 5). Three items showed fit parameters outside the normally expected and accepted range. Seven of nine items had disordered thresholds, meaning that their response categories were not functioning as intended. Ten out of the 54 response categories (19%) had no or very few (>10) observations. These null response categories were located in the lowest categories of six items. Category threshold curves for items 6 and 30 are illustrated in Fig. 2.

Fig. 2

Category probability curves. In item 30 (uppergraph), all response options are ordered. In item 6 (bottom graph), response options 1, 2 and 3 are disordered.

Following rescoring, overall fit slightly improved and all items displayed ordered categories. At this stage, response dependency was apparent between items 6, 7, 27 and 33. No evidence of any form for DIF was found. The summary fit statistics at this stage are presented in Table 5. Individual item locations on the logit scale, sources of item misfit and rescore codes are summarized in Table 6.

The scale-to-sample targeting was adequate (Fig. 2). The mean person location (0.36 logits, SD 0.73, range –1.93 to 2.36) indicated that the subscale was targeted at slightly lower levels of biopsychosocial treatment orientation possessed by responders in this sample. The item thresholds covered a wide range of the underlying construct (range –2.84 to 3.43), exceeding the person measures. However, there were gaps between items near the mean person location, indicating a deficiency in measurement capacity. A limited spread of the trait in the population could be read from the nine class intervals which ranged from –0.75 to +2.01, covering less than 3 logits.

3.5 Scale refinement of the BPSS-9

A satisfactory resolution A could not be reached. Despite attempts to account for local dependency, the items set continued to display misfit parameters and weak fit to the model. Resolution B was reached following removal of three items (items 6, 7 and 27). The remaining set of six items was strictly unidimensional, invariant and free from any form of misfit. Reliability remained insufficient (PSI = 0.61). Summary fit statistics are presented in Table 5 and reasons for item removal in Table 4.

3.6 Inclusion of initially discarded items into the biopsychosocial subscale

The biopsychosocial subscale (BPSS-9) was supplemented with seven items distracted from the original 36-items pool. The resulting item set (BPSIS-16) failed to meet Rasch model expectations (Table 5). Twelve out of sixteen items displayed disordered thresholds. Following rescoring, all items displayed ordered thresholds. At this stage, sizeable response dependency was apparent between five items, whereas item 36 displayed DIF by gender. Five items showed fit parameters outside the accepted range. Details on sources of individual item misfit and rescore codes are summarized in the supplementary material of this paper.

Scale improvement resulted in a resolution B that was reached after successive removal of nine misfitting or biased items (Table 5). The remaining seven items (core item set 11, 12, 17, 29, 33, 34 in addition to re-introduced item 19 If ADL activities cause more back pain, this is not dangerous) were strictly unidimensional, conforming to the Rasch model and free from any form of misfit and with a PSI of 0.64.

All analyses were repeated using two randomized split half samples (n = 336 and n = 331) for validation reasons. Comparable fit statistics and resolutions were also found in these two smaller samples, suggesting that sample size did not influence fit to the model in this population.

3.7 Transformation of ordinal raw scores to interval scaling

After fit to the Rasch model was achieved for modifications of the two subscales, we were able to produce transformation tables (see Appendix) that can be used to convert raw ordinal-level scores to interval-level scores. These transformation tables can be used in parametric data analyses when there are no missing data and distributions are appropriate.

4.1 Biomedical subscale

A set of seven items appeared to be the optimal solution for a unidimensional biomedical subscale of the Norwegian PABS version. Item 4 of the original 36-items version qualified for reintroduction. This item has previously been included in another study [34]. Multidimensionality of the original subscale was found to be related to the effects of local response dependency of individual misfitting items. Apparently, responses to these items were dependent on the responses to other items and not just on the persons’ trait level. This dependency may be explained by several items concerning tissue damage being too similar to each other. Local response dependency is known to be a factor which spuriously inflates reliability when not addressed [48] and this may explain why the internal consistency of our unidimensional 7-items solution (Cronbach’s alpha = 0.67) is lower than values reported in other studies, ranging from alpha = 0.72 to 0.84 [17, 19, 26, 32, 33, 59]. Reliability values for a bi-factor solution which retained all items, were even lower (PSI = 0.60).

4.2 Biopsychosocial subscale

Our analysis indicated that a set of seven items was the optimal solution for a unidimensional version of the Norwegian PABS. Item 19 of the original 36-items version was re-introduced. This item has previously been included in another study [32]. Removal of misfitting items 6, 7 and 27 was found to improve reliability and rendered a unidimensional biopsychosocial subscale. Apparently, the responses to these items were unrelated to the responses to the remaining items and the underlying trait. Items 7 and 27 seemed to represent an attitude challenging the traditional biomedical perspective and appeared hard to endorse, whereas the other items rather seem to validate a biopsychosocial approach, addressing the beneficial influence of activity despite pain, exercises and positive coping.

Most items of the biopsychosocial subscale were found to have incorrectly ordered response categories that needed to be modified [52]. Rescoring of items was necessary to obtain reliable parameter estimates [52], but had the follow-on effect of reducing the total scale score. With all response options in place, the biopsychosocial solution would be scored 7 to 42, whereas after rescoring the total scale score should be contracted to 7–32. Consequently, information values of any observation will be reduced and the precision of measurement decreased [45, 51]. We also found that response categories, mainly in the lower response options of the items, were not fully utilized by the responders. This was evident in our final 7-items solution, where only 15.6% of item responses (range 0.9% to 28.3%) were found in the lower three options of all seven items, indicating large agreement levels on biopsychosocial issues among physiotherapists.

Although our analysis improved reliability of the original 9-items biopsychosocial subscale, separation indexes were still below recommended values for two-group comparisons. The low internal consistency of the biopsychosocial subscale found in other studies has been related to the low number of items [26, 32]. However, in this study, the low PSI seems to be a function of the homogeneous group of physiotherapists to which the scale was administered, as shown by the limited distribution range of person locations along the scale (Fig. 2) and the limited variation on the levels of treatment orientation. The PSI depends on how well-targeted the scale is, but moreover on the “ability” distribution of the respondents, as it is harder to separate persons when they are close in ability [57]. Whether the high agreement levels among Norwegian physiotherapists on biopsychosocial issues also account to other countries or other health care professions is unknown and would be a subject for further research.

4.3 Distribution of scores

Although the person item threshold distributions (Figs. 1 and 3) suggested that both subscales were well-targeted, some item thresholds on the end range of the continuum were found to have no discriminative function, as there were no persons at that item locations [60]. Gaps between biopsychosocial items where the largest part of the sample was found indicate that persons cannot be sufficiently discriminated from each other. Apparently, items are needed at the middle/higher range of biopsychosocial orientation, as this is the range where researchers would be more apt to evaluate change over time.

Fig. 3

Upper graph: Person-item threshold distribution graph of the rescored original 9-items biopsychosocial subscale (BPSS-9). Lower graph: Person-item threshold distribution graph of the refined, unidimensional 7-items biopsychosocial solution.

The transformation tables (see Appendix) allow for simple conversion to interval level scores. The use of PABS interval subscales holds important consequences for responsiveness and calculations of aspects such as minimal important change (MIC), since change here is linear throughout the range of the scale [43]. Conversely, small changes in the ordinal score will be more relevant at the margins than in the middle of the scale. As can be seen from the biopsychosocial transformation table, a raw score change of 3 points represents an interval-level change of at least 3.2 points at the upper or lower margins of the scale, but only 1.4 points in the middle range of the scale. Thus, the interval subscales appear more stable across the construct.

4.4 Strengths and limitations

Previous studies employed factor analysis to reduce the number of items and to determine the dimensionality of the PABS [17, 19, 21, 32, 33, 34]. The inconsistent number of items included by these studies might be due to limitations associated with using ordinal data in factor analysis [61, 62]. As Rasch analysis constructs linearity out of ordinality and provides item and person location on the variable, it seemed necessary to perform Rasch analysis of a large sample.

One other important reason for using Rasch modelling to evaluate item performance is the ability to examine whether response categories are correctly ordered, i.e. to evaluate whether the responders utilized the response categories as they were intended to (as logical increasing levels of treatment orientation) and that all response categories are utilized [57]. Conversely, classical test theory a priori assumes that response thresholds are ordered. Using a scale with disordered thresholds to detect the effect of interventions is problematic, since it will be difficult or even impossible to evaluate a change in categories [46]. Disordered response thresholds appeared to be a major problem in the biopsychosocial subscale and may have contributed to the limited distribution range of our population and the low reliability. Although all item disordering was resolved by collapsing the disordered thresholds, no substantial improvement in the fit to the model was seen. Hence, disordering could perhaps be explained by the presence of null response categories and not by the responders having problems discriminating between the categories [57].

The large sample size (n = 667) provided very robust estimation of the threshold parameters and consequently the response category disordering. However, our convenience sample was not representative for the source population of Norwegian physiotherapists.

An obvious selection bias was present with an overrepresentation of specialized physiotherapists [34]. Conversely, a wide variation in responses from a diversity of clinicians was required in order to avoid clustering of scores [60]. Our sample with different specialties seemed to provide this variability sufficiently.