This study comprised a study of measurement properties using cross-sectional data. The measurement theories and analysis used in this study follow the guidelines outlined by Costello and Osborne,27 Prinsen et al.25 (COSMIN group), and Worthington and Whittaker.26

The sample consists of 672 workers from three previous studies10,29 – 192 blue-collar workers from the manufacturing industry (n = 141 females and n = 51 males, mean ± standard deviation (SD) age 34.5 ± 8.3 years);10 370 call center operators from a private electrical company (n = 168 females and n = 202 males, mean age 26.8 ± 7.7 years);29 and 110 office workers from a public university in Brazil (n = 63 females and n = 47 males, mean age 33.8 ± 9.5 years; unpublished data). The studies were conducted in accordance with the Helsinki declaration, and all participants provided their written informed consent prior to entering in each study. The Human Ethics Committee of the Universidade Federal de São Carlos approved the studies by the registration processes #0054.0.135.000.07, #1080.0.000.135–10, and #0068.0.135.000–10.

Collected outcome measures used in this study include the Br-NFR scale10 and basic demographic information. The Br-NFR scale is an 11-item Likert scale with four response categories: “Never” = 0; “Sometimes” = 1; “Often” = 2; and “Always” = 3. The individual sum of these scores is converted to an index from 1 to 100, where 100 indicates the maximum requirement for recovery. All items included in the Br-NFR scale and their English version are provided in Table 1.

For the analyses we used a split-half validation method in which the participants of these three groups were randomly divided into two groups – a development sample and a validation sample – each with 336 participants.23,26 The random division was performed using the ‘sample’ function of the software R v4.1.1,30 which allocates each worker to one of the two groups without using any external variables. Thus, avoiding introducing any bias in sample allocation. We then used the development sample to explore the factor structure using EFA. We used the validation sample to check the structural validity, criterion validity, and internal consistency of the structure identified by the EFA using CFA, ICC, and Cronbach's alpha.

Descriptive data summaries are presented by means of absolute number [percentage] or mean ± standard deviation. In addition, the response distributions and the correlation matrix are shown for each item of the scale for the complete sample as descriptive data, as well as the missing responses for each sample.

The EFA was used to examine the factor structure of the Br-NFR with the development sample. Suitability (factorability) of the data set for EFA was evaluated based on Bartlett's test of sphericity – used to assess whether the correlation between items was adequate based on a criterion of p<0.05, and the Kaiser-Meyer-Olkin (KMO) statistic – used to measure sample adequacy based on a criterion of ≥0.80.26

Through EFA, the factor structure was explored with the implementation of a polychoric correlation matrix with the Minimum Rank Factor Analysis (MRFA) extraction method.31 If a structure with more than one factor was found, the solutions for the factors would be examined using the direct Oblimin (oblique) rotation to ensure independence of the items, otherwise, the rotation would be set to none. We identified the number of factors to be retained through parallel analysis with random permutation of the observed data.26,31

To the best of our knowledge, there is no consensus on the criteria for item deletion or retention. Therefore, for a one-factor structure, items with a factor loading <0.32 and communality <0.40 were considered candidates for deletion.26,27 On the other hand, for a structure with two- or more factors, items with factor loading <0.32; communalities <0.40; cross-loadings <0.15 in difference from the highest factor loading among an item; factor loading >0.32 in two or more factors were considered candidates for deletion.26,27 It is noteworthy that care has been taken in applying all of these criteria, because EFA is an interactive, multi-step process. Moreover, the highest factor load was used to decide whether an item loaded on a specific factor.

The structural validity of the factor structure extracted during EFA was evaluated using CFA on the validation sample. The CFA was performed with the implementation of a polychoric matrix and the Robust Diagonally Weighted Least Squares (RDWLS) extraction method.32 Considering that the Br-NFR is a Likert-type scale with four response categories, the use of RDWLS is more suitable for ordinal data than other extraction methods (e.g., maximum likelihood).32 The model fit of the factor structure was evaluated using several indices: Pearson's chi-square (χ2) test; chi-square divided by the degree of freedom (χ2/df ratio; <3.00 is adequate, <5.00 is acceptable); Comparative Fit Index (CFI) and Tucker-Lewis Index (TLI; >0.90 is acceptable, >0.95 is excellent); the Root Mean Square Error of Approximation (RMSEA) with the 90% confidence interval (RMSEA; <0.08 is acceptable, <0.06 is excellent); and standardized root mean square residual (SRMR; <0.08 is acceptable).26,33 Additionally, Akaike Information Criterion (AIC) and Bayesian Information Criterion (BIC) were considered to compare the structures tested in this study (see the description below) and the lowest value was considered the most appropriate.26,33 Moreover, factor loads ≥0.40 were considered adequate for the domain.

The structure extracted by EFA (i.e., Brazilian structure) was compared with five structures found in the literature: Dutch,3,9 Swedish,16 Iranian,14 and Chinese13 (11-item [Dutch being the original structure]); French11 (10-item, item 10 was excluded); Italian12 (10-item, item 4 was excluded); and Danish15 (9-item, because items 2 and 3 were merged on the original scale, we compared two structures of the Danish scale with our own – one without items 2 and 11 [Danish 1] and one without items 3 and 11 [Danish 2]). All these structures contain the same concept with a one-factor structure that considers all items belonging to a domain of need for recovery. Moreover, the evaluation of the structures involved checking the quality of the goodness-of-fit indices. In addition, the standardized residual covariance matrix and the modification indices of the Brazilian structure were verified, then theoretical decisions were made to item deletion or retention, to identify which items would be more likely to represent the Br-NFR concepts.26,33

The total scores of the previously specified structures (i.e., Brazilian structure validated through CFA, French, Italian, Danish 1 and 2) were compared with the total score of the original scale of 11-item using the ICC2,1 (two-way random, single measurement, absolute agreement) to assess criterion validity.23 For the purposes of validation, it was decided a priori that the criterion validity between the scales should be excellent (i.e., ICC ≥0.95). This comparison was performed to assess whether the structures are an adequate reflection of the original structure of the NFR scale.23,25 According to COSMIN group,23,25 the original long version can be considered the “gold standard”. Internal consistency of the final structure was evaluated with Cronbach's alpha.23 As well as in the criterion validity, it was decided a priori that the final structure must have good internal consistency (>0.70).Finally, all analyses were performed using the software R v4.1.130 using ‘EFA.MRFA’ v1.1.2,34 ‘psych’ v2.1.635 and ‘lavaan’ v0.6–936 packages.

The complete sample consisted of 672 workers (528 females [78.6%] and 144 males [21.4%]) with a mean age of 30.7 ± 9.0 years and NFR index of 37.5 ± 17.5. After the split-half validation method, these variables were consistent across the development and validation samples (development sample: 259 females [77.1%], 77 males [22.9%], mean age of 30.6 ± 9.0 years, mean NFR index of 37.6 ± 17.2; validation sample: 269 females [80.1%], 67 males [19.9%], mean age of 30.8 ± 8.9 years, mean NFR index of 37.3 ± 17.8). Among the 672 participants, 11 cases with missing responses were identified, seven in the development sample and four in the validation sample. These cases were treated by means of list-wise deletion (i.e., the individual with one or more missing item was excluded). Thus, 661 participants were included in the data analysis. The distribution of the missing responses is presented in Table S1 of Supplementary Online Materials, as well as the distribution of responses for each item in Fig. S1 of Supplementary Online Materials. The correlation matrix for each item of the scale, of the complete sample, is presented in Table S2 of Supplementary Online Materials.

The development sample data set met the assumptions to proceed with the EFA. The Bartlett's test of sphericity yielded significant results, χ2(55)=1852.6, p<0.001, and the KMO test yielded a statistic of 0.92 (ranging from 0.57 to 0.95 between items; see Table S3 of the Supplementary Online Materials). The parallel analysis (Fig. 1) that determined the number of factors to be extracted showed support for a one-factor structure.

Fig. 1.

Parallel analysis. The one-factor structure is shown by the vertical black dashed line. Of note – factors above the line of the simulated data (95th percentile) indicate the number of factors in the structure.

(0.06MB).

The factor loads and communalities for the 11 items of the scale are shown in Table 2. The one-factor structure accounts for 49% of the total variance. In general, the items had factor loadings ranging from 0.61 to 0.83 and communalities ranging from 0.45 to 0.69 (all above the pre-specified criteria). The exceptions were item 4, which demonstrated low value of factor loading (0.11) and communality (0.01) and item 11, which showed a low communality value (0.38). As previously mentioned, EFA is an interactive and multi-step process, therefore, the entire analysis was redone without item 4, then without item 11, and finally without both items. The same result of a construct with one-factor structure was confirmed in this analysis (data not shown). Thus, the structure without items 4 and 11 extracted by EFA was used in the following analyses.

When examining the CFA results through the goodness-of-fit indices, all structures evaluated in the study demonstrated excellent to acceptable fit on all indices (χ2/df <5.00, CFI >0.95, TLI >0.95, and SRMR <0.08), except the RMSEA which was above the acceptable cut-off point of <0.08 (Table 3). Inspections of the standardized residual covariance matrix and the modification indices of the Brazilian structure showed that items 2 and 5 have high standardized residual covariances with several other items, suggesting they should be excluded. Furthermore, a comparison of AIC and BIC statistics among the structures revealed that the Brazilian structure without items 2, 4, 5, and 11 demonstrated superior fit compared to the other structures evaluated in the study (i.e., lower values; cf. Brazilian structure, Table 3), proving to be the most adequate structure tested. The parameter estimates of the Brazilian structure had moderate to high standardized coefficients (i.e., >0.67) for all items. The standardized coefficients and associated data of the Brazilian structure are presented in Table S4 of Supplementary Online Materials.

When comparing the criterion validity of the various pre-specified structures (final version of the Brazilian structure with 7 items, French, Italian, Danish 1 and 2) to the original 11-item scale, our analysis showed excellent ICC values with all structures meeting our criteria for validation (ICC ≥0.95, decided a priori) (Table 4). Moreover, the results suggest the idea that the Brazilian structure obtained by CFA (cf. Brazilian structure, Table 3) is equivalent to the original scale of 11 items. Furthermore, the Cronbach's alpha for the Brazilian structure was 0.85, showing good internal consistency.

The hypothesis of the unidimensionality of Br-NFR was confirmed by the EFA. This result is in agreement with previous studies that selected the number of factors based on the eigenvalue-over-one criterion that is less robust than the parallel analysis employed in this study.9,11-13,16,28 On the other hand, this result differed to some extent from the content validity conducted in the study by Stevens and collaborators,15 which suggested that the scale has a two-factor structure. However, the EFA performed in the study15 showed that the scale is unidimensional, which is consistent with our results and previous literature. Furthermore, our structure explained 49% of the total variance of the scale, while the original9 and French11 versions explained 48% and 44% of the total variance, respectively.

The Brazilian structure demonstrated acceptable to excellent fit and lower values of AIC and BIC, compared to the other pre-specified structures evaluated in the study (original3,9 [Dutch], Swedish,16 Iranian,14 Chinese,13 French,11 Italian,12 Danish15 1 and 2), confirming our initial hypothesis that the structure extracted in this study would present acceptable goodness-of-fit indices. Thus, the Brazilian 7-item structure is a singular one compared to the literature regarding excluded items; i.e., the Italian12 version deleted item 4, the French11 version excluded item 10, and the Danish15 version, merged items 2 and 3 and excluded item 11.

Regarding the criterion validity, all tested structures proved to be equivalent to the original scale of 11-item as they met our a priori criteria (i.e., ICC ≥0.95). The internal consistency of 0.85 found in this study was similar to that of other NFR (i.e., original3,9 [Dutch], Swedish,16 Chinese,13 French,11 Italian12) versions which ranged between 0.78 and 0.90.

The validation of the Br-NFR contributes to the improvement of a more valid measure, giving greater confidence in the results of future studies that use the scale, as well as for its use by occupational health services. In addition, decreasing the instrument length from 11 to 7 items reduces the response time, decreasing the burden on the respondent.37,38 In particular, the removal of items 4 and 5 is helpful, as these items are confusing for those who work night shift and return home during the day, and for workers who work six days a week without the possibility of having two days off. Thus, the exclusion of items 2, 4, 5, and 11 is an advantage, creating greater clarity and ease of applying the Br-NFR.

Strengths of the present study are the adequate sample size and the analysis conducted according to COSMIN consensus25; the implementation of the polychoric correlation matrix when using polytomous data31,32; and a small number of missing values in the dataset. However, some limitations need to be considered. First, the data used to calculate the criterion validity came from the same sample that responded to the 11-item scale, which could influence the results. Nevertheless, as the scale has a short length, we believe this would be unlikely to influence the results. In addition, for this study, only blue-collar workers, call center operators, and office workers were included, limiting the use of this validated structure for other occupational groups. However, this is still a very diverse sample (e.g., different shifts, working hours, working days per week), ensuring wide external validity. Notwithstanding these limitations, our study offers a contribution to research in the field of ergonomics due to the high methodological quality and the validation of the NFR structure in a large population.

We recommend that future studies should evaluate other measurement properties of the Br-NFR with 7 items, such as cross-cultural validity/measurement invariance, reliability, measurement error, hypothesis testing for construct validity, and responsiveness.25 We also suggest that the structure validated in this study should be tested in other languages using CFA and that the use of a polychoric correlation matrix and coherent extraction methods for ordinal data should also be considered.31,32