This study was designed to develop and assess the measurement properties of a tool to measure pain impact in children in grades 4 to 6 (aged 9 to 12) – the Pediatric MSK Pain Impact summary score. The methods of this study were based on the COnsensus-based Standards for the selection of health status Measurement INstruments (COSMIN).17 We opted to include the set of items as part of the data collection of an ongoing cluster randomized controlled trial (ACTRN12616001228471), which we considered as a pragmatic approach. The randomized cluster trial was conducted following the Declaration of Helsinki and approved by the Hunter New England Human Research Ethics Committee (Ref. No. 06/07/26/4.04), University of Newcastle (Ref. No. H-2008–0343), and the Maitland-Newcastle Catholic Schools Office. Informed consent was obtained from all parents from all children involved in the study. According to the Committee on Health and Research Ethics, written informed consent has been obtained.

The definition of MSK pain impact for this study was "pain associated with a significant disability"9 from the MSK system (including bones, muscles, and joints)18 resulting in impairment of the functional status (activity and participation) as per the ICF in primary school-aged children aged 9 to 12.8 We proposed a summary score to measure how well the indicators of pain severity; impact on day-to-day activities, on sporting activities, and school absence reflect the construct of MSK pain impact.

We conceptualized the measurement of pain impact in a reflective model as the items (pain severity, activity limitation, limited participation) are common manifestations of pain impact (effect indicator)11 and, therefore, are expected to change when the construct (pain impact) changes.19,20

We collated pre-existing questions typically used to assess the presence of pain, pain severity, and the impact of pain on physical activity, day-to-day activities, and school absence. We included the questions in the baseline survey of a cluster randomized trial that aimed to assess the effectiveness of a school-based physical activity and nutrition intervention (Supplementary material A).21,22 We used the FPS-R to measure pain severity. The FPS-R is a valid instrument to measure pain severity in children aged 5 to 12.23-25 The FPS-R item responses are presented on a six-point scale ranging from 1 (no pain) to 6 (a lot of pain). We obtained permission from the developers to use the FPS-R as part of this summary score.

To measure activity limitations and participation restrictions following the ICF framework,8 we used three pain impact items capturing different aspects of the impact of MSK pain – restrictions in everyday activity, school, and sports. We adapted the wording of items from previous studies on adolescents with LBP to reflect the widespread MSK pain impact on primary school-aged children.7 The items asked children to indicate whether their pain caused them to; miss school, stop sports or physical activities, or interfered with everyday activity. Each item was rated on a Likert-type scale with the options; "often", "once in a while", "once or twice", and "never".

The scoring of the Pediatric MSK Pain Impact summary score is a sum of each response option from the four included items: the 6-point Likert pain severity scale (FPS-R) and the three items of pain impact (on 4-point Likert scales). The FPS-R scale score ranges from 1 (no pain) to 6 (a lot of pain). The three pain impact items 4-point Likert scale ranging from 1 (never) to 4 (often). A higher overall Pediatric MSK Pain Score summary score indicates greater functional impact due to MSK pain.

Classical test theory approach was applied to assess the measurement properties of the Pediatric MSK Pain Impact summary score (Appendix - www.spoergeskemaer.dk/pediatric-msk-impact). We used confirmatory factor analysis (CFA), proposing a one-factor structure, to examine the validity of the four-items-of-pain impact related to functional status.

We used baseline data from children enrolled in a cluster randomized trial conducted in 12 Catholic primary (elementary) schools in the Hunter New England Region, New South Wales, Australia. All schools were stratified by size (small, < 300 students; or large, >300 students) and placed in random order. To be eligible, the schools had to have an enrolment of greater than 120 students, be current users of the school mobile communication app (Skoolbag), and not participate in other nutrition or physical activity-based research studies. We aimed to assess the measurement properties of the pain impact questions in children in grades 4 to 6 (aged 9 to 12).

All students (aged 9 to 12) attending the 12 schools were invited to participate in the data collection component of the cluster randomized trial via a package sent to their parents, who were asked to provide written consent. Students with parental consent to participate were invited to complete a baseline survey (Supplementary material A). We included students in grades 4 to 6 (aged 9 to 12) who indicated in the baseline survey: 1) the presence of pain or aches in their body, selecting from response options "often," "once in a while," or "once or twice", 2) answered questions relating to the area of their body where they typically experienced pain, and 3) the items about the impact of their pain on their school activities.

• a.

  Item characteristics: We assessed descriptive properties (frequency, means, standard deviations, range, skewness, and kurtosis), the percentage of missing responses for each item, and the use of the distribution of responses for each item. We analyzed each item to assess their feasibility and acceptability to the construct of MSK pain impact. We assessed the correlation between each pair of items to identify possible redundancy. Polychoric correlations were used due to the ordinal nature of the items.16,26 Items with a polychoric correlation > 0.8 were examined for possible exclusion due to high correlation.27 Polychoric correlations can be weak (0.2–0.39), moderate (0.4–0.59), strong (0.6–0.79), or very strong (0.8–1).28,29

• b.

  Reliability: We assessed internal consistency as an indicator of the reliability of the questionnaire. Internal consistency is "the degree of the interrelatedness among the items" or the full scale.16,20 We used standardized Cronbach's alpha coefficient (adequate score: ≥ 0.70) and item correlations.27,30 We calculated a standardized Cronbach's alpha due to the differences between the response options for pain severity (six response options) and the impact questions (four response options).

• c.

  Construct validity: We used the following definition: "the degree to which the scores of a measurement instrument are consistent with hypotheses."16 We evaluated three indicators of construct validity: structural validity, convergent validity (hypotheses testing), and discriminative validity.

  • •

    Structural validity is the degree to which scores on an instrument adequately reflect the dimensionality of the construct.16 A CFA and weighted least-squares in complete cases (students completing the four items) were used to assess the overall adequacy of the model. We estimated the following CFA model fit indices and recommended criteria were estimated: Standardized Root Mean Square Residual (SRMR) < 0.0831,32; Comparative Fit Index (CFI) > 0.9531,32; Root Mean Square Error of Approximation (RMSEA) < 0.0632; Bentler-Bonett Normed Fit Index (NFI) > 0.9531; Model Chi-squared p-value > 0.0527,31; Goodness of Fit Index (GFI) > 0.9031; Adjusted GFI > 0.9031; and Parsimonious GFI, for which there is no accepted threshold, but values around 0.5 have been found adequate when other goodness of fit indices are above 0.90.31

  • •

    Convergent validity (via hypotheses testing) is the degree to which two measures, that are theoretically related, provide similar results.33 We formulated a priori hypotheses about the strength and direction of correlations to examine relationships between the pain impact measurement score and the comparative measures.16 Only students who answered all four items were used to calculate the overall score. We used Spearman rank correlation coefficients to assess the relationship between the overall summary score and the comparative measures. The comparative measures, collected at baseline in the trial, were: Pediatric quality of life Inventory (PedsQL) 4.0 total score and physical functioning scale23-25,34,35; care-seeking7; medication use7; and physical activity and sedentary behavior, both measured by accelerometers.36 The description of each comparative measure and each hypothesis is presented in Supplementary material B.

  • •

    Discriminative validity is the degree to which scores on an instrument distinguish differences between known groups.16 We evaluated discriminative validity by comparing pain impact scores by groups of students known to differ in frequency of MSK pain episodes. We used mixed linear regression to assess whether students with more frequent episodes of MSK pain scored higher on the Pediatric MSK Pain Impact summary score than those with infrequent pain.37 We selected from the included participants the students who experienced pain or aches in their body "often" and reported having pain in the last week (frequent pain). We compared them to those students who responded "once in a while" or "once or twice" and did not report pain in the last week (infrequent pain). The model included a random intercept for schools to account for clustering by school.

  • •

    Sensitivity analyses: We conducted post-hoc sensitivity analyses to compare the primary analysis results for consistency. First, we replicated the primary analysis with only the students who indicated they had pain over one week ("y/n"). We performed a second sensitivity analysis using the same inclusion criteria from the primary analysis but only included students who had experienced pain or aches in their body "often".

The internal consistency estimate measured by the standardized Cronbach's alpha coefficient was 0.63, below the threshold value of 0.70.27,30

• •

  Structural validity: all but two indices met the recommended criteria for a good fit, with the Chi-squared statistic and Bentler-Bonett NFI below the recommended threshold (Table 3).

  Table 3.

  Structural validity indices – confirmatory factor analysis.

  Type of Index	Fit index	Fit value	Recommended criteria
  Absolute	Pr > Chi-Square†	0.015	> 0.0531
  	Standardized RMR (SRMR)#	0.030	< 0.08 acceptable31,32
  	Goodness of Fit Index (GFI) #	0.993	> 0.9031,32
  Incremental	Bentler Comparative Fit Index#	0.955	> 0.95 ideal31
  	Bentler-Bonett NFI†	0.943	> 0.9531
  Parsimony	Adjusted GFI (AGFI)#	0.964	> 0.9031
  	Parsimonious GFI (PGFI)†	0.331	No criteria*,31
  	RMSEA Estimate#	0.073	≤ 0.0632

  †

  Indices below the recommended thresholds;.

  #

  Indicative of fair fit;.

  ⁎

  No accepted threshold; however, the penalties applied to this index often result in considerably lower values compared to other goodness of fit indices, PGFI values around 0.5 have been found when other indices are above 0.9031; GFI: Goodness of Fit Index; NFI: Normed Fit Index; RMSEA: Root Mean Square Error of Approximation; SRMR: Standardized Root Mean Square Residual.

• •

  Convergent validity (hypothesis testing): The total Pediatric MSK Pain Impact summary score was calculated from complete cases (n = 599). There were 16 students with at least one missing response on the pain impact items. The correlations with PedsQL 4.0 physical functioning scale (H2), physical activity (H4), and sedentary behavior (H5) did not meet the a priori hypotheses (Table 4). Three out of the six (50%) hypotheses were not confirmed.

  Table 4.

  Construct validity hypothesis.

  Hypothesis (h)		Correlation		p-value	Confirmed
  		Expected	Observed (95% CI)
  H1	PedsQL 4.0 total score	Moderate negative	Moderate negative−0.33 (−0.40, −0.26)	<0.0001	Yes
  H2	PedsQL 4.0 Physical Funct	Moderate negative	Weak negative−0.20 (−0.27, −0.12)	<0.0001	No
  H3	Care seeking	Moderate positive	Moderate positive0.45 (0.39, 0.51)	<0.0001	Yes
  H4	Medication intake	Moderate positive	Moderate positive0.37 (0.30, 0.43)	<0.0001	Yes
  H5	School Day MVPA	Moderate negative	Weak positive0.02 (−0.06, 0.10)	0.5410	No
  H7	School sedentary behavior	Moderate positive	Weak negative−0.05 (−0.13, 0.03)	0.1139	No

  95% CI, 95% confidence interval; H, hypothesis; Physical Funct, physical functioning; r, spearman correlation coefficient. Spearman correlation indices: weak: 〈 0.3, moderate: 0.3 to 0.5, strong: 〉 0.6.

• •

  Discriminative validity: 322 students were included in the analysis (122 with frequent pain and 200 with infrequent pain). The discriminative validity showed a mean difference of 2.93 (95% CI: 2.36, 3.50) in the Pediatric MSK Pain Impact summary score between children with frequent and infrequent MSK pain. The percentage of total variance in the Pediatric MSK Pain Impact summary score that could be attributed to the between-school variation was 0.9% (ICC: 0.009 [95% CI: 0.001, 0.127]; p<.0001) Students with frequent pain episodes scored 11 (95% CI: 10.5, 11.5), and students with infrequent pain scored 8 (95% CI: 7.7, 8.5) in the Pediatric MSK Pain Impact summary score.

The results of both sensitivity analyses (Supplementary material C) did not modify the main findings substantially.

We found that the Pediatric MSK Pain Impact summary score had insufficient reliability (internal consistency) and marginal construct validity to assess the impact of MSK pain in children aged 9 to 12. The internal consistency was minimal, with a Cronbach's alpha (0.63) below the recommended criteria (> 0.70).16,27 The polychoric item correlations were adequate. The construct validity may be considered minimal even though estimates met agreed thresholds for most structural validity indices, convergent validity (50% of hypotheses confirmed) and discriminative validity (2.93; 95% CI: 2.36, 3.50). Despite the shortcomings, the instrument showed a reasonable discriminative validity as it could discriminate pain impact levels between children with frequent and infrequent pain.

The Cronbach's alpha (0.63) was below the recommended threshold of 0.70, suggesting limited internal consistency. Therefore, internal consistency for this measure should be interpreted as minimal/emerging.38 This is because it is possible that Cronbach's alpha results may be due to the few items of the instrument. Instruments with less than eight items can have lower Cronbach's alpha without necessarily meaning low interrelatedness of the items.27,30 Also, greater Cronbach's alpha estimates are expected in larger samples, meaning that small changes are less likely to affect the overall score, and because our sample is large (n = 815), our confidence in the internal consistency result remains fairly strong.39 Nevertheless, future investigation into the internal consistency of the measure should be considered.

The analysis of structural validity suggests that the summary score reflects the unidimensionality of the construct of pain impact in school-aged children (aged 9 to 12). It is considered best practice to report at least three goodness-of-fit indices, such as SRMR, RMSEA, and CFI.16,31,32 In our study, the goodness-of-fit indices met the recommended criteria. However, we acknowledge that most indices were very close to the recommended threshold, which may still be insufficient compared to other measures.27

The Pediatric MSK Pain Impact summary score showed moderate convergent validity but poor divergence with other constructs. Two of the three expected positive correlations (care seeking and medication intake) met our pre-specified correlation threshold. Whereas one of three expected negative correlations (quality of life) met the pre-specified threshold. However, physical function showed only a weak negative correlation. The remaining two hypotheses were rejected due to a weak unexpected positive correlation (physical activity) and an unexpected negative correlation (sedentary behavior). While this unexpected cross-over (or lack of) with other constructs may undermine the Pediatric MSK Pain Impact summary independence as a construct, we found the score to have good discrimination of "known group" who would be expected to have a higher burden of pain.40

We acknowledge the limitations of our study. First, we opted for a pragmatic approach to develop and assess the measurement properties of the instrument. Following the recommended development steps19 could have provided valuable insights for item reformulation or additions to the instrument items. We selected and adapted pre-existing items used in low back pain studies7 and embedded the items into an existing trial without conducting pilot and field-testing steps. Additionally, the skewness observed in the response items may suggest the presence of a potential floor effect at the item level, which possibly reflects limited content validity and is likely to limit the potential responsiveness of the measure. Due to feasibility and capacity constraints, we were unable to evaluate additional measurement properties such as Standard Error of Measurement (SEM), Minimal Detectable Change (MDC), responsiveness, and Minimal Clinically Important Difference (MCID). Future research should consider these aspects to provide a more comprehensive evaluation of the instrument's performance.

Further research should be conducted to expand the applicability of the instrument. First, future studies could improve the content validity of the instrument by asking the perspectives of children, parents, and teachers about the relevance, comprehensiveness, and comprehensibility of the items, response options, and instructions. Furthermore, future research should investigate how emotional, cognitive, and social aspects of pain can be integrated into this short instrument, as these aspects can potentially affect children's activity and participation.6 However, it is important to emphasize that while considering adding new items to the instrument in the future, maintaining its brevity is important for reducing respondent burden while ensuring reliable and valid estimates. Lengthy outcome measures can be particularly hard for children to complete.11 Finally, it is recommended to assess the measurement properties of this instrument in other age ranges. As children's cognitive development evolves with age, their perceptions of subjective pain constructs may differ. Exploring this across various age groups will yield valuable insights into the measure's validity and applicability across the developmental spectrum.41,42