Participation of children with disabilities is an indicator of social inclusion, health, and well-being, and its evaluation needs to be included in the clinical practice of rehabilitation professionals.
ObjectivesTo investigate the measurement properties of the Participation and Environment Measure - Children and Youth Brazilian version (PEM-CY Brazil).
MethodsWe used the PEM-CY to evaluate participation and environment of children with and without disabilities in the home, school, and community settings. Based on COSMIN guidelines, we analyzed two measurement properties: internal consistency for all domains using Cronbach's alpha; and construct (known-groups) validity, i.e., the degree to which instrument scores identify differences between groups. T-tests, Mann-Whitney, or Chi-square tests compared children with and without disabilities.
Results101 Brazilian children (mean age=9.31 years) with (n = 62) and without (n = 39) disabilities were included. Internal consistency ranged from appropriate (0.70) to excellent (0.95) for all domains in all settings, except for Resources in the environment of the home setting (0.53). Regarding construct validity, PEM-CY participation scores were significantly different between groups in most domains. Children with disabilities were significantly less involved and participated in a smaller number of activities in all settings, in comparison to those without disabilities. The PEM-CY scores in all settings of the environment were significantly higher in the group of children without disabilities.
ConclusionPreliminary support was provided for the internal consistency and construct (known-groups) validity of the PEM-CY Brazil to measure participation and the environment in the Brazilian context. The PEM-CY can therefore be used by rehabilitation professionals in Brazil.
Participation of children with disabilities is an indicator of social inclusion, health, and well-being, and this population presents lower participation rates when compared to children without disabilities.1 The literature shows that children with cerebral palsy, the most prevalent cause of physical disability in childhood, participate less frequently in activities carried out at home, school, and in the community.2,3 Similarly, children with Down syndrome face barriers to participations such as negative behaviors, and unavailability of specialized programs and professionals, which are associated with lower participation.4 Children with autism spectrum disorders, which prevalence is on the rise, also have a lower participation frequency, are less involved in activities, and their parents identify more barriers in the environment.5 Participation restrictions, which last into adulthood, could be minimized by providing adequate support to families during childhood, delivering timely interventions, and by enhancing autonomy.6
The International Classification of Functioning, Disability and Health: Children and Youth Version (ICF-CY) defines participation as “involvement in a life situation”, which for children means to participate in activities at home, in school, and in the community.7 Participation is a complex multidirectional construct,8 influenced by social, attitudinal, and physical aspects of the environment.9 Imms et al.8 have further proposed a family of participation-related constructs (fPRC), which includes attendance and involvement in relevant activities, to capture the participatory experiences of children.8 Attendance is an objective component of the family of constructs, defined as being present in a situation.8,10 Involvement is a subjective component, defined as the experience of participation including elements of motivation, persistence, social connection, and affect.8,10
In recent years novel measurement tools have been developed to enable the evaluation of participation in specific environments.11,12 The Participation and Environment Measure - Children and Youth (PEM-CY) was developed based on the theoretical framework of the ICF-CY, and through extensive conceptual work with parents of children with and without disabilities.13 The PEM-CY provides data on the patterns of participation of children and young people with and without disabilities, as well as the environmental factors that support or hinder participation in daily activities.13,14 Differently from other tools,15–17 this instrument simultaneously evaluates aspects of participation and of the environment in the home, school, and community settings. Another advantage of the PEM-CY is that it captures the elements of attendance (frequency domain) and involvement (involvement domain), thus considering the advances in the understanding of participation provided by the fPRC.8,10
The original version of the PEM-CY presents good indexes of internal consistency, test-retest reliability, and concurrent validity.14,18 The PEM-CY also has the ability to differentiate the participation and the environment between groups with different characteristics, such as children with and without disabilities, and from families across a varied range of income.14,18 Before using a tool developed in another language and context, it is necessary to perform a transcultural adaptation process, and to establish the measurement properties of the adapted version.19,20 The PEM-CY has been successfully translated and validated for other languages and cultures, including in Korea,21 China,22 Germany,23 India,24 and Turkey.25
Two recent literature reviews on children with CP showed that in Brazil, and in other low- and middle-income countries, participation is less frequently studied and explored as a rehabilitation outcome when compared to outcomes related to body structures and functions.26,27 Another recent study investigated the participation of Brazilian children with disabilities using the Children's Assessment of Participation and Enjoyment and the Preferences for Activities (CAPE & PAC), two companion measures of children's participation, but they are not available yet for use in clinical practice in Brazil.28 The availability of valid and reliable assessment tools is essential so participation starts to be used as a rehabilitation outcome. The Brazilian-Portuguese version of the PEM-CY (PEM-CY Brazil) was translated and cross-culturally adapted to Brazilian Portuguese following a structured multi-phase process,29 but its measurement properties have not yet been tested in the Brazilian population. The objective of the present study was to establish measurement properties of the PEM-CY Brazil in terms of: 1) reliability (internal consistency), and 2) construct validity (known-groups validity).
MethodStudy designThis measurement property study was based on the taxonomy, terminology, and definitions of measurement properties of the COnsensus-based Standards for the selection of health status Measurement INstruments (COSMIN), by Mokkink et al.19 The reliability was examined by analyzing the internal consistency of the PEM-CY items, that is, the degree of the interrelatedness among the items.19,30 Under COSMIN, construct validity is the degree to which the instrument scores are consistent with hypotheses based on the assumption that the instrument validly measures the construct to be measured.19 Construct validity was tested by examining the ability of the tool to distinguish between groups with and without disabilities,19 also named discriminative or known-groups validity.30 The COSMIN checklist30 considers appropriate a sample size greater than 50 participants for internal consistency, and greater than 50 participants per group for known-groups validity.
The study involved researchers from four Brazilian Universities: Federal University of Rio Grande do Norte (UFRN), Federal University of Paraíba (UFPB), Federal University of São Carlos (UFSCar), and Federal University of São Paulo (UNIFESP). Each center obtained approval from its respective Research Ethics Committee. Parents and/or guardians were informed about the study, and signed an informed consent form.
ParticipantsParents and/or caregivers of children and adolescents between 5 and 17 years, with or without disabilities, participated in the study. Participants were conveniently recruited from five urban cities in three Brazilian states (Paraíba, Rio Grande do Norte, and São Paulo). To be included, participants needed to be parents or caregivers for a child/youth aged 5–17 years; and able to understand and answer the questions asked in Brazilian-Portuguese. Parents of children with disabilities were recruited from public and private rehabilitation centers in the participating cities. We tried to achieve a diverse sample in terms of diagnosed condition (e.g., motor and intellectual impairments), socioeconomic status, and child age, because previous research suggests that restrictions in participation are not necessarily related to the underlying diagnosis, rather to a variety of child- and context-related factors.31–34 Parents of children without disabilities were recruited by sending invitations to public and private schools or delivered through snowball sampling. Participants who did not complete the questionnaire were excluded.
MeasuresThe PEM-CY was developed as proxy measure using the perception of parents/caregivers to determine participation of children and young people with and without disabilities in activities at home (10 items), school (5 items), and in the community (10 items) settings, as well as the environmental characteristics pertinent to each setting (12–17 items). The original version of the instrument presented moderate to good internal consistency (alpha > 0.59), and test-retest reliability (ICC>0.58).14
The instrument is self-explanatory; for each group of activities the respondent rates the frequency of the child's participation, the degree of involvement, and if they would like to see some kind of change in their child's participation.13 Various characteristics related to the physical, social, and attitudinal environment are evaluated as to whether they support or hinder children's participation.35 The PEM-CY includes an item scoring and application sheet available for purchase at the CanChild website.20Table 1 summarizes PEM-CY domains, and how to calculate the scores.14,36 Each domain has an independent score, the higher it is, the greater the participation or characteristic of the environment (Table 1).
Participation and environment measure - children and youth summary scores.
This information was extracted from the original User's Guide32 and from the study of Coster et al., 2011.13
At the beginning of this study, the research team perceived challenges in calculating the scores manually, a fact that could limit the clinical and scientific use of the instrument. We therefore developed the PEM-CY Brazil Calculator, initially for the purposes of this study, which calculates the results automatically.37 This calculator was created in Excel by a team of five researchers, one of whom had programming experience and the others who had participated in the cross-cultural adaptation of PEM-CY Brazil.29 To calculate all the scores (participation and environment), we carefully followed the scoring instructions from the manual of the original version.
The calculator includes an electronic version of the PEM-CY that is formatted with the same layout, question, and response options as the original PEM-CY. It was designed so data provided by respondents can be entered directly into the PEM-CY Brazil Calculator, which records the data in an Excel spreadsheet. With this spreadsheet, it is possible to store the results for a group of subjects and perform statistical analyses. The open answers are also stored, allowing qualitative analyses. In addition, the PEM-CY Brazil Calculator generates individual graphs and tables, which allow a simplified visualization of the results obtained by a subject. An example of a final report displaying the score sheet and graphs can be viewed as supplementary material (Supplementary online material). It enables knowledge sharing and facilitates clinical practice by making the information more accessible to family members and professionals who work with the child. The calculator will be made available for use in Brazil as soon as a decision regarding copyright aspects is made.
ProceduresParents/guardians who agreed to participate completed a sociodemographic and clinical data form. The questions addressed the children's sex, age, diagnosis, and school attendance; respondent's relation with the child, sex, marital status, and education; family income; and city of residence. The PEM-CY Brazil questionnaire was used to evaluate participation and environmental factors of all participants. The original version of PEM-CY is self-administered; however, the data in the present study were collected using three different administration formats to account for the Brazilian reality in which caregivers may have difficulty with reading and interpretation. The format, based on the preference and comprehension level of the respondent, included: self-completion of the PEM-CY independently (n = 59 parents/caregivers); self-completion with help (n = 25 respondents requested some kind of support for completing the questionnaires); and 17 interviews (the evaluator asked the questions and recorded the responses). An explanatory sheet to assist in understanding how to answer the questionnaire was developed by Galvão t al.29 and was used with all participants in this study.
Data analysisWe used descriptive statistics, such as percentages, means, and standard deviation (SD) to characterize the sample, and Mann-Whitney tests compared groups in terms of sex, age, respondent education, and family income. PEM-CY Brazil results were entered into the PEM-CY Brazil Calculator to obtain the final score of each domain for all settings. We used Cronbach's alpha to calculate the internal consistency of the PEM-CY Brazil domains within the participation section (number of activities in which the child/youth participated in, frequency, involvement, and desire for change) and within the environment section (supports, barriers, environmental helpfulness, environmental resources, and overall environmental support) in the home, school, and community settings, using data from all participants. Alpha values starting from 0.70 were considered appropriate. Although high values (>0.90) are desired, they may indicate redundancy in items.38
One aspect of construct validity is the ability of an instrument to distinguish target summary scores that should differ between groups.19 For this study, differences in participation and environment scores were examined among groups of children with and without disabilities. Four participation domain scores (i.e., number of activities the child/youth participated in; mean frequency of participation considering all activities; mean involvement; and number of activities the respondent desired change in) were compared between groups for each setting (home, school, and community); as well as the five environment domain scores listed above. Non-parametric Mann-Whitney U tests compared PEM-CY scores across groups, as the data did not present a normal distribution based on Kolmogorov–Smirnov test. SPSS version 22 was used to perform all the analyses at a significance level of p < 0.05.
ResultsSample characteristicsA total of 112 participants agreed to participate in the study, however 11 (10 %) were excluded due to incomplete data. A final sample of 101 parents/caregivers of children and youth who where between 5 and 16 years of age participated in this study. The overall mean age of the children/youth was 9.36 (SD=3.47) years. The mean age of the group with disabilities (n = 62) was 9.31 years and of those without disabilities (n = 37) was 9.44 years. Table 2 describes the characteristics of the study participants. The groups did not differ (Mann-Whitney test) in age (p = 0.989) or sex (p = 0.188).
Characteristics of the participants.
*Approximately 265 dollars per month.
The internal consistency of all participation and environment PEM-CY Brazil domains, considering data from all 101 participants, ranged from appropriate (0.70) to excellent (0.95), except in one environment domain of the home setting (Environment Resources), where the alpha was 0.53 (Table 3).
Internal consistency values of the PEM-CY domains for all 101 participants.
PEM-CY, participation and environment measure - children and youth.
Table 4 shows PEM-CY participation and environment scores for the groups of children with and without disabilities. The results show that in all PEM-CY domains, children with disabilities showed poorer scores than their peers without disabilities. With respect to the participation domain, we found significant differences (p < 0.01) between groups for involvement and number of activities performed in all settings; for participation frequency in the community; and for desired change in school. Regarding the environment domain, we found significant differences (p ≤ 0.01) between groups in all domains in the home, school, and community settings.
Differences between groups in the PEM-CY domains.
PEM-CY, participation and environment measure - children and youth. *Statistically significant difference according to the Mann-Whitney U test.
The purpose of this study was to provide evidence on the initial measurement properties of the Brazilian version of the PEM-CY. In summary, the results suggest that the PEM-CY Brazil has appropriate to high indexes of internal consistency and can discriminate the characteristics of participation and environment between children and young people with and without disabilities.
We found appropriate internal consistency indexes for all PEM-CY domains (with the exception of environmental resources in the home setting), which means that the instrument items within each of the domains measure the same construct, for both participation and environment domains. Furthermore, only two items (environment helpfulness, and overall environmental support), in the school setting, presented an alpha value greater than 0.9, which may indicate a redundancy of items in this setting.38 Our results were similar to the ones from the original version, showing indexes of internal consistency ranging from α=0.59 to α>0.80.14 The Chinese, Indian, Korean and Turkish versions of PEM-CY have also shown good internal consistency (α≥0.55, α≥0.61, α≥0.67, and α≥0.67, respectively) for all PEM-CY participation and environment settings and domains.22,24,25,39 The results of these studies performed in different countries and cultures, with their respective translated and adapted versions of the PEM-CY, as well as the present study, reinforce that the instrument is reliable for assessing participation and environment of children and young people with or without disabilities.
Construct validity is the degree to which the scores of an instrument are consistent with hypotheses, such as differences between groups (called known group validity), based on the assumption that the instrument validly measures the construct of interest.19 The literature shows that children with disabilities participate less frequently in different activities and settings than children without disabilities.40–45 Based on this assumption, and on the PEM-CY original measurement properties,14 we hypothesized that the PEM-CY Brazil would be valid to distinguish the participation and environment of Brazilian children with and without disabilities, which was confirmed by the results of this study. We identified that Brazilian children with disabilities were less involved and participated in a smaller number of activities in all settings; and have a decreased frequency of participation in the community setting compared with children without disabilities. Also, caregivers of children with disabilities desired more change in the school setting, and perceived lower levels of environmental support, more barriers and less availability of resources and environmental helpfulness in all settings compared to the group of children without disabilities. Corroborating our results, the measurement properties studies of the Chinese and Korean versions of the PEM-CY found significant differences between the participation and the environment of children with and without disabilities.22,39 Similarly, these studies found no differences between groups in the frequency of participation at home, nor in the desire for change in the home and community.22,39
The lack of differences between groups in the frequency of participation at home can be explained by the characteristics of the activities included in the PEM-CY, and by the conceptual difference between attendance and involvement.8 Activities such as “indoor play and games”, “using computer/video games”, “watching TV/videos”, and “socializing using technology” are often performed by children with disabilities in their homes, however with less involvement than their peers without disabilities.22,34,46 Indeed, the validity study of the Korean PEM-CY version found no differences in frequency of participation in any of the three settings, but did find a difference in involvement.39
Regarding the desire for change domain, it is important to consider that in the Brazilian context the concept of participation is only now being acknowledged, as it had previously been disregarded as a relevant outcome.47 This may have affected the ability of Brazilian parents in both groups to express their desire for change, given that other outcomes, such as walking, are valued more highly.48 In low- and middle-income countries (LMICs) such as Brazil, the importance of participation may be perceived differently when compared to high-income countries, where most studies still focus only on the attendance component.47 This may explain the difference between the results of our study and the original PEM-CY measurement properties study, which found differences in the desire for change between parents of children with and without disabilities.14 Our results show that parents of children with disabilities perhaps are not so optimistic for their child's to be able to participate.
In the present study, we calculated the number of activities the child participated in for each setting to explore the participation patterns of children with and without disabilities. We found that children with disabilities participated in a smaller number of activities in all settings when compared to their peers without disabilities. This coincides with studies conducted in other countries and cultures,14,15,24,49,50 and adds to the construct validity of the PEM-CY in Brazil.
It is widely recognized that participation patterns are shaped by the opportunities and supports provided by the environment.18,34,46 Corroborating with previous studies, our results identified that caregivers of children with disabilities perceive less environmental support, more barriers, and less availability of resources and environmental helpfulness compared to the group of children without disabilities in all settings,14,18,24,51,52 demonstrating that the PEM-CY Brazil is also valid to evaluate the environment where Brazilian children are.
Given the above, we suggest that the PEM-CY Brazil is a reliable and valid instrument to measure the participation and the environment of Brazilian children and youth with and without disabilities. The results obtained with PEM-CY can be used to track the participation levels and aspects of the environment through a comprehensive analysis. Such information can be instrumental in facilitating therapeutic goals being set with the family to improve the participation of children and young people at home, in school, and in the community. Considering specifically the Brazilian context, the availability of PEM-CY could contribute to modifying physical therapy practices for children with disabilities with CP, where less than 1 % of interventions aim for the outcome of participation.27
We highlight the clinical implications and the potential utility of the measure in clinical practice. For example, at the micro-level (e.g., individual client), the PEM-CY may be used to facilitate goal setting by building on the desire for change domain, and intervention-related changes in PEM-CY may be tracked; at the meso¿;level, it may guide program evaluation, educational settings/schools; and at the macro-level, the PEM-CY may be used to measure participation and environmental barriers across districts in Brazil to inform local policies. It should be noted that parents of children without disabilities perceived greater availability of resources and general support of the environment for community participation, which reinforces the importance of increasing context-focused interventions that target the provision of environmental supports for children and youth with disabilities to improve participation outcomes53,54 and well as the need for policy related changes to support children and families.
We recognize some limitations of the study, mainly with regard to the sample size of the group of children without disabilities and because we did not verify other measurement properties, such as test-retest reliability. Despite these limitations, the study demonstrated that PEM-CY was able to detect differences between groups in most of the domains investigated. To confirm the findings of this study, research recruiting a larger sample, and investigating other measurement properties should be performed. Another limitation was the heterogeneity in administration methods (self-administered with or without help, interview), but we think that this difference did not influence the results, as only a small proportion of participants used the interview method. Including participants who have difficulty filling out the form alone was a choice made by the study group as it is more inclusive, but we recommend that future studies explore the potential role of these methods in the results.
ConclusionResults indicate that the PEM-CY Brazil presents good internal consistency and construct (known-groups) validity to evaluate the participation and the environment of Brazilian children and youth between 5 and 17 years of age with and without disabilities.
This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.