Participation is considered a central outcome of rehabilitation and needs to be appropriately assessed from a perspective that values individual values and preferences. The Brazilian version of the Impact on Participation and Autonomy (IPA-Br4) is the only adequate instrument to assess participation from a subjective perspective in Brazil.
ObjectiveTo investigate the test-retest reliability and internal structure (structural validity and internal consistency) of the digital version of IPA-Br4 for individuals with physical disabilities.
MethodsTest-retest reliability was assessed with Intraclass Correlation Coefficients (ICC), structural validity with Confirmatory Factor Analysis (CFA), and internal consistency with Cronbach α.
ResultsThirty (43.25 ± 10.6 years old, 63.3% female) and 130 (47.51 ± 15.73 years old, 58.5% male) individuals participated in the test-retest reliability and structural validity assessments, respectively. ICCs for each domain were: Autonomy indoors = 0.84; Family role = 0.87; Social life and relationships = 0.87; Autonomy outdoors = 0.84. The CFA model for four factors showed appropriate fit with factor loadings indicating that all items loaded onto their respective factors, with magnitudes > 0.30 (p < 0.05). All Cronbach α values (internal consistency) were above 0.82.
ConclusionThe digital version of IPA-Br4 has adequate test-retest reliability and internal structure to be used with adults with physical disabilities.
Participation should be assessed as a rehabilitation outcome using instruments with adequate measurement properties. Participation is considered the 'ultimate aim of rehabilitation'1 and a fundamental right according to the Convention on the Rights of Persons with Disabilities.2,3 The International Classification of Functioning, Disability, and Health (ICF) defines participation as 'involvement in life situations', encompassing domestic life, work, education, social and community involvement, civic activities, and interpersonal relationships.4,5 Although this concept was defined 22 years ago, its assessment with standardized instruments is still challenging2,6,7 because participation is multidimensional and includes not only objective but also subjective aspects2,8,9 such as personal meanings, preferences, and individual satisfaction with participation.9,10
Most existing participation measures primarily focus on objective, quantifiable performance indicators including factors like the frequency, intensity, duration, and diversity of activities within social contexts.11-14 These measures, however, fail to capture the individual's subjective perspective. The personal significance attributed to participation outcomes cannot be accurately inferred from metrics centered solely on objective criteria because objective and subjective aspects of participation are only weakly correlated.10,15 Notably, it is the subjective, rather than the objective dimensions, that demonstrate a stronger correlation with overall well-being and quality of life.15,16
Instruments that capture subjective aspects of participation are necessary to define rehabilitation goals and monitor treatment outcomes according to individual values and preferences,2,9,10 but they are not commonly used in Brazil. A recent systematic review analyzed 10 general measures for assessing subjective aspects of participation for adults with disabilities.17 Three had good coverage of experiential aspects (Participation Enfranchisement,9 Community Integration Measure,18 and Measure of Experiential Aspects of Participation19) but were not translated and adapted into Brazilian Portuguese.17 Only two instruments were translated and adapted for the Brazilian population: the Participation Scale (P-scale) and the Impact on Participation and Autonomy (IPA) questionnaire.17
The P-scale was developed simultaneously in three countries, including Brazil,20-24 to measure social participation in low- and middle-income countries.20 It is listed as a reference instrument for assessing and monitoring patients with Hansen's disease by the Ministry of Health in Brazil.25 However, there is high-quality evidence that it is not a unidimensional measure,17 with participation and activity items mixed in a single scale. Thus, although the P-scale aims to assess participation, its final score does not reflect a single construct and should not be interpreted as an indicator of participation.17
The Brazilian version of IPA questionnaire may better characterize the participation of adults with disabilities in Brazil. The IPA was developed in the Netherlands and was the first instrument assessing participation from the perspective of personal autonomy (i.e., the opportunity to make choices and exert control over one's own life).8,26 It was later translated and adapted in eight countries: Iran,27 Sweden,28 Canada,29 Denmark,30 Thailand,31 Finland,32 United Kingdom,33 and Brazil.34 It is a person-centered instrument as it allows the individual to assess participation based on their choices and values, rather than on a normative or ideal social standard.27 It was developed for individuals with physical disabilities regardless of their diagnosis and has 5 subscales.8 The systematic review indicated that three subscales (Autonomy outdoors, Social life and relationships, and Work and education) adequately cover experiential aspects of participation, while two (Autonomy indoors and Family role) are activity-related subscales.17 However, structural validity (the factor structure, indicating the degree to which the scores of the instrument are an adequate reflection of the constructs being measured) results for the IPA were inconsistent in the systematic review (two studies reported insufficient results and three reported sufficient results, but with varied – rather than unique – factor structures).17 IPA's internal structure (i.e., structural validity and internal consistency) should therefore be further investigated.17
The Brazilian version of IPA, named IPA-Br4 as it lacks one of the 5 original subscales (Work and education), is the only potentially adequate measure to capture subjective aspects of participation in Brazil. It was translated from the English version and adapted to Brazilian Portuguese, with an online digital version tested for floor and ceiling effects, internal consistency, and factor structure, showing adequate results.34 However, it was tested only for individuals with spinal cord injury.34 A reliable digital version with appropriate internal structure may facilitate assessing participation in varied clinical groups in Brazil. Thus, this study aimed to investigate the test-retest reliability and internal structure of the IPA-Br4 for Brazilian individuals with physical disabilities due to diverse health conditions.
MethodsThis cross-sectional study was conducted in five units of the SARAH Network of Rehabilitation Hospitals (Belo Horizonte, Brasília-Center, São Luiz, Salvador, and Rio de Janeiro) and approved by the research ethics committee at the Universidade Federal de Minas Gerais, Belo Horizonte, Minas Gerais, Brazil (CAAE/32860220.0.0000.0022).
ParticipantsIndividuals with chronic physical disabilities who were receiving inpatient or outpatient care in the SARAH Network were invited to take part in the study and signed an informed consent form, constituting a convenience sample. Individuals who, as judged by their reference healthcare professional in the hospital, were illiterate, had difficulty using smartphones (necessary to answer the digital IPA-Br4), or had rapidly progressing diseases, were excluded from the study. Thirty individuals participated in the reliability assessment and 130 (a minimum of five individuals for each of the 26 items in the instrument) participated in the structural validity assessment of the IPA to ensure adequate methodological quality.35,36
InstrumentsThe IPA-Br4 consists of 26 items assessing perception of participation from an autonomy perspective in four domains: Autonomy indoors (7 activity items), Family role (7 items, most of activity), Social life and relationships (7 participation items), and Autonomy outdoors (5 participation items).8,26,34 The fifth domain in the original IPA, Work and education (6 items), was not included in the Brazilian version tested for structural validity because the proportion of individuals who could provide data for questions in this domain was very low (only 85 of 198 participants) as most did not work or study.34
Example items of the IPA-Br4 include 'My chances of getting around in my house where I want to:' (autonomy indoors) and 'My chances of using leisure time the way I want to are:' (autonomy outdoors). Response options and scores for each item vary from 'very good (0)' to 'very poor (4)'. Each domain score is calculated as the mean of item scores; high values indicate a worse perception of participation. Also, eight items such as “if your health or your disability affects your chances of getting around where and when you want, to what extent does this cause you problems?” address problems experienced in participation. Responses options range from 'no problems (0)' to 'major problems (2)', with each item assessed individually. The participant must respond to at least 75% of the items in each domain to calculate domain scores.8,26 The complete instrument can be obtained from the original translation and adaptation study. 34
ProceduresThe digital version of the IPA-Br4 questionnaire was made available in Google Forms. Online meetings with research assistants (physical therapists of the SARAH Network) were conducted to standardize the procedures (instructions to be given to participants on how to access and fill in the form with their smartphones), review the form and instrument items, and address any questions the physical therapists may have had about data collection. After training, the research assistants showed patients how to access a link to the questionnaire in their personal smartphones. Individuals were instructed to complete the questionnaire in a calm environment in the hospital or at home, with time to read the items. In case of doubts, participants could ask the research assistant for clarification or send a message to the main researcher via WhatsApp.
IPA-Br4 data for the internal structure analysis were collected at the five SARAH hospitals, while data for the test-retest reliability analysis (two IPA-Br4 applications separated by a time interval) were collected in Belo Horizonte. The test-retest time interval should be long enough to prevent recall bias (7 days at least), and short enough to ensure that patients have not changed on the construct to be measured (their perception of participation). This study used a time interval of about 2 weeks, considered appropriate for the evaluation of test-retest reliability of patient-reported outcome measures.37
All Google Forms data were accessible only to the main researcher. Demographic and health condition data (including International Classification of Diseases codes) were collected from each individual's electronic medical records.
The assessment of internal structure followed recommendations from the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) for self-reported outcome measures.35,36 Based on previous literature on the IPA, it was expected that the 26 items would form four separate factors corresponding to the four subscales of the instrument. Therefore, a confirmatory factor analysis (CFA), recommended to verify if items are separated as expected into different domains,8,26 was used. Also, the sample size should include at least 100 individuals and five times the number of items in the instrument (26 in the IPA-Br4) for adequate methodological quality. Thus, a minimum of 130 individuals were required.35,36
Statistical analysisQuantitative data were summarized with means and standard deviations. The test-retest reliability for each of the four IPA-Br4 perception of participation domains was assessed with intraclass correlation coefficients (ICC) for single measurements and absolute agreement38,39; ICC values above 0.75 indicate excellent reliability.40 The Statistical Package for the Social Sciences (version 18.0) was used for all analyses.
Confirmatory factor analysis (CFA) with a weighted least squares estimator41,42 was conducted using the R (version 4.2.2) lavaan package43,44 to verify the adjustment of the 26 items across the four domains of the IPA-Br4 (Autonomy indoors, Family role, Social life and relationships, and Autonomy outdoors). The following fit indexes were used in this study: χ2, in which values should be non-significant; χ2/degrees of freedom (df) ratio, with values ≤ 5 but preferably ≤ 3 considered appropriate; comparative fit index (CFI), and Tucker-Lewis index (TLI), with values ≥ 0.90 but preferably ≥ 0.95 considered adequate; root mean square error of approximation (RMSEA), with values ≤ 0.08 or preferably ≤ 0.06 and confidence interval (CI, upper limit) ≤ 0.10; and standardized root mean square residual (SRMR), with values ≤ 0.08 considered adequate.45-47 Next, the internal consistency in each domain was assessed using Cronbach's α coefficient38,48 with values > 0.70 considered satisfactory.
ResultsThe test-retest reliability sample consisted of 30 individuals (43.25 ± 10.6 years old, 63.3% female) with the following diagnoses: sequelae of spinal cord injury (17), sequelae of poliomyelitis (3), sequelae of COVID-19 (2), stroke (1), and others (7). The retest for all patients was administered at least 7 days and no more than 20 days after the test (the test-retest interval ranged between 7 and 20 days).39 The following ICC values were observed for each domain: Autonomy indoors = 0.84; Family role = 0.87; Social life and relationships = 0.87; and Autonomy outdoors = 0.84.
The CFA sample consisted of 130 individuals (47.51 ± 15.73 years, 58.5% male). The most prevalent diagnosis was sequelae of spinal cord injury (72), followed by sequelae of stroke (15), COVID-19 (polyneuropathy) (15), poliomyelitis (9), myelomeningocele (8), polyneuropathies (8), ataxias (7), Human T-lymphotropic virus type 1 (4), Parkinson's disease (4), multiple sclerosis (4), neuromuscular diseases (4), osteoarthritis (3), rheumatologic diseases (3), traumatic brain injury (2), congenital malformations (1), and cerebral palsy (1). Table 1 presents sample characteristics.
Sociodemographic characteristics of participants.
The CFA model for the IPA-Br4 domains (Autonomy indoors, Family role, Social life and relationships, and Autonomy outdoors) presented an appropriate fit: χ2(293) = 211.727, p = 1.000, χ2/df = 0.722, CFI = 1.000, TLI = 1.012, SRMR = 0.089, and RMSEA = 0.057 (90% CI = 0.044, 0.069). Factor loadings indicated that all items loaded onto their respective factors, with magnitudes > 0.30 (p < 0.05). Cronbach α values (internal consistency) were 0.91 (Autonomy indoors), 0.90 (Autonomy outdoors), 0.90 (Family role), and 0.82 (Social life and relationships). Fig. 1 shows the factorial structure of the IPA-Br4.
Discussion
The use of instruments that assess subjective aspects of participation assists in defining outcomes valued by the individual, helping to guide interventions according to their personal interests.1,2,4,7,9,10,26,35,49 It is important, therefore, to define measurement properties of instruments that capture experiential aspects of participation for Brazilian populations. The objective of this study was to investigate the test-retest reliability and internal structure of the IPA-Br4 for Brazilian individuals with physical disabilities due to diverse health conditions. Results show that the IPA-Br4 has adequate test-retest reliability, structural validity, and internal consistency when used with adults with physical disabilities in Brazil.
Measurement properties are not inherent attributes of an instrument. They result from an interaction between the instrument, the tested group, and the testing conditions.50 We tested the IPA-Br4 in digital format with a diverse sample of individuals with disabilities and various diagnoses. Reliability values for the four IPA-Br4 domains were similar to those observed for the original version of the IPA, with ICC values ranging between 0.84 (Autonomy indoors and outdoors) and 0.87 (Family role and Social life and relationships).8,26 Other reliability studies for different IPA versions corroborate these results, with ICC values ranging from 0.70 to 0.97.8,26,28,30-32 In the factor analysis, results confirmed that the 26 items were divided into the four domains as expected. According to current criteria for assessing structural validity,17,36,37 CFI/TLI values > 0.95, RMSEA < 0.06, or SRMR < 0.08 are sufficient. This study obtained a CFI = 1.000, TLI = 1.012, and RMSEA = 0.057, with only the SRMR showing a slight deviation from the cutoff point (0.09). Therefore, the criterion for positive confirmation of structural validity was met.
The present findings contribute to the body of evidence about the IPA. A systematic review17 revealed inconsistent structural validity results leading to a recommendation of caution in using and interpreting the scores of the instrument.17 In the review, two studies,29,51 reported negative results, whereas three28,30,34 reported sufficient structural validity (similarly to the present study) but with different factor structures (i.e., different distributions of items among domains). This variation was possibly due to the interaction between different instrument versions (different languages) and target populations of different countries. Because participation is sensitive to context,17 item stability should not be expected among different versions of IPA. This reinforces the importance of establishing favorable results for the structural validity of the digital version of IPA-Br4, as in the present study.
Additionally, once appropriate item distribution was evidenced by the confirmatory factor analysis, results also indicated sufficient internal consistency for all domains. Together, the structural validity and internal consistency results indicate that the digital IPA-Br4 has adequate internal structure, which implies that the scores of each domain are appropriate indicators of a subjective sense of autonomy in different activity (Autonomy indoors, Family role) and participation (Autonomy outdoors, Social life and relationships) situations.
This study is limited in that it did not attain the criteria for maximum quality of test methods due to recruiting difficulties. Following COSMIN criteria, the quality of the reliability methods can be considered "adequate", as this study ensured sample stability between tests, an appropriate interval between measurements, similar test and retest administrations, and description of the type of ICC (adequate for the study). The maximum quality score ("very good") would have required a sample of 50 individuals for the test-retest assessment. The method for testing structural validity can also be considered "adequate" because a sample of five individuals per item was ensured for CFA. The maximum quality score ("very good") would have required a sample of seven individuals per item (182 individuals).8,26 The internal consistency testing method can be considered "very good" because Cronbach's α values for each domain were calculated after confirming structural validity. Therefore, to establish high quality evidence of sufficient test-retest reliability and internal structure of the IPA-Br4, other studies with sufficient samples and adequate methodological quality will be required. Additionally, other measurement properties such as content validity, criterion validity, hypothesis testing, measurement error, and responsiveness will need to be investigated for the IPA-Br4. Finally, as the IPA-Br4 version tested in this study is an online form version of the print questionnaire, application was limited to patients who were familiar with smartphones. One can expect that answering the questionnaire in print would be easier for most patients. Using a print version for a broader population of patients with diverse physical disabilities should therefore not interfere with the good indices of test-retest reliability and internal structure reported in this study.
ConclusionAccording to the results of this study, the IPA-Br4 has adequate test-retest reliability and adequate internal structure to be used in rehabilitation settings with adults with physical disabilities. 35,36
This study was financed in part by the Coordenação da Aperfeiçoamento de Pessoal de Nível Superior - Brasil (CAPES) - Finance Code 001.
