Accurate interpretation of hip radiographic imaging is critical for the early detection of hip displacement, the second most common impairment observed in children with cerebral palsy (CP). We aimed to evaluate the measurement properties of the HipScreen for assessing hip migration percentage (MP) in Brazilian children/adolescents with CP.
MethodMP was measured on pelvis radiographs of children and adolescents with CP by eight independent raters using the HipScreen app and traditional Reimer’s measurement method. Concurrent validity (Spearman correlation), sensitivity, specificity, and accuracy were established by the correlation between the MP values obtained from the HipScreen and the traditional Reimers method. Intra and inter-rater reliability were evaluated using the intraclass correlation coefficient (ICC). All raters assessed the clinical utility of the HipScreen app with a standardized questionnaire.
ResultsSixty-eight children (136 hips) were included, with a mean age of 8.09 years and 67.5% were classified in the Gross Motor Function Classification System IV or V levels. Notably, 49.3% of hips had an MP > 30%. Intra and inter-rater reliability were excellent (0.89 < ICC < 0.95). The HipScreen app presented strong to very strong correlations (0.82 < r < 0.91) with the traditional Reimers method with 80% sensitivity, 75% specificity, and 77% accuracy. All examiners considered the Hipscreen app easy and very useful for clinical practice.
ConclusionThe HipScreen provides reliable and valid measurements of MP in Brazilian children with CP, facilitating the implementation of hip surveillance programs in the Brazilian context.
Cerebral palsy (CP) is the most common motor childhood disability, and it is defined as an early-onset, non-degenerative, lifelong neurodevelopmental condition resulting from dysplasia or injury to the fetal or infant brain.1–3 It is characterized by activity limitations owing to impaired development of movement and posture, although the manifestations may evolve with age.3 Individuals with cerebral palsy (CP) often experience primary and secondary impairments across various domains of development and functioning, including weakness, coordination deficits, and variations in muscle tone.1,2,4 Among the secondary impairments, such as contractures, bone deformities, and joint stiffness, hip dysplasia is the second most commonly observed impairment in children with CP.2,4–7
The Gross Motor Function Classification System (GMFCS) is used to classify mobility performance in children with CP, ranging from GMFCS level I (ambulatory with good coordination) to GMFCS level V (wheelchair mobility with difficulty maintaining antigravity postures).8 The prevalence and severity of hip displacement are strongly associated with GMFCS level, with a risk of hip displacement up to 90% in GMFCS level V children.6,9 Hip displacement is associated with significant disabilities such as pain, pelvic obliquity, scoliosis, joint stiffness, activity limitations, and participation restrictions.6,7,10
The negative impact of hip displacement underscores the importance of implementing preventative strategies.11 In high-income countries, hip surveillance programs have been highly successful in preventing hip dislocation and its consequences in children with CP.11–13 These programs identify children at risk of hip displacement by implementing regular physical examinations and obtaining anteroposterior radiographs of the hip.12 Such action requires the collaborative involvement of healthcare professionals and family members, with the frequency of hip surveillance contingent upon age and GMFCS levels.11,12,14
Accurate interpretation of hip radiographic imaging is critical for the early detection of hip displacement in children with CP. The migration percentage (MP) established by Reimers, or the percentage of the ossific nucleus of the femoral head outside the lateral edge of the acetabulum, is the most important radiographic measurement needed to perform hip surveillance.12,15 However, the quantification of MP in radiology reports is inconsistent and inaccurate even in countries with centralized medical systems and established hip surveillance guidelines, making implementation of hip surveillance challenging.13,16,17 Digital technology can be used to train health professionals other than radiologists to measure the MP. The HipScreen App, developed as a knowledge translation tool for hip surveillance (www.hipscreen.org, Shriners Children's, Sacramento, CA, USA), is a free, award-winning, and validated tool for the measurement of MP using a mobile device.18,19 HipScreen has excellent intra- (ICC = 0.83) and inter-rater reliability (ICC>0.90), and a sensitivity and specificity of 94% and 66%, respectively, in its original studies.18,19
In Brazil, a hip surveillance program for children with CP has not been implemented as a public health policy. The prevalence of hip dysplasia in all Brazilian children with CP is not known, but is estimated to be very high based on the percentage of children classified in the highest levels of the GMFCS.20–22 Initial data from the Brazilian Cerebral Palsy Registry (available on the website https://registropcbr.com/publicacoes-e-ferramentas) indicated a 32% prevalence of hip dislocation in the population of Brazilian children with CP. A recent study on the functional profile of Brazilian children with CP demonstrated that 48.8% are at GMFCS levels IV or V, representing a high risk for hip dislocation.21 Children with CP in Brazil encounter considerable challenges in accessing health services and assistive technologies,21,22 underscoring the need for implementing community-based models for hip surveillance that can empower all health professionals to participate. Furthermore, a prevalence-based cost-of-illness study identified that “spastic quadriplegic CP” accounts for 31.3% of hospital admissions and 41.7% of outpatient visits, and is associated with a higher cost of healthcare in the Brazilian healthcare public system.23 The present study aimed to contribute to the clinical management of hip dislocation in Brazilian children with CP by examining intra-rater reliability, inter-rater reliability, concurrent validity, sensitivity, specificity, accuracy, and the applicability of the HipScreen App.
MethodThis study on measurement properties was conducted following the COnsensus-based Standards for the selection of health status Measurement INstruments (COSMIN) taxonomy, terminology, and definitions of measurement properties.24 The institution ethics committee from Universidade de Brasília (Brazil) approved the study (CAAE: 28,540,620.6.1001.5133). The children and adolescents carers provided written informed consent to participate in the research and for the publication of data from their medical records. The study was conducted from September 2022 to February 2023.
ParticipantsThis study included anteroposterior radiographs of the pelvis of children and adolescents between 2 and 17 years diagnosed with CP obtained from the medical records of university hospitals affiliated with the Universidade de Brasília, Universidade Federal de Juiz de Fora, and Universidade de São Paulo. To be included in the first analysis, the children's medical records corresponding to the selected radiographs had to contain information about age, clinical type, and GMFCS level. Then, two examiners (GRB and KMAA) excluded radiographs that did not meet the criteria for the following features of a well-positioned pelvis radiographs: symmetric pelvic wings, neutral femoral positioning, and symmetric and oval obturator foramen.19 The COSMIN considers appropriate a sample size > 50 participants for validity and reliability studies.24
MeasuresReimers migration percentageThe original description of the Reimers migration percentage uses reference lines on the pelvic radiograph to obtain the measurement.15 The initial line of interest is the Hilgenreiner line (H line), which is a horizontal line that crosses both triradiate cartilages. The second line, the Perkin line (P line), is a vertical line drawn perpendicular to the H line and tangent to the lateral edge of the ossified acetabulum. The Reimers migration percentage is calculated as the ratio of the femoral head lateral to the P line to the total femoral head width, expressed as a percentage.15 Mathematically, this is represented by the formula Migration Percentage = A/B x100% (Fig. 1).
HipScreen app
The HipScreen App (version 1.4.5) is available for free download (www.hipscreen.org) for use on iOS or Android smartphones. HipScreen prompts the examiner to capture a pelvic radiograph using the device's camera or select an existing image from the photo gallery. The “rotate” function enables the positioning of the triradiate cartilage in alignment with the H line. Subsequently, the “ruler” function enables the measurement of the femoral head position in relation to the lateral (white) and medial (black) lines (Fig. 2). By enlarging the image displayed on the touchscreen, precise positioning can be achieved.
Once the correct position has been determined, the ruler image overlay divides the femoral head into 10 equal vertical sections (illustrated by gray lines), with each section representing 10% of MP. To ascertain the migration percentage, the examiner counts the number of gray lines that appear between the white line and point of intersection between the rim of the acetabulum and aforementioned line. A migration percentage exceeding 30% is indicated if the lateral border of the acetabulum is in contact with the red line. A visual representation of the HipScreen layout is provided in Fig. 2 for illustrative purposes.
ProceduresA total of eight examiners participated in the study. One physical therapy fellowship training with experience in hip X-ray examination of children with CP (examiner I), three pediatric physical therapists with > 20 years of experience (examiners II-IV), two physical therapy master's students with no experience in hip X-ray examination of children with CP (examiners V and VI), and one medical student with no experience in hip X-ray examination of children with CP (examiners VII) were responsible for the evaluation of the X-rays using the HipScreen App. A fellowship trained pediatric orthopedic surgeon (examiner VIII) evaluated the radiographs using the Reimers method.
Prior to the evaluation of the X-ray images using the HipScreen, the seven HipScreen examiners (I-VII) were required to complete a training session comprising two components: 1) watching the video “Fundamentals of Hip X-ray Interpretation,” accessible via https://youtu.be/2rGSdVmjaus, and 2) achievement of a score of at least 80% on a competency test from www.hipscreen.org. All seven examiners successfully completed the training, achieving an average score of 98.2% on the assessment. From training as a pediatric orthopedic surgeon, examiner VIII was facile in the Reimers method examination using a Picture Archiving and Communication System (PACS).
To evaluate inter-rater reliability, all seven HipScreen examiners calculated the MP for both hips on each x-ray independently. The HipScreen examiners were categorized into two groups, designated as “expert” and “novice,” based on their experience in hip X-ray examination of children with CP in clinical settings. The expert group comprised examiners I-IV, whereas the novice group comprised examiners V-VII. The inter-rater reliability was determined for the within-group and overall examiners. To assess intra-rater reliability, the examiner I calculated the migration percentage for both hips (right and left) on each X-ray twice, with a thirty-day interval between assessments.
To assess concurrent validity, the results obtained from the HipScreen App by each examiner (examiners I-VII) were subsequently compared to those derived from the Reimers method (examiner VIII). Examiners were blinded to the identities of the participants, clinical features, and the results obtained by other examiners.
Following the completion of all evaluations, a questionnaire was administered to all HipScreen examiners to evaluate the HipScreen App utility in clinical practice. The questionnaire comprised 14 questions which included the difficulty in accessing and using the tutorial, the ease of handling HipScreen and its tools, suggestions for potential modifications to HipScreen, applicability in a clinical setting, and extent to which any health professional, following training, would be able to use HipScreen (Supplementary material). The questionnaire responses were collected online and on an individual basis from the seven HipScreen examiners.
Statistical analysesThe collected data were imported into the Statistical Package for the Social Sciences software (version 29.0.1.0; IBM Corp., Armonk, NY, USA). We used descriptive statistics, such as percentages, means, and standard deviation (SD) to characterize the sample.
The intra-rater reliability was evaluated through the calculation of the intraclass correlation coefficient (ICC3.1), two-way mixed analysis, and 95% confidence interval (CI). The inter-rater reliability between the scores assigned by the seven examiners was determined using the ICC2.1, absolute agreement, and two-way random effects analysis. The reliability coefficients were interpreted as: ICC < 0.50, poor; 0.50–0.75, moderate; 0.75–0.90, good; and > 0.90, excellent correlation.25 Owing to the non-normal distribution of the data, Spearman's rank correlation coefficient was employed to evaluate the concurrent validity between each HipScreen examiners scores and the Reimers examiner score. Spearman's test was interpreted as: < 0.10, negligible; 0.10–0.39, weak; 0.40–0.69, moderate; 0.70–0.89, strong; and > 0.90, very strong correlation.26
To detect the accuracy of the HipScreen App as a screening test for children at high risk of hip displacement, a hip with an MP ≥ 30% as measured by Examiner I using the traditional Reimer’s method was considered a “positive” case for sensitivity and specificity analysis. Responses from the questionnaire about the utility of the HipScreen App were consolidated into an Excel spreadsheet and expressed as percentages.
ResultsSeventy-seven radiographs (154 hips) were initially selected for the study. Among these, 9 were excluded for not meeting the criteria for well-positioned pelvic radiographs. Consequently, 68 radiographs (136 hips) were included in the final analysis. The mean age of the children was 8.09 years (SD ± 0.50), with the highest proportion from children GMFCS Level IV and spastic bilateral type. “Positive cases” with an MP > 30% was present in 49.3% of hips. Participants classified in GMFCS V had the highest mean MP and greatest number of positive cases. Demographic characteristics and migration percentage data of all participants are presented in Table 1. MP was calculated based on examiner I results.
Demographic characteristics of the 68 children with cerebral palsy included in the study.
n: number of participants; SD: standard deviation; *age of the one participant; MP: migration percentage measured through the HipScreen App.
The intra-rater reliability was excellent (ICC = 0.95; 95% CI 0.93–0.96). The inter-rater reliability among all examiners and within groups was good and excellent (Table 2). The Reimers method yielded a mean MP of 27±23.1° The concurrent validity analysis revealed strong to very strong correlations between the traditional Reimer’s measurement scores and measurement obtained by the seven examiners using the HipScreen (Table 3). The HipScreen app was found to have a sensitivity of 80%, specificity of 75%, and an accuracy of 77% when comparing HipScreen examiner I compared with the traditional Reimer’s measurement examiner VIII.
Results of inter-rater reliability of the HipScreen app.
| Media (SD) | ICC | CI | |
|---|---|---|---|
| Between all raters | 32.5 (21.9) | 0.90 | 0.87–0.92 |
| Between experts | 30.0 (22.3) | 0.89 | 0.87–0.92 |
| Between novices | 30.0 (21.4) | 0.92 | 0.90–0.94 |
SD: standard deviation; ICC: intraclass correlation coefficient; CI: confidence interval.
Concurrent validity of HipScreen App vs. Reimers migration percentage results.
SD: standard deviation; *p-value: < 0.05.
In response to the standardized questionnaire, all examiners considered the Hipscreen easy to use and very useful in clinical practice. All examiners (100%) reported no difficulties in accessing the instructional video, downloading the App to their device or smartphone, understanding the terms and commands written in English, and positioning the radiograph image in the rotate and ruler functions. The examiners (57%) reported difficulty in identifying the margins of anatomical structures and adjusting the rulers over them after zooming in on some images within the app; they suggested that this difficulty was owing to the quality of the photo taken from the radiograph.
DiscussionThis study verifies the excellent properties of the HipScreen App for measuring hip MP in Brazilian children and adolescents with CP. The results demonstrated an excellent intra-rater reliability and good to excellent inter-rater reliability, indicating consistent and accurate measurements across different users. Additionally, inter-rater reliability is excellent for HipScreen across novice and experienced examiners. A strong correlation between the HipScreen measurements and traditional Reimer’s measurement underscore the validity of the HipScreen app. HipScreen exhibited higher sensitivity than specificity, suggesting a greater capacity to identify positive cases, and has good clinical applicability with minor usage challenges.
Our study corroborates the results of other studies.18,19,27 The first study that verified the measurement properties of the HipScreen involved three pediatric orthopedic examiners and 20 radiographs of children with CP.18 The study identified that the results of MP obtained by the HipScreen are similar to the results obtained by PACS measurement or radiographic templating software. The study found excellent intra- and inter-examiner reliability (ICC>0.97) and found the time spent to measure the MP using the HipScreen was significantly less than the time spent by PACS.18
A study conducted in the UK27 involving five examiners with different expertise (pediatric orthopedics, physiotherapist, radiographer, and pediatrician in neurodisability) and 20 radiographs of children with CP, also investigated reliability and validity of the HipScreen. The study identified that the HipScreen app has excellent inter and intra-rater reliability and its results are strongly correlated with the results obtained using the PACS method (r = 0.90),27 similar to the results found in our study.
A more recent study aimed to assess its accuracy, reliability, and discriminatory ability for the measurement of MP in 40 hip surveillance radiographs of children with CP.19 The study involved 37 examiners with different expertise, subdivided into expert and novice groups, and demonstrated good intra-rater reliability (ICC = 0.83), with no significant difference between the two groups.19 Compared to the results of the PACS method, the app showed good accuracy (mean absolute error = 5.72%), 94% of sensitivity, 66% of specificity, and excellent discriminatory ability in identifying positive cases.19 As in our study, physical therapists were the professionals with the greatest representation among the examiners, corroborating that they can be trained to accurately use the HipScreen.
Despite the difference in socioeconomic contexts, the findings of this study performed in a middle-income country aligns with the three previous studies from high-income countries. The radiographs analyzed in this study were sourced from the medical records of public hospitals lacking a dedicated hip surveillance program. Consequently, the technicians were not specifically trained to acquire images optimized for this purpose, unlike the previous studies.19,27 Even so, HipScreen demonstrated excellent measurement properties in the Brazilian context and was considered easy to apply by the examiners. The challenges encountered by the examiners in accurately identifying the margins of anatomical structures after zooming the X-ray images may be attributed to the overall quality of the radiographic examinations, as poor radiographic technique could affect the ability of a HipScreen user to accurately identify the landmarks needed for MP calculation.19
This study does have some limitations. First, the radiographs chosen for inclusion had acceptable subject positioning and image quality, creating the potential for worse results if a broader range of radiographs was evaluated. Second, all of the radiographic evaluators were fluent in English, as the HipScreen app only provides instructional videos in English. This language barrier could influence the ability to reproduce the results of this study among non-English speaking healthcare personnel. Another limitation was the limited number of evaluators for the application of the Reimers method examination using the PACS, making it impossible to evaluate reliability data for this method. Finally, most of the radiographic evaluators in this study were physical therapists. Future research could include a broader range of healthcare professionals to evaluate the reliability of HipScreen across more user groups.
ConclusionIn conclusion, this study confirms that the HipScreen app is a valid and reliable tool to measure hip migration percentage in children with CP in Brazil. With freely available online training, users can be effectively trained to identify children at high risk for hip dislocation, facilitating the implementation of hip surveillance programs in resource-poor settings.
Acknowledgments and Funding: This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors. This study was financed in part by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior – Brasil (CAPES) – finance code 001. And we would like to thank Coordenação de Aperfeiçoamento de Pessoal de Nível Superior – Brasil (CAPES) and Fundação de Amparo à Pesquisa do Estado de Minas Gerais (FAPEMIG) for graduate scholarship (JSC)
The authors declare no competing interest.
