Journal Information
Visits
78
Original Research
Full text access
Immersive virtual reality versus cervical mobilization: The effects on joint position sense and pain for chronic neck pain - a randomized controlled trial
Visits
78
Hatice Cetina,
Corresponding author
haticebitirim@hacettepe.edu.tr

Corresponding author at: Hacettepe University, Faculty of Physical Therapy and Rehabilitation, Samanpazarı 06100, Ankara, Turkey.
, Selenay Aydogdua, Birol Onalb, Esra Dulgerc, Sevil Bilgina, Halil Kamil Oged, Nezire Kosea
a Faculty of Physical Therapy and Rehabilitation, Hacettepe University, Ankara, Turkey
b Faculty of Health Sciences, Physical Therapy and Rehabilitation Department, Ataturk University, Erzurum, Turkey
c Faculty of Health Sciences, Physical Therapy and Rehabilitation Department, Gaziosmanpasa University, Tokat, Turkey
d Department of Neurosurgery, The Faculty of Medicine, Hacettepe University, Ankara, Turkey
Highlights

  • Virtual reality when added to traditional exercises provided no additional.benefit for joint position sense.

  • Cervical mobilization when added to traditional exercises provided no additional benefit for joint position sense.

  • Virtual reality when added to traditional exercises provided no additional benefit for pain relief.

  • Cervical mobilization when added to traditional exercises provided no additional benefit for pain relief.

This item has received
Article information
Abstract
Full Text
Bibliography
Download PDF
Statistics
Figures (1)
Tables (4)
Table 1. Baseline sociodemographic and pain characteristics of the groups.
Tables
Table 2. Primary outcomes (joint position sense and pain) at baseline and posttreatment.
Tables
Table 3. Secondary outcomes (baropodometric outcomes and balance measurements) at baseline and posttreatment.
Tables
Table 4. Secondary outcomes (GPE, NDI, and SF-36) at baseline and posttreatment.
Tables
Show moreShow less
Abstract
Background

Joint position sense (JPS) deficits and pain are among symptoms observed in patients with chronic neck pain (CNP).

Objective

To examine the effects of immersive virtual reality (VR) and cervical mobilization (CM) on JPS and pain in patients with CNP.

Methods

This randomized controlled study included 45 patients with CNP, aged 18 to 65. Patients were divided into three groups: VR+traditional exercise, CM+traditional exercise, and traditional exercise alone. The primary outcomes were JPS and pain. The secondary outcomes were balance, functional disability, global perceived effect, and quality of life. Balance outcomes included baropodometric outcomes, 10-meter walking test, four-square step test, and single leg stance test. All patients completed 10 treatment sessions, with outcomes measured before and after the intervention.

Results

There were no statistically significant additional effects of VR or CM on improving JPS or pain relief when added to traditional exercises (p > 0.05). The changes in all balance outcomes, functional disability, and global perceived effect were also similar for the three groups (p > 0.05). The combination of CM or VR and traditional exercises resulted in greater improvement in social function quality of life compared to traditional exercises alone (MD = 21.15, 95 % CI: 9.46, 32.85, Cohen’s d = 1.52).

Conclusion

The addition of VR or CM to traditional exercises offers no additional benefits for JPS, pain, balance, and function, for individuals with CNP.

Keywords:
Cervical mobilization
Joint position sense
Neck pain
Rehabilitation
Virtual reality
Full Text
Introduction

Chronic neck pain (CNP) is a common health problem, affecting an estimated 10 % to 20 % of the population each year and causes a substantial economic burden to healthcare systems and absence from work worldwide.1 CNP has a frequently recurrent nature; this situation can be attributed to the complex pathoanatomical and physiological structure of the cervical spine.2

The patients diagnosed with CNP exhibit symptoms such as muscle weakness, pain, fatigue, cervical muscles strength imbalance, abnormal postural control, and sensorimotor impairments.3 One of the main sensory impairments and a topic that has been investigated in recent research studies is impairment of joint position sense (JPS) of the cervical spine.4 The cervical spine has a high density of mechanoreceptors and is crucial in providing proprioceptive input.5 The functions of cervical mechanoreceptors may be impaired due to fat infiltration in the cervical muscles or muscular fiber type changes as a result of the recurrence and chronicity of neck pain.6,7 Therefore, CNP impairs the sense of cervical JPS and sensorimotor control.8 The deficits observed in the cervical proprioceptive system, which is one of the three basic parameters that controls balance,9 can lead to impairments on postural balance in people with CNP.10–12

Evidence to date suggests that several proprioceptive retraining approaches, primarily tailored to provide sensorimotor control, could be effective in the management of sensorimotor control deficits due to CNP.13,14 Various treatment approaches, including conventional physical therapy,15 eye tracking and eye–head-neck coordination exercises,16 deep neck flexor muscle training,17 gaze direction recognition exercise,18 oculomotor exercises,19 kinematic training,20 or cervical mobilization (CM) techniques21 can be used alone or in combination for improving JPS.

Virtual reality (VR) has also started to appear in the literature as an approach used for various clinical applications.22 VR technology can be immersive and non-immersive. Non-immersive VR systems display virtual environments on standard monitors with interaction via mouse, joystick, or remote control. In contrast, immersive VR systems use head-mounted displays to surround the user with a 3D computer-generated environment, enabling naturalistic sensory-motor interaction.23 VR, which is primarily used to relieve pain in the field of health, has been also used in recent studies to improve JPS in individuals with CNP.20,24 The physiological mechanism by which VR provides pain inhibition is explained through pain distraction and multisensory input.25

Bahat et al.20 compared the effects of cervical kinematic training with and without interactive VR training on kinematic measures such as head movement velocity and accuracy. The authors indicated better improvement of movement velocity for the cervical kinematic training with VR group. Cetin et al.24 stated in their study, which examined the effects of motor control exercises with and without VR, that neither intervention was superior in terms of effects on pain intensity but VR was superior for improvement in JPS. The improvement of JPS with VR was attributed to the stimulation of joint proprioceptors as a result of more range of motions.22 CM, is another approach for improving JPS, it is believed to increase the sensitivity of the muscle spindles within the gamma motor neurons by stimulating the mechanoreceptors, thus increasing proprioception.26 The neuromuscular and physiological mechanism of VR in improving proprioception is similar to CM techniques. However, to our knowledge, there are no studies comparing the effectiveness of the two approaches.

Given the limited research comparing the effects of VR and CM on JPS and pain, the primary aim of this study was to compare the effects of VR, CM, and traditional exercises on JPS and pain. Additionally, the secondary aim was to compare their effects on balance, functional disability, global perceived effect, and quality of life.

MethodsStudy design

The study was designed as a single-blind, randomized-controlled trial. The Tokat Gaziosmanpasa University, Clinical Research Ethics Committee (Approval number: 22- KAEF-270) approved the protocol for this study. The trial was registered on https://clinicaltrials.gov/ web site (Trial number: NCT05829564). All participants were informed about the study's purpose and the interventions before the study. All participants signed consent forms.

Participants

Individuals with CNP who were referred to the physical therapy program at Hacettepe University Faculty of Physical Therapy and Rehabilitation, were offered to be enrolled in the study. The inclusion criteria were as follows: neck pain persisting for at least 3 months, between the ages of 18–65 years, sedentary life style (not receiving physical therapy in the last 6 months), a score of 10 or more out of 50 on the Neck Disability Index, individuals who can read and write. Exclusion criteria were; history of previous spinal surgery, neurological deficit (motor deficits), vestibular pathology, neurological, cardiopulmonary, musculoskeletal system conditions affecting physical performance, pathology in the shoulder joint, history of spinal trauma, pregnancy.

Outcome measures

The sociodemographic characteristics included age, body mass index, gender, education, and occupation. Additionally, pain characteristics were recorded based on location (upper neck, lower neck, right, left), duration (3–6 months, 6 months-1 year, 1–2 years, 2–5 years, and > 5 years), and frequency (<1 and 1 time, 2–3 times, >3 times per week) for all participants. Outcome measures were recorded pre and post interventions.

Primary outcomesJoint position sense error (JPSE)

Patients were asked to sit upright in a comfortable position with arms at the side. First, using a Cervical Range of Motion 3 (CROM) device, cervical range of motion were recorded in 6 different directions: flexion, extension, right-left lateral flexion, and right-left rotation. The CROM device is a valid and reliable method for assessing cervical JPSEs.27 Subsequently, with the participant’s eyes closed, the physical therapist passively positioned the patients' head to 65 % of the previously recorded range of motion value. This position was held for 3 s and the patients were asked to feel this position. After the patient's neck was repositioned to neutral, the patient was asked to actively bring their head back to the previous position. The difference between the point positioned by the patient and the previously determined reference point was recorded in degrees. For each cervical range of motion, three measurements were performed, and the average of three measurements was recorded as the JPS value for each direction.27

Pain

Visual Analog Scale was used to assess pain. The participants were asked to mark the average pain they felt in the last week on a 100 mm line. On this scale, 0 means no pain and 100 means unbearable pain.28

Secondary outcomesBalance

Baropodometry and stabilometry were measured using the FreeMed baropodometric platform (Sensor Medica, Rome, Italy), which is 60 cm x 50 cm and uses Free Step software with a 400 Hz sampling frequency. Participants stood with bare feet in a relaxed position, heels 5 cm apart. Baropodometry assessed total, forefoot, and hindfoot pressure distribution percentages over 5 s. Stabilometry evaluated Center of Gravity deviation (misalignment C), oscillations in the X (mediolateral) and Y (anteroposterior) axes (DeltaX and DeltaY), and sway velocity (Avspeed) over 20 s. Misalignment C, DeltaX, and DeltaY were measured in millimeters, and Avspeed in degrees.29

10-Meter Walking Test (10MWT): A 10-meter distance is marked in a corridor. Participants are instructed to "Walk quickly without running." Using a chronometer, the total time (in seconds) to walk the entire 10 m was recorded. The test is repeated 3 times, and the average time is recorded in seconds.30

The Four-Square Step Test (FSST): Two plus-shaped lines were drawn on the floor. Starting at square 1, participants stepped to square 2 (right), square 3 (back), square 4 (left), and back to square 1 (front). Then, without pausing, they stepped to squares 4, 3, 2, and 1 in reverse order. The test ended when they returned to square 1. The test was repeated three times, and the best performance time was recorded.31

The Single Leg Stance Test (SLST): The single-leg stance duration was recorded for both legs with eyes closed. The test ended if the other foot touched the ground, the person hopped, or needed support. The test was also terminated after 30 s, as this indicates good balance.32

Functional disability

Neck Disability Index was used to assess disability. This index includes 10 parameters that include personal care, lifting, reading, working, driving, sleeping, resting, headache, pain, and concentration and assesses pain-related disability. Each item is scored between 0–5.33

Global perceived effect

The Global Perceived Effect Scale assesses individuals' perceived change in their condition. At the beginning of the treatment, participants are asked to rate how they currently feel regarding their neck problem. At the end of the treatment, they are asked to compare their current condition to their condition at the beginning of the treatment. The scale ranges from −5 to +5, where −5 indicates much worse, 0 indicates no change, and +5 indicates much better.34

Quality of life

The Short Form Health Questionnaire-36 (SF-36) was used to assess quality of life, encompassing 8 sub-parameters: physical function, role limitations due to physical and emotional problems, energy, emotional well-being, social functioning, pain, and health change. Each parameter is scored from 0 to 100, with 0 indicating the worst possible health state and 100 the best.35

Interventions

All participants received 10 sessions at a frequency of 2–3 times a week for 4 weeks. The study included three groups; 1) VR group (VR+traditional exercise), 2) CM group (cervical mobilization+traditional exercise), 3) traditional exercise group (Fig. 1).

Fig. 1.

Flow diagram of the study.

(0.54MB).
VR group

The "Oculus Go '' device (developed by Meta Reality Labs (formerly Facebook Technologies) in partnership with Qualcomm and Xiaomi) was used for VR applications in the study. The Oculus Go device (weight 470 g) can be controlled by the individuals with the remote controller. Also, The Oculus Go device has a 5.5-inch display with a resolution of 2560 × 1440 pixels. In this study, the "Ocean Rift'' application was preferred because it is a convenient application for performing neck movements in all directions and does not contain fear and action. The individuals can select and follow sea creatures such as whales, sharks, and dolphins within the application. VR was experienced for 10 sessions, 20 min per session. During all sessions, the VR environment was monitored over the phone synchronously and individuals were encouraged to perform cervical motions in all directions. The statements such as "Look! There are starfish down there.", "The dolphins are moving upwards, watch." were used. VR was experienced in the same calm and quiet environment for all participants, sitting in a chair that allows 360-degree movement.24

CM group

The CM techniques were used with the Cyriax method. CM techniques consisted of manual traction, bridging technique, rotation with traction, anterior/posterior gliding with traction, and lateral gliding. The physical therapist with 10 years of experience applied all techniques with the patient in the supine position. Each technique was performed for 10 repetitions, with a 1-minute rest between techniques. Total mobilization time was 20 min.36

Traditional exercise group

All groups were treated with a Traditional Exercise program which consisted of three components: 1) breathing, 2) stretching, and 3) postural reeducation and strengthening exercises. The exercise program started with diaphragmatic breathing exercises performed for 10 repetitions. Then, stretching exercises involving trapezius, scalenes, and pectoral muscles were performed. Finally, in the posture correction and strengthening exercises section, correct posture and spinal alignment were taught. Scapular adduction and depression, shoulder rolling, and chin tuck exercises were performed. Each exercise was performed for 10 repetitions. The exercise program lasted approximately 20 min. Also, all individuals were informed about the ergonomics of cervical spine and correcting forward head, thoracic kyphosis, and forward position of the shoulder girdle.15,37

Sample size calculation, randomization, and blinding

The sample size was calculated with G*Power software (version 3.1) based on a repeated measures design with two time points (baseline and 4 weeks later) and three groups. An effect size of 0.25, which was derived from the pain intensity data of a reference study,38 was estimated, with the significance level set at 5 % and the statistical power set at 80 %. The analysis indicated a total sample size of 36 participants. Considering a 20 % drop-out rate, we planned to enroll 15 participants per group, resulting in a total of 45 participants for the study.

A researcher who was separate from the intervention team selected 45 numbers from a random number table and randomly divided them into the three groups. Then, the paper with numbers was placed into an opaque envelope. The envelopes were scrambled and numbered in turn. After inclusion, patients received the numbered envelopes consecutively and were divided into the corresponding groups according to the number on the paper within the envelope. Assessors who were responsible for the measurements were unaware of the group assignments and remained distant from the intervention.

Statistical analysis

Statistical analyses were performed using SPSS Version 26.0 (IBM Corp., Armonk, NY, USA). Normality and homogeneity of variances were tested using the Shapiro-Wilk and Levene tests, respectively. Descriptive data were presented as mean (standard deviation); mean (95 % CI), and number (frequency). GPE scores, which were not normally distributed, were compared among the three groups using the Kruskal-Wallis test. For analysis of primary and secondary outcome results, Generalized Estimating Equations (GEE) were used to analyze changes over time and interactions between groups. Time, group, and time*group interaction were included as independent variables in the model. Bonferroni correction was used for multiple comparisons. Changes in the outcome variables were calculated by subtracting the pre-intervention value from the post-intervention value. Standardized effect sizes for the differences between groups were calculated using Cohen's d formula. Cohen's d was classified as small (0.2–0.5), moderate (0.5–0.8), and large (≥0.8).39 The significance level for all statistical tests was set at p < 0.05.

Results

The study was completed with a total of 41 participants (Fig. 1). The sociodemographic and pain characteristics of the three groups are provided in Table 1. A total of 70 individuals was assessed for eligibility, with 25 excluded (23 not meeting criteria, 2 for other reasons). The remaining 45 were randomized into three groups. A total of 45 participants were initially allocated: 15 in the VR group (no dropouts), 15 in the CM group (2 dropouts due to health and family reasons), and 15 in the TE group (13 received the intervention). In total, 41 participants completed the study.

Table 1.

Baseline sociodemographic and pain characteristics of the groups.

Variable  VR Group (n = 15) Mean ± SD  CM Group (n = 13) Mean ± SD  TE Group (n = 13) Mean ± SD 
Age (years)  26.9 ± 7.9  31.2 ± 10.7  27.7 ± 6.3 
BMI (kg/m2)  23.9 ± 5.1  24.0 ± 4.2  22.1 ± 4.1 
  n (%)  n (%)  n (%) 
Gender       
Female  11(73.4)  9 (69.3)  8 (66.7) 
Male  4 (26.6)  4 (30.7)  4 (33.3) 
Education       
Secondary School  0 (0.0)  1 (7.7)  0 (0.0) 
High School  7 (47.0)  5 (38.5)  5 (41.7) 
University  8 (11.8)  5 (38.5)  3 (25.0) 
Post graduate  0 (0.0)  2 (15.3)  4 (33.3) 
Occupation       
Workers  10 (66.7)  8 (61.5)  7 (58.4) 
Student  5 (33.3)  5 (38.5)  5 (41.6) 
Pain location       
Upper neck  4 (26.7)  2 (15.4)  5 (41.7) 
Lower neck  5 (33.3)  2 (15.4)  4 (33.3) 
Upper/Lower neck  6 (40.0)  9 (69.2)  3 (25.0) 
Pain location       
Right  2 (13.3)  5 (38.4)  1 (8.4) 
Left  2 (13.3)  1 (7.6)  1 (8.4) 
Right/ Left  11 (73.4)  7 (54.0)  10 (83.2) 
Pain duration       
3–6 months  2 (13.3)  1 (7.6)  1 (8.4) 
6–12 months  2 (13.3)  2 (15.4)  0 (0.0) 
1–2 years  4 (26.6)  2 (15.4)  4 (33.3) 
2–5 years  4 (26.6)  4 (30.8)  6 (50.0) 
>5 years  3 (21.2)  4 (30.8)  1 (8.4) 
Pain frequency       
≤ 1 and once a week  0 (0.0)  2 (15.4)  1 (8.4) 
2–3 times a week  6 (40.0)  3 (23.2)  7 (58.4) 
>3 times a week  9 (60.0)  8 (61.5)  4 (33.3) 

BMI, body mass index; CM, cervical mobilization; SD, standard deviation; TE, traditional exercise; VR, virtual reality.

Primary outcomes

The baseline and post-treatment values for JPSE and pain are shown in Table 2. The group*time interactions for JPSE values in six cervical directions were not statistically significant between groups (p > 0.05). The comparison between VR and CM shows a negligible effect size of 0.01 for JPSE-flexion; 0.02 for JPSE-extension; 0.12 for JPSE-right rotation; and 0.16 for JPSE-left rotation. For JPSE-RLF and JPSE-LLF, the effect size is respectively 0.54 and 0.53, indicating a moderate difference between VR and CM.

Table 2.

Primary outcomes (joint position sense and pain) at baseline and posttreatment.

  Group  Baseline X±SD  PosttreatmentX±SD  Mean difference (95 % CI)  Effect sizeGroupTime  Group*Time 
JPSE-Flexion (°)  VR  3.44 ± 1.64  2.73 ± 1.97  0.71 (−0.51, 1.93)  VR-TE  0.31  0.017      0.213  0.687 
  CM  4.63 ± 1.82  3.93 ± 1.31  0.70 (−0.78, 2.19)  CM-TE  0.29  VR-TE      CM-TE  VR-CM 
  TE  3.64 ± 1.98  3.60 ± 0.52  0.03 (−1.36, 1.42)  VR-CM  0.01  0.166      0.107  0.005 
JPSE Extension (°)  VR  4.25 ± 1.46  3.43 ± 1.20  0.81 (−0.02, 1.65)  VR-TE  0.43  0.852      0.072  0.372 
  CM  4.20 ± 2.02  3.35 ± 0.99  0.84 (−0.48, 2.17)  CM-TE  0.37           
  TE  3.64 ± 1.52  3.67 ± 0.86  −0.03 (−1.39, 1.33)  VR-CM  0.02           
JPSE-RLF (°)  VR  4.57 ± 2.35  2.62 ± 1.28  1.95 (0.43, 3.46)  VR-TE  0.74  <0.001      0.276  0.165 
  CM  5.38 ± 2.65  5.01 ± 1.72  0.37 (−1.47, 2.22)  CM-TE  0.23  VR-TE      CM-TE  VR-CM 
  TE  6.22 ± 3.45  6.64 ± 3.54  −0.42 (−2.74, 1.89)  VR-CM  0.54  <0.001      0.124  0.003 
JPSE-LLF (°)  VR  4.83 ± 2.80  2.18 ± 1.57  2.65 (1.20, 4.10)  VR-TE  0.62  0.010      <0.001  0.310 
  CM  4.76 ± 2.62  3.53 ± 1.83  1.23 (−0.45, 2.90)  CM-TE  0.07  VR-TE      CM-TE  VR-CM 
  TE  5.71 ± 2.50  4.66 ± 1.29  1.05 (−0.55, 2.65)  VR-CM  0.53  0.003      0.070  0.275 
JPSE-RR (°)  VR  5.51 ± 2.97  3.62 ± 1.10  1.89 (0.17, 3.60)  VR-TE  0.62  0.727      0.002  0.110 
  CM  6.01 ± 3.38  3.72 ± 0.85  2.28 (0.32, 4.24)  CM-TE  0.75           
  TE  5.05 ± 2.28  4.92 ± 1.14  0.13 (−1.43, 1.69)  VR-CM  0.12           
JPSE-LR (°)  VR  5.33 ± 2.58  3.99 ± 1.63  1.34 (−0.32, 3.00)  VR-TE  0.23  0.404      0.017  0.631 
  CM  5.23 ± 2.97  3.38 ± 1.39  1.84 (−0.10, 3.79)  CM-TE  0.35           
  TE  5.49 ± 3.70  4.96 ± 0.87  0.54 (−2.06, 3.13)  VR-CM  0.16           
VAS (0–10)  VR  5.00± 1.51  2.33 ± 1.45  2.67 (1.67, 3.66)  VR-TE  0.05  0.011      <0.001  0.563 
  CM  5.69 ± 1.38  2.23 ± 1.88  3.46 (2.40, 4.53)  CM-TE  0.45  VR-TE      CM-TE  VR-CM 
  TE  4.33 ± 0.98  1.58 ± 1.08  2.75 (1.89, 3.61)  VR-CM  0.44  0.032      0.007  0.467 

Data are mean±SD. CI, confidence interval; CM, cervical mobilization; JPSE, joint position sense error; LLF, left lateral flexion; LR, left rotation; RLF, right lateral flexion; RR, right rotation; SD, standard deviation; TE, traditional exercise; VAS, visual analog scale; VR, virtual reality; X, mean.

Similarly, for the VAS, the group*time interaction was not statistically significant (p > 0.05). The comparison between VR and CM shows a small to moderate effect size of 0.44 for VAS.

Secondary outcomes

The baseline and post-treatment values for the baropodometry measurements, 10MWT, FSST, and SLST assessments are shown in Table 3. There were no statistically significant differences between the groups regarding the baropodometry measurements, 10MWT, SLST, and FFST (p > 0.05).

Table 3.

Secondary outcomes (baropodometric outcomes and balance measurements) at baseline and posttreatment.

  Group  Baseline  Post-treatment X±SD  Mean difference (95 % CI)  Effect sizeGroupTime  Group*Time 
Rweight %  VR  50.47 ± 2.45  50.6 ± 3.72  −0.13 (−1.58,1.31)  VR-TE  0.13  0.717      0.268  0.629 
  CM  47.42 ± 14.15  51.38 ± 2.66  −3.96 (−12.59,4.66)  CM-TE  0.32           
  TE  50.67 ± 4.52  51.25 ± 2.09  −0.58 (−3.19,2.02)  VR-CM  0.39           
Lweight %  VR  49.47 ± 2.45  49.4 ± 3.72  0.07 (−1.45,1.58)  VR-TE  0.15  0.374      0.458  0.553 
  CM  44.96 ± 13.41  48.62 ± 2.66  −3.65 (−11.97,4.67)  CM-TE  0.30           
  TE  49.33 ± 4.52  48.75 ± 2.09  0.58 (−2.02,3.19)  VR-CM  0.39           
Rforefoot weight %  VR  25.47 ± 2.70  26.07 ± 2.60  −0.6 (−1.68,0.48)  VR-TE  0.27  0.472      0.387  0.918 
  CM  24.17 ± 7.83  24.69 ± 2.46  −0.52 (−5.79,4.75)  CM-TE  0.11           
  TE  25.33 ± 3.75  26.58 ± 3.20  −1.25 (−3.07,0.57)  VR-CM  0.01           
Lforefoot weight %  VR  24.93 ± 3.10  25.67 ± 3.77  −0.73 (−2.05,0.58)  VR-TE  0.20  0.326      0.672  0.840 
  CM  23.25 ± 7.64  23.92 ± 1.93  −0.67 (−5.45,4.10)  CM-TE  0.07           
  TE  24.58 ± 3.70  24.33 ± 2.64  0.25 (−1.26,1.76)  VR-CM  0.25           
Rrearfoot weight %  VR  25.07 ± 3.35  24.53 ± 3.80  0.53 (−0.55,1.62)  VR-TE  0.07  0.972      0.405  0.194 
  CM  23.25 ± 7.41  26.69 ± 2.25  −3.44 (−7.43,0.54)  CM-TE  0.57           
  TE  25.33 ± 3.42  24.67 ± 2.53  0.67 (−0.55,1.89)  VR-CM  0.84           
Lrearfoot weight %  VR  24.47 ± 2.13  23.67 ± 2.50  0.8 (−0.33,1.93)  VR-TE  0.16  0.535      0.481  0.241 
  CM  21.71 ± 7.40  24.69 ± 2.53  −2.98 (−7.2,1.25)  CM-TE  0.47           
  TE  24.75 ± 3.57  24.42 ± 3.40  0.33 (−2.03,2.70)  VR-CM  0.76           
MisalignmentC (mm)  VR  0.67 ± 0.40  0.87 ± 0.58  −0.2 (−0.51,0.10)  VR-TE  0.90  0.586      0.900  0.164 
  CM  0.77 ± 0.36  0.77 ± 0.47  0 (−0.28,0.29)  CM-TE  0.54           
  TE  0.98 ± 0.31  0.75 ± 0.44  0.23 (−0.01,0.48)  VR-CM  0.39           
DeltaX (mm)  VR  5.04 ± 4.06  4.91 ± 3.43  0.13 (−1.26,1.52)  VR-TE  0.25  0.116      0.506  0.778 
  CM  5.54 ± 3.55  6.14 ± 4.85  −0.6 (−2.72,1.52)  CM-TE  0.03           
  TE  6.21 ± 2.46  7.31 ± 3.09  −0.71 (−3.44,2.01)  VR-CM  0.24           
DeltaY (mm)  VR  4.95 ± 4.45  4.21 ± 2.18  0.74 (−1.37,2.84)  VR-TE  0.33  0.138      0.457  0.431 
  CM  5.67 ± 3.61  7.97 ± 6.77  −2.3 (−6.16,1.57)  CM-TE  0.32           
  TE  7.7 ± 4.09  8.29 ± 4.22  −0.59 (−3.24,2.06)  VR-CM  0.59           
Avspeed (°)  VR  6.98 ± 1.69  9.04 ± 4.00  −2.06 (−4.55,0.43)  VR-TE  0.64  0.100      0.089  0.216 
  CM  7.49 ± 1.97  8.17 ± 2.88  −0.69 (−2.8,1.43)  CM-TE  0.31           
  TE  6.96 ± 1.42  6.86 ± 1.73  0.1 (−0.53,0.73)  VR-CM  0.34           
10MWT  VR  5.51 ± 0.62  5.13 ± 0.64  0.38 (0.09,0.67)  VR-TE  0.19  0.088      0.003  0.648 
  CM  6.24 ± 1.65  5.38 ± 0.55  0.86 (0.01,1.70)  CM-TE  0.38           
  TE  5.46 ± 0.82  4.99 ± 0.57  0.47 (0.21,0.73)  VR-CM  0.45           
FSST  VR  8.09 ± 1.83  6.96 ± 1.54  1.14 (0.12,2.15)  VR-TE  0.32  0.001      0.003  0.747 
  CM  9.07 ± 1.63  7.88 ± 1.85  1.18 (0.19,2.18)  CM-TE  0.38  VR-TE      CM-TE  VR-CM 
  TE  7.28 ± 1.38  6.61 ± 0.95  0.67 (0.11,1.23)  VR-CM  0.02  0.123      <0.001  0.032 
  Group  Baseline  Post-treatment X±SD  Mean difference (95 % CI)  Effect sizeGroupTime  Group*Time 
SLST                       
Right side eyes closed  VR  19.60 ± 12.35  21.75 ± 8.17  2.15 (−4.78,9.09)  VR-TE  0.23  0.357      0.124  0.472 
  CM  14.93 ± 11.67  16.15 ± 11.81  1.22 (−4.99,7.42)  CM-TE  0.16           
  TE  18.11 ± 10.48  17.87 ± 11.59  −0.23 (−4.88,4.41)  VR-CM  0.08           
Left side eyes closed  VR  16.12 ± 12.01  16.03 ± 11.74  −0.09 (−7.63,7.44)  VR-TE  0.24  0.226      0.183  0.376 
  CM  17.37 ± 11.92  19.72 ± 11.49  2.35 (−3.46,8.16)  CM-TE  0.02           
  TE  14.42 ± 11.82  16.96 ± 11.88  2.54 (−1.52,6.59)  VR-CM  0.02           

Data are mean±SD. CI, confidence interval; CM, cervical mobilization; FSST, four square step test; L, left; MWT, meter walking test; R, right; SD, standard deviation; SLST, single leg stance test; TE, traditional exercise; VR, virtual reality; X, mean.

The baseline and post-treatment values for the NDI and SF-36 assessments for the groups are shown in Table 4. The group*time interaction was not statistically significant except SF-36social functioning (p > 0.05). The mean difference in social functioning is 21.15 with a 95 % CI of 9.46, 32.85, indicating a significant improvement in CM group (Cohen’s d = 1.52). Additionally, no statistically significant difference was observed in GPE scores among the three groups at post-treatment (p > 0.05) Table 4.

Table 4.

Secondary outcomes (GPE, NDI, and SF-36) at baseline and posttreatment.

  Group  Baseline X±SD  PosttreatmentX±SD  Mean difference (95 % CI)  Effect sizeGroupTime  Group*Time 
NDI (0–50)  VR  17.59 ± 6.05  10.07 ± 3.53  7.52 (4.34, 10.70)  VR-TE  0.91  0.099      <0.001  0.09 
  CM  16.41 ± 3.08  7.74 ± 3.80  8.67 (6.80, 10.54)  CM-TE  1.84           
  TE  13.22 ± 2.65  9.87 ± 3.62  3.36 (1.65, 5.06)  VR-CM  0.24           
SF-36                       
Physical functioning  VR  80.33 ± 16.20  86.00 ± 11.37  5.67 (−0.69, 12.02)  VR-TE  0.85  0.005      0.373  0.345 
  CM  83.46 ± 11.62  86.15 ± 9.82  2.69 (−4.02, 9.41)  CM-TE  0.55  VR-TE      CM-TE  VR-CM 
  TE  92.08 ± 7.82  90.00 ± 7.39  −2.08 (−5.25, 1.08)  VR-CM  0.26  0.006      0.013  0.607 
PRL  VR  63.33 ± 37.64  73.33 ± 38.34  10.00 (−18.08, 38.08)  VR-TE  0.27  0.897      0.439  0.739 
  CM  63.46 ± 37.66  73.08 ± 36.03  9.62 (−14.70, 33.93)  CM-TE  0.30           
  TE  72.92 ± 29.11  70.83 ± 31.68  −2.08 (−25.99, 21.82)  VR-CM  0.01           
ERL  VR  40.00 ± 32.38  74.11 ± 41.47  34.11(9.24, 58.98)  VR-TE  0.93  0.133      0.066  0.147 
  CM  35.90 ± 44.02  56.42 ± 47.89  20.52 (−10.84, 51.09)  CM-TE  0.56           
  TE  72.23 ± 39.78  66.67 ± 40.21  −5.56 (−30.84, 19.72)  VR-CM  0.28           
Energy  VR  50.00 ± 17.42  53.67 ± 16.53  3.67 (5.06, 12.36)  VR-TE  0.34  0.288      0.163  0.347 
  CM  38.46 ± 19.62  51.15 ± 17.46  12.69 (1.15, 24.24)  CM-TE  0.90           
  TE  50.83 ± 15.05  50.00 ± 16.92  −0.83 (−6.87, 5.20)  VR-CM  0.52           
Emotional well-being  VR  65.07 ± 18.61  64.53 ± 18.13  −0.53 (−12.07, 11.00)  VR-TE  0.03  0.159      0.377  0.379 
  CM  51.69 ± 18.07  62.15 ± 14.20  10.46 (1.94, 18.98)  CM-TE  0.81           
  TE  63.67 ± 15.49  63.67 ± 19.29  0.00 (−7.43, 7.43)  VR-CM  0.61           
Social functioning  VR  70.00 ± 23.53  72.50 ± 20.70  2.5 (−13.46, 18.46)  VR-TE  0.47  0.119      0.289  0.015 
  CM  55.77 ± 17.39  76.92 ± 17.56  21.15 (9.46, 32.85)  CM-TE  1.52           
  TE  82.29 ± 20.27  72.92 ± 24.33  −9.38 (−22.55, 3.80)  VR-CM  0.75           
Pain  VR  56.17 ± 20.98  73.17 ± 16.41  17.00 (5.79, 28.21)  VR-TE  0.77  0.454      <0.001  0.200 
  CM  50.77 ± 13.78  67.69 ± 17.69  16.92 (7.17, 26.68)  CM-TE  0.91           
  TE  60.83 ± 15.72  64.79 ± 13.46  3.96 (−3.72, 11.63)  VR-CM  0.01           
Health Change  VR  57.33 ± 18.50  65.67 ± 18.41  8.33 (−0.83, 17.50)  VR-TE  0.52  0.552      0.150  0.686 
  CM  53.85 ± 16.48  62.31 ± 14.81  8.46 (0.82, 16.10)  CM-TE  0.62           
  TE  62.92 ± 20.72  63.33 ± 20.71  0.42 (−8.09, 8.93)  VR-CM  0.01           
GPE (−5, +5 points)  VR  2.07 ± 1.53    0.596a  –  –     
  CM  2.38 ± 2.02             
  TE  1.17 ± 2.41             

Data are mean±SD. a= Kruskal-Wallis Test, Each SF-36 domain ranges from 0 to 100. CI, confidence interval, CM, cervical mobilization; ERL, role limitations due to emotional health; GPE, global perceived effect; NDI, neck disability index; PRL, role limitations due to physical health; SD, standard deviation; SF-36, short form-36; TE, traditional exercise; VR, virtual reality; X, mean.

One participant had mild dizziness in the first session in the VR group, however dizziness disappeared within 5 min. In the CM group, two participants had mild pain in the first session. No observed pain in the other sessions.

Discussion

This randomized controlled study compared the effects of adding immersive VR and CM to traditional exercises on JPS and pain, as well as balance, global perceived effect, functional disability, and quality of life. The addition of VR and CM to exercises showed no significant benefits. Secondarily, improvements in balance outcomes, global perceived effect, and functional disability were similar across all groups. Furthermore, of the quality-of-life parameters, only the social functioning parameter demonstrated a significant difference between the groups, with the advantage in favor of the CM group.

JPS and pain

Based on this study, which started from the premise that VR and CM applications have similar physiological mechanisms, the addition of VR and CM to traditional exercises did not provide additional benefits for JPS. Similar to the current study, Bahat et al.20 investigated the effects of cervical kinematic training with or without VR, and found no difference in effects.20 Guo et al.37 also stated that there was no difference of improvement in proprioception between VR therapy combined with conventional rehabilitation and conventional rehabilitation alone.37 In another study about comparison of the effect of VR and conventional proprioceptive training, no significant group-by-time interaction were seen in all directions of JPSE.40 In contrast, Cetin et al. compared the effects of VR and motor control exercises on JPS, and they indicated that JPS showed more improvement in the VR group.24

Similarly, adding VR and CM to traditional exercises did not lead to additional benefits. Those results are consistent with those from the studies by Bahat et al.20 and Guo et al. .24 In contrast, Guo et al. ,24 in a different study, indicated that VR therapy provided more pain reduction compared to conventional rehabilitation.37 In a 2024 review study on the management of neck pain using VR, it was noted that combining VR therapy with conventional rehabilitation is effective in alleviating pain.41 A meta-analysis study indicated that moderate evidence supports VR as a beneficial nonpharmacological approach to reducing neck pain intensity, but limited and heterogeneous studies limit the strength of evidence.42 In another study, CM has been shown to be effective in reducing pain in patients with CNP compared to routine physical therapy alone.43 We can say that VR is a promising tool for chronic pain management; however, there is a lack of VR intervention design consistency, follow-up reporting, and large sample sizes.44 These results show that there are studies in the literature presenting different outcomes on this topic, and further research is needed. According to the results of our study, there was no benefit to adding VR and CM to traditional exercises in improving JPS and reducing pain.

Balance, functional disability, global perceived effect and quality of life

Changes in baropodometric measurements, 10MWT, SLST, and FSST were similar for all 3 groups. In a previous study comparing CerviGame®-VR and conventional proprioceptive training, neither intervention was superior in improving dynamic balance.16 Also, Bahat et al.20 indicated that no difference was observed between the kinematic training with or without VR groups in terms of eyes closed balance and single leg stance values.20 Our study also include parameters such as foot pressure distribution and postural oscillations that are not widely researched in the literature. In a study conducted to compare the peak pressure distribution in individuals with CNP compared to healthy participants, the authors stated that there was no difference in peak plantar pressure or center of pressure oscillation values between groups.11 The only notable finding was for the greater difference in quality of life for those receiving a combination of traditional exercises and CM or VR compared to traditional exercises alone.

The study has several limitations. The VR application in the study is not an application specifically developed for patients with CNP. There is a need for applications to be developed specifically for this population. Additionally, the absence of long-term follow-up data further restricts the generalizability of the findings.

Conclusion

Our findings indicate that adding either VR or CM to traditional exercises provides no additional benefits over using traditional exercises alone for pain and JPS in individuals with CNP.

References
[1]
R. Fejer, K.O. Kyvik, J. Hartvigsen.
The prevalence of neck pain in the world population: a systematic critical review of the literature.
Euro Spine J, 15 (2006), pp. 834-848
[2]
N. Vij, H. Tolson, H. Kiernan, V. Agusala, O. Viswanath, I. Urits.
Pathoanatomy, biomechanics, and treatment of upper cervical ligamentous instability: a literature review.
Orthop Rev (Pavia), 14 (2022),
[3]
P.R. Blanpied, A.R. Gross, J.M. Elliott, L.L. Devaney, D. Clewley, D.M. Walton, et al.
Neck pain: revision 2017: clinical practice guidelines linked to the international classification of functioning, disability and health from the orthopaedic section of the American Physical Therapy Association.
J Orthopaedic Sports Phys Therapy, 47 (2017), pp. A1-A83
[4]
B. Peng, L. Yang, Y. Li, T. Liu, Y. Liu.
Cervical proprioception impairment in neck pain-pathophysiology, clinical evaluation, and management: a narrative review.
Pain Ther, 10 (2021), pp. 143-164
[5]
L. Yang, C. Yang, X. Pang, D. Li, H. Yang, X. Zhang, et al.
Mechanoreceptors in diseased cervical intervertebral disc and vertigo.
Spine (Phila Pa 1976), 42 (2017), pp. 540-546
[6]
T. Mitsutake, M. Sakamoto, Y. Chyuda, S. Oka, H. Hirata, T. Matsuo, et al.
Greater cervical muscle fat infiltration evaluated by magnetic resonance imaging is associated with poor postural stability in patients with cervical spondylotic radiculopathy.
Spine (Phila Pa 1976), 41 (2016), pp. E8-E14
[7]
S. Amiri Arimi, L. Ghamkhar, A.H Kahlaee.
The relevance of proprioception to chronic neck pain: a correlational analysis of flexor muscle size and endurance, clinical neck pain characteristics, and proprioception.
Pain Med, 19 (2018), pp. 2077-2088
[8]
S. Field, J. Treleaven, G. Jull.
Standing balance: a comparison between idiopathic and whiplash-induced neck pain.
Man Ther, 13 (2008), pp. 183-191
[9]
B.K. Humphreys.
Cervical outcome measures: testing for postural stability and balance.
J Manipulative Physiol Ther, 31 (2008), pp. 540-546
[10]
J.-H. Kang, R.-Y. Park, S.-J. Lee, J.-Y. Kim, S.-R. Yoon, K.-I. Jung.
The effect of the forward head posture on postural balance in long time computer based worker.
Ann Rehabil Med, 36 (2012), pp. 98-104
[11]
A.V. Dibai-Filho, Jesus de, R.R. Guirro, M.S. Rezende, F.B. Rangon, V.T.K. Ferreira, E.C de Oliveira Guirro.
Analysis of peak plantar pressure and center of pressure oscillation in individuals with chronic neck pain: a cross-sectional study.
J Back Musculoskelet Rehabil, 30 (2017), pp. 1259-1264
[12]
K. Otadi, M.-R. Hadian, S. Talebian, A. Shadmehr, S. Emamdoost, G. Shahriar.
The effect of myofascial neck pain on postural control: visual deprivation.
J Back Musculoskelet Rehabil, 26 (2013), pp. 375-380
[13]
J. Treleaven.
Sensorimotor disturbances in neck disorders affecting postural stability, head and eye movement control—Part 2: case studies.
Man Ther, 13 (2008), pp. 266-275
[14]
M.A. McCaskey, C. Schuster-Amft, B. Wirth, Z. Suica, E.D. de Bruin.
Effects of proprioceptive exercises on pain and function in chronic neck-and low back pain rehabilitation: a systematic literature review.
BMC Musculoskelet Disord, 15 (2014), pp. 1-17
[15]
M. Saadat, R. Salehi, H. Negahban, M.J. Shaterzadeh, M. Mehravar, M. Hessam.
Traditional physical therapy exercises combined with sensorimotor training: the effects on clinical outcomes for chronic neck pain in a double-blind, randomized controlled trial.
J Bodyw Mov Ther, 23 (2019), pp. 901-907
[16]
I. Rezaei, M. Razeghi, S. Ebrahimi, S. Kayedi.
A novel virtual reality technique (Cervigame®) compared to conventional proprioceptive training to treat neck pain: a randomized controlled trial.
J Biomed Phys Eng, 9 (2019), pp. 355
[17]
T.G. Izquierdo, D. Pecos-Martin, E.L. Girbés, G. Plaza-Manzano, R.R. Caldentey, R.M. Melús, et al.
Comparison of cranio-cervical flexion training versus cervical proprioception training in patients with chronic neck pain: a randomized controlled clinical trial.
J Rehabil Med, 48 (2016), pp. 48-55
[18]
S. Nobusako, A. Matsuo, S. Morioka.
Effectiveness of the gaze direction recognition task for chronic neck pain and cervical range of motion: a randomized controlled pilot study.
Rehabil Res Pract, 2012 (2012),
[19]
M. Sremakaew, G. Jull, J. Treleaven, S. Uthaikhup.
Effectiveness of adding rehabilitation of cervical related sensorimotor control to manual therapy and exercise for neck pain: a randomized controlled trial.
Musculoskeletal Sci Pract, 63 (2023),
[20]
H.S. Bahat, H. Takasaki, X. Chen, Y. Bet-Or, J. Treleaven.
Cervical kinematic training with and without interactive VR training for chronic neck pain–a randomized clinical trial.
Man Ther, 20 (2015), pp. 68-78
[21]
C.M. Petersen, C.L. Zimmermann, R. Tang.
Proprioception interventions to improve cervical position sense in cervical pathology.
Int J Ther Rehabil, 20 (2013), pp. 154-163
[22]
Tieri G., Morone G., Paolucci S., Iosa M. Virtual reality in cognitive and motor rehabilitation: facts, fiction and fallacies. Expert review of medical devices. 2018; 15:107–17. https://doi.org/10.1080/17434440.2018.1425613.
[23]
A. Fusco, G. Tieri.
Challenges and perspectives for clinical applications of immersive and non-immersive virtual reality.
J Clin Med, 11 (2022), pp. 4540
[24]
H. Cetin, N. Kose, H.K. Oge.
Virtual reality and motor control exercises to treat chronic neck pain: a randomized controlled trial.
Musculoskeletal Sci Pract, 62 (2022),
[25]
A. Li, Z. Montaño, V.J. Chen, J.I. Gold.
Virtual reality and pain management: current trends and future directions.
Pain Manag, 1 (2011), pp. 147-157
[26]
A. Gross, T. Kay, C. Kennedy, D. Gasner, L. Hurley, K. Yardley, et al.
Clinical practice guideline on the use of manipulation or mobilization in the treatment of adults with mechanical neck disorders.
Man Ther, 7 (2002), pp. 193-205
[27]
I. Audette, J.-P. Dumas, J.N. Côté, S.J. De Serres.
Validity and between-day reliability of the cervical range of motion (CROM) device.
J Orthopaedic Sports Phys Therapy, 40 (2010), pp. 318-323
[28]
A.M. Boonstra, H.R.S. Preuper, G.A. Balk, R.E. Stewart.
Cut-off points for mild, moderate, and severe pain on the visual analogue scale for pain in patients with chronic musculoskeletal pain.
Pain®, 155 (2014), pp. 2545-2550
[29]
M. Długosz-Boś, K. Filar-Mierzwa, R. Stawarz, A. Ścisłowska-Czarnecka, A. Jankowicz-Szymańska, A. Bac.
Effect of three months pilates training on balance and fall risk in older women.
Int J Environ Res Public Health, 18 (2021), pp. 3663
[30]
N.M. Salbach, K.K. O’Brien, D. Brooks, E. Irvin, R. Martino, P. Takhar, et al.
Reference values for standardized tests of walking speed and distance: a systematic review.
Gait Posture, 41 (2015), pp. 341-360
[31]
M. Moore, K. Barker.
The validity and reliability of the four square step test in different adult populations: a systematic review.
[32]
G. Amaral, H. Martins, A.G. Silva.
Postural control in subclinical neck pain: a comparative study on the effect of pain and measurement procedures.
Scand J Pain, 18 (2018), pp. 295-302
[33]
H. Vernon, S. Mior.
The Neck Disability Index: a study of reliability and validity.
J Manipulative Physiol Ther, 14 (1991), pp. 409-415
[34]
R. Evans, G. Bronfort, M. Maiers, C. Schulz, J. Hartvigsen.
I know it’s changed”: a mixed-methods study of the meaning of global perceived effect in chronic neck pain patients.
Euro Spine J, 23 (2014), pp. 888-897
[35]
J.E. Brazier, R. Harper, N. Jones, A. O'cathain, K. Thomas, T. Usherwood, et al.
Validating the SF-36 health survey questionnaire: new outcome measure for primary care.
Br Med J, 305 (1992), pp. 160-164
[36]
S.T. Celenay, T. Akbayrak, D.O. Kaya.
A comparison of the effects of stabilization exercises plus manual therapy to those of stabilization exercises alone in patients with nonspecific mechanical neck pain: a randomized clinical trial.
J Orthopaedic Sports Phys Therapy, 46 (2016), pp. 44-55
[37]
Q. Guo, L. Zhang, L.L. Han, C. Gui, G. Chen, C. Ling, et al.
Effects of virtual reality therapy combined with conventional rehabilitation on pain, kinematic function, and disability in patients with chronic neck pain: randomized controlled trial.
JMIR Serious Games, 12 (2024),
[38]
G. Kjellman, B. Oberg.
A randomized clinical trial comparing general exercise, McKenzie treatment and a control group in patients with neck pain.
J Rehabil Med, 34 (2002), pp. 183-190
[39]
Cohen J. Statistical power analysis for the behavioral sciences: routledge; 2013. https://doi.org/10.4324/9780203771587.
[40]
S. Shabakhti, M. Razeghi, I. Rezaei, N. Jenabi.
Comparison of the effect of virtual reality and conventional proprioceptive training in the neck region on pain, functional variables and balance in patients with cervical radiculopathy. A randomized controlled trial.
J Biomed Phys Eng, 11 (2021),
[41]
Hao J., He Z., Chen Z., Remis A. Virtual reality training versus conventional rehabilitation for chronic neck pain: a systematic review and meta-analysis. PM&R. 2024. https://doi.org/10.1002/pmrj.13158.
[42]
Q. Guo, L. Zhang, C. Gui, G. Chen, Y. Chen, H. Tan, et al.
Virtual reality intervention for patients with neck pain: systematic review and meta-analysis of randomized controlled trials.
J. Med Internet Res., 25 (2023),
[43]
M.N. Farooq, M.A. Mohseni-Bandpei, S.A. Gilani, M. Ashfaq, Q. Mahmood.
The effects of neck mobilization in patients with chronic neck pain: a randomized controlled trial.
J Bodyw Mov Ther, 22 (2018), pp. 24-31
[44]
G. Ye, R.G. Koh, K. Jaiswal, H. Soomal, D. Kumbhare.
The use of virtual reality in the rehabilitation of chronic nonspecific neck pain: a systematic review and meta-analysis.
Clin J Pain, 39 (2023), pp. 491-500
Copyright © 2025. Associação Brasileira de Pesquisa e Pós-Graduação em Fisioterapia
Download PDF
Idiomas
Brazilian Journal of Physical Therapy
Article options
Tools
en pt
Cookies policy Política de cookies
To improve our services and products, we use "cookies" (own or third parties authorized) to show advertising related to client preferences through the analyses of navigation customer behavior. Continuing navigation will be considered as acceptance of this use. You can change the settings or obtain more information by clicking here. Utilizamos cookies próprios e de terceiros para melhorar nossos serviços e mostrar publicidade relacionada às suas preferências, analisando seus hábitos de navegação. Se continuar a navegar, consideramos que aceita o seu uso. Você pode alterar a configuração ou obter mais informações aqui.