This cross-sectional reliability study was carried out at an outpatient physical therapy clinic of the Ghent University Hospital (Ghent, Belgium), between September 2017 and February 2018. Ethical approval to conduct this study was granted by the Ethics Committee of the Ghent University Hospital (Ghent, Belgium) in accordance with the guidelines of the Declaration of Helsinki.

Subjects were referred by general practitioners and physical therapists from local hospitals, general practitioners’ clinics, and outpatient physical therapists’ clinics. The referring practitioners were informed on the inclusion and exclusion criteria, prior to the start of the study. All volunteers of any race or sex, aged between 18 and 75 years old, who experienced acute, subacute or chronic neck pain (±arm pain) were considered eligible for inclusion. Subjects were excluded if they were diagnosed with any serious pathology (i.e. cancer, metastasis, untreated fractures, history of diabetes or pathology of the (central) nervous system, non-musculoskeletal neck pain) that could compromise proper assessment or affect clinical reasoning. After the first screening for eligibility by the referring practitioners, subjects who expressed interest in participating were double-checked for eligibility by the researchers before enrolment in the study.

All participants were informed about the aims and procedure of the study and gave signed informed consent before testing. Forty-eight patients were recruited based on sample size calculations for detecting a kappa coefficient of .5 with a two-tailed test at an alpha level of .05 with 80% power.21 Based on the sample of 48 participants, a lower limit of .351 can be expected for the 95% CI of the .5 kappa coefficient.

The raters were 2 experienced physical therapists (first 2 authors), with both a postgraduate degree in Manual Therapy and respectively 13 and 6 years of clinical experience.

A simultaneous examiner design was used to collect the data. Each patient was interviewed and examined by examiner 1, while examiner 2 observed the assessment to assure that both examiners had the same information. Several other reliability studies have used this approach to avoid variable responses or results due to repeated testing.20,22,23 The assessment was performed based on accepted clinical practice.24 The subjective examination consisted of questions related to demographic data; localization, intensity, quality, onset and evolution, circadian rhythm, provocation, and reduction of the pain; associated complaints (movement restrictions, headache, dizziness, nausea, other); medical diagnosis and history; results of technical examinations (medical imagery, blood tests, etc.), and questionnaires (Neck Disability Index and Central Sensitization Inventory); general health; red and yellow flags; restrictions in activities and participation; and medication and/or previous therapy. The physical examination included a postural and movement assessment, and was complemented with additional articular, myofascial, neurological, and/or sensorimotor control assessments. Following the assessment, both raters classified the patient's NSNP description on 3 outcome measures: (1) predominant pain mechanism (i.e. nociceptive, peripheral neuropathic or central/nociplastic pain)14,16,17,25; (2) predominant pain pattern (i.e. input, processing or output)15; and (3) predominant DP (i.e. articular DP, myofascial DP, neural DP, central/nociplastic DP, and sensorimotor control DP).8 The definitions of the DPs and the descriptions of the criteria on which the examiners decided to classify the patients’ NSNP are outlined in Tables 1–3.

Both researchers had 6 years of previous experience with the proposed clinical reasoning model. Before participation, however, the raters studied an assessment manual with definitions and guidelines for the standardized assessment and thoroughly discussed the interpretation of all subjective and physical examination criteria. The assessment protocol was piloted in 2 ‘dummy trials’ to ascertain equal understanding of the diagnostic procedure. To ensure that the raters were blind to each other's judgements, both researchers independently registered the participants’ classification in separate databases, without knowing each other's decision.

The data analysis was performed using RStudio for Windows version R 3_4_1 (RStudio, Boston, MA, USA). To assess interrater reliability of the classification, Cohen's kappa and percentage of overall agreement with 95% confidence intervals (CIs) were calculated. Percentage of overall agreement was defined as the percentage of cases in which both raters agreed on the condition being present in the total study population (n=48). To explore the nature of agreements in the physical therapists’ judgements, identification agreement and exclusion agreement were examined. Identification agreement was defined as the percentage of cases in which both raters agreed on the condition being present, in the subset of cases that were labeled by at least one rater as being present. Finally, exclusion agreement was defined as the percentage of cases in which both raters agreed on the condition being absent, in the subset of cases that were labeled by at least one rater as being absent.

Interpretations of kappa values were based on categories outlined by Landis and Koch.26 For the purpose of this study ‘clinically acceptable’ reliability was defined as kappa >.60 or in the absence of kappa, a percentage agreement of ≥80%.20,27,28

Consistent with a previous study investigating the reliability of clinical judgements associated with the predominant pain mechanisms in LBP patients,20 the current study found clinically acceptable reliability for this outcome measure.

Studying the sub-categories, judgments related to the peripheral neuropathic pain (PNP) category seemed to have perfect agreement. As Cohen stated, differentiating PNP from nociceptive pain is probably the most important clinical distinction to make.39 Yet, this must be interpreted with caution, as there were only 2 out of 48 subjects identified with a dominance of PNP (as mentioned in Table 5). This high degree of clustering patients in non-PNP categories has most likely inflated the kappa value for the PNP category. Other reliability studies exploring the identification of clinical indicators for PNP (in LBP patients) found only fair to moderate levels of interrater reliability.20,40

The examiners in this study seemed to identify patients’ pain states more reliable as predominant nociceptive as compared to the exclusion of predominant nociceptive pain. The opposite was found for the central/nociplastic pain state. This could be supported by the fact that a dominance of nociceptive pain typically has clear criteria on which physical therapists base their judgements (e.g. ‘pain localized to the area of injury/dysfunction’, ‘clear, proportionate mechanical/anatomical nature to aggravating and easing factors’, ‘pain is usually intermittent and sharp with movement/mechanical provocation’, etc.).41 On the other hand, a predominant central/nociplastic pain mechanism is usually characterized by the lack of a consistent clinical pattern (e.g. ‘disproportionate, non-mechanical, unpredictable pattern of pain provocation in response to multiple/nonspecific aggravating/easing factors’).42 Caution is warranted when interpreting these assumptions, since the indicators for pain mechanisms in LBP patients may not generally apply to patients with neck pain. Nevertheless, the ‘central/nociplastic’ features discernible in NSNP patients most commonly agreed upon in the Delphi-survey by Dewitte et al.8 were similar to those identified in LBP patients by Smart et al.42

The pain pattern classification and DP classification show comparable, clinically acceptable kappa values. The agreement levels seem to reveal a trend in that exclusion agreements are higher than identification agreements. From a probability point of view, it seems evident that the exclusion of a condition by 2 raters yields better conformity, compared to the identification of a condition. For the identification of the predominant pain pattern, both raters had to agree on 1 out of 3 categories (i.e. input, processing or output), resulting in a 3/9 chance to agree. On the other hand, for the exclusion agreement they had 4/9 chances to agree on the absence of the prevailing pain pattern. Analogue reflections apply to the predominant DP classification. The 2 physical therapists had only 5/25 chances to agree on the identified DP, in contrast to the 16/25 chances to agree on the absence of the remaining DPs.

It is important that a dominance of central/nociplastic DP/processing mechanism, underlying a patient's NSNP, can be ruled out, since these patients require a different approach.43 Because the processing pain pattern and central/nociplastic DP are defined in a similar way, they both attained identical, clinically acceptable agreement levels.

The input and output pain patterns did not reach the percentage agreement level of ≥80% for clinical acceptance, neither did the myofascial and sensorimotor control DPs. Possibly this could be explained by the interrelationship between input and output patterns, and the considerable overlap in clinical indicators that define the myofascial and sensorimotor control DPs.8 Prolonged sensorimotor control dysfunctions may induce myofascial dysfunctions,44–46 but changes in myofascial structure and function may generate sensorimotor control problems.47 This may render it difficult to clearly distinguish between a dominant myofascial DP or sensorimotor control DP, as both ‘dysfunctions’ appear to merge.

This study has several limitations. Firstly, the simultaneous examiner design may have caused bias toward overestimated interrater agreement values. Indeed, this design excludes the possible variability in assessments and patient–therapist interactions, that could occur in an independent-examiner design, which might result in lower levels of agreement.20

Secondly, the raters in this study had 6 years of experience with the classification under study. As such, the study results may not generalize to novice therapists or clinicians less familiar with mechanisms-based approaches.

Thirdly, the sample studied in the current study showed a preponderance of female patients (72.92%), and patients with headache (79.17%). A purposive, intentionally diverse sample might have been better to support the robustness of the reliability estimates. However, these patient characteristics appear to be typical for patients presenting with NSNP and correspond with the prevalence reported in the available literature.1,2,39,48–51

Finally, we did not test reliability of unique criteria. Consequently, we cannot comment on the capacity of a stand-alone criterion for a definitive decision regarding the classification. This is, however, beyond the scope of this study. In practice, most clinicians do not make clinical decisions based on a single test finding. Using clusters provide more promising findings and are more closely associated to clinical decision making.52