Participants in the LIKP study were asked to complete, amongst other things, three outcome questionnaire measures at both pre- and (6 months) post-intervention – the WOMAC (Western Ontario and McMaster Universities Osteoarthritis Index), EQ-5D and SF-36 (the latter was used to calculate the SF-6D score). The WOMAC was chosen as primary outcome measure within the LIKP study as the pain subscale of the WOMAC was considered to be the best way of capturing knee pain severity.

The WOMAC measures the amount of pain (5 questions), stiffness (2 questions), and difficulty in physical functioning (17 questions), where the response options are none (0), mild (1), moderate (2), severe (3) or extreme (4) 16. Scores can thereby range between 0 and 20 on the pain scale, 0 and 8 on the stiffness scale, 0 and 68 on the functioning scale, and 0 to 96 on the total WOMAC (WOMAC₉₆) scale, where higher scores denote a worse response 17. Previous evidence of the adequate performance of the WOMAC has been shown for construct validity 18 and responsiveness 1920.

The EQ-5D has five questions, where the respondent is asked to report the level of problems they have (no problems, some/moderate problems, and severe/extreme problems) with regard to mobility, self-care, usual activities, pain, and anxiety/depression 5. Responses to these five dimensions are converted into one of 243 different EQ-5D health state descriptions, which range between no problems on all five dimensions (11111) and severe/extreme problems on all five dimensions (33333). A utility score was assigned to each health state using the York A1 tariff 21, which was based on the preferences elicited from a survey of 3395 UK residents – EQ-5D scores range between -0.594 and 1 (full health).

In a similar way, responses to eleven of the questions on the SF-36 22 were used to estimate a score on the SF-6D 7. The SF-6D is composed of six dimensions (physical functioning, role limitations, social functioning, pain, mental health and vitality) which have between four and six levels. We used the consistent 23 version of the SF-6D algorithm 8 to estimate utility scores for each of 18,000 potential health states – SF-6D scores range between 0.296 and 1.00.

There are many approaches to assessing validity and, as Fitzpatrick et al. 11 point out, these criteria are not uniformly described. Indeed, Streiner & Norman 24 suggest that the myriad of terms that are used to describe such approaches means that one of the most difficult aspects of validity testing is the terminology. In the light of this, we attempt to provide clear definitions of the type of validity that we are testing for in order to avoid the possibility of misinterpretation. References to previous studies which have used similar techniques are also provided. Finally, rather than assessing the predictive ability of certain variables 25, it should be noted that we focus solely on the relationship between the WOMAC and the utility measures of the EQ-5D and SF-6D as, as far as we are aware, such relationships have not been previously investigated. This is in contrast to a number of previous studies e.g. 2627 which have looked at the effect that different socio-demographic characteristics and clinical conditions have on measures of utility.

Practicality was assessed in terms of completion rates, where the SF-36 appeared before the EQ-5D in the ascertainment questionnaire. We assessed whether sufficient information was provided in order to calculate a utility score for the EQ-5D and SF-6D, as outlined by Gerard et al. 28.

Validity was assessed in terms of both construct and convergent validity. Construct validity relates to whether a measure can discriminate between two patient groups, one which has a certain trait, and the other which does not 24. This has also been referred to as known groups validity 2930, based on the principle that certain specified groups of patients may be expected to score differently from one another. We assessed whether the EQ-5D and SF-6D could discriminate between participants with different levels of (pre-intervention) severity on the WOMAC. In accordance with the WOMAC response options (none (0), mild (1), moderate (2), severe (3) or extreme (4)), four severity levels were created on the overall WOMAC₉₆ score – i) none to mild (total score of 0 to ≤24 on the WOMAC₉₆ scale), ii) >mild to moderate (total score of >24 to ≤48 on the WOMAC₉₆ scale), iii) >moderate to severe (total score of >49 to ≤72 on the WOMAC₉₆ scale), and iv) >severe to extreme (total score of >72 to ≤96 on the WOMAC₉₆ scale). Similar severity levels were also created for each of the three sub-scales: i) none to mild was denoted by total scores of 0 to ≤5 (pain), 0 to ≤2 (stiffness), and 0 to ≤17 (functioning), ii) >mild to moderate was denoted by total scores of >5 to ≤10 (pain), >2 to ≤4 (stiffness), and >17 to ≤34, iii) >moderate to severe was denoted by total scores of >10 to ≤15 (pain), >4 to ≤6 (stiffness), and >34 to ≤51 (functioning), and iv) >severe to extreme was denoted by total scores of >15 to ≤20 (pain), >6 to ≤8 (stiffness), and >51 to ≤68 (functioning). On the overall WOMAC₉₆ score, and each of the three WOMAC subscales, in order to assess whether there were significant (p < 0.05) differences between the utility scores of participants in each of these four severity levels a one-way analysis of variance (ANOVA) was conducted. This analysis is akin to that conducted previously 3031.

Convergent validity is determined by how closely a measure is related to other measures of the same construct 24. Thus, in line with previous studies 303233, we assessed whether (pre-intervention) scores on the EQ-5D and SF-6D were significantly correlated with the WOMAC₉₆ score according to the Spearman rank test.

Responsiveness, which is different to sensitivity 29, is determined by the ability an instrument has to detect a meaningful or clinically important change 34, where one seeks to discriminate between those who change a lot and those who change a little 24. Previously, a 20% improvement on each of the three subscales of the WOMAC has been deemed to equate to the minimum clinically important difference 353637. Thus, we sought to assess whether the EQ-5D and SF-6D could discriminate between three groups: i) those who did not improve according to the WOMAC (≤0% change post-intervention), ii) those who improved by <20% (>0% to <20% change post-intervention), and iii) those who improved by ≥20% (≥20% change post-intervention) – the change was estimated by subtracting the pre-intervention score from the post-intervention score, and those who had a worse WOMAC score post-intervention were included in the first group. The mean change scores for the EQ-5D and SF-6D were calculated for each of these three post-intervention groups, for both the overall WOMAC₉₆ scale and each of the three WOMAC subscales, and the paired t-tests was conducted to assess whether there was a significant change in the mean utility score. For both the EQ-5D and the SF-6D a one-way ANOVA was also conducted to assess whether there was a significant difference between the mean change in utility across each of the three groups. The above analysis is in line with that undertaken previously 30.

Pre-intervention the EQ-5D was wholly completed by 378 of the 389 participants (97.2%). Four of these participants were categorised as being in full health (11111), and a total of 36 different EQ-5D health states were reported – 29 participants had health states rated as worse than death, the lowest score was -0.239 (22333), and the mean score was 0.550 (95% confidence interval 0.521 to 0.578).

Pre-intervention, SF-6D scores could be calculated for 366 of the 389 participants (94.1%). None were categorised in full health, but one person did report that they had the lowest score on all six dimensions (645655). SF-6D scores ranged from 0.296 to 0.948, 194 different health states were reported, and the mean score was 0.646 (95% confidence interval 0.631 to 0.660).

In terms of construct validity the results in Table 1 show that, for the 359 participants for whom the WOMAC₉₆ could be calculated (pre-intervention), participants who had higher scores (increased severity) tended to have lower utility scores on both the EQ-5D and SF-6D – the mean EQ-5D (SF-6D) utility score for those with a WOMAC₉₆ score between 0 and 24 was 0.722 (0.731), compared to 0.069 (0.460) for those with a WOMAC₉₆ score between 73 and 96. These differences were significant according to the one-way ANOVA, and similar results were also obtained for each of the three WOMAC subscales (Table 1). With regard to convergent validity, scores on both the EQ-5D and SF-6D scores were highly correlated (p < 0.001) with scores on the WOMAC₉₆ scale (r = -0.576 and r = -0.501, respectively).

Pre- and post-intervention WOMAC₉₆ scores could be calculated for 324 participants. Post-intervention the WOMAC₉₆ score did not improve for 33.8% of the participants, for 25.2% the WOMAC₉₆ score improved by <20%, and for 40.9% the WOMAC₉₆ score improved by ≥20%. The mean change in utility (post-intervention) for each of these three groups was -0.036, 0.091 and 0.127 for the EQ-5D, compared to 0.021, 0.023 and 0.053 for the SF-6D (Table 2). The one-way ANOVA showed that the differences between these three groups were significant according to the EQ-5D (p < 0.001), but not the SF-6D (p = 0.084). Similar results were also obtained across the three subscales of the WOMAC (see Table 2).