How well do surveys on adherence to pandemic policies assess actual behaviour: Measurement properties of the Dutch COVID-19 adherence to prevention advice survey (CAPAS).

Background: Survey data on adherence to COVID-19 prevention measures have often been used to inform policy makers and public health professionals. Although behavioural survey data are often considered to suffer from biases, there is a lack of studies critically examining the validity, reliability and responsiveness of population-survey data on behaviour throughout the COVID-19 pandemic.

Aim: We studied the measurement properties of the COVID-19 Adherence to Prevention Advice Survey (CAPAS), a novel questionnaire implemented in a repeated cross-sectional (i.e., 'Trend') Study and a Cohort Study in the Netherlands during the COVID-19 pandemic.

Methods: The CAPAS is a novel questionnaire developed in March 2020, with the aim to assess social activity and adherence to COVID-19 prevention measures. Items were formulated to minimise social desirability and aid memory retrieval. Based on the COSMIN framework, we selected the most suitable test for each behavioural question. We investigated criterion validity of vaccination, testing behaviour and mobility by comparing (aggregate) trends of self-reported behaviour to trends in objective data. Responsiveness of mobility and ventilation behaviour was assessed by studying whether self-reported behaviour changed following contextual (e.g., policy) changes. Test-retest reliability of hygienic behaviour, wearing face masks, ventilation behaviour and social distancing was examined during a period in which the context was stable.

Results: Overall, aggregate trends in self-reported behaviour closely corresponded to trends in external objective data. Self-reported behaviours were responsive to contextual changes and test-retest reliabilities were adequate. For infrequent behaviours reliability improved when measures were dichotomised. We were able to examine national representativeness for vaccination, which suggested a modest overestimation of on average 3.7%.

Conclusions: This study supports the suitability of using carefully designed, self-reported surveys (and the CAPAS specifically) to study changes in protective behaviours in a dynamic context.