Survey Reliability: Models, Methods, and Findings.

J Surv Stat Methodol

is with Westat Inc., Survey Methodology Unit, USA.

Published: November 2021

Although most survey researchers agree that reliability is a critical requirement for survey data, there have not been many efforts to assess the reliability of responses in national surveys. In addition, there are quite different approaches to studying the reliability of survey responses. In the first section of the Lecture, I contrast a psychological theory of over-time consistency with three statistical models that use reinterview data, multi-trait multi-method experiments, and three-wave panel data to estimate reliability. The more sophisticated statistical models reflect concerns about memory effects and the impact of method factors in reinterview studies. In the following section of the Lecture, I examine some of the major findings from the literature on reliability. Despite the differences across methods for exploring reliability, the findings mostly converge, identifying similar respondent and question characteristics as major determinants of reliability. The next section of the paper looks at the correlations among estimates of reliability derived from the different methods; it finds some support for the validity of the measures from traditional reinterview studies. The empirical claims motivating the more sophisticated methods for estimating reliability are not strongly supported in the literature. Reliability is, in my judgment, a neglected topic among survey researchers, and I hope the Lecture spurs further studies of the reliability of survey questions.

Download full-text PDF

Source
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC8665769PMC
http://dx.doi.org/10.1093/jssam/smaa021DOI Listing

Publication Analysis

Top Keywords

reliability
11
survey researchers
8
reliability survey
8
statistical models
8
reinterview studies
8
literature reliability
8
survey
6
survey reliability
4
reliability models
4
methods
4

Similar Publications

Want AI Summaries of new PubMed Abstracts delivered to your In-box?

Enter search terms and have AI summaries delivered each week - change queries or unsubscribe any time!