Psychol Methods. 2021 Aug 26. doi: 10.1037/met0000403. Online ahead of print.
ABSTRACT
Curbstoning, the willful fabrication of survey responses by outside data collectors, threatens the integrity of the inferences drawn from data. Researchers who outsource data collection to survey collection panels, field interviewers, or research assistants should validate whether each collection agent actually collected the data. Our review of the survey auditing literature demonstrates a consistent presence of curbstoning, even at professional levels. This study proposes several general simple survey questions that have statistical distributions known a priori, as a method to detect curbstoning. By exploiting common deficiencies in statistical understanding, survey collectors imputing data to these questions can leverage empirically known distributions to determine deviation from the expected distribution of responses. We examined both authentic and fabricated surveys that included these questions and we compared the observed distributions with the expected distributions. The majority of the proposed methods had Type I error rates near or below the specified alpha level (.05). The methods demonstrated the ability to detect false responses correctly 48%-90% of the time across two samples when surveying at least 50 participants. While the methods varied in effectiveness, combining these methods demonstrated the highest statistical power, with Type I error rates lower than 1%. Additionally, even in situations with smaller sample sizes (e.g., N = 30), combining these methods allows them to be effective in detecting curbstoning. These methods provide a simple and generalizable way for researchers not present during data collection to possess accurate data. (PsycInfo Database Record (c) 2021 APA, all rights reserved).
PMID:34435808 | DOI:10.1037/met0000403
