Bots in web survey interviews

A showcase

authored by: Jan Karem Höhne, Joshua Claassen, Saijal Shahania, David Broneske
Abstract: Cost- and time-efficient web surveys have progressively replaced other survey modes. These efficiencies can potentially cover the increasing demand for survey data. However, since web surveys suffer from low response rates, researchers and practitioners start considering social media platforms as new sources for respondent recruitment. Although these platforms provide advertisement and targeting systems, the data quality and integrity of web surveys recruited through social media might be threatened by bots. Bots have the potential to shift survey outcomes and thus political and social decisions. This is alarming since there is ample literature on bots and how they infiltrate social media platforms, distribute fake news, and possibly skew public opinion. In this study, we therefore investigate bot behavior in web surveys to provide new evidence on common wisdom about the capabilities of bots. We programmed four bots – two rule-based and two AI-based bots – and ran each bot N = 100 times through a web survey on equal gender partnerships. We tested several bot prevention and detection measures, such as CAPTCHAs, invisible honey pot questions, and completion times. The results indicate that both rule- and AI-based bots come with impressive completion rates (up to 100%). In addition, we can prove conventional wisdom about bots in web surveys wrong: CAPTCHAs and honey pot questions pose no challenges. However, there are clear differences between rule- and AI-based bots when it comes to web survey completion.
Organisation(s): Sociology Department
External Organisation(s): German Centre for Higher Education Research and Science Studies (DZHW)
Otto-von-Guericke University Magdeburg
Type: Article
Journal: International Journal of Market Research
No. of pages: 10
ISSN: 1470-7853
Publication date: 15.11.2024
Publication status: E-pub ahead of print
Peer reviewed: Yes
ASJC Scopus subject areas: Business and International Management, Economics and Econometrics, Marketing
Electronic version(s): https://doi.org/10.1177/14707853241297009 (Access: Open)

BibTeX

@article{b45f056b83834c55a42cf33c886000e5,
title = "Bots in web survey interviews: A showcase",
abstract = "Cost- and time-efficient web surveys have progressively replaced other survey modes. These efficiencies can potentially cover the increasing demand for survey data. However, since web surveys suffer from low response rates, researchers and practitioners start considering social media platforms as new sources for respondent recruitment. Although these platforms provide advertisement and targeting systems, the data quality and integrity of web surveys recruited through social media might be threatened by bots. Bots have the potential to shift survey outcomes and thus political and social decisions. This is alarming since there is ample literature on bots and how they infiltrate social media platforms, distribute fake news, and possibly skew public opinion. In this study, we therefore investigate bot behavior in web surveys to provide new evidence on common wisdom about the capabilities of bots. We programmed four bots – two rule-based and two AI-based bots – and ran each bot N = 100 times through a web survey on equal gender partnerships. We tested several bot prevention and detection measures, such as CAPTCHAs, invisible honey pot questions, and completion times. The results indicate that both rule- and AI-based bots come with impressive completion rates (up to 100%). In addition, we can prove conventional wisdom about bots in web surveys wrong: CAPTCHAs and honey pot questions pose no challenges. However, there are clear differences between rule- and AI-based bots when it comes to web survey completion.",
keywords = "AI-based bots, completion behavior, data integrity, data quality, rule-based bots, web surveys",
author = "H{\"o}hne, {Jan Karem} and Joshua Claassen and Saijal Shahania and David Broneske",
note = "Publisher Copyright: {\textcopyright} The Author(s) 2024.",
year = "2024",
month = nov,
day = "15",
doi = "10.1177/14707853241297009",
language = "English",
journal = "International Journal of Market Research",
issn = "1470-7853",
publisher = "Market Research Society",
}