Question order effects

how robust are survey measures on political solidarities with reference to Germany and Europe?

verfasst von: Jan Karem Höhne, Achim Goerres
Abstract: The measurement of political solidarities and related concepts is an important endeavor in numerous scientific disciplines, such as political and social science research. European surveys, such as the Eurobarometer, frequently measure these concepts for people’s home country and Europe raising questions with respect to the order of precedence. Research has shown that the order of asking questions can have a profound impact on answer behavior compromising data quality. In this study, we therefore investigated the occurrence of question order effects in a German-European context using two questions on political solidarities. For this purpose, we conducted an experiment in a German online panel (N = 874) and analyzed response behavior and effort in terms of response times. In contrast to previous research, we found no empirical evidence for question order effects impacting people’s responses. Even though there were no response time differences between the question order conditions, the first question always took longer to respond to than the remaining one. Overall, our findings indicate the robustness of questions on political solidarities against question order effects. One potential explanation is that people have comparatively strong (or crystallized) attitudes when it comes to political solidarities.
Organisationseinheit(en): Institut für Soziologie
Externe Organisation(en): Universität Duisburg-Essen
Typ: Artikel
Journal: International Journal of Social Research Methodology
Band: 27
Seiten: 249-254
Anzahl der Seiten: 6
ISSN: 1364-5579
Publikationsdatum: 03.2024
Publikationsstatus: Veröffentlicht
Peer-reviewed: Ja
ASJC Scopus Sachgebiete: Allgemeine Sozialwissenschaften
Elektronische Version(en): https://doi.org/10.1080/13645579.2023.2227011 (Zugang: Offen)

BibTeX

@article{3d025b03d1794a3190ccc0ff9b753505,
title = "Question order effects: how robust are survey measures on political solidarities with reference to Germany and Europe?",
abstract = "The measurement of political solidarities and related concepts is an important endeavor in numerous scientific disciplines, such as political and social science research. European surveys, such as the Eurobarometer, frequently measure these concepts for people{\textquoteright}s home country and Europe raising questions with respect to the order of precedence. Research has shown that the order of asking questions can have a profound impact on answer behavior compromising data quality. In this study, we therefore investigated the occurrence of question order effects in a German-European context using two questions on political solidarities. For this purpose, we conducted an experiment in a German online panel (N = 874) and analyzed response behavior and effort in terms of response times. In contrast to previous research, we found no empirical evidence for question order effects impacting people{\textquoteright}s responses. Even though there were no response time differences between the question order conditions, the first question always took longer to respond to than the remaining one. Overall, our findings indicate the robustness of questions on political solidarities against question order effects. One potential explanation is that people have comparatively strong (or crystallized) attitudes when it comes to political solidarities.",
keywords = "data quality, non-probability panel, Response behavior, survey question order, web survey",
author = "H{\"o}hne, {Jan Karem} and Achim Goerres",
note = "Funding Information: The inspection of the response distributions shows that they are skewed. However, the order of the questions did not substantially impact people{\textquoteright}s response behavior. There is no higher willingness (in the form of agreeing answers) to endorse governmental support depending on the question order. This impression was supported by the results of two chi-square tests: Germany [χ = 3.80, df = 4, p-value = 0.43] and Europe [χ = 3.36, df = 4, p-value = 0.50]. Both tests indicated no statically significant differences. ",
year = "2024",
month = mar,
doi = "10.1080/13645579.2023.2227011",
language = "English",
volume = "27",
pages = "249--254",
journal = "International Journal of Social Research Methodology",
issn = "1364-5579",
publisher = "Taylor and Francis Ltd.",
number = "2",
}