Generating a Question Answering Dataset About Geographic Changes in a Knowledge Graph

authored by: Michalis Mitsios, Dharmen Punjani, Sara Abdollahi, Simon Gottschalk, Eleni Tsalapati, Elena Demidova, Manolis Koubarakis
Abstract: Most studies on semantic question answering (QA) are predominantly focused on encyclopedic knowledge graphs like DBpedia and Wikidata. These studies cover, if at all, the spatial and temporal characteristics of geospatial entities in isolation, not addressing them simultaneously. In this paper, we introduce a pipeline for creating question answering datasets for evaluating the reasoning capabilities of QA models in the context of geographic changes over time. This pipeline generates questions, GeoSPARQL queries, and corresponding answers by leveraging subgraph and query template extraction techniques. We exemplify this pipeline with the creation of the GeoChangesQA dataset with questions over a knowledge graph of US counties and states and their changes from 1629 to 2000. By evaluating GeoChangesQA using a Transformer-based model, we demonstrate that historical geospatial questions pose a substantial challenge for semantic question answering.
Organisation(s): L3S Research Centre
External Organisation(s): University of Athens
Université Jean Monnet Saint-Étienne
Deutsche Akademie der Technikwissenschaften (acatech)
Athens Technology Center
University of Bonn
Lamarr Institute for Machine Learning and Artificial Intelligence
Archimedes/Athena RC
Type: Conference contribution
Pages: 471-489
No. of pages: 19
Publication date: 2025
Publication status: Published
Peer reviewed: Yes
ASJC Scopus subject areas: Theoretical Computer Science, General Computer Science
Electronic version(s): https://doi.org/10.1007/978-3-031-77792-9_28 (Access: Unknown)

BibTeX

@inproceedings{f6f9ab36ae914310a249e767f914dd43,
title = "Generating a Question Answering Dataset About Geographic Changes in a Knowledge Graph",
abstract = "Most studies on semantic question answering (QA) are predominantly focused on encyclopedic knowledge graphs like DBpedia and Wikidata. These studies cover, if at all, the spatial and temporal characteristics of geospatial entities in isolation, not addressing them simultaneously. In this paper, we introduce a pipeline for creating question answering datasets for evaluating the reasoning capabilities of QA models in the context of geographic changes over time. This pipeline generates questions, GeoSPARQL queries, and corresponding answers by leveraging subgraph and query template extraction techniques. We exemplify this pipeline with the creation of the GeoChangesQA dataset with questions over a knowledge graph of US counties and states and their changes from 1629 to 2000. By evaluating GeoChangesQA using a Transformer-based model, we demonstrate that historical geospatial questions pose a substantial challenge for semantic question answering.",
author = "Michalis Mitsios and Dharmen Punjani and Sara Abdollahi and Simon Gottschalk and Eleni Tsalapati and Elena Demidova and Manolis Koubarakis",
note = "Publisher Copyright: {\textcopyright} The Author(s), under exclusive license to Springer Nature Switzerland AG 2025.; 24th International Conference on Knowledge Engineering and Knowledge Management, EKAW 2024 ; Conference date: 26-11-2024 Through 28-11-2024",
year = "2025",
doi = "10.1007/978-3-031-77792-9_28",
language = "English",
isbn = "9783031777912",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
publisher = "Springer Science and Business Media Deutschland GmbH",
pages = "471--489",
editor = "Mehwish Alam and Marco Rospocher and {van Erp}, Marieke and Laura Hollink and Gesese, {Genet Asefa}",
booktitle = "Knowledge Engineering and Knowledge Management - 24th International Conference, EKAW 2024, Proceedings",
address = "Germany",
}