Identification of Speaker Roles and Situation Types in News Videos

verfasst von: Gullal S. Cheema, Judi Arafat, Chiao I. Tseng, John A. Bateman, Ralph Ewerth, Eric Müller-Budack
Abstract: The proliferation of news sources on the web amplifies the problem of disinformation and misinformation, impacting public perception and societal stability. These issues necessitate the identification of bias in news broadcasts, whereby the analysis and understanding of speaker roles and news contexts are essential prerequisites. Although there is prior research on multimodal speaker role recognition (mostly) in the news domain, modern feature representations have not been explored yet, and no comprehensive public dataset is available. In this paper, we propose novel approaches to classify speaker roles (e.g., “anchor," “reporter," “expert") and categorise scenes into news situations (e.g., “report," “interview") in news videos, to enhance the understanding of news content. To bridge the gap of missing datasets, we present a novel annotated dataset for various speaker roles and news situations from diverse (national) media outlets. Furthermore, we suggest a rich set of features and employ aggregation and post-processing techniques. In our experiments, we compare classifiers like Random Forest and XGBoost for identifying speaker roles and news situations in video segments. Our approach outperforms recent state-of-the-art methods, including end-to-end multimodal deep network and unimodal transformer-based models. Through detailed feature combination analysis, generalisation and explainability insights, we underscore our models’ capabilities and set new directions for future research.
Organisationseinheit(en): Forschungszentrum L3S
Externe Organisation(en): Göteborgs Universitet
Universität Bremen
Technische Informationsbibliothek (TIB) Leibniz-Informationszentrum Technik und Naturwissenschaften und Universitätsbibliothek
Typ: Aufsatz in Konferenzband
Seiten: 506-514
Anzahl der Seiten: 9
Publikationsdatum: 07.06.2024
Publikationsstatus: Veröffentlicht
Peer-reviewed: Ja
ASJC Scopus Sachgebiete: Computergrafik und computergestütztes Design, Mensch-Maschine-Interaktion, Software
Elektronische Version(en): https://doi.org/10.1145/3652583.3658101 (Zugang: Offen)

BibTeX

@inproceedings{771cdd7b89354b07b839c08ba7edc63b,
title = "Identification of Speaker Roles and Situation Types in News Videos",
abstract = "The proliferation of news sources on the web amplifies the problem of disinformation and misinformation, impacting public perception and societal stability. These issues necessitate the identification of bias in news broadcasts, whereby the analysis and understanding of speaker roles and news contexts are essential prerequisites. Although there is prior research on multimodal speaker role recognition (mostly) in the news domain, modern feature representations have not been explored yet, and no comprehensive public dataset is available. In this paper, we propose novel approaches to classify speaker roles (e.g., “anchor,{"} “reporter,{"} “expert{"}) and categorise scenes into news situations (e.g., “report,{"} “interview{"}) in news videos, to enhance the understanding of news content. To bridge the gap of missing datasets, we present a novel annotated dataset for various speaker roles and news situations from diverse (national) media outlets. Furthermore, we suggest a rich set of features and employ aggregation and post-processing techniques. In our experiments, we compare classifiers like Random Forest and XGBoost for identifying speaker roles and news situations in video segments. Our approach outperforms recent state-of-the-art methods, including end-to-end multimodal deep network and unimodal transformer-based models. Through detailed feature combination analysis, generalisation and explainability insights, we underscore our models{\textquoteright} capabilities and set new directions for future research.",
keywords = "news situations, news videos, speaker roles, video classification",
author = "Cheema, {Gullal S.} and Judi Arafat and Tseng, {Chiao I.} and Bateman, {John A.} and Ralph Ewerth and Eric M{\"u}ller-Budack",
note = "Publisher Copyright: {\textcopyright} 2024 Copyright held by the owner/author(s).; 2024 International Conference on Multimedia Retrieval, ICMR 2024 ; Conference date: 10-06-2024 Through 14-06-2024",
year = "2024",
month = jun,
day = "7",
doi = "10.1145/3652583.3658101",
language = "English",
pages = "506--514",
booktitle = "ICMR '24",
publisher = "Association for Computing Machinery, Inc",
}