Identification of Speaker Roles and Situation Types in News Videos

authored by: Gullal S. Cheema, Judi Arafat, Chiao I. Tseng, John A. Bateman, Ralph Ewerth, Eric Müller-Budack
Abstract: The proliferation of news sources on the web amplifies the problem of disinformation and misinformation, impacting public perception and societal stability. These issues necessitate the identification of bias in news broadcasts, whereby the analysis and understanding of speaker roles and news contexts are essential prerequisites. Although there is prior research on multimodal speaker role recognition (mostly) in the news domain, modern feature representations have not been explored yet, and no comprehensive public dataset is available. In this paper, we propose novel approaches to classify speaker roles (e.g., “anchor," “reporter," “expert") and categorise scenes into news situations (e.g., “report," “interview") in news videos, to enhance the understanding of news content. To bridge the gap of missing datasets, we present a novel annotated dataset for various speaker roles and news situations from diverse (national) media outlets. Furthermore, we suggest a rich set of features and employ aggregation and post-processing techniques. In our experiments, we compare classifiers like Random Forest and XGBoost for identifying speaker roles and news situations in video segments. Our approach outperforms recent state-of-the-art methods, including end-to-end multimodal deep network and unimodal transformer-based models. Through detailed feature combination analysis, generalisation and explainability insights, we underscore our models’ capabilities and set new directions for future research.
Organisation(s): L3S Research Centre
External Organisation(s): University of Gothenburg
University of Bremen
German National Library of Science and Technology (TIB)
Type: Conference contribution
Pages: 506-514
No. of pages: 9
Publication date: 07.06.2024
Publication status: Published
Peer reviewed: Yes
ASJC Scopus subject areas: Computer Graphics and Computer-Aided Design, Human-Computer Interaction, Software
Electronic version(s): https://doi.org/10.1145/3652583.3658101 (Access: Open)

BibTeX

@inproceedings{771cdd7b89354b07b839c08ba7edc63b,
title = "Identification of Speaker Roles and Situation Types in News Videos",
abstract = "The proliferation of news sources on the web amplifies the problem of disinformation and misinformation, impacting public perception and societal stability. These issues necessitate the identification of bias in news broadcasts, whereby the analysis and understanding of speaker roles and news contexts are essential prerequisites. Although there is prior research on multimodal speaker role recognition (mostly) in the news domain, modern feature representations have not been explored yet, and no comprehensive public dataset is available. In this paper, we propose novel approaches to classify speaker roles (e.g., “anchor,{"} “reporter,{"} “expert{"}) and categorise scenes into news situations (e.g., “report,{"} “interview{"}) in news videos, to enhance the understanding of news content. To bridge the gap of missing datasets, we present a novel annotated dataset for various speaker roles and news situations from diverse (national) media outlets. Furthermore, we suggest a rich set of features and employ aggregation and post-processing techniques. In our experiments, we compare classifiers like Random Forest and XGBoost for identifying speaker roles and news situations in video segments. Our approach outperforms recent state-of-the-art methods, including end-to-end multimodal deep network and unimodal transformer-based models. Through detailed feature combination analysis, generalisation and explainability insights, we underscore our models{\textquoteright} capabilities and set new directions for future research.",
keywords = "news situations, news videos, speaker roles, video classification",
author = "Cheema, {Gullal S.} and Judi Arafat and Tseng, {Chiao I.} and Bateman, {John A.} and Ralph Ewerth and Eric M{\"u}ller-Budack",
note = "Publisher Copyright: {\textcopyright} 2024 Copyright held by the owner/author(s).; 2024 International Conference on Multimedia Retrieval, ICMR 2024 ; Conference date: 10-06-2024 Through 14-06-2024",
year = "2024",
month = jun,
day = "7",
doi = "10.1145/3652583.3658101",
language = "English",
pages = "506--514",
booktitle = "ICMR '24",
publisher = "Association for Computing Machinery, Inc",
}