Online Optimization of Curriculum Learning Schedules using Evolutionary Optimization

verfasst von: Mohit Jiwatode, Leon Schlecht, Alexander Dockhorn
Abstract: We propose RHEA CL, which combines Curriculum Learning (CL) with Rolling Horizon Evolutionary Algorithms (RHEA) to automatically produce effective curricula during the training of a reinforcement learning agent. RHEA CL optimizes a population of curricula, using an evolutionary algorithm, and selects the best-performing curriculum as the starting point for the next training epoch. Performance evaluations are conducted after every curriculum step in all environments. We evaluate the algorithm on the DoorKey and DynamicObstacles environments within the Minigrid framework. It demonstrates adaptability and consistent improvement, particularly in the early stages, while reaching a stable performance later that is capable of outperforming other curriculum learners. In comparison to other curriculum schedules, RHEA CL has shown to yield performance improvements for the final Reinforcement learning (RL) agent at the cost of additional evaluation during training.
Organisationseinheit(en): Institut für Informationsverarbeitung
Typ: Aufsatz in Konferenzband
Publikationsdatum: 05.08.2024
Publikationsstatus: Veröffentlicht
Peer-reviewed: Ja
ASJC Scopus Sachgebiete: Artificial intelligence, Computergrafik und computergestütztes Design, Maschinelles Sehen und Mustererkennung, Mensch-Maschine-Interaktion, Software
Elektronische Version(en): https://doi.org/10.48550/arXiv.2408.06068 (Zugang: Offen)
https://doi.org/10.1109/CoG60054.2024.10645570 (Zugang: Geschlossen)

BibTeX

@inproceedings{a2d3981b66c8426aa66e811e2679164a,
title = "Online Optimization of Curriculum Learning Schedules using Evolutionary Optimization",
abstract = "We propose RHEA CL, which combines Curriculum Learning (CL) with Rolling Horizon Evolutionary Algorithms (RHEA) to automatically produce effective curricula during the training of a reinforcement learning agent. RHEA CL optimizes a population of curricula, using an evolutionary algorithm, and selects the best-performing curriculum as the starting point for the next training epoch. Performance evaluations are conducted after every curriculum step in all environments. We evaluate the algorithm on the DoorKey and DynamicObstacles environments within the Minigrid framework. It demonstrates adaptability and consistent improvement, particularly in the early stages, while reaching a stable performance later that is capable of outperforming other curriculum learners. In comparison to other curriculum schedules, RHEA CL has shown to yield performance improvements for the final Reinforcement learning (RL) agent at the cost of additional evaluation during training.",
keywords = "Curriculum Learning, Evolutionary Algorithms, Reinforcement Learning, Rolling Horizon Algorithms",
author = "Mohit Jiwatode and Leon Schlecht and Alexander Dockhorn",
note = "Publisher Copyright: {\textcopyright} 2024 IEEE.; 6th Annual IEEE Conference on Games, CoG 2024 ; Conference date: 05-08-2024 Through 08-08-2024",
year = "2024",
month = aug,
day = "5",
doi = "10.48550/arXiv.2408.06068",
language = "English",
isbn = "979-8-3503-5068-5",
series = "IEEE Conference on Computatonal Intelligence and Games, CIG",
publisher = "IEEE Computer Society",
booktitle = "Proceedings of the 2024 IEEE Conference on Games, CoG 2024",
address = "United States",
}