Online Optimization of Curriculum Learning Schedules using Evolutionary Optimization

authored by: Mohit Jiwatode, Leon Schlecht, Alexander Dockhorn
Abstract: We propose RHEA CL, which combines Curriculum Learning (CL) with Rolling Horizon Evolutionary Algorithms (RHEA) to automatically produce effective curricula during the training of a reinforcement learning agent. RHEA CL optimizes a population of curricula, using an evolutionary algorithm, and selects the best-performing curriculum as the starting point for the next training epoch. Performance evaluations are conducted after every curriculum step in all environments. We evaluate the algorithm on the DoorKey and DynamicObstacles environments within the Minigrid framework. It demonstrates adaptability and consistent improvement, particularly in the early stages, while reaching a stable performance later that is capable of outperforming other curriculum learners. In comparison to other curriculum schedules, RHEA CL has shown to yield performance improvements for the final Reinforcement learning (RL) agent at the cost of additional evaluation during training.
Organisation(s): Institute of Information Processing
Type: Conference contribution
Publication date: 05.08.2024
Publication status: Published
Peer reviewed: Yes
ASJC Scopus subject areas: Artificial Intelligence, Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition, Human-Computer Interaction, Software
Electronic version(s): https://doi.org/10.48550/arXiv.2408.06068 (Access: Open)
https://doi.org/10.1109/CoG60054.2024.10645570 (Access: Closed)

BibTeX

@inproceedings{a2d3981b66c8426aa66e811e2679164a,
title = "Online Optimization of Curriculum Learning Schedules using Evolutionary Optimization",
abstract = "We propose RHEA CL, which combines Curriculum Learning (CL) with Rolling Horizon Evolutionary Algorithms (RHEA) to automatically produce effective curricula during the training of a reinforcement learning agent. RHEA CL optimizes a population of curricula, using an evolutionary algorithm, and selects the best-performing curriculum as the starting point for the next training epoch. Performance evaluations are conducted after every curriculum step in all environments. We evaluate the algorithm on the DoorKey and DynamicObstacles environments within the Minigrid framework. It demonstrates adaptability and consistent improvement, particularly in the early stages, while reaching a stable performance later that is capable of outperforming other curriculum learners. In comparison to other curriculum schedules, RHEA CL has shown to yield performance improvements for the final Reinforcement learning (RL) agent at the cost of additional evaluation during training.",
keywords = "Curriculum Learning, Evolutionary Algorithms, Reinforcement Learning, Rolling Horizon Algorithms",
author = "Mohit Jiwatode and Leon Schlecht and Alexander Dockhorn",
note = "Publisher Copyright: {\textcopyright} 2024 IEEE.; 6th Annual IEEE Conference on Games, CoG 2024 ; Conference date: 05-08-2024 Through 08-08-2024",
year = "2024",
month = aug,
day = "5",
doi = "10.48550/arXiv.2408.06068",
language = "English",
isbn = "979-8-3503-5068-5",
series = "IEEE Conference on Computatonal Intelligence and Games, CIG",
publisher = "IEEE Computer Society",
booktitle = "Proceedings of the 2024 IEEE Conference on Games, CoG 2024",
address = "United States",
}