From Optimal Control to Reinforcement Learning”, Konstantinos Hatzyligeroudis, Assistant Professor, Electrical and Computer Engineering Department – Τμήμα Μηχανικών Η/Υ και Πληροφορικής

Τίτλος:  Agile Robot Behaviors: From Optimal Control to Reinforcement Learning

Ομιλητής: Κωνσταντίνος Χατζηλυγερούδης, Επίκουρος Καθηγητής, Τμήμα Ηλεκτρολόγων Μηχανικών και Τεχνολογίας Υπολογιστών, Πανεπιστήμιο Πατρών

Ημερομηνία-χώρος: Παρασκευή 30 Mαΐου,  3:15-5μμ, ΤΜΗΥΠ, αμφιθέατρο Γ

Abstract: Generating,learning and executing agile robot behaviors remains one of the most ambitious challenges in robotics. In this talk, we make the case that learning primitive behaviors directly from unstructured video (e.g. YouTube videos) is an effective way of gaining insights on agility from humans and animals. We first discuss how to generate non-agile robot behaviors and dynamically feasible trajectories via optimal control. Next, we dig into the core challenge of agility: rapid reorientation and forceful maneuvers require precise contact and ground force reaction reasoning, and showcase real-world applications from the animation industry. We, then, present our automatic contact-generation framework that discovers support sequences and frictional interactions for acrobatic tasks, then demonstrate examples of agile movement generation. Despite accurate models, purely model-based pipelines struggle with real-world imperfections, and real-world deployment requires fast reactions. To bridge this gap, we integrate a learning-based correction module that adapts online for unmodeled dynamics, and how it can be integrated in several model-based control methods (e.g. real-time and MPC controllers). We then show that optimal control and reinforcement learning share a common mathematical backbone—both solve sequential decision making problems under different assumptions—and illustrate how insights from one paradigm translate to the other. Finally, we tackle sim-to-real transfer on a quadruped platform: by combining domain randomization, curriculum learning, and intelligent input representations, we are able to transfer robot controllers zero-shot on real hardware.

About the speaker: Konstantinos Chatzilygeroudis received the Integrated Master degree (Engineering Diploma) in computer science and engineering from the University of Patras, Patras, Greece, in 2014, and the Ph.D. degree in robotics and machine learning from Inria Nancy-Grand Est, France and the University of Lorraine, Nancy, France in 2018. From 2018 to 2020 he was a Postdoctoral Fellow with the LASA Team with the Swiss Federal Institute of Technology Lausanne (EPFL), Lausanne, Switzerland. He is a recipient of an H.F.R.I. Grant for Post-doctoral Fellows (2022-2024): he is the Principal Investigator of the project “Novel Optimization Methods for Autonomous Skill Learning in Robotics” that is being implemented within the Department of Mathematics, University of Patras, Greece. He has also taught and is still teaching several undergraduate and post-graduate courses on Artificial Intelligence, Computer Science and Robotics at University of Patras, Greece. He has also co-supervised several undergraduate and master theses. He is currently serving as an Associate Co-Chair of the IEEE Technical Committee on Model-based Optimization for Robotics, while he has served as an Associate Editor for several years at the International Conference on Intelligent Robotics (IROS) and the International Conference on Humanoid Robots (Humanoids), actively participated in the organization committee (as a Chair responsible for the virtual part of the conference) of the International Conference on Robot Learning (CoRL) 2021, and serving as an Area Chair for NeurIPS (2024-). His work has been published in top-tier journals and conferences in the fields artificial intelligence, machine learning and robotics, and he has received a Best Paper Award at GECCO 2022. He has also actively collaborated with industrial partners: he was the Leader of the R&D Computer Vision Team at Metargus, a pre-seed funded startup (based in Patras, Greece), and he was the Lead Robotics Engineer at Ragdoll Dynamics (company based in London, UK). His research interests include the area of artificial intelligence and focus on reinforcement learning, fast robot adaptation, evolutionary computation and autonomous skill discovery.