Sheila McIlraith

University of Toronto

Talk title: Building Taskable Reinforcement Learning Agents via Formal Languages and Automata.

Abstract

Reinforcement Learning (RL) is proving to be a powerful technique for building sequential decision-making systems in cases where the complexity of the underlying environment is difficult to model. Two challenges that face RL are reward specification and sample efficiency. Specification of a reward function---typically, a mapping from state to numeric value---can be challenging, particularly when reward-worthy behaviour is complex and temporally extended. Further, when reward is sparse, it can require millions of exploratory episodes for an RL agent to converge to a reasonable quality policy. In this talk I'll show how formal languages and automata can be used to represent complex non-Markovian reward functions. I'll present the notion of a Reward Machine, an automata-based structure that provides a normal form representation for reward functions, exposing function structure in a manner that can be exploited by tailored learning algorithms to learn more efficiently. Finally, I'll show how these machines can be generated via symbolic planning---to build taskable goal-directed agents---or learned from data, solving (deep) RL problems that otherwise could not be solved.

Bio

Bio: Dr. Sheila McIlraith is a Professor in the Department of Computer Science, University of Toronto, Canada CIFAR AI Chair (Vector Institute for Artificial Intelligence), and Associate Director and Research Lead of the Schwartz Reisman Institute for Technology and Society. Prior to joining U of T, McIlraith spent six years as a Research Scientist at Stanford University, and one year at Xerox PARC.

McIlraith is the author of over 100 scholarly publications in the area of knowledge representation, automated reasoning and machine learning. Her work focuses on AI sequential decision making broadly construed, through the lens of human-compatible AI. McIlraith is a fellow of the ACM, a fellow of the Association for the Advancement of Artificial Intelligence (AAAI) and a past President of KR Inc., the international scientific foundation concerned with fostering research and communication on knowledge representation and reasoning. She is currently serving on the Standing Committee of the Stanford One Hundred Year Study on Artificial Intelligence (AI100).

McIlraith is an associate editor of the Journal of Artificial Intelligence Research (JAIR), a past associate editor of the journal Artificial Intelligence (AIJ), and a past board member of Artificial Intelligence Magazine. In 2018, McIlraith served as program co-chair of the 32nd AAAI Conference on Artificial Intelligence (AAAI-18). She also served as program co-chair of the International Conference on Principles of Knowledge Representation and Reasoning (KR2012), and the International Semantic Web Conference (ISWC2004). McIlraith's early work on Semantic Web Services has had notable impact. In 2011 she and her co-authors were honoured with the SWSA 10-year Award, a test of time award recognizing the highest impact paper from the International Semantic Web Conference, 10 years prior, and in 2022 McIlraith and co-authors were honoured with the 2022 ICAPS Influential Paper Award, recognizing a significant and influential paper published 10 years prior at the International Conference on Automated Planning and Scheduling.

Matthew Gombolay

Georgia Institute of Technology

Title: Democratizing Reinforcement Learning: Interpretable and Interactive – not Tabula Rasa

Abstract

Reinforcement learning (RL) is a powerful tool in the hands of expert researchers, achieving impressive success from the virtual world of Gran Turismo Sport to the physical world of Drone Racing. Many researchers are on a quest for true RL tabula rasa – learning without the need for inductive biases given by human intelligence. While an interesting academic exercise, the true power of RL lies in empowering non-expert end-users to intuitively shape the behavior of robotic systems. To democratize RL, end-users will need the ability to understand and interact with collaborative RL systems. In this talk, I will share our making RL accessible with interpretable RL and neurosymbolic methods along with human-in-the-loop RL both in RL from human feedback (RLHF) and human-guided exploration. At the end of my talk, I will lay out my vision for the future of democratizing RL moving out of the laboratory and into the hands of end-users.

Bio

Dr. Matthew Gombolay is an Assistant Professor of Interactive Computing at the Georgia Institute of Technology and was named the Anne and Alan Taetle Early-career Assistant Professor in 2018. Dr. Gombolay is the director of the Cognitive Optimization and Relational (CORE) Robotics Lab, which seeks to place the power of robots in the hands of everyone by developing new computational methods and human factors insights that enable robots to learn from interaction with diverse, non-expert end-users to perform assistive tasks and coordinate in human-robot teams in applications from healthcare to manufacturing.

Dr. Gombolay’s laboratory has received multiple best paper awards and nominations, including at the 2022 ACM/IEEE International Conference on Human-Robot Interaction, the 2020 Conference on Robot Learning (CoRL), and the 2020 American Controls Conference (ACC). Dr. Gombolay was awarded a NASA Early Career Fellowship and was selected as a DARPA Riser. Dr. Gombolay is an Associate Editor of Autonomous Robots and the ACM Transactions on Human-Robot Interaction.

Dana Kulić

Monash University

Talk title: Reinforcement Learning for HRI: the role of simulations

Abstract

TBD

Bio

Prof. Dana Kulić conducts research in robotics, learning and human-robot interaction (HRI). Dana Kulić received the combined B. A. Sc. and M. Eng. degree in electro-mechanical engineering, and the Ph. D. degree in mechanical engineering from the University of British Columbia, Canada, in 1998 and 2005, respectively. From 2006 to 2009, Dr. Kulić was a JSPS Post-doctoral Fellow and a Project Assistant Professor at the Nakamura-Yamane Laboratory at the University of Tokyo, Japan. From 2009 - 2018, Dr. Kulić led the Adaptive System Laboratory at the University of Waterloo, Canada, conducting research in human robot interaction, human motion analysis for rehabilitation and humanoid robotics. Since 2019, Dr. Kulić is a professor and director of Monash Robotics at Monash University, Australia. In 2020, Dr. Kulić was awarded the ARC Future Fellowship.

Johannes Stork

Örebro University

Talk title: Towards reinforcement learning with priorities

Abstract

Reinforcement learning for complex tasks is challenging because we must design a scalar-valued reward function that induces the desired behavior and we must expensively learn each new task from scratch. Instead, it might be more suitable to model complex tasks as several simpler subtasks and more efficient to reuse or adapt subtask solutions that were learned previously. This talk is about how we can take inspiration for multi-objective control methods to develop reinforcement learning algorithms that can combine several RL problems to solve more complex RL problems. There are several open questions: How do we resolve conflicts between the subtasks? Is there an equivalent scalar reward function? Does this lead to a practical and efficient algorithm?

Bio

Johannes A. Stork is Associate Senior Lecturer and WASP Professor of machine learning at Örebro University and works at the Center for Applied Autonomous Sensor Systems (AASS). He joined the Autonomous Mobile Manipulation Lab (AMM) as a founding member. His main research interests are in machine learning and autonomous intelligent systems. Starting spring of 2020 he is WASP Professor as part of the Wallenberg AI, Autonomous Systems and Software Program (WASP), Sweden’s largest ever individual research program and major national initiative for strategic basic research, education, and faculty recruitment.

From 2018 to 2020, he held a position as Associate Senior Lecturer as part of the vice chancellor's strategic initiative for faculty development. Before joining AASS in 2018, he was a post-doctoral researcher and before that PhD. student in Computer Science (Computer Vision and Robotics) at the Computer Vision and Active Perception Lab (CVAP) at the Royal Institute of Technology (KTH) in Stockholm. He did his post-graduate research as a member of Professor Danica Kragic’s research group where his co-supervisors were Dr. Carl Henrik Ek and Dr. Yasemin Bekiroglu. Before that, he spent several years of his undergraduate studies at the University of Freiburg, Germany, as a student research assistant at the Social Robotics Lab of Professor Kai O. Arras. He holds a MSc. and a BSc. degree in Computer Science with concentration in Artificial Intelligence and Robotics from Freiburg University.