CIT 11-Safe robot learning

safe-control-gym+: A Unified sim2real Benchmark Pipeline for Safe Robot Learning and Control

CIT - Electrical and Computer Engineering

TUEILSY Chair of Safety, Performance and Reliability for Learning Systems (Prof. Schoellig)

Robotics, Machine Learning, System Control

Computer ScienceComputer Science/ InformaticsElectrical EngineeringMechanical Engineering

Planned project location:

Theresienstr. 90, 80333 Munich, Germany

Prof.

Angela

Schoellig

angela.schoellig@tum.de

+49 (89) 289 - 25336

Dr.

SiQi

Zhou

siqi.zhou@tum.de

+49 (89) 289 - 25334

Lukas

Brunke

lukas.brunke@tum.de

+49 (89) 289 - 25334

1. Introduction and Background: Robots are playing an increasingly important role in our lives. Examples include, but are not limited to, autonomous driving, drone delivery, and service robots. In real-world deployments, robots are required to safely operate and learn despite uncertainties such as unknown road and weather conditions or unknown room layouts. Our recent survey paper [1] summarizes the advances in safe decision-making from both the controls and reinforcement learning communities. While algorithms proposed by both communities have shown ever-increasing capabilities of guaranteeing safety under uncertainties, the algorithms are rarely compared to each other on well-defined benchmark problems. To this end, we recently released a simulation benchmark suite, called "safe-control-gym" [2], to examine the performance, data efficiency, and safety of traditional model-based control, learning-based control, and safe reinforcement learning approaches. The benchmark suite currently includes two simulation environments (cart-pole and quadrotor), 12 control/learning-based decision-making algorithms, and a set of metrics for evaluating the algorithms' data efficiency, robustness, and performance.

2. Project Goal and Expectations: The goal of this project is to expand the safe-control-gym and develop a pipeline for transferring the results to real-world experiments. The scope of the project includes adding a state estimation module to the benchmark suite, implementing additional simulation environments (e.g., autonomous cars, robot manipulators, and legged robots), identifying sim2real transfer gaps, and implementing an interface to facilitate the transfer from simulation to the real world. Possible tasks encompass a literature review on sim2real approaches and results in robotics, coding of simulation environments and control and reinforcement learning approaches in Python, and real-world robot experimentation and testing.

3. Outcome: The results of this project will be summarized and published in a robotics journal, and the code will be released as an extended component of the open-source "safe-control-gym" benchmark suite [2].

4. References:
[1] L. Brunke*, M. Greeff*, A. W. Hall*, Z. Yuan*, S. Zhou*, J. Panerati, and A. P. Schoellig. "Safe learning in robotics: From learning-based control to safe reinforcement learning." Annual Review of Control, Robotics, and Autonomous Systems 5 (2022): 411-444. Available at https://arxiv.org/pdf/2108.06266.pdf.

[2] Z. Yuan, A. W. Hall, S. Zhou, L. Brunke, M. Greeff, J. Panerati, and A. P. Schoellig. "safe-Control-Gym: A Unified Benchmark Suite for Safe Learning-Based Control and Reinforcement Learning in Robotics." IEEE Robotics and Automation Letters (2022). Available at https://arxiv.org/pdf/2109.06325.pdf. Code repository: https://github.com/utiasDSL/safe-control-gym.

40

2 years of bachelor studies completed

Robotics, control theory, dynamic systems, reinforcement learning, and programming (Python and C++)

Preferred experience in Robot Operating System (ROS), PyTorch, control or reinforcement learning algorithm implementations, and optimization