Tutorials

The first part of the tutorial will focus on two recent topics in Markov decision processes (MDPs). We will first discuss the construction of data-driven MDPs that combine the tasks of estimating the system's behavior and selecting a policy that performs well out of sample. We will then discuss the exploitation of problem structure to scale to large problem sizes via weakly coupled MDPs that combine a potentially large number of MDPs via a small number of linking constraints. The second part of the tutorial will provide an introduction to distributionally robust optimization with Wasserstein ambiguity sets. In particular, we will address robust Linear-Quadratic-Gaussian (LQG) control problems, where the noise distributions are unknown and belong to Wasserstein ambiguity sets centered at nominal Gaussian distributions. We will derive structural properties of the optimal primal and dual solutions and develop an efficient Frank-Wolfe algorithm to solve robust LQG problems.

Requirements are inherent to systems. This is because systems are always defined as tradeoffs between multiple competing specifications such as, e.g., stability, safety, robustness, and efficiency. Learning to satisfy requirements is, however, antithetical to the standard ML practice of minimizing individual losses. To close this gap, we develop the theory and practice of constrained learning. This tutorial provides an overview theoretical and algorithmic developments that show when and how it is possible to learn with constraints. We describe how theoretical guarantees and viable learning algorithms are hindered by lack of convexity of the resulting optimization problems and explain how a near-duality theory circumvents this challenge. Throughout the tutorial we explore supervised learning, robust learning, and reinforcement learning with constraints. We put emphasis on showcasing the breadth of potential applications by discussing fairness, robust classification, federated learning, learning under invariance, and safe reinforcement learning. Attendees will be prepared to start conducting research in this emerging research frontier.

Further information here: https://luizchamon.com/l4dc/

Autonomous systems impose numerous challenging requirements on the underlying control algorithms. In particular, while ensuring safety is often critical in the design, the complexity and uncertainty of the dynamics and the multifaceted nature of the control objective make it hard to achieve. Classical control methods are often unsuited to apply in such unstructured tasks with potentially conflicting safety specifications. While impressive progress has been achieved through learning-based policies, missing safety certificates often still prohibit the widespread application of these techniques outside of research environments. This tutorial presents safety filter techniques, providing general and modular approaches to augment any control policy with safety guarantees in the form of constraint satisfaction. The first part provides a comprehensive overview of three main classes of safety filters and introduces invariance-based methods relying on Hamilton-Jacobi reachability, control barrier functions, and predictive control techniques. The second part then focuses on extensions towards improved practical applicability including data-driven enhancements and reduced-order models. The tutorial aims at providing a comprehensive introduction for researchers of different backgrounds. It provides the main concepts, theoretical guarantees and addresses practical aspects, accompanied by application examples.