Safe Motion Planning and Control Under Uncertainty
Robots usually need to leverage a robot-environment interaction model to properly plan their motions to avoid hazardous states and achieve desired behaviors. However, models (learned and hand-crafted) generally cannot exactly capture the real-world interactions and often involves high levels of uncertainties. This research aims at directly integrating the uncertainty estimation into the planning stage for generating robust and safe closed-loop motion plans (or policies).
Value Function Approximation in Continuous MDPs
At the core of this problem involves solving a continuous state and action Markov Decision Process (MDP). We propose a series of work that approximates and solves the continuous value function [RSS 2021] and integrates the proposed solver into a system that connects from perception to planning. This system has been heavily tested in real-world unstructured environments [IJRR 2024].
Causal Inference for Motion Model Learning
The above work computes the value function and policy using probabilistic motion models realized by a Gaussian distribution, with parameters learned from historical offline driving data. While effective, learning from offline data poses unique challenges due to confounding variables that can bias the robot's understanding of action-state relationship. In a follow-up work published in ICRA 2023, we address this limitation using causal inference to learn more sophisticated motion models.
Safe Value Function
This work directly bakes the safety and task constraints into the boundary conditions of the second-order Hamilton-Jacobi Belllman (HJB) Equation. As a result, the solution to this HJB equation is a safe value function that sharpens the distinction between safe and unsafe states and can guide the policy to achieve goals safely. By treating safety and task as boundary conditions, we move from complex, dense rewards to more straightforward, sparse constraints. Additionally, we also propose a hybrid model that combines a mesh-based function approximator for accurately computing boundary conditions with a meshless method, such as neural networks or kernel functions, to enhance computational efficiency. This work is accepted by the International Journal of Robotics Research (IJRR) [Paper].