Junhong Xu
About Me
I am interested in developing methods that enable physical and virtual agents to safely explore their environments through interaction and enhance their understanding of the world. By continuously improving their world model, these agents can more effectively assist humans in completing tasks reliably and safely in uncertain, human-centered environments.
Currently, I am working on safe reinforcement learning algorithms at Nuro, addressing challenging, safety-critical decision-making and planning problems in the realm of self-driving vehicles. I obtained my PhD from Indiana University, where my research centered on safe decision-making under uncertainty for real-world robotic systems. My work spans over multiple disciplines, including constrained stochastic optimal control, reinforcement learning, causal inference, game theory, and control as inference.
Prior to my Ph.D., I also worked on reducing DAgger’s manual labeling in imitation learning and multi-task imitation learning for mobile robot navigation in indoor environments.
Selected Research
Safe Value Function This work directly bakes the safety and task constraints into the boundary conditions of the second-order Hamilton-Jacobi Belllman (HJB) Equation. As a result, the solution to this HJB equation is a safe value function that sharpens the distinction between safe and unsafe states and can guide the policy to achieve goals safely. By treating safety and task as boundary conditions, we move from complex, dense rewards to more straightforward, sparse constraints. Additionally, we also propose a hybrid model that combines a mesh-based function approximator for accurately computing boundary conditions with a meshless method, such as neural networks or kernel functions, to enhance computational efficiency. This work is accepted by the International Journal of Robotics Research (IJRR) [Paper]. |
Causal Inference for Dynamics Learning Learning a causal model of how an intervention (action) changes the state of a system is crucial for decision-making. When we only have observational (offline) data, learning such a model is challenging due to the presence of confounders - variables that can bias the estimated effects of actions on state transitions. In this work, we leverage the potential outcome causal inference framework to control for the confounding variables and learn a robot motion model for downstream decision-making. This work is accepted by ICRA 2023 [Paper]. |
Control as Inference for Decision-Making Among Strategic Agents Our paper "Decision-Making Among Bounded Rational Agents" on explicit modeling of computational limits in multi-agent motion planning using control as inference (a.k.a., information-theoretic bounded rationality) is accepted by DARS 2022. |
Motion Planning and Control in Extreme Off-Road Environments Planning and control of an autonomous vehicle in extreme off-road environments pose many challenges due to the presence of highly uncertain interactions between the robot and the environment. Our team is 1 of 4 teams selected (others are MIT, CMU and University of Washington) by the Army Research Laboratory (ARL) to enhance unmanned ground vehicle's navigation and control capability in off-road environments. We have successfully deployed our stochastic motion planner based on our previous works to various off-road environments. This work has been published to the International Journal of Robotics Research (IJRR) [Paper]. |