Trustworthy Interaction-Aware Decision Making and Planning

Trustworthy Interaction-Aware Decision Making and Planning
For autonomous systems, having the ability to predict the evolution of their surroundings is essential for safe, reliable, and efficient operation. Prediction is especially important when the autonomous system must interact and “negotiate” with humans, whether it be in settings that are cooperative, adversarial, or anywhere in between. This line of research involves quantifying the relative likelihoods of multiple, possibly highly distinct futures for interactive scenarios, planning strategies such that the autonomous agent is cognizant of how the human may respond, developing models that are offer transparency into the autonomous agent’s decision making process, and designing safe human-in-the-loop testing methodologies to validate our models and planning algorithms.

Keywords: Human-Robot Interaction, Deep Learning, Motion Planning

Students: Edward Schmerling, Karen Leung, Boris Ivanovic, Sandeep Chinchali

Related Works

Conference Articles

  1. B. Ivanovic, E. Schmerling, K. Leung, and M. Pavone, “Generative Modeling of Multimodal Multi-Human Behavior,” in IEEE/RSJ Int. Conf. on Intelligent Robots & Systems, Madrid, Spain, 2018.

    Abstract: This work presents a methodology for modeling and predicting human behavior in settings with N humans interacting in highly multimodal scenarios (i.e. where there are many possible highly-distinct futures). A motivating example includes robots interacting with humans in crowded environments, such as self-driving cars operating alongside human-driven vehicles or human-robot collaborative bin packing in a warehouse. Our approach to model human behavior in such uncertain environments is to model humans in the scene as nodes in a graphical model, with edges encoding relationships between them. For each human, we learn a multimodal probability distribution over future actions from a dataset of multi-human interactions. Learning such distributions is made possible by recent advances in the theory of conditional variational autoencoders and deep learning approximations of probabilistic graphical models. Specifically, we learn action distributions conditioned on interaction history, neighboring human behavior, and candidate future agent behavior in order to take into account response dynamics. We demonstrate the performance of such a modeling approach in modeling basketball player trajectories, a highly multimodal, multi-human scenario which serves as a proxy for many robotic applications.

      author = {Ivanovic, B. and Schmerling, E. and Leung, K. and Pavone, M.},
      title = {Generative Modeling of Multimodal Multi-Human Behavior},
      booktitle = {{IEEE/RSJ Int. Conf. on Intelligent Robots \& Systems}},
      year = {2018},
      address = {Madrid, Spain},
      month = oct,
      url = {},
      owner = {borisi},
      timestamp = {2018-10-14}
  2. K. Leung, E. Schmerling, M. Chen, J. Talbot, J. C. Gerdes, and M. Pavone, “On Infusing Reachability-Based Safety Assurance within Probabilistic Planning Frameworks for Human-Robot Vehicle Interactions,” in Int. Symp. on Experimental Robotics, 2018.


      author = {Leung, K. and Schmerling, E. and Chen, M. and Talbot, J. and Gerdes, J. C. and Pavone, M.},
      title = {On Infusing Reachability-Based Safety Assurance within Probabilistic Planning Frameworks for Human-Robot Vehicle Interactions},
      booktitle = {{Int. Symp. on Experimental Robotics}},
      year = {2018},
      url = {../wp-content/papercite-data/pdf/Leung.Schmerling.Chen.ea.ISER18.pdf},
      owner = {mochen72},
      timestamp = {2018-10-13}