Research

Adversarial Option-Aware Hierarchical Imitation Learning

ICML

Authors

  • Mingxuan Jing
  • Wenbing Huang
  • Xiaojian Ma
  • Fuchan Sun
  • Tao Kong
  • Chuang Gan
  • Lei Li

Published on

06/10/2021

It has been a challenge to learning skills for an agent from long-horizon unannotated demonstrations. Existing approaches like Hierarchical Imitation Learning(HIL) are prone to compounding errors or suboptimal solutions. In this paper, we propose Option-GAIL, a novel method to learn skills at long horizon. The key idea of Option-GAIL is modeling the task hierarchy by options and train the policy via generative adversarial optimization. In particular, we propose an Expectation-Maximization(EM)-style algorithm: an E-step that samples the options of expert conditioned on the current learned policy, and an M-step that updates the low- and high-level policies of agent simultaneously to minimize the newly proposed option-occupancy measurement between the expert and the agent. We theoretically prove the convergence of the proposed algorithm. Experiments show that Option-GAIL outperforms other counterparts consistently across a variety of tasks.

This paper has been published at ICML 2021

Please cite our work using the BibTeX below.

@misc{jing2021adversarial,
      title={Adversarial Option-Aware Hierarchical Imitation Learning}, 
      author={Mingxuan Jing and Wenbing Huang and Fuchun Sun and Xiaojian Ma and Tao Kong and Chuang Gan and Lei Li},
      year={2021},
      eprint={2106.05530},
      archivePrefix={arXiv},
      primaryClass={cs.LG}
}
Close Modal