Collective Model Fusion for Multiple Black-Box Experts



  • Minh Hoang
  • Nghia Hoang
  • Bryan Kian Hsiang Low
  • Carleton Kingsford

Published on


Model fusion is a fundamental problem in collective machine learning (ML) where independent experts with heterogeneous learning architectures are required to combine expertise to improve predictive performance. This is particularly challenging in information-sensitive domains where experts do not have access to each other’s internal architecture and local data. This paper presents the first collective model fusion framework for multiple experts with heterogeneous black-box architectures. The proposed method will enable this by addressing the key issues of how black-box experts interact to understand the predictive behaviors of one another; how these understandings can be represented and shared efficiently among themselves; and how the shared understandings can be combined to generate high-quality consensus prediction. The performance of the resulting framework is analyzed theoretically and demonstrated empirically on several datasets.

Please cite our work using the BibTeX below.

  title = 	 {Collective Model Fusion for Multiple Black-Box Experts},
  author =       {Hoang, Minh and Hoang, Nghia and Low, Bryan Kian Hsiang and Kingsford, Carleton},
  booktitle = 	 {Proceedings of the 36th International Conference on Machine Learning},
  pages = 	 {2742--2750},
  year = 	 {2019},
  editor = 	 {Chaudhuri, Kamalika and Salakhutdinov, Ruslan},
  volume = 	 {97},
  series = 	 {Proceedings of Machine Learning Research},
  month = 	 {09--15 Jun},
  publisher =    {PMLR},
  pdf = 	 {},
  url = 	 {}
Close Modal