Noether networks: meta-learning useful conserved quantities
Authors
Authors
- Ferran Alet
- Dylan Doblar
- Allan Zhou
- Joshua Tenenbaum
- Kenji Kawaguchi
- Chelsea Finn
Authors
- Ferran Alet
- Dylan Doblar
- Allan Zhou
- Joshua Tenenbaum
- Kenji Kawaguchi
- Chelsea Finn
Published on
12/14/2021
Categories
Progress in machine learning (ML) stems from a combination of data availability, computational resources, and an appropriate encoding of inductive biases. Useful biases often exploit symmetries in the prediction problem, such as convolutional networks relying on translation equivariance. Automatically discovering these useful symmetries holds the potential to greatly improve the performance of ML systems, but still remains a challenge. In this work, we focus on sequential prediction problems and take inspiration from Noether’s theorem to reduce the problem of finding inductive biases to meta-learning useful conserved quantities. We propose Noether Networks: a new type of architecture where a meta-learned conservation loss is optimized inside the prediction function. We show, theoretically and experimentally, that Noether Networks improve prediction quality, providing a general framework for discovering inductive biases in sequential problems.
Please cite our work using the BibTeX below.
@inproceedings{
alet2021noether,
title={Noether Networks: meta-learning useful conserved quantities},
author={Ferran Alet and Dylan Doblar and Allan Zhou and Joshua B. Tenenbaum and Kenji Kawaguchi and Chelsea Finn},
booktitle={Advances in Neural Information Processing Systems},
editor={A. Beygelzimer and Y. Dauphin and P. Liang and J. Wortman Vaughan},
year={2021},
url={https://openreview.net/forum?id=_NOwVKCmSo}
}