SNAKE: Shape-aware Neural 3D Keypoint Field



  • Chuang Gan
  • Chengliang Zhong
  • Peixing You
  • Xiaoxue Chen
  • Hao Zhao
  • Fuchun Sun
  • Guyue Zhou
  • Xiaodong Mu
  • Wenbing Huang

Published on




Detecting 3D keypoints from point clouds is important for shape reconstruction, while this work investigates the dual question: can shape reconstruction benefit 3D keypoint detection? Existing methods either seek salient features according to statistics of different orders or learn to predict keypoints that are invariant to transformation. Nevertheless, the idea of incorporating shape reconstruction into 3D keypoint detection is under-explored. We argue that this is restricted by former problem formulations. To this end, a novel unsupervised paradigm named SNAKE is proposed, which is short for shape-aware neural 3D keypoint field. Similar to recent coordinate-based radiance or distance field, our network takes 3D coordinates as inputs and predicts implicit shape indicators and keypoint saliency simultaneously, thus naturally entangling 3D keypoint detection and shape reconstruction. We achieve superior performance on various public benchmarks, including standalone object datasets ModelNet40, KeypointNet, SMPL meshes and scene-level datasets 3DMatch and Redwood. Intrinsic shape awareness brings several advantages as follows. (1) SNAKE generates 3D keypoints consistent with human semantic annotation, even without such supervision. (2) SNAKE outperforms counterparts in terms of repeatability, especially when the input point clouds are down-sampled. (3) the generated keypoints allow accurate geometric registration, notably in a zero-shot setting. Codes are available at

Please cite our work using the BibTeX below.

title={{SNAKE}: Shape-aware Neural 3D Keypoint Field},
author={Chengliang Zhong and Peixing You and Xiaoxue Chen and Hao Zhao and Fuchun Sun and Guyue Zhou and Xiaodong Mu and Chuang Gan and Wenbing Huang},
booktitle={Advances in Neural Information Processing Systems},
editor={Alice H. Oh and Alekh Agarwal and Danielle Belgrave and Kyunghyun Cho},
Close Modal