About

Chuang Gan

Principal Research Staff Member

Website Google Scholar

Who they work with

Categories

Computer Vision Graph Deep Learning Multimodal Learning Neuro-Symbolic AI

I am a principal research staff member at MIT-IBM Watson AI Lab. I am also a visiting research scientist at MIT, working closely with Prof. Antonio Torralba and Prof. Josh Tenenbaum. Before that, I completed my PhD with the highest honor at Tsinghua University, where I was supervised by Prof. Andrew Chi-Chih Yao. I primarily focus on video understanding, including representation learning, neural-symbolic visual reasoning, audio-visual scene analysis and embodied intelligence. My research work have been recognized by Microsoft Fellowship, Baidu Fellowship, and media coverage from CNN, BBC, The New York Times, WIRED, Forbes, and MIT Tech Review.

Publications with the MIT-IBM Watson AI Lab

3D Concept Grounding on Neural Fields

NeurIPS

Learning Neural Acoustic Fields

NeurIPS

Weakly-Supervised Multi-Granularity Map Learning for Vision-and-Language Navigation

NeurIPS

Learning Active Camera for Multi-Object Navigation

NeurIPS

Music Gesture for Visual Sound Separation

Finding Fallen Objects Via Asynchronous Audio-Visual Integration

CVPR

Fixing Malfunctional Objects With Learned Physical Simulation and Functional Prediction

CVPR

AutoGPart: Intermediate Supervision Search for Generalizable 3D Part Segmentation

CVPR

Temporal and Object Quantification Networks

IJCAI

STAR: A Benchmark for Situated Reasoning in Real-World Videos

NeurIPS

ThreeDWorld: A Platform for Interactive Multi-Modal Physical Simulation

NeurIPS

Dynamic Visual Reasoning by Learning Differentiable Physics Models from Video and Language

NeurIPS

Memory-efficient Patch-based Inference for Tiny Deep Learning

NeurIPS

PTR: A Benchmark for Part-based Conceptual, Relational, and Physical Reasoning

NeurIPS

When does Contrastive Learning Preserve Adversarial Robustness from Pretraining to Finetuning?

NeurIPS

Network Augmentation for Tiny Deep Learning

ICLR

Curious Representation Learning for Embodied Intelligence

ICCV

FALCON: Fast Visual Concept Learning by Integrating Images, Linguistic descriptions, and Conceptual Relations

ICLR

HAT: Hardware-Aware Transformers for Efficient Natural Language Processing

Computation and Language Machine Learning

Foley Music: Learning to Generate Music from Videos

ECCV Computer Vision

DataMix: Efficient Privacy-Preserving Edge-Cloud Inference

ECCV

On Fast Adversarial Robustness Adaptation in Model-Agnostic Meta-Learning

ICLR Machine Learning

Found a Reason for me? Weakly-supervised Grounded Visual Question Answering using Capsules

CVPR Computer Vision

PlasticineLab: A Soft-Body Manipulation Benchmark with Differentiable Physics.

ICLR Machine Learning

Grounding Physical Concepts of Objects and Events Through Dynamic Visual Reasoning

ICLR Computer Vision

Learning Task Decomposition with Ordered Memory Policy Network

ICLR

RSPNet: Relative Speed Perception for Unsupervised Video Representation Learning.

AAAI Computer Vision

Augmenting Policy Learning with Routines Discovered from a Demonstration

AAAI Machine Learning

MVFNet: Multi-View Fusion Network for Efficient Video Recognition

AAAI Computer Vision

AGENT: A Benchmark for Core Psychological Reasoning

ICML Artificial Intelligence

Interactive Fiction Game Playing as Multi-Paragraph Reading Comprehension with Reinforcement Learning

EMNLP Computation and Language

Adversarial Option-Aware Hierarchical Imitation Learning

ICML Machine Learning

Tiny Transfer Learning: Towards Memory-Efficient On-Device Learning

Efficient AI

MCUNet: Tiny Deep Learning on IoT Devices

Deep Learning Efficient AI

Dense Regression Network for Video Grounding

CVPR

CLEVRER: The first video dataset for neuro-symbolic reasoning

Neuro-Symbolic AI Computer Vision

Once for All: Train One Network and Specialize it for Efficient Deployment

Efficient AI ICLR

Deep Audio Priors Emerge From Harmonic Convolutional Networks

Audio Processing Deep Learning

Self-supervised Moving Vehicle Tracking with Stereo Sound

Multimodal Learning Computer Vision

The sound of motions

Computer Vision

TSM: Temporal Shift Module for Efficient Video Understanding

Efficient AI Computer Vision

Graph Convolutional Networks for Temporal Action Localization

Graph Deep Learning Computer Vision

Imitation learning from observations

Deep Learning

Visual Concept-Metaconcept Learning

Neuro-Symbolic AI

Cross-channel Communication Networks

Deep Learning Computer Vision

Watch, Reason and Code: Learning to Represent Videos Using Program

Computer Vision

Defensive Quantization: When Efficiency Meets Robustness

Efficient AI Robustness

Beyond RNNs: Positional Self-Attention with Co-Attention for Video Question Answering

Computer Vision Time Series

The Neuro-Symbolic Concept Learner: Interpreting Scenes, Words, and Sentences From Natural Supervision

Neuro-Symbolic AI Computer Vision

Weakly Supervised Dense Event Captioning in Videos

Computer Vision

StNet: Local and Global Spatial-Temporal Modeling for Action Recognition

Computer Vision Time Series

Neural-Symbolic VQA: Disentangling Reasoning from Vision and Language Understanding

Neuro-Symbolic AI Computer Vision

Controllable Image-to-Video Translation: A Case Study on Facial Expression Generation

Generative Models

Geometry Guided Convolutional Neural Networks for Self-Supervised Video Representation Learning

CVPR

Sparse, Smart Contours to Represent and Edit Images

CVPR

Attention Clusters: Purely Attention Based Local Feature Integration for Video Classification

CVPR

The Sound of Pixels

Multimodal Learning Computer Vision