ALP: Action-Aware Embodied Learning for Perception

Abstract

Current methods in training and benchmarking vision models exhibit an over-reliance on passive, curated datasets. Although models trained on these datasets have shown strong performance in a wide variety of tasks such as classification, detection, and segmentation, they fundamentally are unable to generalize to an ever-evolving world due to constant out-of-distribution shifts of input data. Therefore, instead of training on fixed datasets, can we approach learning in a more human-centric and adaptive manner? In this paper, we introduce Action-Aware Embodied Learning for Perception (ALP), an embodied learning framework that incorporates action information into representation learning through a combination of optimizing a reinforcement learning policy and an inverse dynamics prediction objective. Our method actively explores in complex 3D environments to both learn generalizable task-agnostic visual representations as well as collect downstream training data. We show that ALP outperforms existing baselines in several downstream perception tasks. In addition, we show that by training on actively collected data more relevant to the environment and task, our method generalizes more robustly to downstream tasks compared to models pre-trained on fixed datasets such as ImageNet.

Method Overview

We present ALP, an embodied learning framework based on active exploration for visual representation learning and downstream perception tasks. Our framework consists of two stages.

In Stage 1, we allow the agent to actively explore in visual environments from intrinsic motivation to discover diverse observations. We propose a coupled approach to incorporate action information and learn a set of shared visual representations that jointly optimizes a reinforcement learning objective as indirect action signals and an inverse dynamics prediction loss as explicit action signals.

In Stage 2, we label a small subset of collected samples from active exploration to obtain better training data for downstream perception tasks. We initialize from the pretrained representation in Stage 1 to improve the performance of the perception model.

Visualizations

We combine ALP with baseline exploration methods and visualize learned policies in environment maps using the Gibson dataset of the Habitat simulator. RND-ALP and CRL-ALP show wider coverage of environments and longer trajectory movements than exploration baselines.

RND (left) and RND-ALP (right)

CRL (left) and CRL-ALP (right)

BibTeX

@article{liang2023alp,
      title={ALP: Action-Aware Embodied Learning for Perception},
      author={Liang, Xinran and Han, Anthony and Yan, Wilson and Raghunathan, Aditi and Abbeel, Pieter},
      journal={arXiv preprint arXiv:2306.10190},
      year={2023}
    }