Subhransu S. Bhattacharjee, Dylan Campbell, Rahul Shome
A single egocentric image typically captures only a small portion of the floor, yet a complete metric traversability map of the surroundings would better serve applications such as indoor navigation. We introduce FlatLands, a dataset and benchmark for single-view indoor floor-map completion in metric bird's-eye view (BEV), evaluating eleven models: both deterministic and generative. FlatLands contain 270,575 observations from 17,656 scene records across six metric 3D datasets, with aligned observation, visibility, validity, and ground-truth maps and both in- and out-of-distribution protocols. We compare training-free, deterministic, ensemble, and generative methods on shared geometry-derived inputs and, without fine-tuning, on maps estimated from real egocentric RGB images. Our results show that predicting floor almost everywhere can achieve high IoU when most evaluated cells are floor, while generative sample sets outperform the best single-output model under a proper scoring rule and support safer risk-weighted planning. FlatLands establishes a rigorous testbed for evaluating both floor-map accuracy and how effectively alternative completions represent the ambiguity of unseen space.
Object interaction tasks have been a focus of advances in imitation learning. End-to-end methods, dominated by generative variants have shown leaps in performance but tend to be black boxes. Recent indications hint at demonstration lookup as the operating process underneath generative policy performance. In this paper, we revisit a few modern imitation learning benchmarks for object interactions, with the aim of composing a simple framework from first principles. We repurpose theories of manipulation modes and apply them to retrieve and transform demonstration datasets, creating a minimal strategy that preserves performance and robustness while remaining interpretable. We identify pose estimation as a key module that governs information about the object. This informs a manipulation mode-aware retrieval of a demonstration, a mode-aware transformation, a replan step that connects to the retrieval point while preserving mode constraints, returning a transformed demonstration for rollout. These compose four key steps of the proposed RE4 framework, evaluated over state-based and image-based benchmarks in simulated Push-T, Robomimic, and MimicGen as well as a real robot experiment. Robustness is measured in an adversarial study that evaluates sparse data regimes, a study with visual distractors, and the addition of an obstructing collision geometry. RE4 outperforms baselines, while notably preserving interpretability. The results overwhelmingly motivate the benefits of identifying simple interpretable building blocks to learn manipulation skills for object interactions.