Is this you? Claim this profile to correct it, add a bio and choose the work people see first.
Claim this profileAcademic lineage
Possible advisorsa guess from early papers, not confirmed
- Cordelia SchmidSuggested from co-authorship
Is this you? Claim this profile to confirm or dismiss it.
Works52 from public data
- Zero-1-to-3: Zero-shot One Image to 3D Object1,901
Zero-1-to-3, a framework for changing the camera viewpoint of an object given just a single RGB image, is introduced and significantly outperforms state-of-the-art single-view 3D reconstruction and novel view synthesis models by leveraging Internet-scale pre-training.
- Learning Video Object Segmentation with Visual Memory345
A novel two-stream neural network with an explicit memory module to achieve the task of segmenting moving objects in unconstrained videos and provides an extensive ablative analysis to investigate the influence of each component in the proposed framework.
- Learning Motion Patterns in Videos279
The core of this approach is a fully convolutional network, which is learned entirely from synthetic video sequences, and their ground-truth optical flow and motion segmentation, which outperforms the top method on the recently released DAVIS benchmark dataset by 5.6%.
- Learning to Track with Object Permanence258
This work introduces an end-to-end trainable approach for joint object detection and tracking that outperforms the state of the art on KITTI, and MOT17 datasets thanks to its robustness to occlusions.
- TAO: A Large-Scale Benchmark for Tracking Any Object246
It is shown that existing single- and multi-object trackers struggle when applied to this scenario in the wild, and that detection-based, multi- object trackers are in fact competitive with user-initialized ones.
- Learning Compositional Representations for Few-Shot Recognition135
This work introduces a simple regularization technique that allows the learned representation to be decomposable into parts, and demonstrates the value of compositional representations on three datasets and shows that they require fewer examples to learn classifiers for novel categories.
- Dreamitate: Real-World Visuomotor Policy Learning via Video Generation126
A visuomotor policy learning framework that fine-tunes a video diffusion model on human demonstrations of a given task and demonstrates that harnessing internet-scale generative models allows the learned policy to achieve a significantly higher degree of generalization than existing behavior cloning approaches.
- A Structured Model for Action Detection118
A standard I3D network is augmented with a tracking module to aggregate long-term motion patterns, and a graph convolutional network is used to reason about interactions between actors and objects, simplifying optimization.
- Learning to Segment Moving Objects101
This work designs a framework with three cues: independent object motion between a pair of frames, which complements object recognition, object appearance, which helps to correct errors in motion estimation, and temporal consistency, which imposes additional constraints on the segmentation.
- 99
- Towards Segmenting Anything That Moves98
This work proposes a simple learning-based approach for spatio-temporal grouping that leverages motion cues from optical flow as a bottom-up signal for separating objects from each other, and shows that this model matches top-down methods on common categories, while significantly out-performing both top- down and bottom- up methods on never-before-seen categories.
- Video Generators are Robot Policies96
This paper proposes Video Policy, a modular framework that combines video and action generation that can be trained end-to-end, and shows strong generalization to unseen objects, backgrounds, and tasks, both in simulation and the real world.
- Standing Between Past and Future: Spatio-Temporal Modeling for Multi-Camera 3D Multi-Object Tracking91
- Breaking the “Object” in Video Object Segmentation70
This work collects a new dataset for Video Object Segmentation under Transformations (VOST) and extensively evaluates state-of-the-art VOS methods, showing that existing methods struggle when applied to this novel task and that their main limitation lies in over-reliance on static appearance cues.
- One click mining: interactive local pattern discovery through implicit preference and performance learning69
A fundamentally new approach towards user involvement that relies exclusively on the implicit feedback available from the natural analysis behavior of the user, and at the same time allows the user to work with a multitude of pattern classes and discovery algorithms simultaneously without even knowing the details of each algorithm is presented.
- Weakly-Supervised Semantic Segmentation Using Motion Cues56
This work presents motion-CNN (M-CNN), a novel FCNN framework which incorporates motion cues and is learned from video-level weak annotations, and demonstrates that the performance of M-CNN learned with 150 weak video annotations is on par with state-of-the-art weakly-supervised methods trained with thousands of images.
- Discovering Objects that Can Move52
This paper simplifies the recent auto-encoder based frameworks for unsuper-vised object discovery, and augment the resulting model with a weak learning signal from general motion segmentation algorithms, which is enough to generalize to segment both moving and static instances of dynamic objects.
- Heterogeneous-Agent Trajectory Forecasting Incorporating Class Uncertainty41
HAICU, a method for heterogeneous-agent trajectory forecasting that explicitly incorporates agents' class probabilities, is presented, demonstrating that incorporating class probabilities in trajectory forecasting significantly improves performance in the face of uncertainty, and enables new forecasting capabilities such as counterfactual predictions.
- Object Permanence Emerges in a Random Walk along Memory35
It is shown that object permanence can emerge by optimizing for temporal coherence of memory: a Markov walk along a space-time graph of memories, where the states in each time step are non-Markovian features from a sequence encoder.
- Relational linear programming32
RLP's relational nature allows a more intuitive representation of optimization problems, in particular over relational domains, and is illustrated empirically by experiments on approximate inference in Markov logic networks using LP relaxations, on solving Markov decision processes, and on collective inference using LP support vector machines.
- Object Discovery from Motion-Guided Tokens31
This work augments the auto-encoder representation learning framework with two key components: motion-guidance and mid-level feature tokenization, and introduces a new transformer decoder showing that their benefits can compound thanks to motion-guided vector quantization.
- 30
- Understanding Video Transformers via Universal Concept Discovery28
This work proposes an efficient approach for unsupervised identification of units of video transformer representations - concepts, and ranking their importance to the output of a model, and discovers that some of these mechanism are universal in video transformers.
- Unsupervised Learning of Video Representations via Dense Trajectory Clustering22
This paper proposes to adapt two top performing objectives in this class - instance recognition and local aggregation, to the video domain, and forms clusters in the IDT space, using heuristic-based IDT descriptors as a an unsupervised prior in the iterative local aggregation algorithm.
- Tracking Through Containers and Occluders in the Wild21
TCOW is introduced, a new benchmark and model for visual tracking through heavy occlusion and containment, and a task where the goal is to segment both the projected extent of the target object, as well as the surrounding container or occluder whenever one exists.
- 15
- Learning Semantic Segmentation with Weakly-Annotated Videos13
This work presents motion-CNN (M-CNN), a novel FCNN framework which incorporates motion cues and is learned from video-level weak annotations, and demonstrates that the performance of M-CNN learned with 150 weak video annotations is on par with state-of-the-art weakly-supervised methods trained with thousands of images.
- 12
- Zero-Shot Open-Vocabulary Tracking with Large Pre-Trained Models11
This paper combines an open-vocabulary detector, segmenter, and dense optical flow estimator into a model that tracks and segments any object in 2D videos, and outperforms previous state-of-the-art in UVO and BURST, benchmarks for open-world object tracking and segmentation, despite never being explicitly trained for tracking.
- 11
- 11
- A careful examination of large behavior models for multitask dexterous manipulation8
It is found that multitask pretraining made the policies more successful and robust and enabled teaching complex new tasks more quickly, using a fraction of the data when compared with single-task baselines, and performance predictably increased as pretraining scale and diversity grows.
- GRIN: Zero-Shot Metric Depth with Pixel-Level Diffusion8
This paper presents GRIN, an efficient diffusion model designed to ingest sparse unstructured training data, and uses image features with 3D geometric positional encodings to condition the diffusion process both globally and locally, generating depth predictions at a pixel-level.
- 6
- Robot Critics that Sweat the Small Stuff5
A method to fine-tune critics by constructing pairwise progress supervision using success and failure rollouts obtained from a policy, which results in a fine-tuned critic that excels at fine-grained progress reasoning and subtle failure detection, outperforming prior progress reasoning baselines.
- 5
- Capturing Visual Environment Structure Correlates with Control Performance3
This study provides insight into the representational properties that support generalizable manipulation, suggesting that learning to encode the latent physical state of the environment is a promising objective for control.
- RoboDream: Compositional World Models for Scalable Robot Data Synthesis3
This work presents a generalizable embodiment-centric world model that achieves scalable data generation by synthesizing photorealistic demonstrations with novel objects, in novel scenes, and from novel viewpoints, and demonstrates with real-world experiments that the generated data consistently improves downstream policy performance and significantly reduces real-world data requirements across diverse manipulation tasks.
- 3
- Understanding Complexity in VideoQA via Visual Program Generation3
An algorithm for estimating question complexity from code is proposed, which identifies fine-grained primitives that correlate with the hardest questions for any given set of models, making it easy to scale to new approaches in the future.
- Learning from motion3
This thesis attempts to bridge the gap between weakly- supervised and fully-supervised methods by utilizing motion information by proposing a learning-based method for motion segmentation: a convolutional neural network that takes optical flow as input and is trained to segment objects that move independently from the camera.
- Learning Generalizable Representations via Diverse Supervision.2
The ADE-FewShot benchmark is introduced, which stands upon the ADE dataset for scene parsing that features a realistic, long-tail distribution of categories as well as a diverse set of annotations and is turned into a realistic few-shot classification benchmark.
- 1
- Generative 4D Scene Gaussian Splatting with Object View-Synthesis Priors1
GenMOJO, a novel approach that integrates rendering-based deformable 3D Gaussian optimization with generative priors for view synthesis, generates more realistic novel views of scenes and produces more accurate point tracks compared to existing approaches.
- 1
- 1
- –
- UniGen-AR: Unifying Visual Generation with Auto-Regressive Modeling–
Unigen-AR, a framework that pairs a general-purpose multi-modal language model (MLLM) with an efficient next-scale visual auto-regressive (VAR) decoder, is proposed, establishing visual auto-regressive modeling as a compelling and efficient backbone for unified visual generation.
- –
- –
- –
- –
Publication data from OpenAlex, with missing venues and authors filled in from Crossref; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-15. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.
Report an error
Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.