Is this you? Claim this profile to correct it, add a bio and choose the work people see first.
Claim this profileWorks96 from public data
- Learning Continuous Image Representation with Local Implicit Image Function1,024
This paper proposes Local Implicit Image Function (LIIF), which takes an image coordinate and the 2D deep features around the coordinate as inputs, predicts the RGB value at a given coordinate as an output and builds a bridge between discrete and continuous representation in 2D.
- 882
- GroupViT: Semantic Segmentation Emerges from Text Supervision783
A hierarchical Grouping Vision Transformer (GroupViT), which goes beyond the regular grid structure representation and learns to group image regions into progressively larger arbitrary-shaped segments and performs competitively to state-of-the-art transfer-learning methods requiring greater levels of supervision.
- Generative Face Completion645
This paper demonstrates qualitatively and quantitatively that the proposed effective face completion algorithm is able to deal with a large area of missing pixels in arbitrary shapes and generate realistic face completion results.
- Open-Vocabulary Panoptic Segmentation with Text-to-Image Diffusion Models512
ODISE is presented, which unifies pre-trained text-image diffusion and discriminative models to perform open-vocabulary panoptic segmentation and outperforms the previous state of the art by significant margins on both open- VocabularyPanoptic and semantic segmentation tasks.
- 508
- 295
- COLMAP-Free 3D Gaussian Splatting273
This paper leverages both the explicit geometric representation and the continuity of the input video stream to perform novel view synthesis without any SfM preprocessing and significantly improves over previous approaches in view synthesis and camera pose estimation under large motion changes.
- NVILA: Efficient Frontier Visual Language Models260
NVILA, a family of open VLMs designed to optimize both efficiency and accuracy, is introduced by improving its model architecture by first scaling up the spatial and temporal resolutions, and then compressing visual tokens, which enables NVILA to efficiently process high-resolution images and long videos.
- Deep Cascaded Bi-Network for Face Hallucination240
A novel framework for hallucinating faces of unconstrained poses and with very low resolution, which allows exceptional hallucination quality on in-the-wild low-res faces with significant pose and illumination variations is presented.
- Learning Dual Convolutional Neural Networks for Low-Level Vision221
The proposed DualCNN consists of two parallel branches, which respectively recovers the structures and details in an end-to-end manner, and can generate the target signals according to the formation model for each particular application.
- Semi-Supervised 3D Hand-Object Poses Estimation with Interactions in Time216
This work builds a joint learning framework where the spatial-temporal consistency in large-scale hand-object videos as a constraint for generating pseudo labels in semi-supervised learning and substantially improves hand pose estimation in challenging real-world dataset.
- Self-supervised Single-View 3D Reconstruction via Semantic Consistency194
This work is the first to try and solve the single-view reconstruction problem without a category-specific template mesh or semantic keypoints, and demonstrates that the unsupervised method performs comparably if not better than existing category-specific reconstruction methods learned with supervision.
- Synthesizing Long-Term 3D Human Motion and Interaction in 3D Scenes178
This paper presents a hierarchical generative framework to synthesize long-term 3D human motion conditioning on the 3D scene structure and further enforce multiple geometry constraints between the human mesh and scene point clouds via optimization to improve realistic synthesis.
- CamCo: Camera-Controllable 3D-Consistent Image-to-Video Generation175
This work introduces CamCo, which allows fine-grained Camera pose Control for image-to-video generation and significantly improves 3D consistency and camera control capabilities compared to previous models while effectively generating plausible object motion.
- Learning Recursive Filters for Low-Level Vision via a Hybrid Neural Network169
Experimental results show that many low-level vision tasks can be effectively learned and carried out in real-time by the proposed algorithm and is significantly smaller and faster in comparison with a deep CNN based image filter.
- SCOPS: Self-Supervised Co-Part Segmentation162
This work proposes a self-supervised deep learning approach for part segmentation, where several loss functions are devised that aids in predicting part segments that are geometrically concentrated, robust to object variations and are also semantically consistent across different object instances.
- Cosmos 3: Omnimodal World Models for Physical AI150
This report introduces Cosmos 3, a family of omnimodal world models designed to jointly process and generate language, image, video, audio, and action sequences within a unified mixture-of-transformers architecture and demonstrates omnimodal world models as scalable, general-purpose backbones for embodied agents.
- 143
- Affordance Diffusion: Synthesizing Hand-Object Interactions140
This work proposes a two-step generative approach: a LayoutNet that samples an articulation-agnostic hand-object-interaction layout, and a ContentNet that synthesizes images of a hand grasping the object given the predicted layout, built on top of a large-scale pretrained diffusion model to make use of its latent representation.
- 139
- RGBD Objects in the Wild: Scaling Real-World 3D Object Learning from RGB-D Videos133
WildRGB-D comprises large-scale category-level RGB-D object videos, which are taken using an iPhone to go around the objects in 360 degrees and shows that the large-scale capture of RGB-D objects provides a large potential to advance 3D object learning.
- Putting Humans in a Scene: Learning Affordance in 3D Indoor Environments129
This work builds a fully automatic 3D pose synthesizer that fuses semantic knowledge from a large number of 2D poses extracted from TV shows as well as 3D geometric knowledge from voxel representations of indoor scenes to predict semantically plausible and physically feasible human poses within a given scene.
- 119
- Describe Anything: Detailed Localized Image and Video Captioning107
The Describe Anything Model (DAM), a model designed for detailed localized captioning (DLC), is introduced and sets new state-of-the-art on 7 benchmarks spanning keywordlevel, phrase-level, and detailed multi-sentence localized image and video captioning.
- RegionGPT: Towards Region Understanding Vision Language Model105
It is demonstrated that a universal RGPT model can be effectively applied and significantly enhancing performance across a range of region-level tasks, including but not limited to complex region descriptions, reasoning, object classification, and referring expressions comprehension.
- Communication-Efficient Collaborative Perception via Information Filling with Codebook98
Results show that CodeFilling outperforms previous SOTA Where2comm on DAIR-V2X/OPV2VH+ with 1,333/1,206× lower communication volume.
- A Unified Approach for Text-and Image-Guided 4D Scene Generation95
This work proposes Dream-in-4D, a novel two-stage approach for text-to-4D synthesis, which features a novel two-stage approach for text-to-4D synthesis, leveraging 3D and 2D diffusion guidance to effectively learn a high-quality static 3D asset in the first stage.
- 81
- 70
- Face Parsing via Recurrent Propagation67
This paper proposes a face parsing algorithm that combines hierarchical representations learned by a CNN, and accurate label propagations achieved by a spatially variant recurrent neural network (RNN).
- AGG: Amortized Generative 3D Gaussians for Single Image to 3D64
An Amortized Generative 3D Gaussian framework (AGG) is introduced that instantly produces 3D Gaussians from a single image, eliminating the need for per-instance optimization.
- 53
- 51
- 49
- Compositional Text-to-Image Generation with Dense Blob Representations48
This work develops a blob-grounded text-to-image diffusion model, termed BlobGEN, and introduces a new in-context learning approach to generate blob representations from text prompts to leverage the compositionality of large language models (LLMs).
- 48
- HOIDiffusion: Generating Realistic 3D Hand-Object Interaction Data46
This model is a conditional diffusion model that takes both the 3D hand-object geometric structure and text description as inputs for image synthesis, which offers a more control-lable and realistic synthesis as it can specify the structure and style inputs in a disentangled manner.
- Self-Supervised Viewpoint Learning From Image Collections44
This work proposes a novel learning framework which incorporates an analysis-by-synthesis paradigm to reconstruct images in a viewpoint aware manner with a generative network, along with symmetry and adversarial constraints to successfully supervise the authors' viewpoint estimation network.
- Regularizing Meta-learning via Gradient Dropout42
This paper introduces a simple yet effective method to alleviate the risk of overfitting for gradient-based meta-learning by randomly dropping the gradient in the inner-loop optimization of each parameter in deep neural networks, such that the augmented gradients improve generalization to new tasks.
- Video Autoencoder: self-supervised disentanglement of static 3D structure and motion40
The proposed video autoencoder can be trained directly using a pixel re-construction loss, without any ground truth 3D or camera pose annotations, and can be applied to a range of tasks, including novel view synthesis, camera pose estimation, and video generation by motion following.
- Switchable Temporal Propagation Network39
This paper proposes a learnable unified framework for propagating a variety of visual properties of video images, including but not limited to color, high dynamic range (HDR), and segmentation information, where the properties are available for only a few key-frames.
- 34
- 3D Aware Region Prompted Vision Language Model32
Spatial Region 3D is presented, a aware vision-language model that connects single-view 2D images and multi-view 3D data through a shared visual token space and achieves state-of-the-art performance, underscoring its effectiveness for unifying 2D and 3D representation space on scene understanding.
- Rendering Portraitures from Monocular Camera and Beyond30
This work introduces an automatic system that achieves portrait DoF rendering for monocular cameras using Convolutional Neural Networks and trains a spatially-variant Recursive Neural Network to learn and accelerate this rendering process.
- Scaling Vision Pre-Training to 4K Resolution29
This work introduces PS3 that scales CLIP-style vision pre-training to 4K resolution with a near-constant cost, and introduces VILA-HD, a new benchmark of image QA at 4K resolution on which VILA-HD outperforms all previous MLLMs, including a 14.5% improvement over GPT-4o, and a 3.96× speedup over Qwen2-VL.
- 29
- 28
- Learning to Track Instances without Video Annotations28
This work introduces a novel semisupervised framework by learning instance tracking networks with only a labeled image dataset and unlabeled video sequences with an instance contrastive objective, and shows that even when only trained with images, the learned feature representation is robust to instance appearance variations, and is thus able to track objects steadily across frames.
- CoordGAN: Self-Supervised Dense Correspondences Emerge from GANs26
Coordinate GAN (CoordGAN) is introduced, a structure-texture disentangled GAN that learns a dense correspondence map for each generated image and achieves better structure and texture disentanglement compared to existing approaches.
- 26
- Generalizable One-shot Neural Head Avatar24
Experiments show that the proposed approach generalizes well to unseen validation datasets, surpassing SOTA baseline methods by a large margin on head avatar reconstruction and animation.
- Omni-RGPT: Unifying Image and Video Region-level Understanding via Token Marks23
Token Mark, a set of tokens highlighting the target regions within the visual feature space that are directly embedded into spatial regions using region prompts and simultaneously incorporated into the text prompt to specify the target, establishes a direct connection between visual and text tokens.
- Zero-shot Pose Transfer for Unrigged Stylized 3D Characters20
A zero-shot approach that requires only the widely available deformed non-stylized avatars in training, and deforms stylized characters of significantly different shapes at inference, and generalizes to categories with scarce annotation, such as stylized quadrupeds is presented.
- BlobGEN-Vid: Compositional Text-to-Video Generation with Blob Video Representations19
This work develops a blob-grounded video diffusion model named BlobGEN-Vid, a general representation for controllable video generation that allows users to control object motions and fine-grained object appearance and introduces a masked 3D attention module that effectively improves regional consistency across frames.
- 19
- Learning Contrastive Representation for Semantic Correspondence17
It is shown that image-level contrastive learning is a key component to encourage the convolutional features to find correspondence between similar objects, while the performance can be further enhanced by regularizing cross-instance cycle-consistency at intermediate feature levels.
- 16
- 16
- 15
- Hierarchical Contrastive Motion Learning for Video Action Recognition15
H hierarchical contrastive motion learning is presented, a new self-supervised learning framework to extract effective motion representations from raw video frames that bridges the semantic gap between low-level motion cues and high-level recognition tasks, and promotes the fusion of appearance and motion information at multiple levels.
- Long-Horizon Manipulation via Trace-Conditioned VLA Planning13
LoHo-Manip is presented, a modular framework that scales short-horizon VLA execution to long-horizon instruction following via a dedicated task-management VLM, demonstrating strong gains in long-horizon success, robustness, and out-of-distribution generalization.
- 13
- 12
- Generalizable One-shot 3D Neural Head Avatar12
Experiments show that the proposed approach generalizes well to unseen validation datasets, surpassing SOTA baseline methods by a large margin on head avatar reconstruction and animation.
- 12
- TUVF: Learning Generalizable Texture UV Radiance Fields11
This paper introduces Texture UV Radiance Fields (TUVF) that generate textures in a learnable UV sphere space rather than directly on the 3D shape, which provides a more efficient and accurate representation of textures than traditional texture maps.
- SSE: Multimodal Semantic Data Selection and Enrichment for Industrial-scale Data Assimilation10
This work quantitatively shows that the Semantic Selection and Enrichment framework (SSE) can a) successfully maintain model performance with a smaller training dataset and b) improve model performance by enriching the smaller dataset without exceeding the original dataset size.
- Self-Supervised Object Detection via Generative Image Synthesis10
SSOD is presented, the first end-to-end analysis-by-synthesis framework with controllable GANs for the task of self-supervised object detection, which shows that SSOD outperforms the prior state-of-the-art purely image-based self- Supervised Object detection method Wetectron.
- 10
- CosAE: Learnable Fourier Series for Image Restoration8
This paper introduces Cosine Autoencoder (CosAE), a novel, generic Au-toencoder that seamlessly leverages the classic Fourier series with a feed-forward neural network, highlighting its capability to learn a generalizable representation for image restoration.
- Scraping Textures from Natural Images for Synthesis and Editing8
This paper aims to scrape textures directly from natural images of everyday objects and scenes, build texture models, and employ them for texture synthesis, texture editing, etc, and shows that the learned textures capture many natural and man-made textures in real images, and can be applied to tasks like texture synthesis, texture editing and texture swapping.
- 8
- Learning Video-Story Composition via Recurrent Neural Network7
The coherence between video clips from real videos via the Recurrent Neural Network is learned via the RNN that jointly incorporates the spatial-temporal semantics and motion dynamics to generate smooth and relevant compositions.
- Parallel Sequence Modeling via Generalized Spatial Propagation Network5
With learnable, input-dependent weights and no reliance on positional embeddings, GSPN achieves superior spatial fidelity and state-of-the-art performance in vision tasks, including ImageNet classification, class-guided image generation, and text-to-image generation.
- 3D Reconstruction with Generalizable Neural Fields using Scene Priors5
This work introduces training generalizable Neural Fields incorporating scene Priors (NFPs), which can be trained on large-scale datasets, allowing for fast adaptation to the reconstruction of a new scene with fewer views.
- 5
- 5
- 5
- Physics-based Indirect Illumination for Inverse Rendering4
A physics-based inverse rendering method that learns the illumination, geometry, and materials of a scene from posed multi-view RGB images and utilizes the Leibniz’s integral rule to resolve non-differentiability in the proposed illumination model caused by boundary lights inspired by differentiable irradiance in computer graphics.
- 3
- 4D-RGPT: Toward Region-level 4D Understanding via Perceptual Distillation3
This work introduces 4D-RGPT, a specialized MLLM designed to capture 4D representations from video inputs with enhanced temporal perception and introduces R4D-Bench, a benchmark for depth-aware dynamic scenes with region-level prompting.
- BlobGEN-3D: Compositional 3D-Consistent Freeview Image Generation with 3D Blobs3
This work proposes BlobGEN-3D, a novel approach that decouples compositional 3D scene representation from 2D image generation, enabling direct controllability in the 3D space while fully leveraging the capabilities of 2D diffusion models.
- 2
- 4DP-QA: Scalable QA for 4D Perception in Vision Language Models1
A QA generation pipeline that focuses on motion-related scene understanding, taking particular care of the entanglement of camera and object motion by casting tracking in both the traditional way and in a novel, fixed reference system, dubbed True-Motion Tracking, which provides an intuitive description of motion.
- Grounded 3D-Aware Spatial Vision-Language Modeling1
GR3D achieves consistent improvements across grounded and non-grounded spatial benchmarks, demonstrating grounding as an effective inductive bias for strengthening spatial understanding in VLMs.
- Temporal Prompting Matters: Rethinking Referring Video Object Segmentation1
This work decomposes the RVOS task into referring, video, and segmentation factors, and proposes a Temporal Prompt Generation and Selection (Tenet) framework to address the referring and video factors while leaving the segmentation problem to foundation models.
- 1
- Synthesizing Consistent Novel Views Via 3D Epipolar Attention Without Re-Training1
This work proposes to use epipolar geometry to locate and retrieve overlapping information from the input view, and extends the utilization of epipolar attention to a multi-view setting, allowing retrieval of overlapping information from the input view and other target views.
- GroPrompt: Efficient Grounded Prompting and Adaptation for Referring Video Object Segmentation1
This work aims to efficiently adapt foundation segmentation models for addressing RVOS from weak supervision with the proposed Grounded Prompting (GroPrompt) framework and proposes Text-Aware Prompt Contrastive Learning (TAP-CL) to enhance the association between the position prompts and the referring sentences with only box supervisions.
- Moment Matching Q-Learning–
By accelerating the action sampling process for flow-based policies, MoMa QL demonstrates superior performance in offline-to-online RL tasks because of faster and stronger adaptability for online interactive finetuning.
- Scaling Parallel Sequence Models to Foundation-Scale Vision Encoders–
C-GSPN is presented, a foundation-scale vision encoder based on 2D spatial propagation that makes the operator practical through three improvements: a fast GSPN CUDA kernel that fuses per-step launches into a single warp-specialized implementation with shared-memory tiling, coalesced access, and a compact multi-channel propagation.
- –
- Learning Propagation for Arbitrarily-Structured Data–
It is shown that SGPN consistently improves the performance of both pixel and point cloud segmentation, compared to networks that do not contain this module, and suggests an effective way to model the global pairwise relations for arbitrarily-structured data.
- –
- Learning Affinity to Parse Images–
A novel multi-objective learning method is developed that optimizes a single unified deep convolutional network with two distinct non-structured loss functions: one encoding the unary label likelihoods and the other encoding the pairwise label dependencies.
Publication data from OpenAlex, with missing venues and authors filled in from Crossref; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-14. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.
Report an error
Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.