Is this you? Claim this profile to correct it, add a bio and choose the work people see first.
Claim this profileWorks224 from public data
- Learning to See in the Dark1,622
A pipeline for processing low-light images is developed, based on end-to-end training of a fully-convolutional network that operates directly on raw sensor data and replaces much of the traditional image processing pipeline, which tends to perform poorly on such data.
- Photographic Image Synthesis with Cascaded Refinement Networks1,013
It is shown that photographic images can be synthesized from semantic layouts by a single feedforward network with appropriate structure, trained end-to-end with a direct regression objective.
- VideoCrafter1: Open Diffusion Models for High-Quality Video Generation633
This model is the first open-source I2V foundation model capable of transforming a given image into a video clip while maintaining content preservation constraints, and can generate realistic and cinematic-quality videos with a resolution of $1024 \times 576.
- FateZero: Fusing Attentions for Zero-shot Text-based Video Editing594
F FateZero, a zero-shot text-based editing method on real-world videos without per-prompt training or use-specific mask, is proposed, which is the first one to show the ability of zero-shot text-driven video style and local attribute editing from the trained text-to-image model.
- Defending ChatGPT against jailbreak attack via self-reminders515
This work systematically documents the threats posed by jailbreak attacks, introduces and analyses a dataset for evaluating defensive interventions and proposes the psychologically inspired self-reminder technique that can efficiently and effectively mitigate against jailbreaks without further training.
- 495
- Involution: Inverting the Inherence of Convolution for Visual Recognition418
The proposed involution operator could be leveraged as fundamental bricks to build the new generation of neural networks for visual recognition, powering different deep learning models on several prevalent benchmarks, including ImageNet classification, COCO detection and segmentation, together with Cityscapes segmentation.
- RODIN: A Generative Model for Sculpting 3D Digital Avatars Using Diffusion405
The RODIN model brings much-needed computational efficiency while preserving the integrity of 3D diffusion by using 3D-aware convolution that attends to projected features in the 2D plane according to their original relationships in 3D.
- Single Image Reflection Separation with Perceptual Losses404
The approach uses a fully convolutional network trained end-to-end with losses that exploit low-level and high-level image information and proposes a novel exclusion loss that enforces pixel-level layer separation.
- FFB6D: A Full Flow Bidirectional Fusion Network for 6D Pose Estimation379
This work proposes FFB6D, a Full Flow Bidirectional fusion network designed for 6D pose estimation from a single RGBD image, which learns to combine appearance and geometry information for representation learning as well as output representation selection.
- Fast Image Processing with Fully-Convolutional Networks360
This work presents an approach to accelerating a wide variety of image processing operators using a fully-convolutional network that is trained on input-output pairs that demonstrate the operator’s action, and demonstrates that the presented approach is significantly more accurate than prior approximation schemes.
- 339
- High-Fidelity GAN Inversion for Image Attribute Editing311
A novel highfidelity generative adversarial network (GAN) inversion framework that enables attribute editing with image-specific details well-preserved with a clear improvement in both inversion and editing quality is presented.
- 310
- TextDiffuser: Diffusion Models as Text Painters294
This work introduces TextDiffuser, focusing on generating images with visually appealing text that is coherent with backgrounds, and contributes the first large-scale text images dataset with OCR annotations, MARIO-10M, containing 10 million image-text pairs with text recognition, detection, and character-level segmentation annotations.
- Zoom to Learn, Learn to Zoom288
This paper shows that when applying machine learning to digital zoom, it is beneficial to operate on real, RAW sensor data, and shows how to obtain such ground-truth data via optical zoom and contribute a dataset, SR-RAW, for real-world computational zoom.
- Enhanced Invertible Encoding for Learned Image Compression263
This paper proposes an enhanced Invertible Encoding Network with invertible neural networks (INNs) to largely mitigate the information loss problem for better compression, and shows that this method outperforms the existing learned image compression methods and compression standards, including VVC (VTM 12.1), especially for high-resolution images.
- Latent Video Diffusion Models for High-Fidelity Long Video Generation260
This work introduces lightweight video diffusion models that synthesize high-fidelity and arbitrary-long videos from pure noise that outperforms previous methods on 3D pixel space when under a limited computational budget and introduces conditional latent perturbation to reduce performance degradation during generating long-duration videos.
- 223
- 222
- Pretraining is All You Need for Image-to-Image Translation219
Examination of the proposed pretraining-based image-to-image translation (PITI) is shown to be capable of synthesizing images of unprecedented realism and faithfulness, and an adversarial training to enhance the texture synthesis in the diffusion model training is proposed.
- TPCN: Temporal Point Cloud Networks for Motion Forecasting218
The Temporal Point Cloud Networks (TPCN) is proposed, a novel and flexible framework with joint spatial and temporal learning for trajectory prediction that achieves state-of-the-art results on the Argoverse motion forecasting benchmark.
- Full Flow: Optical Flow Estimation By Global Optimization over Regular Grids201
It is shown that one-shot global optimization of a classical Horn-Schunck-type objective over regular grids at a single resolution is sufficient to initialize continuous interpolation and achieve state-of-the-art performance on challenging modern benchmarks.
- Follow-Your-Emoji: Fine-Controllable and Expressive Freestyle Portrait Animation199
Follow-Your-Emoji, a diffusion-based framework for portrait animation, which animates a reference portrait with target landmark sequences, is presented and a facial fine-grained loss is proposed to improve the model’s ability of subtle expression perception and reference portrait appearance reconstruction.
- 191
- Semi-Parametric Image Synthesis178
A semi-parametric approach to photographic image synthesis from semantic layouts that combines the complementary strengths of parametric and nonparametric techniques is presented.
- Invertible Image Signal Processing176
An Invertible Image Signal Processing (InvISP) pipeline is designed, which not only enables rendering visually appealing sRGB images but also allows recovering nearly perfect RAW data.
- Speech Denoising with Deep Feature Losses175
An end-to-end deep learning approach to denoising speech signals by processing the raw waveform directly, which outperforms the state-of-the-art in objective speech quality metrics and in large-scale perceptual experiments with human listeners.
- Seeing and Hearing: Open-domain Visual-Audio Generation with Diffusion Latent Aligners152
This work proposes a mul-timodality latent aligner with the pre-trained ImageBind model that shares a similar core as the clas-sifier guidance that guides the diffusion denoising process during inference time, and shows the superior performance of the method on joint video-audio generation, visual-steered audio generation, and audio-steered vi-sual generation tasks.
- Polarized Reflection Removal With Perfect Alignment in the Wild152
This work identifies the misalignment issues of existing reflection removal datasets where the collected reflection-free images are not perfectly aligned with input mixed images due to glass refraction and proposes a polarized reflection removal model with a two-stage architecture.
- ScaleCrafter: Tuning-free Higher-Resolution Visual Generation with Diffusion Models150
This work investigates the capability of generating images from pre-trained diffusion models at much higher resolutions than the training image sizes and proposes the dispersed convolution and noise-damped classifier-free guidance, which can enable ultra-high-resolution image generation.
- CoDeF: Content Deformation Fields for Temporally Consistent Video Processing141
CoDeF is able to lift image-to-image translation to video-to-video translation and lift keypoint detection to keypoint tracking without any training, and achieves superior cross-frame consistency in processed videos compared to existing video-to-video translation approaches, and even manage to track non-rigid objects like water and smog.
- FS6D: Few-Shot 6D Pose Estimation of Novel Objects140
This work studies a new open set problem; the few-shot 6D object poses estimation: estimating the 6D pose of an unknown object by a few support views without extra training and proposes a large-scale RGBD photorealistic dataset (ShapeNet6D) for network pre-training.
- PLUTO: Pushing the Limit of Imitation Learning-based Planning for Autonomous Driving135
Impressively, PLUTO achieves state-of-the-art closed-loop performance, beating other competing learning-based methods and surpassing the current top-performed rule-based planner for the first time.
- Fully Automatic Video Colorization With Self-Regularization and Diversity128
This work presents a fully automatic approach to video colorization with self-regularization and diversity, which uses a perceptual loss with diversity to differentiate various modes in the solution space.
- 124
- 116
- Shape from Polarization for Complex Scenes in the Wild109
This work proposes a learning-based framework with a multi-head self-attention module and viewing encoding, which is designed to handle increasing polarization ambiguities caused by complex materials and non-orthographic projection in scene-level SfP.
- Animate-A-Story: Storytelling with Retrieval-Augmented Video Generation106
This work develops a framework comprised of two functional modules, which provides video candidates with desired scene or motion context described by query texts, and generates plot-aligned videos under the guidance of motion structure and text prompts.
- DiffSHEG: A Diffusion-Based Approach for Real-Time Speech-Driven Holistic 3D Expression and Gesture Generation104
This work proposes DiffSHEG, a Diffusion-based approach for Speech-driven Holistic 3D Expression and Gesture generation with arbitrary length, and introduces an outpainting-based sampling strategy for arbitrary long sequence generation in diffusion models, offering flexibility and computational efficiency.
- Future Video Synthesis With Object Motion Prediction103
An approach to predict future video frames given a sequence of continuous video frames in the past by decoupling the background scene and moving objects and shows that this model outperforms the state-of-the-art in terms of visual quality and accuracy.
- MetaPortrait: Identity-Preserving Talking Head Generation with Fast Personalized Adaptation99
This work proposes an ID-preserving talking head generation framework, which surpasses prior generation fidelity on established benchmarks, and adaptively fuse the source identity during synthesis, so that the network better preserves the key characteristics of the image portrait.
- High-fidelity 3D GAN Inversion by Pseudo-multi-view Optimization89
This work proposes a novel pipeline that builds on the pseudo-multi-view estimation with visibility analysis and achieves advantageous reconstruction and novel view synthesis quality over prior work, even for images with out-of-distribution textures.
- Gaussian Shell Maps for Efficient 3D Human Generation83
This work introduces Gaussian Shell Maps (GSMs) as a framework that connects SOTA generator network archi-tectures with emerging 3D Gaussian rendering primitives using an articulable multi shell-based scaffold and demonstrates that GSMs suc-cessfully generate 3D humans when trained on single-view datasets, including SHHQ and DeepFashion.
- Scaling Instruction-Based Video Editing with a High-Quality Synthetic Dataset82
Ditto is introduced, a holistic framework that fuses the creative diversity of a leading image editor with an in-context video generator, overcoming the limited scope of existing models and resolves the prohibitive cost-quality trade-off.
- 82
- Federated Domain Generalization for Image Recognition via Cross-Client Style Transfer81
This paper proposes a novel domain generalization method for image recognition under federated learning through cross-client style transfer (CCST) without exchanging data samples, which can lead to more uniform distributions of source clients, and make each local model learn to fit the image styles of all the clients to avoid the different model biases.
- 78
- AniFaceGAN: Animatable 3D-Aware Face Image Generation for Video Avatars77
To achieve meaningful control over facial expressions via deformation, this paper proposes a 3D-level imitative learning scheme between the generator and a parametric 3D face model during adversarial training of the3D-aware GAN.
- Self-supervised Dance Video Synthesis Conditioned on Music74
A self-supervised approach with pose perceptual loss for automatic dance video generation and a new cross-modal evaluation metric to evaluate the dance quality, which is able to estimate the similarity between two modalities (music and dance).
- Follow-Your-Click: Open-domain Regional Image Animation via Motion Prompts73
This paper proposes a practical framework, named Follow-Your-Click, to achieve image animation with a simple user click and a motion prompt, and proposes flow-based motion magnitude control to control the speed of target movement more precisely.
- Follow-Your-Creation: Empowering 4D Creation through Video Inpainting70
This work reformulates 4D video creation as a video inpainting task, enabling the model to fill in missing content caused by camera trajectory changes or user edits, and generates composite masked inpainting video data to effectively fine-tune the model for 4D video generation.
- Follow-Your-Click: Open-domain Regional Image Animation via Short Prompts68
This paper proposes a practical framework, named Follow-Your-Click, to achieve image animation with a simple user click and a short motion prompt and has simpler yet precise user control and better generation performance than previous methods.
- Dual-Camera Super-Resolution with Aligned Attention Modules68
This work proposes a self-supervised domain adaptation strategy for real-world images that generalizes the standard patch-based feature matching with spatial alignment operations and further explores the dual-camera super-resolution that is one promising application of RefSR.
- ASCEND: A Spontaneous Chinese-English Dataset for Code-switching in Multi-turn Conversation67
ASCEND (A Spontaneous Chinese-English Dataset) is a high-quality Mandarin Chinese- English code-switching corpus built on spontaneous multi-turn conversational dialogue sources collected in Hong Kong and reported on its design and procedure for collecting the speech data, including annotations.
- Image Inpainting with External-internal Learning and Monochromic Bottleneck67
An external-internal inPainting scheme with a monochromic bottleneck that helps image inpainting models remove artifacts and a novel internal color propagation method with progressive learning strategies for consistent color restoration.
- MagicStick: Controllable Video Editing via Control Handle Transformations66
This method is the first method to show the ability of video property editing from the pre-trained text-to-image model, and compares with shape-aware text-based editing and handcrafted motion video generation, demonstrating the superior temporal consistency and editing capability than previous works.
- Learning to denoise astronomical images with U-nets63
This work proposes Astro U-net, a convolutional neural network for image de-noising and enhancement, which is able to produce images with noise characteristics as if they are obtained with twice the exposure time, and with minimum bias or information loss.
- Robust Reflection Removal with Reflection-free Flash-only Cues62
This work proposes a simple yet effective reflection-free cue for robust reflection removal from a pair of flash and ambient (no-flash) images that outperforms state-of-the-art reflection removal approaches and avoids introducing artifacts.
- VidMuse: A Simple Video-to-Music Generation Framework with Long-Short-Term Modeling61
This work presents a large-scale dataset by collecting 360K video-music pairs, and proposes VidMuse, a simple framework for generating music aligned with video inputs that outperforms existing models in terms of audio quality, diversity, and audio-visual alignment.
- UNIC: Unified In-Context Video Editing58
UNified In-Context Video Editing (UNIC), a simple yet effective framework that unifies diverse video editing tasks within a single model in an in-context manner, and introduces task-aware RoPE to facilitate consistent temporal positional encoding, and condition bias that enables the model to clearly differentiate different editing tasks.
- 58
- 56
- Optimizing Video Prediction via Video Frame Interpolation55
This work presents a new optimization framework for video prediction via video frame interpolation, in which it solves an extrapolation problem based on an interpolation model and outperforms other video prediction methods that require a large amount of training data or extra semantic information.
- Diffusion-Based Visual Art Creation: A Survey and New Perspectives52
This survey explores the emerging realm of diffusion-based visual art creation, examining its development from both artistic and technical perspectives and sheds light on the mechanisms through which AI systems emulate and, possibly, enhance human capacities in artistic perception and creativity.
- MagicQuill: An Intelligent Interactive Image Editing System51
The MagicQuill system starts with a streamlined yet functionally robust interface, enabling users to articulate their ideas with just a few strokes, then monitored by a multimodal large language model (MLLM) to anticipate user intentions in real time, bypassing the need for prompt entry.
- EEdit ⚡: Rethinking the Spatial and Temporal Redundancy for Efficient Image Editing51
This paper rethink that the redundancy in inversion-based image editing exists in both the spatial and temporal dimensions, such as the unnecessary computation in unedited regions and the redundancy in the inversion progress, and proposes an Efficient Editing framework, named EEdit, to achieve efficient image editing.
- Follow-Your-Canvas: Higher-Resolution Video Outpainting with Extensive Content Generation49
This paper explores higher-resolution video outpainting with extensive content generation with extensive content generation by proposing a diffusion-based method called Follow-Your-Canvas, which achieves the best quantitative results across various resolution and scale setups.
- DRINet: A Dual-Representation Iterative Learning Network for Point Cloud Segmentation48
The DRINet work, which serves as the basic network structure for dual-representation learning with great flexibility at feature transferring and less computation cost, especially for large-scale point clouds, is proposed.
- DYNAFED: Tackling Client Data Heterogeneity with Global Dynamics47
An idea to collect and leverage global knowledge on the server without hindering data privacy is put forth, which enjoys the following advantages: it does not rely on any external on-server dataset, which requires no additional cost for data collection and the pseudo data can be synthesized in early communication rounds, which enables Dyna fed to take effect early for boosting the convergence and stabilizing training.
- 47
- 46
- LeviTor: 3D Trajectory Oriented Image-to-Video Synthesis45
This work proposes a pioneering method for 3D trajectory control in image-to-video synthesis by abstracting object masks into a few cluster points, accompanied by the depth information and the instance information, which are finally fed into a video diffusion model as the control signal.
- Blind Video Deflickering by Neural Filtering with a Flawed Atlas44
This work proposes a general flicker removal framework that only receives a single flickering video as input without additional guidance, and achieves satisfying deflickering performance and even outperforms baselines that use extra guidance on a public benchmark.
- One Model to Edit Them All: Free-Form Text-Driven Image Manipulation with Semantic Modulations44
The proposed Free-Form CLIP has a cross-modality semantic modulation module containing semantic alignment and injection, which performs the automatic latent mapping via linear transformations with a cross attention mechanism and can leverage text prompts with multiple semantic meanings for image manipulation.
- Internal Video Inpainting by Implicit Long-range Propagation44
This work proposes a novel framework for video inpainting by adopting an internal learning strategy that can be achieved implicitly by fitting a convolutional neural network to known regions, and designs two regularization terms to preserve high-frequency details and long-term temporal consistency.
- 44
- VideoGen-of-Thought: Step-by-step generating multi-shot video with minimal manual intervention41
VideoGen-of-Thought is introduced, a step-by-step framework that automates multi-shot video synthesis from a single sentence by systematically addressing three core challenges: Narrative fragmentation, identity-aware cross-shot propagation, and transition artifacts.
- Deep Video Prior for Video Consistency and Propagation40
This work shows that temporal consistency can be achieved by training a convolutional network on a video with Deep Video Prior (DVP), and shows its effectiveness in propagating three different types of information (color, artistic style, and object segmentation).
- 39
- 3D-Aware Indoor Scene Synthesis with Depth Priors39
This work proposes a dual-path generator, where one path is responsible for depth generation, whose intermediate features are injected into the other path as the condition for appearance rendering, which eases the 3D-aware synthesis with explicit geometry information.
- Point Cloud Compression with Sibling Context and Surface Priors39
A new entropy model is proposed that explores the hierarchical dependency in an octree using the context of siblings' children, ancestors, and neighbors to encode the occupancy information of each non-leaf octree node into a bitstream.
- PSConv: Squeezing Feature Pyramid into One Compact Poly-Scale Convolutional Layer39
The proposed convolution operation, named Poly-Scale Convolution (PSConv), mixes up a spectrum of dilation rates and tactfully allocate them in the individual convolutional kernels of each filter regarding a single convolutionAL layer.
- 37
- Delving StyleGAN Inversion for Image Editing: A Foundation Latent Space Viewpoint36
The exploration of the foundation latent space W is explored, which improves the representation ability of latent codes in W+ and features in F, which yields state-of-the-art reconstruction fidelity and editability results on the standard benchmarks.
- SPIRE: Semantic Prompt-Driven Image Restoration35
SPIRE is the first framework that supports fine-level instruction through language-based quantitative specification of the restoration strength, without the need for explicit task-specific design, and introduces a novel fusion mechanism that augments the existing ControlNet architecture by learning to rescale the generative prior, thereby achieving better restoration fidelity.
- Dynamic Hierarchical Mimicking Towards Consistent Optimization Objectives35
Dynamic Hierarchical Mimicking, a generic feature learning mechanism, is proposed to advance CNN training with enhanced generalization ability and promotes multi-level interactions among different branches through an optimization formula with probabilistic prediction matching losses, thus guaranteeing a more robust optimization process and better representation ability.
- Unsupervised Portrait Shadow Removal via Generative Priors34
This work proposes the first unsupervised method for portrait shadow removal without any training data, and leverages the generative facial priors embedded in the off-the-shelf pretrained StyleGAN2 to leverage the decomposition process.
- DepthLab: From Partial to Complete33
This work bridges this gap with DepthLab, a foundation depth inpainting model powered by image diffusion priors that demonstrates resilience to depth-deficient regions, and faithfully preserves scale consistency with the conditioned known depth when filling in missing values.
- 4D Panoptic Scene Graph Generation33
To solve PSG-4D, the introduction of 4D Panoptic Scene Graph (PSG-4D), a new representation that bridges the raw visual data perceived in a dynamic 4D world and high-level visual understanding, and a real-world application example to demonstrate how the model can achieve dynamic scene understanding by integrating a large language model into the system.
- IICNet: A Generic Framework for Reversible Image Conversion33
This work develops Invertible Image Conversion Net (IIC-Net) as a generic solution to various RIC tasks due to its strong capacity and task-independent design and maintains a highly invertible structure based on invertable neural networks (INNs) to better preserve the information during conversion.
- Video Depth Estimation by Fusing Flow-to-Depth Proposals33
This paper presents a model for video depth estimation, which consists of a flow-to-depth layer, a camera pose refinement module, and a depth fusion network that outperforms state-of-the-art depth estimation methods, and has reasonable crossdataset generalization ability.
- SkillMimic-V2: Learning Robust and Generalizable Interaction Skills from Sparse and Noisy Demonstrations32
To enable effective RLID with augmented data, an Adaptive Trajectory Sampling strategy for dynamic curriculum generation and a historical encoding mechanism for memory-dependent skill learning are developed, which enables robust skill acquisition that significantly generalizes beyond the reference demonstrations.
- Planning for Sample Efficient Imitation Learning32
EfficientImitate (EI), a planning-based imitation learning method that can achieve high in-environment sample efficiency and performance simultaneously, is proposed and the seemingly incompatible two classes of imitation algorithms can be naturally unified under this framework.
- MangaNinja: Line Art Colorization with Precise Reference Following31
This work incorporates two thoughtful designs to ensure precise character detail transcription, including a patch shuffling module to facilitate correspondence learning between the reference color image and the target line art, and a point-driven control scheme to enable fine-grained color matching.
- HeadArtist: Text-conditioned 3D Head Generation with Self Score Distillation31
Experimental results show that the HeadArtist approach produces high-quality 3D head sculptures with rich geometry and photo-realistic appearance, which significantly outperforms state-of-the-art methods.
- Efficient Point Cloud Segmentation with Geometry-Aware Sparse Networks31
The Geometry-aware Sparse Networks (GASN) is proposed by utilizing the sparsity and geometry of a point cloud in a single voxel representation and achieves state-of-the-art performance on both SemanticKITTI and Nuscenes datasets while running significantly faster and consuming less memory.
- Safety-aware Motion Prediction with Unseen Vehicles for Autonomous Driving31
This approach is the first one that can predict the existence of unseen vehicles in most cases and significantly outperforms the state-of-the-art baselines on the safety-aware motion prediction task.
- LinkGAN: Linking GAN Latents to Pixels for Controllable Image Synthesis30
An easy-to-use regularizer for GAN training, which helps explicitly link some axes of the latent space to a set of pixels in the synthesized image, which can improve the spatial controllability of both 2D and 3D-aware GAN models, barely sacrificing the synthesis performance.
- Real-time Streaming Video Denoising with Bidirectional Buffers30
A novel Bidirectional Buffer Block is introduced as the core module of the BSVD, which makes it possible to achieve high-fidelity real-time denoising for streaming videos with both past and future temporal receptive fields and outperforms previous methods in terms of restoration fidelity and runtime.
- Bootstrap Motion Forecasting With Self-Consistent Constraints29
A novel framework to bootstrap Motion forecastIng with Self-consistent Constraints (MISC) that significantly outperforms the state-of-the-art methods and designs a novel self-ensembling scheme to obtain accurate teacher targets to enforce the self-constraints with multi-modality supervision.
- Composite Photograph Harmonization with Complete Background Cues28
This work investigates the usage of a target background and finds that a complete background plays a vital role in both seamlessly blending and harmonization and develops a network to learn the composition process given an imperfect alpha matte with appearance features extracted from the complete background to adjust color distribution.
- AniPortraitGAN: Animatable 3D Portrait Generation from 2D Image Collections27
An animatable 3D-aware GAN that generates portrait images with controllable facial expression, head pose, and shoulder movements and a dual-camera rendering and adversarial learning scheme is proposed to improve the quality of the generated faces.
- LiDAR-GS:Real-time LiDAR Re-Simulation using Gaussian Splatting26
LiDAR-GS designs a differentiable laser beam splatting, using range-view representation for precise surface splatting by projecting lasers onto micro cross-sections, effectively eliminating artifacts associated with local affine approximations.
- 26
- Learning 3D-Aware Image Synthesis with Unknown Pose Distribution25
This work proposes PoF3D, a pose-free generator and the pose-aware discriminator that frees generative radiance fields from the requirements of 3D pose priors, and demonstrates the feasibility of learning high-quality 3D-aware image synthesis without using 3D posing priors for the first time.
- SAD: Segment Any RGBD25
The Segment Any RGBD (SAD) model is proposed, which is specifically designed to extract geometry information directly from images, inspired by the natural ability of humans to identify objects through the visualization of depth maps, thus providing cues with enhanced geometry information and mitigating the issue of over-segmentation.
- CMDFusion: Bidirectional Fusion Network With Cross-Modality Knowledge Distillation for LiDAR Semantic Segmentation25
This work proposes a Bidirectional Fusion Network with Cross-Modality Knowledge Distillation (CMDFusion) and shows that this scheme achieves the best performance among all fusion-based methods on SemanticKITTI and nuScenes datasets.
- Human MotionFormer: Transferring Human Motions with Vision Transformers25
A hierarchical ViT framework that leverages global and local perceptions to capture large and subtle motion matching, respectively, and set the new state-of-the-art performance both qualitatively and quantitatively is proposed.
- Stereo Matching by Self-supervision of Multiscopic Vision25
A new self-supervised framework for stereo matching utilizing multiple images captured at aligned camera positions is proposed, which obtains better disparity maps than previous unsupervised methods on the KITTI dataset and is comparable to supervised methods when generalized to unseen data.
- Self-supervised Object Tracking with Cycle-consistent Siamese Networks25
This work proposes to integrate a Siamese region proposal and mask regression network in a cycle-consistent self-supervised framework for object tracking so that a fast and more accurate tracker can be learned without the annotation of each frame.
- Robust Reflection Removal With Flash-Only Cues in the Wild23
A dedicated model is proposed that not only utilizes the reflection-free cue but also avoids introducing artifacts, which helps accurately estimate reflection and transmission, and outperforms state-of-the-art reflection removal approaches.
- Improving 3D-aware Image Synthesis with A Geometry-aware Discriminator23
GeoD is proposed through learning a geometry-aware discriminator to improve 3D-aware GANs and is registered as a general framework such that a more capable discriminator can further assist the generator with a better multi-view consistency.
- 22
- AlignGuard: Scalable Safety Alignment for Text-to-Image Generation22
AlignGuard is introduced, a method for safety alignment of T2I models that consistently outperforms the state-of-the-art on many benchmarks and establishes new practices for safety alignment in T2I networks.
- 22
- 22
- Automatic Speech Recognition Datasets in Cantonese: A Survey and New Dataset22
A powerful and robust Cantonese ASR model is created by applying multi-dataset learning on MDCC and Common Voice zh-HK and the results show the effectiveness of the dataset.
- Large Motion Video Autoencoding with Cross-modal Video VAE21
A novel and powerful video autoencoder capable of high-fidelity video encoding with temporal-aware spatial compression to better encode and decode the spatial information and a lightweight motion compression model for further temporal compression.
- ERRA: An Embodied Representation and Reasoning Architecture for Long-Horizon Language-Conditioned Manipulation Tasks21
ERRA is an embodied learning architecture that enables robots to jointly obtain three fundamental capabilities (reasoning, planning, and interaction) for solving long-horizon language-conditioned manipulation tasks based on tightly-coupled probabilistic inferences at two granularity levels.
- A Categorized Reflection Removal Dataset with Diverse Real-world Scenes21
It is shown that state-of-the-art reflection removal methods generally perform well on blurry reflection but fail in obtaining satisfying performance on other types of real-world reflection.
- Robust Depth Enhancement via Polarization Prompt Fusion Tuning20
This work presents a general framework that leverages polarization imaging to improve inaccurate depth measurements from various depth sensors and proposes a Polarization Prompt Fusion Tuning (PPFT) strategy to effectively utilize RGB-based models pre-trained on large-scale datasets.
- Exploring Sparse MoE in GANs for Text-conditioned Image Synthesis19
Aurora, a GAN-based text-to-image generator that employs a collection of experts to learn feature processing, together with a sparse router to adaptively select the most suitable expert for each feature point, closes the performance gap between GANs and industry-level diffusion models, maintaining a fast inference speed.
- 19
- FullDiT2: Efficient In-Context Conditioning for Video Diffusion Transformers18
FullDiT2 is proposed, an efficient in-context conditioning framework for general controllability in both video generation and editing tasks, which innovates from two key perspectives: to address the token redundancy and a selective context caching mechanism to minimize redundant interactions between condition tokens and video latents.
- 18
- 18
- 17
- AvatarArtist: Open-Domain 4D Avatarization16
This work focuses on open-domain 4D avatarization, and selects parametric triplanes as the intermediate 4D representation, and proposes a practical training paradigm that takes advantage of both generative adversarial networks (GANs) and diffusion models.
- OV9D: Open-Vocabulary Category-Level 9D Object Pose and Size Estimation16
Comprehensive quantitative and qualitative experiments demonstrate that the proposed open-vocabulary method, trained on large-scale synthesized data, significantly outperforms the baseline and can effectively generalize to real-world images of unseen categories.
- Automatic Controllable Colorization via Imagination16
The core of the framework lies in an imagination module: by understanding the content within a grayscale image, a pre-trained image generation model is utilized to generate multiple images that contain the same content.
- 16
- Randomized Quantization: A Generic Augmentation for Data Agnostic Self-supervised Learning16
This paper explores the orthogonal channel dimension for generic data augmentation by exploiting precision redundancy in quantization, and significantly surpasses existing generic data augmentation methods, while showing on par performance against modality-specific augmentations.
- Neural Camera Simulators16
This work represents the first attempt to simulate a camera sensor’s behavior leveraging both the advantage of traditional raw sensor features and the power of data-driven deep learning.
- 16
- 15
- Volumetric-based Contact Point Detection for 7-DoF Grasping15
A novel grasp pipeline based on contact point detection on the truncated signed distance function (TSDF) volume to achieve closed-loop 7-degree-of-freedom (7-DoF) grasping on cluttered environments is proposed.
- In-Domain GAN Inversion for Faithful Reconstruction and Editability14
In-domain GAN inversion is proposed, which consists of a domain-guided encoder and a domain-regularized optimizer, to regularize the inverted code in the native latent space of the pre-trained GAN model, to sufficiently reuse the knowledge learned by GANs for image reconstruction, facilitating a wide range of editing applications without any retraining.
- 14
- 14
- Audio-Omni: Extending Multi-modal Understanding to Versatile Audio Generation and Editing13
Beyond its core capabilities, Audio-Omni exhibits remarkable inherited capabilities, including knowledge-augmented reasoning generation, in-context generation, and zero-shot cross-lingual control for audio generation, highlighting a promising direction toward universal generative audio intelligence.
- Real-time 6K Image Rescaling with Rate-distortion Optimization13
A novel frame-work (HyperThumbnail) for real-time 6K rate-distortion-aware image rescaling that minimizes the file size of the embedding LR JPEG thumbnail while maximizing HR reconstruction quality and reconstructs a high-fidelity HR image from the LR one in real time.
- 12
- Enlarging Instance-specific and Class-specific Information for Open-set Action Recognition12
A novel Prototypical Similarity Learning (PSL) framework is proposed to keep the instance variance within the same class to retain more IS information and to enlarge the instance-specific and classspecific information contained in the feature for better performance.
- Learning to Learn Parameterized Classification Networks for Scalable Input Images12
This work employs meta learners to generate convolutional weights of main networks for various input scales and maintain privatized Batch Normalization layers per scale for improved training performance and achieves an improved accuracy-efficiency trade-off during the adaptive inference process.
- 11
- Infinite-Canvas: Higher-Resolution Video Outpainting with Extensive Content Generation11
Infinite-Canvas excels in large-scale video outpainting, e.g., from 512 × 512 to 1152 × 2048 (9×), while producing high-quality and aesthetically pleasing results.
- 11
- Flipbot: Learning Continuous Paper Flipping via Coarse-to-Fine Exteroceptive-Proprioceptive Exploration11
Flipbot allows the robot to capture object physical properties by integrating exteroceptive and proprioceptive perceptions that are indispensable for manipulating deformable objects by incorporating a proposed coarse-to-fine exploration process.
- 11
- Stereo Waterdrop Removal with Row-wise Dilated Attention11
A novel row-wise dilated attention module is proposed to enlarge attention’s receptive field for effective information propagation between the two stereo images to better detect and remove waterdrops from stereo images.
- Edicho: Consistent Image Editing in the Wild10
Edicho steps in with a training-free solution based on diffusion models, featuring a fundamental design principle of using explicit image correspondence to direct editing, and enjoys a plug-and-play nature and is compatible to most diffusion-based editing methods, such as ControlNet and BrushNet.
- Video Waterdrop Removal via Spatio-Temporal Fusion in Driving Scenes10
This work proposes an attention-based framework that fuses the spatio-temporal representations from multiple frames to restore visual information occluded by waterdrops and adopts a cross-modality training strategy that combines synthetic videos and real-world images.
- Hints of Prompt: Enhancing Visual Representation for Multimodal LLMs in Autonomous Driving8
The Hints of Prompt framework is proposed, which introduces three key enhancements: Affinity hint to emphasize instance-level structure by strengthening token-wise connections, Semantic hint to incorporate high-level information relevant to driving-specific cases, and Question hint to align visual features with the query context, focusing on question-relevant regions.
- 8
- 8
- Restorable Image Operators with Quasi-Invertible Networks8
A quasi-invertible model that learns common image processing operators in a restorable fashion is proposed that can generate visually pleasing results with the original content embedded and can be easily applied to practical applications such as restorable human face retouching and highlight preserved exposure adjustment.
- Deep Reinforced Attention Learning for Quality-Aware Visual Recognition8
This paper builds upon the weakly-supervised generation mechanism of intermediate attention maps in any convolutional neural networks and disclose the effectiveness of attention modules more straightforwardly to fully exploit their potential.
- Instruction-Based Image Editing with Planning, Reasoning, and Generation7
This work aims to bridge understanding and generation via a new multi-modality model that provides the intelligent abilities to instruction-based image editing models for more complex cases and has competitive editing abilities on complex real-world images.
- 7
- Neural Image Popularity Assessment with Retrieval-augmented Transformer7
A retrieval-augmented approach that leverages both image features and user-specific statistics for neural image popularity assessment and achieves 79.5% accuracy, which significantly outperforms human and baseline models on the pairwise ranking of images from the Instagram Influencer Dataset.
- Joint Depth and Normal Estimation from Real-world Time-of-flight Raw Data7
This work meticulously constructed the first large-scale dataset with paired raw ToF data and ground-truth high-resolution depth maps provided by an industrial depth camera, and designs a simple but effective framework for joint depth and normal estimation.
- FlyAware: Inertia-Aware Aerial Manipulation via Vision-Based Estimation and Post-Grasp Adaptation5
This study develops an inertia-aware adaptive control strategy based on gain scheduling, and assess its robustness via frequency-domain system identification, and provides new insights into post-grasp control for AMs.
- 5
- MagicScroll: Nontypical Aspect-Ratio Image Generation for Visual Storytelling via Multi-Layered Semantic-Aware Denoising5
This work proposes MagicScroll, a multi-layered, progressive diffusion-based image generation framework with a novel semantic-aware denoising process that showcases promising results in aligning with the narrative text, improving visual coherence, and engaging the audience.
- A Portable Multiscopic Camera for Novel View and Time Synthesis in Dynamic Scenes5
The results show that the portable multiscopic camera system with a dedicated model for novel view and time synthesis in dynamic scenes outperforms alternative solutions qualitatively and quantitatively.
- 5
- 5
- Divide-then-Diagnose: Weaving Clinician-Inspired Contexts for Ultra-Long Capsule Endoscopy Videos4
VideoCAP is introduced, the first CE dataset with diagnosis-driven annotations derived from real clinical reports, and DiCE, a clinician-inspired framework that mirrors the standard CE reading workflow is proposed, which consistently outperforms state-of-the-art methods.
- WorldDirector: Building Controllable World Simulators with Persistent Dynamic Memory4
Experimental results demonstrate that the WorldDirector framework supports the synthesis of complex and extended events with unprecedented controllability and persistent dynamic object memory.
- Threat-Aware UAV Dodging of Human-Thrown Projectiles With an RGB-D Camera4
Drawing inspiration from baseball, in which pitchers’ body movements are analyzed to predict the ball’s trajectory, a novel real-time dodging system that leverages an RGB-D camera is proposed that achieves high prediction accuracy and outperforms the baseline in effective distance and latency.
- Learning Naturally Aggregated Appearance for Efficient 3D Editing4
This work proposes to learn the color field as an explicit 2D appearance aggregation, also called canonical image, with which users can easily customize their 3D editing via 2D image processing, and complement the canonical image with a projection field that maps 3D points onto 2D pixels for texture query.
- ModelGrow: Continual Text-to-Video Pre-training with Model Expansion and Language Understanding Enhancement4
This work explores the continual general pre-training of text-to-video models, enabling the model to "grow" its abilities based on a pre-trained foundation, analogous to how humans acquire new knowledge based on past experiences.
- 4
- 4
- 4
- Distributed Coevolutionary Multidisciplinary Design Optimization: A Flexible Approach4
The result shows that the algorithm is effective in design optimization of co upled sy stems and good approximation of the whole Pareto optimal front can be obtained by DCMDO in one run.
- Response-G1: Explicit Scene Graph Modeling for Proactive Streaming Video Understanding3
Response-G1 is introduced, a novel framework that establishes explicit, structured alignment between the accumulated video evidence and the query's expected response conditions via scene graphs via scene graphs, validating the advantage of explicit scene graph modeling and retrieval in streaming video understanding.
- Enhancing HDR Imaging with Joint Denoising and Deblurring3
A novel and efficient attention-based multi-exposure HDR imaging method, which skillfully selects the optimal information from corrupted LDR inputs by the authors' customized cross-attention mechanism to generate HDR information is proposed.
- Industrial-Grade Sensor Simulation via Gaussian Splatting: A Modular Framework for Scalable Editing and Full-Stack Validation3
This work provides both algorithmic insights and practical validation, establishing GS as a cornerstone for industrial-grade sensor simulation by refactor three crucial components through GS, to leverage its explicit scene representation and real-time rendering.
- SpA2V: Harnessing Spatial Auditory Cues for Audio-driven Spatially-aware Video Generation3
SpA2V is presented, the first framework explicitly exploits spatial auditory cues from audios to generate videos with high semantic and spatial correspondence and develops an efficient and effective approach to seamlessly integrate VSLs as conditional guidance into pre-trained diffusion models, enabling VSL-grounded video generation in a training-free manner.
- 3
- 3
- Cross-Cluster Shifting for Efficient and Effective 3D Object Detection in Autonomous Driving3
This work introduces an intriguing Cross-Cluster Shifting operation to unleash the representation capacity of the point-based detector by efficiently modeling longer-range inter-dependency while including only a negligible overhead.
- 3
- 3
- 2
- 2
- Multitarget Device-Free Localization via Cross-Domain Wi-Fi RSS Training Data and Attentional Prior Fusion2
This study pioneeringly proposed a transformer-based learning method with Wi-Fi RSS as input, and an attentional prior fusion module, to simultaneously locate an unknown number of people at random positions, to overcome the multitarget data collection challenges.
- 2
- Two-Layer Bandit Optimization for Recommendations2
This work introduces an efficient two-layer bandit approach which is contextualized to user cohorts of similar taste and mitigate cannibalization at runtime within a single multi-intent content surfacing platform by formalizing relevant offline evaluation metrics, and by involving the cross-component interactions in the bandit rewards.
- 2
- 2
- 2
- Decentralized Autonomous Video Copyright Protection2
A deep learning based approach for fast and accurate similar video detection, which will help fight against video plagiarism and achieve high detection rate for similar videos, and run in real time is proposed.
- Master Rules from Chaos: Learning to Reason, Plan, and Interact from Chaos for Tangram Assembly1
MRChaos (Master Rules from Chaos), a robust and general solution for learning assembly policies that can generalize to novel objects, indicates that radical generalization in robotic assembly can be achieved by learning in much simpler domains.
- 1
- 1
- HeadArtist-VL: Vision / Language Guided 3D Head Generation with Self Score Distillation1
In the experiments, the HeadArtist-VL produces high-quality 3D head sculptures with rich geometry and photo-realistic appearance, which significantly outperforms state-of-the-art methods.
- 1
- Towards Degradation-Robust Reconstruction in Generalizable NeRF1
The Objaverse Blur Dataset is constructed, comprising 50,000 images from over 1000 settings featuring multiple levels of blur degradation, and a simple and model-agnostic module for enhancing the degradation robustness of GNeRFs is designed.
- Online Overexposed Pixels Hallucination in Videos with Adaptive Reference Frame Selection1
A transformer-based deep neural network (DNN) is proposed to infer the missing HDR details and uses reinforcement learning to train a reference frame selection DNN that decides whether to adopt the current frame as a future reference.
- 1
- Interpreting Class Conditional GANs with Channel Awareness1
Channel awareness is proposed, which quantitatively characterizes how a single channel contributes to the final synthesis of multiple classes in the class conditional generator, and shows promising potential in segmenting the synthesized image and evaluating the category-wise synthesis performance.
- 1
- 1
- 1
- –
- –
- VAInpaint: Zero-Shot Video-Audio inpainting framework with LLMs-driven Module–
VAInpaint is proposed, a novel pipeline that first utilizes a segmentation model to generate masks and guide a video inpainting model in removing objects, and an LLM then analyzes the scene globally, while a region-specific model provides localized descriptions.
- –
- –
- –
- –
- SimMAT: Exploring Transferability from Vision Foundation Models to Any Image Modality–
SimMAT can improve the segmentation performance (mIoU) from 22.15% to 53.88% on average for evaluated modalities and consistently outperforms other baselines and can raise awareness of cross-modal transfer learning and benefit various fields for better results with vision foundation models.
- SimCMF: A Simple Cross-modal Fine-tuning Strategy from Vision Foundation Models to Any Imaging Modality–
In SimCMF, a simple and effective framework to study cross-modal fine-tuning from vision foundation models trained on natural RGB images to other imaging modalities of different physical properties, a novel cross-modal alignment module is proposed to address the modality misalignment problem.
- LDM-ISP: Enhancing Neural ISP for Low Light with Latent Diffusion Models–
LDM-SID aims at inserting a set of proposed taming modules into a frozen pre-trained diffusion model to steer its generating process, and highlights the effectiveness of leveraging a pre-trained diffusion model as a generative prior to the LLIE task.
- –
- –
- –
- Anti-Counterfeit Technology of Halftone Image Watermarking by Digital Screening–
The experimental results showed that the variable angle algorithm of screening anti-counterfeit technology had nice security effect, and could be used in practical application.
- –
- Softwarl Register Code Based on ECDSA–
Based on elliptic curve crypto-system, a software registration scheme is put forward which not only can achieve the effect that single software in a computer has just one register code, but also has more sophisticated eryptosystem than traditional software registration methods due to its security based on the difficulties of elliptic Curve discrete logarithm problems.
- Design of Workflow Management System in Distributed Collaborative Modeling & Simulation Environment–
An object-oriented design based on OMG workflow management facility specification is introduced, which supports soft workflow structure and also supports sub-flows with unlimited levels.
Publication data from OpenAlex, with missing venues and authors filled in from Crossref; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-12. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.
Report an error
Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.