Is this you? Claim this profile to correct it, add a bio and choose the work people see first.
Claim this profileAcademic lineage
Possible advisorsa guess from early papers, not confirmed
- Chi‐Man PunSuggested from co-authorship
- Hao GaoSuggested from co-authorship
Is this you? Claim this profile to confirm or dismiss it.
Works93 from public data
- Uformer: A General U-Shaped Transformer for Image Restoration2,405
Uformer, an effective and efficient Transformer-based architecture for image restoration, in which a hierarchical encoder-decoder network is built using the Transformer block and a learnable multi-scale restoration modulator in the form of a multi- scale spatial bias to adjust features in multiple layers of the Uformer decoder is proposed.
- Generating Human Motion from Textual Descriptions with Discrete Representations798
A simple and must-known conditional generative framework based on Vector Quantised-Variational AutoEncoder (VQ-VAE) and Generative Pre-trained Transformer for human motion generation from textural descriptions shows better performance than competitive approaches, including recent diffusion-based approaches.
- VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Models710
This work explores the training scheme of video models extended from Stable Diffusion and investigates the feasibility of leveraging low-quality videos and synthesized high-quality images to obtain a high-quality video model, resulting in a generic high-quality video model.
- VideoCrafter1: Open Diffusion Models for High-Quality Video Generation633
This model is the first open-source I2V foundation model capable of transforming a given image into a video clip while maintaining content preservation constraints, and can generate realistic and cinematic-quality videos with a resolution of $1024 \times 576.
- FateZero: Fusing Attentions for Zero-shot Text-based Video Editing594
F FateZero, a zero-shot text-based editing method on real-world videos without per-prompt training or use-specific mask, is proposed, which is the first one to show the ability of zero-shot text-driven video style and local attribute editing from the trained text-to-image model.
- SadTalker: Learning Realistic 3D Motion Coefficients for Stylized Audio-Driven Single Image Talking Face Animation542
SadTalker is presented, which generates 3D motion coefficients (head pose, expression) of the 3DMM from audio and implicitly modulates a novel 3D-aware face render for talking head generation and designs PoseVAE via a conditional VAE to synthesize head motion in different styles.
- EvalCrafter: Benchmarking and Evaluating Large Video Generation Models416
This work argues that it is hard to judge the large conditional generative models from the simple metrics since these models are often trained on very large datasets with multi-aspect abilities, and proposes a novel framework and pipeline for exhaustively evaluating the performance of the generated videos.
- DEIM: DETR with Improved Matching for Fast Convergence394
The Matchability-Aware Loss (MAL), a novel loss function that optimizes matches across various quality levels, enhancing the effectiveness of Dense O2O, is proposed.
- Towards Ghost-Free Shadow Removal via Dual Hierarchical Aggregation Network and Shadow Matting GAN295
A shadow matting generative adversarial network (SMGAN) is designed to synthesize realistic shadow mattings from a given shadow mask and shadow-free image to outperforms other state-of-the-art methods by a large margin.
- CodeTalker: Speech-Driven 3D Facial Animation with Discrete Motion Prior286
This paper proposes to cast speech-driven facial animation as a code query task in a finite proxy space of the learned codebook, which effectively promotes the vividness of the generated motions by reducing the cross-modal mapping uncertainty.
- Explicit Visual Prompting for Low-Level Structure Segmentations249
This work takes inspiration from the widely-used pre-training and then prompt tuning protocols in NLP and proposes a new visual prompting model, named Explicit Visual Prompting (EVP), which significantly outperforms other parameter-efficient tuning protocols under the same amount of tunable parameters.
- DepthCrafter: Generating Consistent Long Depth Sequences for Open-world Videos243
Comprehensive evaluations reveal that DepthCrafter achieves state-of-the-art performance in open-world video depth estimation under zero-shot settings and facilitates various downstream applications, including depth-based visual effects and conditional video generation.
- SmartEdit: Exploring Complex Instruction-Based Image Editing with Multimodal Large Language Models213
A Bidirectional Interaction Module (BIM) is proposed that enables comprehensive bidirectional information interactions between the input image and the MLLM output and demonstrates that a small amount of complex instruction editing data can effectively stimulate SmartEdit’ s editing capabilities for more complex instructions.
- 188
- Make-Your-Video: Customized Video Generation Using Textual and Structural Guidance166
This paper explores customized video generation by utilizing text as context description and motion structure (e.g., frame-wise depth) as concrete guidance as concrete guidance in joint-conditional video generation using a Latent Diffusion Model.
- 165
- Improving the Harmony of the Composite Image by Spatial-Separated Attention Module165
This work proposes a novel attention module named Spatial-Separated Attention Module (S2AM) and designs a novel image harmonization framework by inserting the S2AM in the coarser low-level features of the Unet structure by two different ways.
- ScaleCrafter: Tuning-free Higher-Resolution Visual Generation with Diffusion Models150
This work investigates the capability of generating images from pre-trained diffusion models at much higher resolutions than the training image sizes and proposes the dispersed convolution and noise-damped classifier-free guidance, which can enable ultra-high-resolution image generation.
- 124
- Animate-A-Story: Storytelling with Retrieval-Augmented Video Generation106
This work develops a framework comprised of two functional modules, which provides video candidates with desired scene or motion context described by query texts, and generates plot-aligned videos under the guidance of motion structure and text prompts.
- Interactive Story Visualization with Multiple Characters84
A system for generic interactive story visualization, capable of handling multiple novel characters and supporting the editing of layout and local structure is proposed, developed by leveraging the prior knowledge of large language and T2I models, trained on massive corpora.
- DiTCtrl: Exploring Attention Control in Multi-Modal Diffusion Transformer for Tuning-Free Multi-Prompt Longer Video Generation83
DiTCtrl is proposed, a training-free multi-prompt video generation method under MM-DiT architectures for the first time that achieves state-of-the-art performance without additional training and presents MPVBench, a new benchmark specially designed for multi-prompt video generation to evaluate the performance of multi-prompt generation.
- High-Resolution Document Shadow Removal via A Large-Scale Real-World Dataset and A Frequency-Aware Shadow Erasing Net74
This work handles high-resolution document shadow removal directly via a larger-scale real-world dataset and a carefully-designed frequency-aware network, which shows a clearly better performance than previous methods in terms of visual quality and numerical results.
- DPE: Disentanglement of Pose and Expression for General Video Portrait Editing69
This paper introduces a novel self-supervised disentanglement framework to decouple pose and expression without 3DMMs and paired data, which consists of a motion editing module, a pose generator, and an expression generator and proposes a bidirectional cyclic training strategy with well-designed constraints.
- Split then Refine: Stacked Attention-guided ResUNets for Blind Single Image Visible Watermark Removal69
A novel two-stage framework with a stacked attention-guided ResUNets to simulate the process of detection, removal and refinement of digital watermark, which outperforms other state-of-the-art methods by a large margin.
Publication data from OpenAlex; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-12. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.
Report an error
Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.