Is this you? Claim this profile to correct it, add a bio and choose the work people see first.
Claim this profileAcademic lineage
Possible advisorsa guess from early papers, not confirmed
- Suggested from co-authorship
Is this you? Claim this profile to confirm or dismiss it.
Works72 from public data
- AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning1,729
The proposed AnimateDiff is a practical framework for animating personalized T2I models without requiring model-specific tuning and MotionLoRA is a lightweight fine-tuning technique for AnimateDiff that enables a pre-trained motion module to adapt to new motion patterns, such as different shot types, at a low training and data collection cost.
- 1,226
- CameraCtrl: Enabling Camera Control for Text-to-Video Generation496
Experimental results demonstrate the effectiveness of CameraCtrl in achieving precise camera control with different video generation models, marking a step forward in the pursuit of dynamic and customized video storytelling from textual and camera pose inputs.
- Temporal Pyramid Network for Action Recognition456
A generic Temporal Pyramid Network (TPN) at the feature-level is proposed, which can be flexibly integrated into 2D or 3D backbone networks in a plug-and-play manner and shows consistent improvements over other challenging baselines on several action recognition datasets.
- LaVie: High-Quality Video Generation with Cascaded Latent Diffusion Models426
The incorporation of simple temporal self-attentions, coupled with rotary positional encoding, adequately captures the temporal correlations inherent in video data and proves that the process of joint image-video fine-tuning plays a pivotal role in producing high-quality and creative outcomes.
- 386
- Seedance 1.0: Exploring the Boundaries of Video Generation Models276
Compared to state-of-the-art video generation models, Seedance 1.0 stands out with high-quality and fast video generation having superior spatiotemporal fluidity with structural stability, precise instruction adherence in complex multi-subject contexts, native multi-shot narrative coherence with consistent subject representation.
- Seedream 4.0: Toward Next-generation Multimodal Image Generation253
This work introduces Seedream 4.0, an efficient and high-performance multimodal image generation system that unifies text-to-image (T2I) synthesis, image editing, and multi-image composition within a single framework, and develops a highly efficient diffusion transformer with a powerful VAE which allows for efficient training of the model.
- Semantic Hierarchy Emerges in Deep Generative Representations for Scene Synthesis216
This work shows that highly-structured semantic hierarchy emerges in the deep generative representations from the state-of-the-art GANs like StyleGAN and BigGAN, trained for scene synthesis, and quantifies the causality between the layer-wise activations and the semantics occurring in the output image.
- Instance Localization for Self-supervised Detection Pretraining169
This paper proposes a new self-supervised pretext task, called instance localization, based on the inherent difference between classification and detection, and shows that integration of bounding boxes into pretraining promotes better task alignment and architecture alignment for transfer learning.
- Pose Guided Human Video Generation163
A pose guided method to synthesize human videos in a disentangled way: plausible motion prediction and coherent appearance generation and enforce semantic consistency between the generated and ground-truth poses at a high feature level.
- Diffusion Adversarial Post-Training for One-Step Video Generation153
This work proposes Adversarial Post-Training (APT) against real data following diffusion pre-training for one-step video generation, and introduces several improvements to the model architecture and training procedures, along with an approximated R1 regularization objective.
- 3D-aware Image Synthesis via Learning Structural and Textural Representations145
This work proposes a novel framework, termed as VolumeGAN, for high-fidelity 3D-aware image synthesis, through explicitly learning a structural representation and a textural representation in a Generative Adversarial Network.
- Accelerating Diffusion Models via Early Stop of the Diffusion Process142
This work proposes a principled acceleration strategy, referred to as Early-Stopped DDPM (ES-DDPM), which obtains promising acceleration effect and performance improvement over representative baseline methods and demonstrates several attractive properties, including being orthogonal to existing acceleration methods, as well as simultaneously enabling both global semantic and local pixel-level control in image generation.
- 130
- Generative Hierarchical Features from Synthesizing Images126
This work considers the pre-trained StyleGAN generator as a learned loss function and utilizes its layer-wise representation to train a novel hierarchical encoder, termed as Generative Hierarchical Feature (GH-Feat), which has strong transferability to both generative and discriminative tasks.
- Long Context Tuning for Video Generation119
This work introduces Long Context Tuning (LCT), a training paradigm that expands the context window of pre-trained single-shot video diffusion models to learn scene-level consistency directly from data, enabling both joint and auto-regressive shot generation without additional parameters.
- CameraCtrl: Enabling Camera Control for Video Diffusion Models107
This paper builds on improved single-clip capabilities of CameraCtrl II by allowing users to iteratively specify camera trajectories for generating coherent video sequences, and enables extended scene exploration by allowing users to iteratively specify camera trajectories for generating coherent video sequences.
- CameraCtrl II: Dynamic Scene Exploration via Camera-Controlled Video Diffusion Models107
This paper builds on improved single-clip capabilities of CameraCtrl II by allowing users to iteratively specify camera trajectories for generating coherent video sequences, and enables extended scene exploration by allowing users to iteratively specify camera trajectories for generating coherent video sequences.
- Penalizing Top Performers: Conservative Loss for Semantic Segmentation Adaptation107
This work proposes a novel loss function, i.e., Conservative Loss, which penalizes the extreme good and bad cases while encouraging the moderate examples and enables the network to learn features that are discriminative by gradient descent and are invariant to the change of domains via gradient ascend method.
- Mixture of Contexts for Long Video Generation94
This work recast long-context video generation as an internal information retrieval task and proposes a simple, learnable sparse attention routing module, Mixture of Contexts (MoC), as an effective long-term memory retrieval engine.
- 92
- Seaweed-7B: Cost-Effective Training of Video Generation Foundation Model90
This technical report presents a cost-efficient strategy for training a video generation foundation model called Seaweed-7B trained from scratch using 665,000 H100 GPU hours and highlights the key design decisions that enhance the performance of the medium-sized diffusion model.
- DisCoScene: Spatially Disentangled Generative Radiance Fields for Controllable 3D-aware Scene Synthesis83
The proposed model spatially disentangles the whole scene into object-centric generative radiance fields by learning on only 2D images with the global-local discrimination, and obtains the generation fidelity and editing flexibility of individual objects while being able to efficiently compose objects and the background into a complete scene.
- SeedVR: Seeding Infinity in Diffusion Transformer Towards Generic Video Restoration75
SeedVR is a diffusion transformer designed to handle real-world video restoration with arbitrary length and resolution, and achieves highly-competitive performance on both synthetic and real-world benchmarks, as well as AI-generated videos.
- Seedance 1.5 pro: A Native Audio-Visual Joint Generation Foundation Model73
This work introduces an acceleration framework that boosts inference speed by over 10X and distinguishes itself through precise multilingual and dialect lip-syncing, dynamic cinematic camera control, and enhanced narrative coherence, positioning it as a robust engine for professional-grade content creation.
- Towards Text-guided 3D Scene Composition70
SceneWiz3D is introduced - a novel approach to synthesize high-fidelity 3D scenes from text by introducing a hybrid 3D representation - explicit for objects and implicit for scenes, and achieves superior quality over previous approaches.
- One-Shot Generative Domain Adaptation70
This work aims to transfer a Generative Adversarial Network pre-trained on one image domain to another domain referred to as few as just one reference image, and proposes to constrain the diversity of the latent space through truncation in the training process, alleviating the optimization difficulty.
- Learning Where to Focus for Efficient Video Object Detection70
A novel module called Learnable Spatio-Temporal Sampling (LSTS) has been proposed to learn semantic-level correspondences among adjacent frame features accurately and achieves state-of-the-art performance on the ImageNet VID dataset with less computational complexity and real-time speed.
- Cross-Model Pseudo-Labeling for Semi-Supervised Action Recognition66
This work proposes a more effective pseudo-labeling scheme, called Cross-Model Pseudo-Labeling (CMPL), which introduces a lightweight auxiliary network in addition to the primary backbone, and asks them to predict pseudo-labels for each other, and observes that these two models tend to learn complementary representations from the same video clips.
- 58
- Captain Cinema: Towards Short Movie Generation53
To support stable and efficient generation of multi-scene long narrative cinematic works, this work introduces an interleaved training strategy for Multimodal Diffusion Transformers (MM-DiT), specifically adapted for long-context video data.
- SeedVR2: One-Step Video Restoration via Diffusion Adversarial Post-Training51
This work proposes a one-step diffusion-based VR model, termed as SeedVR2, which performs adversarial VR training against real data, and introduces several enhancements to both model architecture and training procedures to handle the challenging high-resolution VR within a single step.
- 51
- Scaling Laws For Diffusion Transformers43
Based on the scaling law, the loss of pretraining DiT also follows a power-law relationship with the involved compute and can not only determine the optimal model size and required data but also accurately predict the text-to-image generation loss given a model with 1B parameters and a compute budget of 1e21 FLOPs.
- 39
- Improving GAN Equilibrium by Raising Spatial Awareness35
This work aims to alleviate the issue of D dominating the competition in GANs, and proposes to align the spatial awareness of G with the attention map induced from D, effectively lessen the information gap between D and G.
- 3DitScene: Editing Any Scene via Language-guided Disentangled Gaussian Splatting33
3DitScene is proposed, a novel and unified scene editing framework leveraging language-guided disentangled Gaussian Splatting that enables seamless editing from 2D to 3D, allowing precise control over scene composition and individual objects.
- 31
- LinkGAN: Linking GAN Latents to Pixels for Controllable Image Synthesis30
An easy-to-use regularizer for GAN training, which helps explicitly link some axes of the latent space to a set of pixels in the synthesized image, which can improve the spatial controllability of both 2D and 3D-aware GAN models, barely sacrificing the synthesis performance.
- InterActHuman: Multi-Concept Human Animation with Layout-Aligned Audio Conditions19
This work discards the single-entity assumption and introduces a novel framework that enforces strong, region-specific binding of conditions from modalities to each identity's spatiotemporal footprint and enables the high-quality generation of human dialogue videos between two to three people or video customization from multiple reference images.
- Exploring Sparse MoE in GANs for Text-conditioned Image Synthesis19
Aurora, a GAN-based text-to-image generator that employs a collection of experts to learn feature processing, together with a sparse router to adaptively select the most suitable expert for each feature point, closes the performance gap between GANs and industry-level diffusion models, maintaining a fast inference speed.
- Towards Smooth Video Composition19
This work investigates modeling the temporal relations for composing video with arbitrary length, from a few frames to even infinite, using generative adversarial networks (GANs), and develops a novel B-Spline based motion representation to ensure temporal smoothness to achieve infinite-length video generation.
- 12
- GLeaD: Improving GANs with A Generator-Leading Task12
This work proposes a new paradigm for adversarial training, which makes G assign a task to D as well, and expects D to extract representative features that can be adequately decoded by G to reconstruct the input.
- Edicho: Consistent Image Editing in the Wild10
Edicho steps in with a training-free solution based on diffusion models, featuring a fundamental design principle of using explicit image correspondence to direct editing, and enjoys a plug-and-play nature and is compatible to most diffusion-based editing methods, such as ControlNet and BrushNet.
- Interspatial Attention for Efficient 4D Human Video Generation9
This paper introduces a new interspatial attention (ISA) mechanism as a scalable building block for modern diffusion transformer (DiT)-based video generation models and achieves state-of-the-art performance for 4D human video synthesis.
- SMaRt: Improving GANs with Score Matching Regularity9
This work revisit the mathematical foundations of GANs, and theoretically reveal that the native adversarial loss for GAN training is insufficient to fix the problem of subsets with positive Lebesgue measure of the generated data manifold lying out of the real data manifold, and finds that score matching serves as a promising solution to this issue.
- Context Unrolling in Omni Models8
Omni, a unified multimodal model natively trained on diverse modalities, is presented, where it is found that such training enables Context Unrolling, where the model explicitly reasons across multiple modal representations before producing predictions.
- 7
- BerfScene: Bev-conditioned Equivariant Radiance Fields for Infinite 3D Scene Generation7
A practical and efficient 3D representation that incorporates an equivariant radiance field with the guidance of a bird's-eye view (BEV) map is proposed that allows for large-scale, even infinite-scale, 3D scenes via synthesizing local scenes and then stitching them with smooth consistency.
- Representation Forcing for Bottleneck-Free Unified Multimodal Models6
This paper proposes Representation Forcing (RF), a technique that forces the decoder to autoregressively predict visual representations as intermediate tokens before pixels; these tokens then stay in context to guide pixel diffusion within the same backbone.
- 5
- 4
- 4
- Improving Out-of-Distribution Robustness of Classifiers via Generative Interpolation4
This paper develops a simple yet effective method called Generative Interpolation to fuse generative models trained from multiple domains for synthesizing diverse OoD samples and explicitly increases the diversity of training domains and achieves consistent improvements over baselines across datasets and multiple different distribution shifts.
- [Severity classification of chronic obstructive pulmonary disease based on deep learning].4
This study provides an effective solution for the application of deep learning method in automatic diagnostic decision making by building an automatic classification algorithm of severity of chronic obstructive pulmonary disease based on deep belief network.
- VTok: A Unified Video Tokenizer with Decoupled Spatial-Temporal Latents3
VTok is proposed to decouple the spatial and temporal representations of videos by retaining the spatial features of a single key frame while encoding each subsequent frame into a single residual token, achieving compact yet expressive video tokenization.
- UniMC: Taming Diffusion Transformer for Unified Keypoint-Guided Multi-Class Image Generation3
A DiT-based framework, named UniMC, is designed to explore unifying controllable multi-class image generation, and a large-scale, high-quality, and diverse dataset designed for keypoint-guided human and animal image generation is proposed.
- Spatial Steerability of GANs via Self-Supervision from Discriminator3
This work designs randomly sampled Gaussian heatmaps to be encoded into the intermediate layers of generative models as spatial inductive bias, and incorporates DragGAN into the framework, which facilitates fine-grained manipulation within a reasonable time and supports a coarse-to-fine editing process.
- GenDeF: Learning Generative Deformation Field for Video Generation3
This work proposes to render a video by warping one static image with a generative deformation field (GenDeF), alleviating the difficulty in producing a video and thereby resulting in better visual quality.
- GH-Feat: Learning Versatile Generative Hierarchical Features From GANs3
This work investigates that a generative feature learned from image synthesis exhibits great potentials in solving a wide range of computer vision tasks, including both generative ones and more importantly discriminative ones, and develops GH-Feat, a hierarchical visual feature with multi-level semantics.
- 2
- 1
- 1
- Learning Motion Priors for Efficient Video Object Detection.1
This paper first initialize some motion priors for each location and then use them to propagate features across frames and achieves state-of-the-art performance on the ImageNet VID dataset with real-time speed.
- –
- –
- –
- –
- Explicit Critic Guidance for Aligning Diffusion Models–
A state-aligned latent actor-critic framework for diffusion post-training is proposed, in which the diffusion model serves as its own timestep-conditioned value function and predicts values directly on noisy latent states, which enables trajectory-level PPO training, and supports stable actor-critic optimization with simple conditioning and value pretraining strategies.
- –
Publication data from OpenAlex; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-15. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.
Report an error
Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.