Is this you? Claim this profile to correct it, add a bio and choose the work people see first.
Claim this profileWorks80 from public data
- DeeperForensics-1.0: A Large-Scale Dataset for Real-World Face Forgery Detection648
The on-going effort of constructing a large- scale benchmark for face forgery detection is presented, with 60, 000 videos constituted by a total of 17.6 million frames, 10 times larger than existing datasets of the same kind.
- MEAD: A Large-Scale Audio-Visual Dataset for Emotional Talking-Face Generation522
An emotional talking-face generation baseline that enables the manipulation of both emotion and its intensity and could be used for a number of research projects including conditional generation, cross-modal understanding and expression recognition.
- Focal Frequency Loss for Image Reconstruction and Synthesis520
It is shown that narrowing gaps in the frequency domain can ameliorate image reconstruction and synthesis quality further, and a novel focal frequency loss is proposed, which allows a model to adaptively focus on frequency components that are hard to synthesize by down-weighting the easy ones.
- Bi-directional Cross-Modality Feature Propagation with Separation-and-Aggregation Gate for RGB-D Semantic Segmentation478
A unified and efficient Cross-modality Guided Encoder to not only effectively recalibrate RGB feature responses, but also to distill accurate depth information via multiple stages and aggregate the two recalibrated representations alternatively.
- Pose-Controllable Talking Face Generation by Implicitly Modularized Audio-Visual Representation474
This paper proposes a clean yet effective framework to generate pose-controllable talking faces whose poses are controllable by other videos and has multiple advanced capabilities including extreme view robustness and talking face frontalization.
- OmniObject3D: Large-Vocabulary 3D Object Dataset for Realistic Perception, Reconstruction and Generation469
OmniObject3D, a large vocabulary 3D object dataset with massive high-quality real-scanned 3D objects, is proposed, facilitating the development of 3D perception, reconstruction, and generation in the real world.
- MotionBERT: A Unified Perspective on Learning Human Motion Representations428
A pretraining stage in which a motion encoder is trained to recover the underlying 3D motion from noisy partial 2D observations is proposed, which achieves state-of-the-art performance on all three downstream tasks by simply finetuning the pretrained motion encoding with a simple regression head, which demonstrates the versatility of the learned motion representations.
- TAM: Temporal Adaptive Module for Video Recognition388
A new temporal adaptive module (TAM) to generate video-specific temporal kernels based on its own feature map that outperforms other temporal modeling methods consistently, and achieves the state-of-the-art performance under the similar complexity.
- Audio-Driven Emotional Video Portraits300
EVP is presented, a system for synthesizing high-quality video portraits with vivid emotional dynamics driven by audios, using the Cross-Reconstructed Emotion Disentanglement technique to decompose speech into two decoupled spaces.
- EAMM: One-Shot Emotional Talking Face via Audio-Based Emotion-Aware Motion Model250
The Emotion-Aware Motion Model (EAMM) is proposed to generate one-shot emotional talking faces by involving an emotion source video and an Implicit Emotion Displacement Learner to represent emotion-related facial dynamics as linearly additive displacements to the previously acquired motion representations.
- ReenactGAN: Learning to Reenact Faces via Boundary Transfer229
The proposed method, known as ReenactGAN, is capable of transferring facial movements and expressions from an arbitrary person's monocular video input to a target person’s video, and can perform photo-realistic face reenactment.
- Semantic-Aware Implicit Neural Audio-Driven Video Portrait Generation177
Semantic-aware Speaking Portrait NeRF (SSP-NeRF), which creates delicate audio-driven portraits using one unified set of NeRF through two semantic-aware modules, and renders more realistic video portraits compared to previous methods.
- Everybody’s Talkin’: Let Me Talk as You Want172
A method to edit a target portrait footage by taking a sequence of audio as input to synthesize a photo-realistic video, which is end-to-end learnable and robust to voice variations in the source audio.
- Learning Hierarchical Cross-Modal Association for Co-Speech Gesture Generation169
This work proposes a novel framework named Hierarchical Audio-to-Gesture (HA2G) for co-speech gesture generation, and develops a contrastive learning strategy based on audio-text alignment for better audio representations.
- 144
- Text2Human141
Extensive quantitative and qualitative evaluations demonstrate that the proposed Text2Human framework can generate more diverse and realistic human images compared to state-of-the-art methods.
- DNA-Rendering: A Diverse Neural Actor Repository for High-Fidelity Human-centric Rendering131
This work presents DNA-Rendering, a large-scale, high-fidelity repository of human performance data for neural actor rendering, and provides a large-scale and quantitative benchmark in full-scale, with multiple tasks to evaluate the existing progress of novel view synthesis, novel pose animation synthesis, and novel identity rendering methods.
- CelebV-Text: A Large-Scale Facial Text-Video Dataset129
The effectiveness and potential of CelebV- Text are shown through extensive self-evaluation, and a benchmark is constructed with representative methods to standardize the evaluation of the facial text-to-video generation task.
- MonoHuman: Animatable Human Neural Field from Monocular Video122
A novel framework MonoHuman is proposed, which robustly renders view-consistent and high-fidelity avatars under arbitrary novel poses and is able to model the deformation field with bi-directional constraints and explicitly leverage the off-the-peg keyframe information to reason the feature correlations for coherent results.
- 122
- TransGaGa: Geometry-Aware Unsupervised Image-To-Image Translation114
A novel disentangle-and-translate framework to tackle the complex objects image-to-image translation task, which disentangles image space into a Cartesian product of the appearance and the geometry latent spaces and supports multimodal translation.
- Aggregation via Separation: Boosting Facial Landmark Detector With Semi-Supervised Style Translation95
A new perspective of facial landmark detection is investigated and it is demonstrated it leads to further notable improvement and surprisingly outperforms the fully-supervised model by a large margin.
- SynBody: Synthetic Dataset with Layered Human Models for 3D Human Perception and Modeling83
A new synthetic dataset, SynBody, with three appealing features: a clothed parametric human model that can generate a diverse range of subjects; the layered human representation that naturally offers high-quality 3D annotations to support multiple tasks; and a scalable system for producing realistic data to facilitate real-world tasks.
- Text2Performer: Text-Driven Human Video Generation78
This work decomposes the VQVAE latent space into human appearance and pose representation in an unsupervised manner by utilizing the nature of human videos, and proposes continuous VQ-diffuser to sample a sequence of pose embeddings.
- Make a Face: Towards Arbitrary High Fidelity Face Manipulation78
This work proposes Additive Focal Variational Auto-encoder (AF-VAE), a novel approach that can arbitrarily manipulate high-resolution face images using a simple yet effective model and only weak supervision of reconstruction and KL divergence losses.
- TransEditor: Transformer-Based Dual-Space GAN for Highly Controllable Facial Editing67
This work proposes TransEditor, a novel Transformer-based framework to enhance interaction in a dual-space GAN for more controllable editing, and develops a new dual- space editing and inversion strategy to provide additional editing flexibility.
- TransMoMo: Invariance-Driven Unsupervised Video Motion Retargeting66
A lightweight video motion retargeting approach TransMoMo that is capable of transferring motion of a person in a source video realistically to another video of a target person by exploiting invariance properties of three orthogonal factors of variation including motion, structure, and view-angle.
- RenderMe-360: A Large Digital Asset Library and Benchmarks Towards High-fidelity Head Avatars57
This paper presents RenderMe-360, a comprehensive 4D human head dataset to drive advance in head avatar research, with 16 state-of-the-art methods performed on five main tasks: novel view synthesis, novel expression synthesis, hair rendering, hair editing, and talking head generation.
- Vid2Sim: Realistic and Interactive Simulation from Video for Urban Navigation51
Vid2Sim is proposed, a novel framework that effectively bridges the sim2real gap through a scalable and cost-efficient real2sim pipeline with 3D scene reconstruction and simulation to enable the reinforcement learning of visual navigation agents in complex urban environments.
- Generalizable Neural Performer: Learning Robust Radiance Fields for Human Novel View Synthesis49
This work presents a simple yet powerful framework, named Generalizable Neural Performer (GNR), that learns a generalizable and robust neural body representation over various geometry and appearance, and introduces an Implicit Geometric Body Embedding strategy to enhance the robustness based on both parametric 3D human body model and multi-view images hints.
- CosmicMan: A Text-to-Image Foundation Model for Humans44
This work proposes to model the relationship between dense text descriptions and image pixels in a decomposed manner, and presents Decomposed-Attention-Refocusing (Daring) training framework, which seamlessly decomposes the cross-attention features in existing text-to-image diffusion model, and enforces attention refocusing without adding extra modules.
- FAB: A Robust Facial Landmark Detection Framework for Motion-Blurred Videos39
A framework named FAB is proposed that takes advantage of structure consistency in the temporal dimension for facial landmark detection in motion-blurred videos, and a structure predictor is proposed to predict the missing face structural information temporally, which serves as a geometry prior.
- Joint Optimization for 4D Human-Scene Reconstruction in the Wild33
A novel optimization-based method for 4D human-scene reconstruction in the wild from monocular videos, JOSH, which achieves better results on both global human motion estimation and dense scene reconstruction by joint optimization of scene geometry and human motion.
- 29
- Progressive Attention on Multi-Level Dense Difference Maps for Generic Event Boundary Detection28
An effective and end-to-end learnable framework (DDM-Net) to tackle the diversity and complicated semantics of event boundaries, and demonstrates the effectiveness of richer motion representation and more sophisticated aggregation, in handling the diversity of GEBD.
- 27
- MoCaNet: Motion Retargeting In-the-Wild via Canonicalization Networks26
A novel framework that brings the 3D motion retargeting task from controlled environments to in-the-wild scenarios and could serve as a disentangled and interpretable representation of human motion that benefits action analysis and motion retrieval.
- 23
- Learning Unified Decompositional and Compositional NeRF for Editable Novel View Synthesis20
A unified Neural Radiance Field (NeRF) framework to effectively perform joint scene decomposition and composition for modeling real-world scenes, outperforming state-of-the-art methods for both novel-view synthesis and editing tasks.
- Audio-Driven Dubbing for User Generated Contents via Style-Aware Semi-Parametric Synthesis20
This paper investigates an audio-driven dubbing method that is more feasible for User Generated Content (UGC) production, and develops a semi-parametric video renderer, which takes full advantage of the limited training data of the unseen speaker via a video-level retrieve-warp-refine pipeline.
- HyperStyle3D: Text-Guided 3D Portrait Stylization via Hypernetworks19
This paper proposes a novel method, dubbed HyperStyle3D, based on 3D-aware GANs for 3D portrait stylization, which offers a strong capacity to handle multiple styles with a single model, but also enables flexible fine-grained stylization that affects only texture, shape, or local part of the portrait.
- 3DHumanGAN: 3D-Aware Human Image Generation with 3D Pose Mapping19
3DHumanGAN is a 3D-aware generative adversarial network that synthesizes photo-like images of full-body humans with consistent appearances under different view-angles and body-poses and can incorporate the 3D human prior and enable pose conditioning.
- UnitedHuman: Harnessing Multi-Source Data for High-Resolution Human Generation18
This work designs a Multi-Source Spatial Transformer that spatially aligns multi-source images to full-body space with a human parametric model and proposes an end-to-end framework, UnitedHuman, that empowers continuous GAN with the ability to effectively utilize multi-source data for high-resolution human generation.
- 18
- ReliTalk: Relightable Talking Portrait Generation from a Single Video17
ReliTalk is proposed, a novel framework for relightable audio-driven talking portrait generation from monocular videos that involves 3D facial priors derived from audio features to predict delicate normal maps through implicit functions and decomposes the portrait’s reflectance from implicitly learned audio-driven facial normals and images.
- MetaUrban: An Embodied AI Simulation Platform for Urban Micromobility16
MetaUrban, a compositional simulation platform for Embodied AI research in urban spaces, is presented and experiments demonstrate that the compositional nature of the simulated environments can substantially improve the generalizability and safety of the trained mobile agents.
- StyleFaceV: Face Video Generation via Decomposing and Recomposing Pretrained StyleGAN315
A principled framework named StyleFaceV, which produces high-fidelity identity-preserving face videos with vivid movements, and a temporal-dependent model is built upon the decomposed latent features, and samples reasonable sequences of motions that are capable of generating realistic and temporally coherent face videos.
- 15
- 13
- 12
- DeepFakes Detection: the DeeperForensics Dataset and Challenge11
This chapter presents the on-going effort of constructing DeeperForensics-1.0, a large-scale forgery detection dataset, and describes the observations obtained from organizing DeeperForensics Challenge 2020, a real-world face forgery detection competition based on DeeperForensics-1.0.
- MEAT: Multiview Diffusion Model for Human Generation on Megapixels with Mesh Attention9
This paper explores human multiview diffusion models at the megapixel level and introduces a solution called mesh attention to enable training at 10242 resolution, and presents valuable insights into applying multiview human motion videos for diffusion training, addressing the longstanding issue of data scarcity.
- OrthoPlanes: A Novel Representation for Better 3D-Awareness of GANs9
This work proposes a hybrid explicit-implicit representation called OrthoPlanes, which encodes fine-grained 3D information in feature maps that can be efficiently generated by modifying 2D StyleGANs.
- Talking Faces: Audio-to-Video Face Generation9
This chapter discusses the definition and underlying challenges of the talking face generation, and presents an overview of recent progress, as well as introducing some widely used datasets and performance metrics.
- 9
- UrbanVerse: Scaling Urban Simulation by Watching City-Tour Videos8
Experiments show that UrbanVerse scenes preserve real-world semantics and layouts, achieving human-evaluated realism comparable to manually crafted scenes, in urban navigation and in urban navigation policies trained in UrbanVerse exhibit scaling power laws and strong generalization.
- Learning to Generate Diverse Pedestrian Movements from Web Videos with Noisy Labels7
A generative model called PedGen that contains a novel context encoder that lifts the 2D scene context to 3D and can incorporate various context factors in generating realistic pedestrian movements in urban scenes and achieves zero-shot generalization in both real-world and simulated environments.
- 7
- 6
- Learning Sidewalk Autopilot from Multi-Scale Imitation with Corrective Behavior Expansion5
This work proposes a framework that advances IL through corrective behavior expansion and multi-scale imitation learning and introduces a multi-scale IL architecture that captures both short-horizon interactive behaviors and long-horizon goal-directed intentions via horizon-based trajectory clustering and hierarchical supervision.
- VLG: General Video Recognition with Web Textual Knowledge5
A unified visual-linguistic framework (VLG) to solve the problem of GVR with an effective two-stage training paradigm is presented, and extensive results show that the VLG obtains the state-of-the-art performance under four settings.
- Parameterization-Driven Neural Surface Reconstruction for Object-Oriented Editing in Neural Rendering4
A novel neural algorithm for parameterizing neural implicit surfaces to simple parametric domains like spheres and polycubes that computes bi-directional deformation between the object and the domain using a forward mapping from the object's zero level set and an inverse deformation for backward mapping.
- 4
- 4
- 3
- Localin Reshuffle Net: Toward Naturally and Efficiently Facial Image Blending3
A novel local-reshuffle-layer is designed to map local patches in the feature space, which can be learned in an end-to-end fashion with dedicated loss and is essential for the proposed Local Reshuffle Network (LRNet).
- SidewalkBench: Benchmarking Visual Navigation on Urban Sidewalks2
This work proposes SidewalkBench, a comprehensive benchmark designed for visual navigation on urban sidewalks, built upon NVIDIA Isaac Sim, which brings GPU-accelerated simulation of diverse, high-fidelity sidewalk environments, including both procedurally generated and real-world scanned scenes.
- AURA: Multimodal Shared Autonomy for Real-World Urban Navigation2
AURA is a new multi-modal framework that decomposes urban navigation into high-level human instruction and low-level AI control, and effectively follows human instructions, reduces manual operation effort, and improves navigation stability, while enabling online adaptation.
- Submission to Generic Event Boundary Detection Challenge@CVPR 2022: Local Context Modeling and Global Boundary Decoding Approach1
A local context modeling sub-network is proposed to perceive diverse patterns of generic event boundaries, and it generates powerful video representations and reliable boundary confidence, and global boundary decoding sub- network is exploited to decode event boundaries from a global view.
- 1
- –
- –
- –
- –
- –
- NJU MCG - Sensetime Team Submission to Pre-training for Video Understanding Challenge Track II–
This paper follows the basic pipeline of temporal segment networks and uses the latest transformer-based architectures, e.g., Swin Transformer, DeiT, CLIP-ViT, to enhance the representation power and derive an ensemble of deep models to attain a high classification accuracy.
- –
- –
- –
- –
Publication data from OpenAlex, with missing venues and authors filled in from Crossref; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-16. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.
Report an error
Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.