Is this you? Claim this profile to correct it, add a bio and choose the work people see first.
Claim this profileWorks42 from public data
- Hallo: Hierarchical Audio-Driven Visual Synthesis for Portrait Image Animation241
This research delves into the complexities of synchronizing facial movements and creating visually appealing, temporally consistent animations within the framework of diffusion-based methodologies, and introduces a hierarchical audio-driven visual synthesis module to enhance the precision of alignment between audio inputs and visual outputs.
- Hallo3: Highly Dynamic and Realistic Portrait Image Animation with Video Diffusion Transformer147
This paper introduces the first application of a pretrained transformer-based video generative model that demonstrates strong generalization capabilities and generates highly dynamic, realistic videos for portrait animation, effectively addressing these challenges.
- Real-Time Neural Radiance Talking Portrait Synthesis via Audio-Spatial Decomposition137
An efficient NeRF-based framework is proposed, which enables faster convergence and real-time synthesizing of stable talking portraits, by utilizing the recent success of grid-based NeRF.
- VRP-SAM: SAM with Visual Reference Prompt131
A novel Visual Reference Prompt encoder that empowers the Segment Any-thing Model (SAM) to utilize annotated reference images as prompts for segmentation, creating the VRP-SAM model, which achieves a breakthrough within the SAM framework by extending its versatility and applicabil-ity while preserving SAM's inherent strengths, thus enhancing user-friendliness.
- Hallo2: Long-Duration and High-Resolution Audio-Driven Portrait Image Animation123
Hallo2, proposed in this paper, is the first method to achieve 4K resolution and generate hour-long, audio-driven portrait image animations enhanced with textual prompts, and achieves 4K resolution portrait video generation.
- A Survey of Reasoning with Foundation Models73
This paper introduces seminal foundation models proposed or adaptable for reasoning, highlighting the latest advancements in various reasoning tasks, methods, and benchmarks and discusses the relevance of multimodal learning, autonomous agents, and super alignment in the context of reasoning.
- Exploring Effective Factors for Improving Visual In-Context Learning68
This paper proposes a simple framework prompt-SelF, which outperformed OSLSM method-based meta-learning in 1-shot segmentation for the first time and indicated the great potential of visual in-context learning.
- Learning to Rematch Mismatched Pairs for Robust Cross-Modal Retrieval40
The potential semantic similarity among unpaired samples makes it possible to excavate useful knowledge from mismatched pairs and the proposed L2RM, a general framework based on Optimal Transport that learns to rematch mismatched pairs is proposed.
- GP-NeRF: Generalized Perception NeRF for Context-Aware 3D Scene Understanding37
Generalized Perception NeRF is proposed, a novel pipeline that makes the widely used segmentation model and NeRF work compatibly under a unified framework, for facilitating context-aware 3D scene perception and introduces transformers to aggregate radiance as well as semantic embedding fields jointly for novel views and facilitates the joint volumetric rendering of both fields.
- Open-sourced Data Ecosystem in Autonomous Driving: the Present and Future35
This comprehensive review, conducted in collaboration with esteemed colleagues from both academia and industry, systematically assesses over seventy open-source autonomous driving datasets from domestic and international sources and offers insights into various aspects, such as the principles underlying the creation of high-quality datasets.
- Re-HOLD: Video Hand Object Interaction Reenactment via adaptive Layout-instructed Diffusion Model24
This work presents a novel video reenactment framework focusing on Human-Object Interaction (HOI) via an adaptive Layout-instructed Diffusion model (Re-HOLD), and demonstrates that the proposed framework significantly outperforms existing methods.
- 24
- VoxelSplat: Dynamic Gaussian Splatting as an Effective Loss for Occupancy and Flow Prediction15
This paper proposes a novel regularization framework called VoxelSplat, which leverages recent developments in 3D Gaussian Splatting to enhance model performance in two key ways: Enhanced Semantics Supervision through 2D Projection and Scene Flow Learning.
- Perception Before Reasoning: Two-Stage Reinforcement Learning for Visual Reasoning in Vision-Language Models14
A two-stage reinforcement learning framework designed to jointly enhance both the perceptual and reasoning capabilities of vision-language models (VLMs) is proposed and PeBR-R1, a vision-language model with significantly enhanced perceptual and reasoning capabilities is obtained.
- 10
- Automatic Classification of Bug Reports Based on Multiple Text Information and Reports’ Intention9
A new automatic classification method for bug reports is proposed that when categorizing bug reports, in addition to using the text information of the report, the intention of the report is also considered, thereby improving the performance of the classification.
- 7
- Continual SFT Matches Multimodal RLHF with Negative Supervision6
The nSFT disentangles this negative supervision in RLHF paradigm, and continually aligns VLMs with a simple SFT loss, which is more memory efficient than multimodal RLHF where 2 or 4 large VLMs are strictly required.
- 5
- 4
- 3
- 3
- 3
- 2
- 1
Publication data from OpenAlex; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-14. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.
Report an error
Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.