Is this you? Claim this profile to correct it, add a bio and choose the work people see first.
Claim this profileAcademic lineage
Possible advisorsa guess from early papers, not confirmed
- Jason J. CorsoSuggested from co-authorship
Is this you? Claim this profile to confirm or dismiss it.
Works173 from public data
- Towards Automatic Learning of Procedures From Web Instructional Videos1,112
A segment-level recurrent network is proposed for generating procedure segments by modeling the dependencies across segments and it is shown that the proposed model outperforms competitive baselines in procedure segmentation.
- TDAN: Temporally-Deformable Alignment Network for Video Super-Resolution655
Experimental results demonstrate that theTDAN is capable of alleviating occlusions and artifacts for temporal alignment and the TDAN-based VSR model outperforms several recent state-of-the-art VSR networks with a comparable or even much smaller model size.
- 494
- Hierarchical Cross-Modal Talking Face Generation With Dynamic Pixel-Wise Loss480
A cascade GAN approach to generate talking face video, which is robust to different face shapes, view angles, facial characteristics, and noisy audio conditions, and compared to a direct audio-to-image approach, this approach avoids fitting spurious correlations between audiovisual signals that are irrelevant to the speech content.
- 333
- Video Understanding With Large Language Models: A Survey332
The emergent capabilities of Vid-LLMs are surprisingly advanced, particularly their ability for open-ended multi-granularity (abstract, temporal, and spatiotemporal) reasoning combined with common-sense knowledge, suggesting a promising path for future video understanding.
- Lip Movements Generation at a Glance331
This paper devise a method to fuse audio and image embeddings to generate multiple lip images at once and propose a novel correlation loss to synchronize lip changes and speech changes and significantly outperforms other state-of-the-art methods extended to this task.
- Learning to Answer Questions in Dynamic Audio-Visual Scenarios295
The results demonstrate that AVQA benefits from multisensory perception and the model outperforms recent A-, V-, andAVQA approaches.
- Streaming Hierarchical Video Segmentation262
This work proposes an approximation framework for streaming hierarchical video segmentation motivated by data stream algorithms: each video frame is processed only once and does not change the segmentation of previous frames.
- Deep Cross-Modal Audio-Visual Generation235
This paper uses conditional generative adversarial networks to achieve cross-modal audio-visual generation of musical performances and demonstrates that the model has the ability to generate one modality from the other modality, i.e., visual/audio, to a good extent.
- Unified Multisensory Perception: Weakly-Supervised Audio-Visual Video Parsing234
Experimental results show that the challenging audio-visual video parsing can be achieved even with only video-level weak labels, and the proposed framework can effectively leverage unimodal and cross-modal temporal contexts and alleviate modality bias and noisy labels problems.
- 231
- Zooming Slow-Mo: Fast and Accurate One-Stage Space-Time Video Super-Resolution214
A one-stage space-time video super-resolution framework is proposed, which directly synthesizes an HR slow-motion video from an LFR, LR video and proposes a deformable ConvLSTM to align and aggregate temporal information simultaneously for better leveraging global temporal contexts.
- Talking-Head Generation with Rhythmic Head Motion213
This work proposes a 3D-aware generative network along with a hybrid embedding module and a non-linear composition module that achieves controllable, photo-realistic, and temporally coherent talking-head videos with natural head movements.
- Weakly-Supervised Action Segmentation with Iterative Soft Boundary Assignment204
A novel action modeling framework is proposed, which consists of a new temporal convolutional network, named Temporal Convolutional Feature Pyramid Network (TCFPN), for predicting frame-wise action labels, and a novel training strategy for weakly-supervised sequence modeling, named Iterative Soft Boundary Assignment (ISBA), to align action sequences and update the network in an iterative fashion.
- 138
- MRI tumor segmentation with densely connected 3D CNN137
This work develops a 3D Convolutional Neural Network (3D CNN) based model to automatically segment gliomas and proposes a novel architecture that hierarchically segments different lesion regions of the necrotic and non-enhancing tumor (NCR/NET), peritumoral edema (ED) and GD-enhanced tumor (ET).
- 130
- Transformer-empowered Multi-scale Contextual Matching and Aggregation for Multi-contrast MRI Super-resolution124
A novel network of innovative Transformer-empowered multi-scale contextual matching and aggregation techniques is proposed, which outperforms state-of-the-art approaches and has great potential to be applied in clinical practice.
- A Whac-A-Mole Dilemma: Shortcuts Come in Multiples Where Mitigating One Amplifies Others114
The Last Layer Ensemble is proposed, a simple-yet-effective method to mitigate multiple shortcuts without Whac-A-Mole behavior, which surface multi-shortcut mitigation as an overlooked challenge critical to advancing the reliability of vision systems.
- Cyclic Co-Learning of Sounding Object Visual Grounding and Sound Separation99
This paper proposes a cyclic co-learning (CCoL) paradigm that can jointly learn sounding object visual grounding and audio-visual sound separation in a unified framework and improves training example sampling for sounding object grounding, which builds a co- learning cycle for the two tasks and makes them mutually beneficial.
- Learning to Generate Scene Graph from Natural Language Supervision96
This paper proposes one of the first methods that learn from image-sentence pairs to extract a graphical representation of localized objects and their relationships within an image, known as scene graph, and designs a Transformer-based model to predict these "pseudo" labels via a masked token prediction task.
- Watch What You Just Said: Image Captioning with Text-Conditional Attention85
The experimental results show that the proposed novel attention mechanism, called text-conditional attention, outperforms state-of-the-art captioning methods on various quantitative metrics as well as in human evaluation, which supports the use of the text- Conditional attention in image captioning.
- Automated classification of big X-ray diffraction data using deep learning models70
Given the critical need for high-throughput automated XRD pattern analysis, this work presents a generalized deep learning model to classify a diverse set of materials’ crystal systems and space groups and optimize model architecture to elicit classification based on Bragg’s Law.
- Procedure Planning in Instructional Videos via Contextual Modeling and Model-based Policy Learning68
This work leverage instructional videos to study humans’ decision-making processes, focusing on learning a model to plan goal-directed actions in real-life videos and proposes novel algorithms to model human behaviors through Bayesian Inference and model-based Imitation Learning.
Publication data from OpenAlex, with missing venues and authors filled in from Crossref; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-16. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.
Report an error
Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.