Is this you? Claim this profile to correct it, add a bio and choose the work people see first.
Claim this profileAcademic lineage
Possible advisorsa guess from early papers, not confirmed
- Qi Chuan TianSuggested from co-authorship
- Jinhui TangSuggested from co-authorship
Is this you? Claim this profile to confirm or dismiss it.
Works63 from public data
- Egocentric Video-Language Pretraining306
The recently released Ego4D dataset is exploited to pioneer Egocentric VLP along three directions, and a novel pretraining objective is proposed, dubbed EgoNCE, which adapts video-text contrastive learning to the egocentric domain by mining egOCentric-aware positive and negative samples.
- All in One: Exploring Unified Video-Language Pre-Training258
This work introduces an end-to-end VLP model, namely all-in-one Transformer, that embeds raw video and textual signals into joint representations using a unified backbone architecture and introduces a novel and effective token rolling operation to encode temporal representations from video clips in a non-parametric manner.
- 239
- 231
- Expansion-Squeeze-Excitation Fusion Network for Elderly Activity Recognition195
This work attempts to effectively aggregate the discriminative information of actions and interactions from both RGB videos and skeleton sequences by attentively fusing multi-modal features by implementing a novel Expansion-Squeeze-Excitation Fusion Network (ESE-FN).
- Participation-Contributed Temporal Dynamic Model for Group Activity Recognition142
A novel Participation-Contributed Temporal Dynamic Model (PC-TDM) to recognize group activity, which mainly consists of a "One" network and a " one to Key" network, which aims at modeling the individual dynamic of each person.
- M3Net: Multi-view Encoding, Matching, and Fusion for Few-shot Fine-grained Action Recognition134
M3Net is proposed, a matching-based framework for FS-FG action recognition, which incorporates multi-view encoding, multi- view matching, and multi-View fusion to facilitate embedding encoding, similarity matching,and decision making across multiple viewpoints.
- Social Adaptive Module for Weakly-Supervised Group Activity Recognition121
This paper presents a new task named weakly-supervised group activity recognition (GAR) which differs from conventional GAR tasks in that only video-level labels are available, yet the important persons within each frame are not provided even in the training data.
- 109
- Object-aware Video-language Pre-training for Retrieval92
Object-aware Transformers is presented, an object-centric approach that extends video-language transformer to incorporate object representations to leverage the bounding boxes and object tags to guide the training process.
- Dual-AI: Dual-path Actor Interaction Learning for Group Activity Recognition87
This work proposes a distinct Dual-path Actor Interaction (Dual-AI) framework, which flexibly arranges spatial and temporal transformers in two complementary orders, enhancing actor relations by integrating merits from different spatio-temporal paths, and introduces a novel Multi-scale Actor Contrastive Loss (MAC-Loss) between two interactive paths of Dual-AI.
- MVP-Shot: Multi-Velocity Progressive-Alignment Framework for Few-Shot Action Recognition69
A novel Multi-Velocity Progressive-alignment (MVP-Shot) framework to progressively learn and align semantic-related action features at multi-velocity levels and outperforms current state-of-the-art methods on multiple standard few-shot benchmarks.
- 63
- Audio Deep Fake Detection System with Neural Stitching for ADD 202233
This paper describes the best system and methodology for ADD 2022: The First Audio Deep Synthesis Detection Challenge and outperforms all other systems with 10.1% equal error rate(EER) in Track 3.2.
- Video-Text Pre-training with Learned Regions for Retrieval30
This work proposes a simple yet effective module for video-text representation learning, namely RegionLearner, which can take into account the structure of objects during pre-training on large-scale video- text pairs and is much more computationally efficient.
- Spatio-Temporal Decoupled Knowledge Compensator for Few-Shot Action Recognition26
DiST, an innovative Decomposition-incorporation framework for FSAR that makes use of decoupled Spatial and Temporal knowledge provided by large language models to learn expressive multi-granularity prototypes, is proposed.
- 24
- GPT4Ego: Unleashing the Potential of Pre-Trained Models for Zero-Shot Egocentric Action Recognition24
A straightforward yet remarkably potent VLM framework, GPT4Ego, designed to enhance the fine-grained alignment of concept and description between vision and language, which can sustainably evolve with the advancement of ever-growing pre-trained foundational models.
- DVF: Advancing Robust and Accurate Fine-Grained Image Retrieval with Retrieval Guidelines21
This paper designs a novel Dual Visual Filtering mechanism for the plain visual transformer, denoted as DVF, to capture subcategory-specific discrepancies and implements a discriminative model training strategy to improve the discriminability and generalization ability of DVF.
- 20
- Look Less Think More: Rethinking Compositional Action Recognition19
A simple yet effective framework, Look Less Think More (LLTM), to reduce the strong association between visual objects and action-level labels (Look Less), and then discover the commonsense relationships between object categories and human actions (Think More).
- DiffusionVMR: Diffusion Model for Joint Video Moment Retrieval and Highlight Detection17
A novel framework, DiffusionVMR, is proposed to redefine the two tasks as a unified conditional denoising generation process by combining the diffusion model, inheriting the advantages of diffusion models that allow for iteratively refined results during inference, enhancing the boundary transition from coarse to fine.
- VideoExpert: Augmented LLM for Temporal-Sensitive Video Understanding16
VideoExpert is a general-purpose MLLM suitable for several temporal-sensitive video tasks and integrates two parallel modules: the Temporal Expert and the Spatial Expert, which handles specially designed spatial tokens and language input, aiming to generate content-related responses.
- 15
- 15
- 15
- 14
- MUP: Multi-granularity Unified Perception for Panoramic Activity Recognition13
A novel Multi-granularity Unified Perception (MUP) framework that perceives different granularity behaviors universally to explore the co-occurrence motion pattern via the same parameters in an end-to-end fashion is proposed.
- 13
- Skip-attention encoder–decoder framework for human motion prediction13
A novel skip-attention encoder–decoder (SAED) framework to model human motion dependences in spatiotemporal space, by utilizing the encoder and decoder to encode the observed motions, and decode the predicted motions, respectively is proposed.
- 12
- 10
- Storyboard relational model for group activity recognition10
A Storyboard Relational Model (SRM) is proposed to address the problem of Group Activity Recognition by splitting and reintegrating the group activity based on the small yet compact Storyboards.
- 9
- Foreground/Background-Masked Interaction Learning for Spatio-temporal Action Detection8
A novel Foreground/Background-masked Interaction Learning framework to learn the multi-actor features by attentively interacting with the hands-down foreground and background frames and a new Actor-guided Feature Aggregation scheme that integrates foreground- and background-interacted actor features with the learnable actor-based weights are presented.
- Egocentric Video-Language Pretraining @ Ego4D Challenge 20228
A pretrained video-language model that is able to transfer its egocentric video-text representation or video-only representation to several video downstream tasks and pioneer Egocentric VLP from pretraining dataset, pretraining objective, and development set is developed.
- 6
- 6
- AdaFPP: Adapt-Focused Bi-Propagating Prototype Learning for Panoramic Activity Recognition6
A novel Adapt-Focused bi-Propagating Prototype learning (AdaFPP) framework to jointly recognize individual, group, and global activities in panoramic activity scenes by learning an adapt-focused detector and multi-granularity prototypes as the pretext tasks in an end-to-end way is proposed.
- Revitalize Region Feature for Democratizing Video-Language Pre-training of Retrieval6
This work revitalizes region features of sparsely sampled video clips to significantly reduce both spatial and temporal visual redundancy towards democratizing VLP research at the same time achieving state-of-the-art results.
- 6
- Leveraging Frame- and Feature-level Progressive Augmentation for Semi-supervised Action Recognition5
A new Progressive Augmentation mechanism that implements the weak/strong augmentations first at the frame level, and further implements the perturbation at the feature level, to obtain abundant four types of augmented features in broader perturbation spaces is designed.
- Slowfast Diversity-aware Prototype Learning for Egocentric Action Recognition5
A new Part-to-Prototype (P2P) scheme to learn prototypes from raw videos covering the interacting objects by refining the semantic information from part level to prototype level is presented, and a new Slow-Fast Context (SFC) mechanism is designed that explores the Up/Down augmentations for the prototype representation at the semantic level.
- 3
- Boosting Weakly-Supervised Image Segmentation via Representation, Transform, and Compensator3
A novel single-stage WSIS method that utilizes a siamese network with contrastive learning to improve the quality of class activation maps (CAMs) and achieve a self-refinement result that significantly outperforms other state-of-the-art methods.
- 3
- Attack-Augmentation Mixing-Contrastive Skeletal Representation Learning3
In A 2 MC, Attack-Augmentation (Att-Aug) is designed to collaboratively perform targeted and untargeted perturbations of skeletons via attack and augmentation respectively, for generating high-quality hard positive features and hard negative features.
- Dilation-erosion for single-frame supervised temporal action localization3
The Snippet Classification model and the Dilation-Erosion module are presented, which relies on the single-frame annotation and the output of the snippet classification, and a new embedding loss is proposed to aggregate the features of action instances with the same label and separate the Features of actions from backgrounds.
- Reliable and Diverse Hierarchical Adapter for Zero-shot Video Classification2
This work proposes a reliable and diverse Hierarchical Adapter for zero-shot video classification, which consists of Frame-level Cache Refiner and Video-level Cache Updater, and updates the cache during inference based on the principle of diversity.
- Egocentric Video-Language Pretraining @ EPIC-KITCHENS-100 Multi-Instance Retrieval Challenge 20222
A pretrained video-language model that is able to transfer its egocentric video-text representation to MIR benchmark is developed and an adaptive multi-instance max-margin loss is devised to effectively fine-tune the model and equip the dual-softmax technique for reliable inference.
- 1
- Continual Learning meets Multimodal Foundation Models: Fundamentals and Advances1
This workshop aims to provide a venue where academic researchers and industry practitioners can come together to discuss the principles, limitations and applications of multimodal foundation models in continual learning for multimedia applications, and promote the understanding of multimodal foundation models in continual learning.
- 1
- 1-Norm Projection Twin Support Vector Machine1
This paper introduces a Tikhonov regularization term to the objective function of projection twin support vector machine (PTSVM) and constructs an unconstrained convex programming problem according to the exterior penalty (EP) theory.
- –
- –
- –
- –
- Time-IC: Empowering MLLM with Interleaved Context for Temporal-Sensitive Video Understanding–
Time-IC is proposed, a multimodal large language model that incorporates video context for improved understanding that integrates both internal context (timestamps, speech transcripts) and external context (titles, author descriptions), providing complementary semantic cues and high-level semantic priors.
- –
- –
- Cross-modal Token Selection for Video Understanding–
This work introduces a novel transformer-based multi-modal architecture that outperforms existing state-of-the-art methods while significantly reducing the computational cost and extensive ablation studies showcase the benefits of the improved method over naive approaches.
- –
Publication data from OpenAlex, with missing venues and authors filled in from Crossref; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-12. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.
Report an error
Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.