Is this you? Claim this profile to correct it, add a bio and choose the work people see first.
Claim this profileWorks79 from public data
- LanguageBind: Extending Video-Language Pretraining to N-modality by Language-based Semantic Alignment490
This work proposes LanguageBind, taking the language as the bind across different modalities because the language modality is well-explored and contains rich semantics, and freezes the language encoder acquired by VL pretraining, then train encoders for other modalities with contrastive learning.
- Recurrent Fusion Network for Image Captioning286
This paper proposes a novel recurrent fusion network (RFNet) for the image captioning task, which can exploit the interactions among the outputs of the image encoders and generate new compact and informative representations for the decoder.
- VideoMoCo: Contrastive Video Representation Learning with Temporally Adversarial Examples274
This paper improves the temporal feature representations of MoCo from two perspectives and improves MoCo temporally based on contrastive learning by empowering the temporal robustness of the encoder and modeling the temporal decay of the keys.
- Bidirectional Attentive Fusion with Context Gating for Dense Video Captioning239
This work proposes a bidirectional proposal method that effectively exploits both past and future contexts to make proposal predictions, and proposes a novel context gating mechanism to balance the contributions from the current event and its surrounding contexts dynamically.
- 236
- Temporally Grounding Language Queries in Videos by Contextual Boundary-Aware Prediction212
This work proposes an end-to-end boundary-aware model, which uses a lightweight branch to predict semantic boundaries corresponding to the given linguistic information, and outperforms its competitors with a clear margin on three public datasets.
- Controllable Video Captioning With POS Sequence Guidance Based on Gated Fusion Network189
A gating strategy is proposed to dynamically and adaptively incorporate the global syntactic POS information into the decoder for generating each word, which not only boosts the video captioning performance but also improves the diversity of the generated captions.
- Learning Modality Interaction for Temporal Sentence Localization and Event Captioning in Videos114
This work proposes a novel method for learning pairwise modality interactions in order to better exploit complementary information for each pair of modalities in videos and thus improve performances on both tasks.
- Regularizing RNNs for Caption Generation by Reconstructing the Past with the Present97
A novel architecture, namely Auto-Reconstructor Network (ARNet), which, coupling with the conventional encoder-decoder framework, works in an end-to-end fashion to generate captions, and can help regularize the transition dynamics of recurrent neural networks (RNNs).
- Learning to Guide Decoding for Image Captioning78
An extension of the encoder-decoder framework by adding a component called guiding network, which models the attribute properties of input images, and its output is leveraged to compose the input of the decoder at each time step.
- Robust dictionary learning with capped l 1 -norm78
Aiming at learning dictionaries resistant to outliers, this paper proposed capped l1-norm based dictionary learning and an efficient iterative re-weighted algorithm to solve the problem.
- VLMixer: Unpaired Vision-Language Pre-training via Cross-Modal CutMix63
A new unpaired VLP method, dubbed as VLMixer, is presented, that integrates CMC with contrastive learning to pull together the uni- modal and multi-modal views for better instance-level alignments among different modalities.
- SWEM: Towards Real-Time Video Object Segmentation with Sequential Weighted Expectation-Maximization61
A novel Sequential Weighted Expectation-Maximization (SWEM) network to greatly reduce the redundancy of memory features and maintains a fixed number of template features in memory, which ensures the stable inference complexity of the VOS system.
- Transfer Spectral Clustering56
This paper proposes a new algorithm called transfer spectral clustering (TSC), which makes use of co-clustering to achieve and control the knowledge transfer among tasks and can greatly improve the clustering performance when compared with other state-of-the-art clustering algorithms.
- 35
- Bidirectional image-sentence retrieval by local and global deep matching33
The proposed local and global deep matching model outperforms the state-of-the-art bidirectional retrieval approaches on the Flickr8K, Flickr30K, and MSCOCO datasets and the image and sentence representations exploited in local andglobal levels are demonstrated to play synergic and complementary roles for biddirectional image-sentence retrieval.
- A Deep Bayesian Tensor-Based System for Video Recommendation32
An empirical study for video recommendation demonstrates the superiority of the proposed deep Bayesian probabilistic tensor framework and indicates that it can better capture the latent patterns of interactions and generates interesting recommendations based on creative tag combinations.
- 30
- The l2,1-Norm Stacked Robust Autoencoders for Domain Adaptation30
The experimental results demonstrate that the proposed method is very effective in multiple cross domain classification datasets which include Amazon review dataset, spam dataset from ECML/PKDD discovery challenge 2006 and 20 newsgroups dataset.
- Hierarchical Photo-Scene Encoder for Album Storytelling29
The proposed model can be trained in an end-to-end manner, which results in an improved performance over the state-of-the-arts on the public visual storytelling (VIST) dataset.
- 28
- Knowledge transfer for spectral clustering26
A novel algorithm called transfer spectral clustering (TSC) is proposed, which assumes the feature embeddings being shared with the auxiliary tasks and utilizes co-clustering to extract useful information from the auxiliary datasets to improve the clustering performance.
- A trace ratio maximization approach to multiple kernel-based dimensionality reduction26
A multiple kernel learning framework for dimensionality reduction based on regularized trace ratio, termed as MKL-TR, which demonstrates its effectiveness in benchmark datasets, for supervised, unsupervised as well as semi-supervised settings.
- 21
- Learning Grounded Vision-Language Representation for Versatile Understanding in Untrimmed Videos17
A grounded vision-language learning framework for untrimmed videos, which automatically detects informative events and effectively excavates the alignments between multi-sentence descriptions and corresponding event segments, and proposes a novel semantic-aware cost to mitigate the sub-optimal matching results caused by ambiguous boundary annotations.
Publication data from OpenAlex, with missing venues and authors filled in from Crossref; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-15. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.
Report an error
Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.