Is this you? Claim this profile to correct it, add a bio and choose the work people see first.
Claim this profileWorks117 from public data
- UNITER: UNiversal Image-TExt Representation Learning2,731
UNITER, a UNiversal Image-TExt Representation, learned through large-scale pre-training over four image-text datasets is introduced, which can power heterogeneous downstream V+L tasks with joint multimodal embeddings.
- Less is More: CLIPBERT for Video-and-Language Learning via Sparse Sampling836
Experiments on text-to-video retrieval and video question answering on six datasets demonstrate that CLIPBERT outperforms (or is on par with) existing methods that exploit full-length videos, suggesting that end- to-end learning with just a few sparsely sampled clips is often more accurate than using densely extracted offline features from full- length videos, proving the proverbial less-is-more principle.
- Emu3: Next-Token Prediction is All You Need811
Emmu3 is introduced, a new suite of state-of-the-art multimodal models trained solely with next-token prediction that outperforms several well-established task-specific models in both generation and perception tasks, surpassing flagship models such as SDXL and LLaVA-1.6.
- HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-training573
HELP, a novel framework for large-scale video+language omni-representation learning that achieves new state of the art on multiple benchmarks over Text-based Video/Video-moment Retrieval, Video Question Answering (QA), Video-and-language Inference and Video Captioning tasks across different domains is presented.
- Large-Scale Adversarial Training for Vision-and-Language Representation Learning561
To enable large-scale training, VILLA adopts the "free" adversarial training strategy, and combines it with KL-divergence-based regularization to promote higher invariance in the embedding space.
- Multispectral Deep Neural Networks for Pedestrian Detection542
This work carefully design four ConvNet fusion architectures that integrate two-branch ConvNets on different DNNs stages, all of which yield better performance compared with the baseline detector, and finds that ConvNet-based pedestrian detectors trained by color or thermal images separately provide complementary information in discriminating human instances.
- Relation-Aware Graph Attention Network for Visual Question Answering400
A Relation-aware Graph Attention Network (ReGAT), which encodes each image into a graph and models multi-type inter-object relations via a graph attention mechanism, to learn question-adaptive relation representations.
- StoryGAN: A Sequential Conditional GAN for Story Visualization312
A new story-to-image-sequence generation model, StoryGAN, based on the sequential conditional GAN framework is proposed, which outperformed state-of-the-art models in image quality, contextual consistency metrics, and human evaluation.
- 240
- TVT: Transferable Vision Transformer for Unsupervised Domain Adaptation188
This paper first comprehensively investigates the performance of ViT on a variety of domain adaptation tasks, and proposes an unified framework, namely Transferable Vision Transformer (TVT), to fully exploit the transferability ofViT for domain adaptation.
- Emu: Generative Pretraining in Multimodality169
Emu, a Transformer-based multimodal foundation model, is presented, which can seamlessly generate images and texts in multi-modality context through a one-model-for-all autoregressive training process and demonstrates superb performance compared to state-of-the-art large multimodAL models.
- Behind the Scene: Revealing the Secrets of Pre-trained Vision-and-Language Models144
VALUE (Vision-And-Language Understanding Evaluation), a set of meticulously designed probing tasks generalizable to standard pre-trained V+L models, aiming to decipher the inner workings of multimodal pre-training.
- Language-Based Image Editing with Recurrent Attentive Models143
A generic modeling framework for two subtasks of LBIE: language-based image segmentation and image colorization using recurrent attentive models to fuse image and language features is proposed.
- Wasserstein Contrastive Representation Distillation121
Wasserstein Contrastive Representation Distillation (WCoRD) is proposed, which leverages both primal and dual forms of Wasserstein distance for KD, yielding a contrastive learning objective that maximizes the lower bound of mutual information between the teacher and the student networks.
- Insulator Faults Detection in Aerial Images from High-Voltage Transmission Lines Based on Deep Learning Model119
The experimental results and analysis demonstrate that the proposed CSPD-YOLO model performs better in insulator fault detection from high-voltage transmission lines with a complex background.
- UC2: Universal Cross-lingual Cross-modal Vision-and-Language Pre-training114
UC2, the first machine translation-augmented framework for cross-lingual cross-modal representation learning, is introduced, to tackle the scarcity problem of multilingual captions for image datasets and facilitate the learning of a joint embedding space of images and all languages of interest.
- Sequential Attention GAN for Interactive Image Editing114
A novel Sequential Attention Generative Adversarial Network (SeqAttnGAN) is proposed, which applies a neural state tracker to encode the previous image and the textual description in each turn of the sequence, and uses a GAN framework to generate a modified version of the image that is consistent with the preceding images and coherent with the description.
- Sequential Attention GAN for Interactive Image Editing via Dialogue.114
A novel Sequential Attention Generative Adversarial Network (SeqAttnGAN) is proposed, which applies a neural state tracker to encode the previous image and the textual description in each turn of the sequence, and uses a GAN framework to generate a modified version of the image that is consistent with the preceding images and coherent with the description.
- Improved YOLOv3 Network for Insulator Detection in Aerial Images with Diverse Background Interference112
Experimental results and analysis validate the claim that the proposed YOLOv3-dense network achieves good performance in the detection of different size insulators amid diverse background interference.
- Multi-step Reasoning via Recurrent Dual Attention for Visual Dialog111
This paper presents a new model for visual dialog, Recurrent Dual Attention Network (ReDAN), using multi-step reasoning to answer a series of questions about an image, and demonstrates that ReDAN can locate context-relevant visual and textual clues via iterative refinement, which can lead to the correct answer step-by-step.
- Target Detection of Forward-Looking Sonar Image Based on Improved YOLOv5109
The experimental results show that the improved model based on YOLOv5 network is superior to the original YOL olv5 network and other popular deep neural networks for target detection in the forward-looking sonar image, which has a reference significance for underwater target detection.
- MTI-YOLO: A Light-Weight and Real-Time Deep Neural Network for Insulator Detection in Complex Aerial Images105
This work proposes the modified YOLO-tiny for insulator (MTI-YOLO) network, which achieves good performance in both complex backgrounds and bright illumination conditions and improves the detection accuracy of different sizes of insulator.
- LightningDOT: Pre-training Visual-Semantic Embeddings for Real-Time Image-Text Retrieval95
A simple yet highly effective approach, LightningDOT is proposed that accelerates the inference time of ITR by thousands of times, without sacrificing accuracy, and achieves superior performance across mainstream ITR benchmarks such as Flickr30k and COCO datasets.
- A Hybrid Retrieval-Generation Neural Conversation Model94
This paper proposes a hybrid neural conversation model that combines the merits of both response retrieval and generation methods and shows that the proposed model outperforms both retrieval- based methods and generation-based methods under both automatic evaluation metrics and human evaluation.
- Contrastive Distillation on Intermediate Representations for Language Model Compression87
CoDIR is proposed, a principled knowledge distillation framework where the student is trained to distill knowledge through intermediate layers of the teacher via a contrastive objective, and achieves superb performance on the GLUE benchmark, outperforming state-of-the-art compression methods.
Publication data from OpenAlex, with missing venues and authors filled in from Crossref; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-14. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.
Report an error
Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.