Is this you? Claim this profile to correct it, add a bio and choose the work people see first.
Claim this profileWorks69 from public data
- UNITER: UNiversal Image-TExt Representation Learning2,731
UNITER, a UNiversal Image-TExt Representation, learned through large-scale pre-training over four image-text datasets is introduced, which can power heterogeneous downstream V+L tasks with joint multimodal embeddings.
- Large-Scale Adversarial Training for Vision-and-Language Representation Learning561
To enable large-scale training, VILLA adopts the "free" adversarial training strategy, and combines it with KL-divergence-based regularization to promote higher invariance in the embedding space.
- UNITER: Learning UNiversal Image-TExt Representations479
UNITER, a UNiversal Image-TExt Representation, learned through large-scale pre-training over four image-text datasets is introduced, which can power heterogeneous downstream V+L tasks with joint multimodal embeddings.
- StoryGAN: A Sequential Conditional GAN for Story Visualization312
A new story-to-image-sequence generation model, StoryGAN, based on the sequential conditional GAN framework is proposed, which outperformed state-of-the-art models in image quality, contextual consistency metrics, and human evaluation.
- Adversarial Robustness: From Self-Supervised Pre-Training to Fine-Tuning301
This work introduces adversarial training into self-supervision, to provide general-purpose robust Pretrained models for the first time, and finds these robust pretrained models can benefit the subsequent fine-tuning in two ways: i) boosting final model robustness; ii) saving the computation cost, if proceeding towards adversarial fine- Tuning.
- Behind the Scene: Revealing the Secrets of Pre-trained Vision-and-Language Models144
VALUE (Vision-And-Language Understanding Evaluation), a set of meticulously designed probing tasks generalizable to standard pre-trained V+L models, aiming to decipher the inner workings of multimodal pre-training.
- A Survey of Attacks on Large Vision–Language Models: Resources, Advances, and Future Trends130
This article provides a comprehensive review of the various forms of existing LVLM attacks, and systematically review the development of LVLM attack methods, such as adversarial attacks that manipulate model outputs, jailbreak attacks that exploit model vulnerabilities for unauthorized actions, and data poisoning that affects model training.
- UC2: Universal Cross-lingual Cross-modal Vision-and-Language Pre-training114
UC2, the first machine translation-augmented framework for cross-lingual cross-modal representation learning, is introduced, to tackle the scarcity problem of multilingual captions for image datasets and facilitate the learning of a joint embedding space of images and all languages of interest.
- Sequential Attention GAN for Interactive Image Editing114
A novel Sequential Attention Generative Adversarial Network (SeqAttnGAN) is proposed, which applies a neural state tracker to encode the previous image and the textual description in each turn of the sequence, and uses a GAN framework to generate a modified version of the image that is consistent with the preceding images and coherent with the description.
- Multi-step Reasoning via Recurrent Dual Attention for Visual Dialog111
This paper presents a new model for visual dialog, Recurrent Dual Attention Network (ReDAN), using multi-step reasoning to answer a series of questions about an image, and demonstrates that ReDAN can locate context-relevant visual and textual clues via iterative refinement, which can lead to the correct answer step-by-step.
- Fine-grained Iterative Attention Network for Temporal Language Localization in Videos106
A Fine-grained Iterative Attention Network (FIAN) that consists of an iterative attention module for bilateral query-video in-formation extraction and a content-oriented localization strategy instead of applying recent anchor-based localization is proposed.
- Memory-Guided Semantic Learning Network for Temporal Sentence Grounding99
A memory-augmented network, called Memory-Guided Semantic Learning Network (MGSL-Net), that learns and memorizes the rarely appeared content in TSG task and substantially improves the accuracy not only on the entire dataset but also on the rare cases.
- Meta Module Network for Compositional Visual Reasoning85
This work proposes Meta Module Network (MMN), a more powerful NMN architecture centered on a novel meta module, which can take in function recipes and morph into diverse instance modules dynamically, inheriting the strong explainability and compositionality of NMN.
- FreeLB: Enhanced Adversarial Training for Language Understanding82
A novel adversarial training algorithm - FreeLB - is proposed, that promotes higher robustness and invariance in the embedding space, by adding adversarial perturbations to word embeddings and minimizing the resultant adversarial risk inside different regions around input samples.
- Violin: A Large-Scale Dataset for Video-and-Language Inference81
A new large-scale dataset, named Violin (VIdeO-and-Language INference), is introduced for this task, which consists of 95,322 video-hypothesis pairs from 15,887 video clips, spanning over 582 hours of video.
- What Makes A Good Story? Designing Composite Rewards for Visual Storytelling76
This paper proposes three assessment criteria: relevance, coherence and expressiveness, which are observed through empirical analysis could constitute a “high-quality” story to the human eye and proposes a reinforcement learning framework, ReCo-RL, with reward functions designed to capture the essence of these quality criteria.
- 72
- Unsupervised Temporal Video Grounding with Deep Semantic Clustering62
This paper proposes a novel Deep Semantic Clustering Network (DSCNet) to leverage all semantic information from the whole query set to compose the possible activity in each video for grounding and demonstrates that the DSCNet achieves competitive performance, and even outperforms most weakly-supervised approaches.
- 58
- Benchmarking all-atom biomolecular structure prediction with FoldBench54
FoldBench is introduced, an extensive benchmark dataset consisting of 1522 biological assemblies categorized into nine distinct prediction tasks, providing crucial insights and benchmarks to inform future model development and refinement, and highlighting significant advancements yet revealing persistent limitations within the field.
- 51
- Adversarial Category Alignment Network for Cross-domain Sentiment Classification50
An adversarial category alignment network (ACAN) is proposed, which attempts to enhance category consistency between the source domain and the target domain, and increases the discrepancy of two polarity classifiers to provide diverse views.
- BachGAN: High-Resolution Image Synthesis From Salient Object Layout44
The proposed Background Hallucination Generative Adversarial Network (BachGAN), which leverages a background retrieval module to first select a set of segmentation maps from a large candidate pool, then encodes these candidate layouts via a background fusion module to hallucinate a suitable background for the given objects.
- Deep Co-Attention Network for Multi-View Subspace Learning39
A deep co-attention network for multi-view subspace learning is proposed, which aims to extract both the common information and the complementary information in an adversarial setting and provide robust interpretations behind the prediction to the end-users via the co-Attention mechanism.
- 39
Publication data from OpenAlex, with missing venues and authors filled in from Crossref; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-15. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.
Report an error
Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.