Is this you? Claim this profile to correct it, add a bio and choose the work people see first.
Claim this profileAcademic lineage
Possible advisorsa guess from early papers, not confirmed
- Tamara L. BergSuggested from co-authorship
Is this you? Claim this profile to confirm or dismiss it.
Works124 from public data
- UNITER: UNiversal Image-TExt Representation Learning2,731
UNITER, a UNiversal Image-TExt Representation, learned through large-scale pre-training over four image-text datasets is introduced, which can power heterogeneous downstream V+L tasks with joint multimodal embeddings.
- Modeling Context in Referring Expressions1,889
This work focuses on incorporating better measures of visual context into referring expression models and finds that visual comparison to other objects within an image helps improve performance significantly.
- MAttNet: Modular Attention Network for Referring Expression Comprehension1,003
This work proposes to decompose expressions into three modular components related to subject appearance, location, and relationship to other objects, which allows for flexibly adapt to expressions containing different types of information in an end-to-end framework.
- TVQA: Localized, Compositional Video Question Answering821
This paper presents TVQA, a large-scale video QA dataset based on 6 popular TV shows, and provides analyses of this new dataset as well as several baselines and a multi-stream end-to-end trainable neural network framework for the TVZA task.
- HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-training573
HELP, a novel framework for large-scale video+language omni-representation learning that achieves new state of the art on multiple benchmarks over Text-based Video/Video-moment Retrieval, Video Question Answering (QA), Video-and-language Inference and Video Captioning tasks across different domains is presented.
- UNITER: Learning UNiversal Image-TExt Representations479
UNITER, a UNiversal Image-TExt Representation, learned through large-scale pre-training over four image-text datasets is introduced, which can power heterogeneous downstream V+L tasks with joint multimodal embeddings.
- Learning to Navigate Unseen Environments: Back Translation with Environmental Dropout422
This paper presents a generalizable navigational agent, trained in two stages via mixed imitation and reinforcement learning, outperforming the state-of-art approaches by a large margin on the private unseen test set of the Room-to-Room task, and achieving the top rank on the leaderboard.
- TVR: A Large-Scale Dataset for Video-Subtitle Moment Retrieval388
The proposed XML model uses a late fusion design with a novel Convolutional Start-End detector (ConvSE), surpassing baselines by a large margin and with better efficiency, providing a strong starting point for future work.
- A Joint Speaker-Listener-Reinforcer Model for Referring Expressions295
A unified framework for the tasks of referring expression comprehension and generation is proposed, composed of three modules: speaker, listener, and reinforcer, which achieves state-of-the-art results for both comprehension andgeneration on three referring expression datasets.
- TVQA+: Spatio-Temporal Grounding for Video Question Answering293
By performing this joint task, the proposed Spatio-Temporal Answerer with Grounded Evidence with STAGE model is able to produce insightful and interpretable spatio-temporal attention visualizations.
- 184
- 148
- Behind the Scene: Revealing the Secrets of Pre-trained Vision-and-Language Models144
VALUE (Vision-And-Language Understanding Evaluation), a set of meticulously designed probing tasks generalizable to standard pre-trained V+L models, aiming to decipher the inner workings of multimodal pre-training.
- Multi-Target Embodied Question Answering137
This work presents a generalization of EQA -- Multi-Target EQA (MT-EQA), and proposes a modular architecture composed of a program generator, a controller, a navigator, and a VQA module that can outperform previous methods and strong baselines by a significant margin.
- Physics-Inspired Garment Recovery from a Single-View Image115
This work proposes a method that is able to compute a 3D model of a human body and its outfit from a single photograph with little human interaction and demonstrates the effectiveness of the algorithm by re-purposing the reconstructed garments for virtual try-on and garment transfer applications and for cloth animation on digital characters.
- AVID: Any-Length Video Inpainting with Diffusion Model105
Any-Length Video Inpainting with Diffusion Model, dubbed as AVID, is introduced, equipped with effective motion modules and adjustable structure guidance, for fixed-length video inpainting and a novel Temporal MultiDiffusion sampling pipeline with middle-frame attention guidance mechanism, facilitating the generation of videos with any desired duration.
- Apollo: An Exploration of Video Understanding in Large Multimodal Models101
A comprehensive study that helps uncover what effectively drives video understanding in Large Multimodal Models (LMMs), and introduces Apollo, a state-of-the-art family of LMMs that achieve superior performance across different model sizes.
- FlowVid: Taming Imperfect Optical Flows for Consistent Video-to-Video Synthesis94
A consistent V2V synthesis framework by jointly leveraging spatial conditions and temporal optical flow clues within the source video to encode the optical flow via warping from the first frame and serve it as a supplementary reference in the diffusion model.
- What is More Likely to Happen Next? Video-and-Language Future Event Prediction89
This work collects a new dataset, named Video-and-Language Event Prediction (VLEP), with 28,726 future event prediction examples (along with their rationales) from 10,234 diverse TV Show and YouTube Lifestyle Vlog video clips, and presents a strong baseline incorporating information from video, dialogue, and commonsense knowledge.
- FAME-ViL: Multi-Tasking Vision-Language Model for Heterogeneous Fashion Tasks86
This work proposes a novel FAshion-focused Multi-task Efficient learning method for Vision-and-Language tasks (FAME-ViL), which applies a single model for multiple heterogeneous fashion tasks, therefore being much more parameter-efficient.
- Single NIR Laser-Activated Multifunctional Nanoparticles for Cascaded Photothermal and Oxygen-Independent Photodynamic Therapy85
The results exhibit that Bi2Se3@AIPH NPs possesses 31.2% photothermal conversion efficiency for enhanced PTT, ideal free radical generation for oxygen-independent PDT, and 37.77 HU mL mg−1 X-ray attenuation coefficient for CT imaging with high quality.
- Violin: A Large-Scale Dataset for Video-and-Language Inference81
A new large-scale dataset, named Violin (VIdeO-and-Language INference), is introduced for this task, which consists of 95,322 video-hypothesis pairs from 15,887 video clips, spanning over 582 hours of video.
- 74
- 70
- Hierarchically-Attentive RNN for Album Summarization and Storytelling69
This work addresses the problem of end-to-end visual storytelling by using the Visual Storytelling dataset and a model composed of three hierarchically-attentive Recurrent Neural Nets to encode the album photos, select representative photos, and compose the story.
Publication data from OpenAlex, with missing venues and authors filled in from Crossref; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-16. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.
Report an error
Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.