Is this you? Claim this profile to correct it, add a bio and choose the work people see first.
Claim this profileAcademic lineage
Possible advisorsa guess from early papers, not confirmed
- Suggested from co-authorship
Is this you? Claim this profile to confirm or dismiss it.
Works175 from public data
- Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations6,796
The Visual Genome dataset is presented, which contains over 108K images where each image has an average of 35 objects and contains dense annotations of objects, attributes, and relationships within each image to learn these models.
- 2,188
- Dense-Captioning Events in Videos1,676
This work proposes a new model that is able to identify all events in a single pass of the video while simultaneously describing the detected events with natural language, and introduces a new captioning module that uses contextual information from past and future events to jointly describe all events.
- Visual Relationship Detection with Language Priors1,268
This work proposes a model that can scale to predict thousands of types of relationships from a few examples and improves on prior work by leveraging language priors from semantic word embeddings to finetune the likelihood of a predicted relationship.
- 1,264
- TIFA: Accurate and Interpretable Text-to-Image Faithfulness Evaluation with Question Answering523
The introduction of TIFA (Text-to-Image Faithfulness evaluation with question Answering), an automatic evaluation metric that measures the faithfulness of a generated image to its text input via visual question answering (VQA), and a comprehensive evaluation of existing text-to-image models using TIFA v1.0.
- Action Genome: Actions As Compositions of Spatio-Temporal Scene Graphs455
This work introduces Action Genome, a representation that decomposes actions into spatio-temporal scene graphs and demonstrates the utility of a hierarchical event decomposition by enabling few-shot action recognition, achieving 42.7% mAP using as few as 10 examples.
- Explanations Can Reduce Overreliance on AI Systems During Decision-Making433
This paper argues that people strategically choose whether or not to engage with an AI explanation, demonstrating empirically that there are scenarios where AI explanations reduce overreliance, and formalizes this strategic choice in a cost-benefit framework, where the costs and benefits of engaging with the task are weighed against the costs of relying on the AI.
- A Hierarchical Approach for Generating Descriptive Image Paragraphs416
A model that decomposes both images and paragraphs into their constituent parts is developed, detecting semantic regions in images and using a hierarchical recurrent neural network to reason about language.
- Generating Semantically Precise Scene Graphs from Textual Descriptions for Improved Image Retrieval375
It is shown that scene graphs can be effectively created automatically from a natural language scene description and that using the output of the parsers is almost as effective as using human-constructed scene graphs.
- Holodeck: Language Guided Generation of 3D Embodied AI Environments374
Large-scale human evaluation shows that annotators prefer Holodeck over manually designed procedural baselines in residential scenes and that Holodeck can produce high-quality outputs for diverse scene types, which is a significant step forward in developing general-purpose embodied agents.
- Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language Models315
Molmo is presented, a new family of VLMs that are state-of-the-art in their class of openness, with a best-in-class 72B model that not only outperforms others in the class of open weight and data models, but also outperforms larger proprietary models.
- 306
- NVILA: Efficient Frontier Visual Language Models260
NVILA, a family of open VLMs designed to optimize both efficiency and accuracy, is introduced by improving its model architecture by first scaling up the spatial and temporal resolutions, and then compressing visual tokens, which enables NVILA to efficiently process high-resolution images and long videos.
- @ CREPE: Can Vision-Language Foundation Models Reason Compositionally?238
A new compositionality evaluation benchmark, @ CREPE, is introduced, which measures two important aspects of compositionality identified by cognitive science literature: system-aticity and productivity.
- 235
- Conceptual Metaphors Impact Perceptions of Human-AI Collaboration192
It is suggested that projecting competence may help attract new users, but those users may discard the agent unless it can quickly correct with a lower competence metaphor, and that users are drawn to systems that project higher competence and warmth.
- AGQA: A Benchmark for Compositional Spatio-Temporal Reasoning177
Using AGQA, modern visual reasoning systems are evaluated, demonstrating that the best models barely perform better than non-visual baselines exploiting linguistic biases and that none of the existing models generalize to novel compositions unseen during training.
- Quilt-LLaVA: Visual Instruction Tuning by Extracting Localized Narratives from Open-Source Histopathology Videos139
This work introduces QUILT-INSTRUCT, a large-scale dataset of107, 131 histopathology-specific instruction question/answer pairs, grounded within diagnostically relevant image patches that make up the WSI, and trains QUILT-LLAVA, which can reason beyond the given single image patch, enabling diagnostic reasoning across patches.
- Information Maximizing Visual Question Generation104
It is argued that a good question is one that has a tightly focused purpose --- one that is aimed at expecting a specific type of response, and a model is built that maximizes mutual information between the image, the expected answer and the generated question.
- Mind Your Outliers! Investigating the Negative Impact of Outliers on Active Learning for Visual Question Answering102
It is shown that active learning sample efficiency increases significantly as the number of collective outliers in the active learning pool decreases, and prescriptive recommendations for mitigating the effects of these outliers are made.
- Referring Relationships97
An iterative model is introduced that localizes the two entities in the referring relationship by modelling predicates that connect the entities as shifts in attention from one entity to another, and it is demonstrated that this model can not only outperform existing approaches on three datasets but also that it produces visually meaningful predicate shifts, as an instance of interpretable neural networks.
- Visual Program Distillation: Distilling Tools and Programmatic Reasoning into Vision-Language Models96
VPD is proposed, an instruction tuning framework that produces a vision-language model (VLM) capable of solving complex visual tasks with a single forward pass and improves the VLM's ability to count, understand spatial relations, and reason compositionally.
- Scene Graph Prediction with Limited Labels94
This paper introduces a semi-supervised method that assigns probabilistic relationship labels to a large number of unlabeled images using few labeled examples and defines a complexity metric for relationships that serves as an indicator for conditions under which the method succeeds over transfer learning, the de-facto approach for training with limited labels.
- 93
Publication data from OpenAlex, with missing venues and authors filled in from Crossref; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-13. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.
Report an error
Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.