Is this you? Claim this profile to correct it, add a bio and choose the work people see first.
Claim this profileAcademic lineage
Possible advisorsa guess from early papers, not confirmed
- Gregory S. ChirikjianSuggested from co-authorship
Is this you? Claim this profile to confirm or dismiss it.
Works218 from public data
- Deep Modular Co-Attention Networks for Visual Question Answering1,023
A deep Modular Co-Attention Network (MCAN) that consists of Modular co-attention layers cascaded in depth that significantly outperforms the previous state-of-the-art models and is quantitatively and qualitatively evaluated on the benchmark VQA-v2 dataset.
- ActivityNet-QA: A Dataset for Understanding Complex Web Videos via Question Answering793
This work introduces ActivityNet-QA, a fully annotated and large scale VideoQA dataset, which consists of 58,000 QA pairs on 5,800 complex web videos derived from the popular ActivityNet dataset and explores various video representation strategies to improve videoQA performance.
- Multi-modal Factorized Bilinear Pooling with Co-attention Learning for Visual Question Answering758
A Multi-modal Factorized Bilinear (MFB) pooling approach to efficiently and effectively combine multi- modal features, which results in superior performance for VQA compared with other bilinear pooling approaches.
- Beyond Bilinear: Generalized Multimodal Factorized High-Order Pooling for Visual Question Answering532
A DNN architecture is designed to integrate all these aforementioned modules into a unified model for achieving superior VQA performance and an ensemble of the authors' MFH models achieve the state-of-the-art performance on the large-scale VQ a data sets and win the runner-up in V QA Challenge 2017.
- Multimodal Transformer With Multi-View Visual Representation for Image Captioning468
Inspired by the success of the Transformer model in machine translation, this work extends it to a Multimodal Transformer (MT) model for image captioning that significantly outperforms the previous state-of-the-art methods.
- Prompting Large Language Models with Answer Heuristics for Knowledge-Based Visual Question Answering211
- Rethinking Diversified and Discriminative Proposal Generation for Visual Grounding150
This paper introduces the diversity and discrimination simultaneously when generating proposals, and in doing so proposes Diversified and Discriminative Proposal Networks model (DDPN), a high performance baseline model for visual grounding and evaluates it on four benchmark datasets.
- Discriminative coupled dictionary hashing for fast cross-media retrieval150
The experiments show that the DCDH and MV-DCDH outperform the state-of-the-art methods significantly on cross-media retrieval and conjecture that a balanced representation is crucial in cross- media retrieval.
- 130
- UC2: Universal Cross-lingual Cross-modal Vision-and-Language Pre-training114
UC2, the first machine translation-augmented framework for cross-lingual cross-modal representation learning, is introduced, to tackle the scarcity problem of multilingual captions for image datasets and facilitate the learning of a joint embedding space of images and all languages of interest.
- Deep Multimodal Neural Architecture Search114
The obtained MMnasNet significantly outperforms existing state-of-the-art approaches across three multimodal learning tasks (over five datasets), including visual question answering, image-text matching, and visual grounding.
- 114
- Strategy and Policy Learning for Non-Task-Oriented Conversational Systems103
A set of generic conversational strategies to handle possible sys-tem breakdowns in non-task-oriented dialog systems and three metrics, which consider both the lo-cal and global quality of the conversation, to evaluate conversation quality are proposed.
- Paraphrase Augmented Task-Oriented Dialog Generation92
A paraphrase augmented response generation (PARG) framework that jointly trains a paraphrase model and a response generation model to improve the dialog generation performance and design a method to automatically construct paraphrase training data set based on dialog state and dialog act labels.
- A Visual Attention Grounding Neural Model for Multimodal Machine Translation87
A novel multimodal machine translation model that utilizes parallel visual and textual information and jointly optimizes the learning of a shared visual-language embedding and a translator that achieves competitive state-of-the-art results on the Multi30K and the Ambiguous COCO datasets.
- Matching and reconstruction of brachytherapy seeds using the Hungarian algorithm (MARSHAL)81
Matching rate and reconstruction accuracy were found to be sufficient for prostate brachytherapy and the algorithm is sufficiently generic to be used for establishing correspondences across any choice of features in different imaging modalities.
- Zero-Shot Dialogue State Tracking via Cross-Task Transfer77
This work proposes TransferQA, a transferable generative QA model that seamlessly combines extractive QA and multi-choice QA via a text-to-text transformer framework, and tracks both categorical slots and non-categorical slots in DST.
- ENHANCING THE PERFORMANCE OF74
Recommendations for the use of Behavioural Markers recommendations for performance measurement in a range of organisational settings, especially in high reliability industries such as nuclear power, rail and maritime transport, and medicine.
- SPRNet: Single-Pixel Reconstruction for One-Stage Instance Segmentation72
This article proposes a one-stage framework, single-pixel reconstruction net (SPRNet), which performs efficient instance segmentation by introducing a single- pixel reconstruction ( SPR) branch to off-the-shelf one- stage detectors.
- 64
- ROSITA: Enhancing Vision-and-Language Semantic Alignments via Cross- and Intra-modal Knowledge Integration63
A new VLP method called ROSITA is introduced, which integrates the cross- and intra-modal knowledge in a unified scene graph to enhance the semantic alignments and introduces a novel structural knowledge masking (SKM) strategy to use the scene graph structure as a priori to perform masked language (region) modeling.
- Open-Ended Long-form Video Question Answering via Adaptive Hierarchical Reinforced Networks61
This paper proposes the adaptive hierarchical encoder network to learn the joint representation of the long-form video contents according to the question with adaptive video segmentation and develops the reinforced decodernetwork to generate the natural language answer for open-ended video question answering.
- A robotic library system for an off-site shelving facility61
An autonomous mobile robotic library system has been developed to retrieve items from bookshelves and carry them to scanning stations located in the off-site shelving facility in order to achieve real-time browsing of printed materials.
- ALICE: Active Learning with Contrastive Natural Language Explanations56
This work proposes Active Learning with Contrastive Explanations (ALICE), an expert-in-the-loop training framework that utilizes contrastive natural language explanations to improve data efficiency in learning and outperform baseline models that are trained with 40-100% more training data.
- Learning Conversational Systems that Interleave Task and Non-Task Content51
Experiments with human users indicate that a system that interleaves social and task content achieves a better task success rate and is also rated as more engaging compared to a pure task-oriented system.
Publication data from OpenAlex, with missing venues and authors filled in from Crossref; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-14. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.
Report an error
Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.