Is this you? Claim this profile to correct it, add a bio and choose the work people see first.
Claim this profileWorks255 from public data
- Vision-and-Language Navigation: Interpreting Visually-Grounded Navigation Instructions in Real Environments2,014
This work provides the first benchmark dataset for visually-grounded natural language navigation in real buildings - the Room-to-Room (R2R) dataset and presents the Matter-port3D Simulator - a large-scale reinforcement learning environment based on real imagery.
- FVQA: Fact-Based Visual Question Answering566
A conventional visual question answering dataset is extended, which contains image-question-answer triplets, through additional image- question-answer-supporting fact tuples, and a novel model is described which is capable of reasoning about an image on the basis of supporting-facts.
- REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environments551
A dataset of varied and complex robot tasks, described in natural language, in terms of objects visible in a large set of real images, and a novel Interactive Navigator-Pointer model is proposed that provides a strong baseline on the task.
- VLN↻BERT: A Recurrent Vision-and-Language BERT for Navigation503
A recurrent BERT model that is time-aware for use in VLN is proposed that can replace more complex encoder-decoder models to achieve state-of-the-art results and can generalised to other transformer-based architectures.
- NavGPT: Explicit Reasoning in Vision-and-Language Navigation with Large Language Models490
The NavGPT is introduced, a purely LLM-based instruction-following navigation agent, to reveal the reasoning capability of GPT models in complex embodied scenes by performing zero-shot sequential action prediction for vision-and-language navigation (VLN).
- What Value Do Explicit High Level Concepts Have in Vision to Language Problems?482
A method of incorporating high-level concepts into the successful CNN-RNN approach is proposed, and it is shown that it achieves a significant improvement on the state-of-the-art in both image captioning and visual question answering.
- Visual question answering: A survey of methods and datasets476
The state of the art by comparing modern approaches to VQA, and the common approach of combining convolutional and recurrent neural networks to map images and questions to a common feature space are examined.
- Image Captioning and Visual Question Answering Based on Attributes and External Knowledge416
A visual question answering model that combines an internal representation of the content of an image with information extracted from a general knowledge base to answer a broad range of image-based questions and allows questions to be asked where the image alone does not contain the information required to select the appropriate answer.
- Medical image classification using synergic deep learning402
The proposed synergic deep learning model using multiple deep convolutional neural networks simultaneously and enabling them to mutually learn from each other achieves the state‐of‐the‐art performance in these medical image classification tasks.
- Fine-Grained Video-Text Retrieval With Hierarchical Graph Reasoning393
A Hierarchical Graph Reasoning (HGR) model is proposed, which decomposes video-text matching into global-to-local levels and generates hierarchical textual embeddings via attention-based graph reasoning.
- Ask Me Anything: Free-Form Visual Question Answering Based on Knowledge from External Sources392
A method for visual question answering which combines an internal representation of the content of an image with information extracted from a general knowledge base to answer a broad range of image-based questions and is specifically able to answer questions posed in natural language, that refer to information not contained in the image.
- Learning Semantic Concepts and Order for Image and Sentence Matching361
A semantic-enhanced image and sentence matching model is proposed, which can improve the image representation by learning semantic concepts and then organizing them in a correct semantic order.
- Explicit Knowledge-based Reasoning for Visual Question Answering295
A method for visual question answering which is capable of reasoning about contents of an image on the basis of information extracted from a large-scale knowledge base is described, addressing one of the key issues in general visual answering.
- Neighbourhood Watch: Referring Expression Comprehension via Language-Guided Graph Attention Networks290
A graph-based, language-guided attention mechanism that represents inter-object relationships, and properties with a flexibility and power impossible with competing approaches, and enables the comprehension decision to be visualizable and explainable.
- Say As You Wish: Fine-Grained Control of Image Caption Generation With Abstract Scene Graphs262
This work proposes a novel ASG2Caption model, which is able to recognise user intentions and semantics in the graph, and therefore generate desired captions following the graph structure, and achieves better controllability conditioning on ASGs than carefully designed baselines on both VisualGenome and MSCOCO datasets.
- Non-Salient Region Object Mining for Weakly Supervised Semantic Segmentation223
A graph-based global reasoning unit is introduced to strengthen the classification network’s ability to capture global relations among disjoint and distant regions and a potential object mining module is proposed to reduce the false-negative rate in pseudo labels.
- 215
- 212
- Multilabel Image Classification With Regional Latent Semantic Dependencies210
The proposed RLSD achieves the best performance compared to the state-of-the-art models, especially for predicting small objects occurring in the images, and can approach the upper bound without using the bounding-box annotations, which is more realistic in the real world.
- Jo-SRC: A Contrastive Approach for Combating Noisy Labels206
A noise-robust approach named Jo-SRC (Joint Sample Selection and Model Regularization based on Consistency) that trains the network in a contrastive learning manner to advance the model generalization performance by introducing consistency regularization.
- Bridging the Gap Between Learning in Discrete and Continuous Environments for Vision-and-Language Navigation194
It is shown that agents navigating in continuous environments with predicted waypoints perform significantly better than agents using low-level actions, which reduces the absolute discrete-to-continuous gap by 11.76% through extensive experiments.
- Scaling Data Generation in Vision-and-Language Navigation184
This work proposes an effective paradigm for generating large-scale data for learning, which applies 1200+ photo-realistic environments from HM3D and Gibson datasets and synthesizes 4.9 million instruction-trajectory pairs using fully-accessible resources on the web.
- Mucko: Multi-Layer Cross-Modal Knowledge Reasoning for Fact-based Visual Question Answering166
A modality-aware heterogeneous graph convolutional network is proposed to capture evidence from different layers that is most relevant to the given question to achieve a new state-of-the-art performance on the FVQA task.
- Parallel Attention: A Unified Framework for Visual Object Discovery Through Dialogs and Queries153
A unified framework, the ParalleL AttentioN (PLAN) network, to discover the object in an image that is being referred to in variable length natural expression descriptions, from short phrases query to long multi-round dialogs is proposed.
- Are You Talking to Me? Reasoned Visual Dialog Generation Through Adversarial Learning148
A novel approach that combines Reinforcement Learning and Generative Adversarial Networks (GANS) to generate more human-like responses to questions to overcome the relative paucity of training data, and the tendency of the typical MLE-based approach to generate overly terse answers.
Publication data from OpenAlex, with missing venues and authors filled in from Crossref; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-12. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.
Report an error
Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.