Is this you? Claim this profile to correct it, add a bio and choose the work people see first.
Claim this profileWorks255 from public data
- Vision-and-Language Navigation: Interpreting Visually-Grounded Navigation Instructions in Real Environments2,014
This work provides the first benchmark dataset for visually-grounded natural language navigation in real buildings - the Room-to-Room (R2R) dataset and presents the Matter-port3D Simulator - a large-scale reinforcement learning environment based on real imagery.
- FVQA: Fact-Based Visual Question Answering566
A conventional visual question answering dataset is extended, which contains image-question-answer triplets, through additional image- question-answer-supporting fact tuples, and a novel model is described which is capable of reasoning about an image on the basis of supporting-facts.
- REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environments551
A dataset of varied and complex robot tasks, described in natural language, in terms of objects visible in a large set of real images, and a novel Interactive Navigator-Pointer model is proposed that provides a strong baseline on the task.
- VLN↻BERT: A Recurrent Vision-and-Language BERT for Navigation503
A recurrent BERT model that is time-aware for use in VLN is proposed that can replace more complex encoder-decoder models to achieve state-of-the-art results and can generalised to other transformer-based architectures.
- NavGPT: Explicit Reasoning in Vision-and-Language Navigation with Large Language Models490
The NavGPT is introduced, a purely LLM-based instruction-following navigation agent, to reveal the reasoning capability of GPT models in complex embodied scenes by performing zero-shot sequential action prediction for vision-and-language navigation (VLN).
- What Value Do Explicit High Level Concepts Have in Vision to Language Problems?482
A method of incorporating high-level concepts into the successful CNN-RNN approach is proposed, and it is shown that it achieves a significant improvement on the state-of-the-art in both image captioning and visual question answering.
- Visual question answering: A survey of methods and datasets476
The state of the art by comparing modern approaches to VQA, and the common approach of combining convolutional and recurrent neural networks to map images and questions to a common feature space are examined.
- Image Captioning and Visual Question Answering Based on Attributes and External Knowledge416
A visual question answering model that combines an internal representation of the content of an image with information extracted from a general knowledge base to answer a broad range of image-based questions and allows questions to be asked where the image alone does not contain the information required to select the appropriate answer.
- Medical image classification using synergic deep learning402
The proposed synergic deep learning model using multiple deep convolutional neural networks simultaneously and enabling them to mutually learn from each other achieves the state‐of‐the‐art performance in these medical image classification tasks.
- Fine-Grained Video-Text Retrieval With Hierarchical Graph Reasoning393
A Hierarchical Graph Reasoning (HGR) model is proposed, which decomposes video-text matching into global-to-local levels and generates hierarchical textual embeddings via attention-based graph reasoning.
- Ask Me Anything: Free-Form Visual Question Answering Based on Knowledge from External Sources392
A method for visual question answering which combines an internal representation of the content of an image with information extracted from a general knowledge base to answer a broad range of image-based questions and is specifically able to answer questions posed in natural language, that refer to information not contained in the image.
- Learning Semantic Concepts and Order for Image and Sentence Matching361
A semantic-enhanced image and sentence matching model is proposed, which can improve the image representation by learning semantic concepts and then organizing them in a correct semantic order.
- Explicit Knowledge-based Reasoning for Visual Question Answering295
A method for visual question answering which is capable of reasoning about contents of an image on the basis of information extracted from a large-scale knowledge base is described, addressing one of the key issues in general visual answering.
- Neighbourhood Watch: Referring Expression Comprehension via Language-Guided Graph Attention Networks290
A graph-based, language-guided attention mechanism that represents inter-object relationships, and properties with a flexibility and power impossible with competing approaches, and enables the comprehension decision to be visualizable and explainable.
- Say As You Wish: Fine-Grained Control of Image Caption Generation With Abstract Scene Graphs262
This work proposes a novel ASG2Caption model, which is able to recognise user intentions and semantics in the graph, and therefore generate desired captions following the graph structure, and achieves better controllability conditioning on ASGs than carefully designed baselines on both VisualGenome and MSCOCO datasets.
- Non-Salient Region Object Mining for Weakly Supervised Semantic Segmentation223
A graph-based global reasoning unit is introduced to strengthen the classification network’s ability to capture global relations among disjoint and distant regions and a potential object mining module is proposed to reduce the false-negative rate in pseudo labels.
- 215
- 212
- Multilabel Image Classification With Regional Latent Semantic Dependencies210
The proposed RLSD achieves the best performance compared to the state-of-the-art models, especially for predicting small objects occurring in the images, and can approach the upper bound without using the bounding-box annotations, which is more realistic in the real world.
- Jo-SRC: A Contrastive Approach for Combating Noisy Labels206
A noise-robust approach named Jo-SRC (Joint Sample Selection and Model Regularization based on Consistency) that trains the network in a contrastive learning manner to advance the model generalization performance by introducing consistency regularization.
- Bridging the Gap Between Learning in Discrete and Continuous Environments for Vision-and-Language Navigation194
It is shown that agents navigating in continuous environments with predicted waypoints perform significantly better than agents using low-level actions, which reduces the absolute discrete-to-continuous gap by 11.76% through extensive experiments.
- Scaling Data Generation in Vision-and-Language Navigation184
This work proposes an effective paradigm for generating large-scale data for learning, which applies 1200+ photo-realistic environments from HM3D and Gibson datasets and synthesizes 4.9 million instruction-trajectory pairs using fully-accessible resources on the web.
- Mucko: Multi-Layer Cross-Modal Knowledge Reasoning for Fact-based Visual Question Answering166
A modality-aware heterogeneous graph convolutional network is proposed to capture evidence from different layers that is most relevant to the given question to achieve a new state-of-the-art performance on the FVQA task.
- Parallel Attention: A Unified Framework for Visual Object Discovery Through Dialogs and Queries153
A unified framework, the ParalleL AttentioN (PLAN) network, to discover the object in an image that is being referred to in variable length natural expression descriptions, from short phrases query to long multi-round dialogs is proposed.
- Are You Talking to Me? Reasoned Visual Dialog Generation Through Adversarial Learning148
A novel approach that combines Reinforcement Learning and Generative Adversarial Networks (GANS) to generate more human-like responses to questions to overcome the relative paucity of training data, and the tendency of the typical MLE-based approach to generate overly terse answers.
- Referring Expression Comprehension: A Survey of Methods and Datasets146
This survey examines the state-of-the-art by comparing modern approaches to the referring expression comprehension problem, and classifies methods by their mechanism to encode the visual and textual modalities.
- CogTree: Cognition Tree Loss for Unbiased Scene Graph Generation143
This work builds a cognitive structure CogTree to organize the relationships based on the prediction of a biased SGG model, and proposes a debiasing loss specially for this cognitive structure, which supports coarse-to-fine distinction for the correct relationships.
- Object-and-Action Aware Model for Visual Language Navigation142
An Object-and-Action Aware Model (OAAM) is proposed that processes these two different forms of natural language based instruction separately and enables each process to match object-centered/action-centered instruction to their own counterpart visual perception/action orientation flexibly.
- 140
- Data Hiding With Deep Learning: A Survey Unifying Digital Watermarking and Steganography130
An overview of recent developments in deep learning techniques deployed for data hiding is provided, categorized systematically according to model architectures and noise injection methods.
- MuKEA: Multimodal Knowledge Extraction and Accumulation for Knowledge-based Visual Question Answering128
This paper proposes MuKEA to represent multimodal knowledge by an explicit triplet to correlate visual objects and fact answers with implicit relations and proposes three objective losses to learn the triplet representations from complementary views: embedding structure, topological relation and semantic space.
- Overcoming Language Priors in VQA via Decomposed Linguistic Representations120
A novel method of language attention-based VQA that learns decomposed linguistic representations of questions and utilizes the representations to infer answers for overcoming language priors is presented.
- Visual Question Answering with Memory-Augmented Networks112
It is shown that memory-augmented neural networks are able to maintain a relatively long-term memory of scarce training exemplars, which is important for visual question answering due to the heavy-tailed distribution of answers in a general VQA setting.
- HOP: History-and-Order Aware Pretraining for Vision-and-Language Navigation108
This work proposes a novel history-and-order aware pre-training paradigm (HOP) with VLN-specific objectives that exploit the past observations and support future action prediction to promote the learning of spatio-temporal visual-textual correspondence and the agent's capability of decision making.
- 107
- Sub-Instruction Aware Vision-and-Language Navigation101
This work provides agents with fine-grained annotations during training and finds that they are able to follow the instruction better and have a higher chance of reaching the target at test time.
- Open-Nav: Exploring Zero-Shot Vision-and-Language Navigation in Continuous Environment with Open-Source LLMs99
Open-Nav is introduced, a novel study that explores open-source LLMs for zero-shot VLN in the continuous environment and employs a spatial-temporal chain-of-thought (CoT) reasoning approach to break down tasks into instruction comprehension, progress estimation, and decision-making.
- Neighbor-view Enhanced Model for Vision and Language Navigation99
This work proposes a multi-module Neighbor-View Enhanced Model (NvEM) to adaptively incorporate visual contexts from neighbor views for better textual-visual matching of existing methods.
- UniMiSS: Universal Medical Self-supervised Learning via Breaking Dimensionality Barrier98
This paper designs a pyramid U-like medical Transformer (MiT), composed of the switchable patch embedding (SPE) module and Transformers, which enables UniMiSS to learn representations from both 2D and 3D images.
- 97
- The VQA-Machine: Learning How to Use Existing Vision Algorithms to Answer New Questions97
The core of the proposed method is a new co-attention model that learns how to exploit a set of external off-the-shelf algorithms to achieve its goal, an approach that has something in common with the Neural Turing Machine.
- The Road to Know-Where: An Object-and-Room Informed Sequential BERT for Indoor Vision-Language Navigation95
This work proposes an object-informed sequential BERT to encode visual perceptions and linguistic instructions at the same fine-grained level, namely objects and words, and enables the visual-textual clues to be interpreted in light of the temporal context, which is crucial to multi-round VLN tasks.
- Cops-Ref: A New Dataset and Task on Compositional Referring Expression Comprehension87
A new dataset for visual reasoning in context of referring expression comprehension with two main features, and a novel expression engine rendering various reasoning logics that can be flexibly combined with rich visual properties to generate expressions with varying compositionality.
- 83
- 80
- 80
- Language-Guided Navigation via Cross-Modal Grounding and Alternate Adversarial Learning75
A cross-modal grounding module is designed, which is composed of two complementary attention mechanisms, to equip the agent with a better ability to track the correspondence between the textual and visual modalities and further exploit the advantages of both these two learning schemes via adversarial learning.
- Towards Accurate Text-based Image Captioning with Content Diversity Exploration74
This work proposes a novel Anchor-Captioner method, which not only achieves SOTA performance but also generates diverse captions to describe images.
- Structured Multimodal Attentions for TextVQA73
This paper proposes an end-to-end structured multimodal attention (SMA) neural network that outperforms the SoTA models on TextVQA dataset and two tasks of ST-VQ a dataset among all models except pre-training based TAP.
- DualVD: An Adaptive Dual Encoding Model for Deep Visual Understanding in Visual Dialogue72
A novel model to depict an image from both visual and semantic perspectives, which can tell which modality has more contribution in answering the current question by visualizing the gate values gives insights in understanding of human cognition in Visual Dialogue.
- 72
- Length-Controllable Image Captioning70
This paper devise a non-autoregressive image captioning approach that outperforms the autoregressive baselines in terms of controllability and diversity, and also significantly improves the decoding efficiency for long captions.
- 68
- March in Chat: Interactive Prompting for Remote Embodied Referring Expression65
A March-in-Chat (MiC) model that can talk to the LLM on the fly and plan dynamically based on a newly proposed Room-and-Object Aware Scene Perceiver (ROASP) and outperforms the previous state-of-the-art by large margins by SPL and RGSPL metrics on the REVERIE benchmark.
- Cascade Reasoning Network for Text-based Visual Question Answering64
A novel Cascade Reasoning Network (CRN) is proposed that consists of a progressive attention module (PAM) and a multimodal reasoning graph (MRG) module that aims to explicitly model the connections and interactions between texts and visual concepts.
- Simple is not Easy: A Simple Strong Baseline for TextVQA and TextCaps61
This work argues that a simple attention mechanism can do the same or even better job without any bells and whistles on state-of-the-art SOTA models, and sets the new baseline for these two OCR text related applications and to inspire new thinking of multi-modality encoder design.
- Diagnosing Vision-and-Language Navigation: What Really Matters59
This work conducts a series of diagnostic experiments to unveil agents’ focus during navigation and shows that indoor navigation agents refer to both object and direction tokens when making decisions, and Transformer-based agents acquire a better cross-modal understanding of objects and display strong numerical reasoning ability than non-Transformer- based agents.
- Prompt Switch: Efficient CLIP Adaptation for Text-Video Retrieval56
This paper proposes to apply an auxiliary video captioning objective to train the frame representations, which facilitates the learning of detailed video semantics by providing fine-grained guidance in the semantic space.
- Semantic Equivalent Adversarial Data Augmentation for Visual Question Answering56
This paper uses adversarial learning to train a classic VQA model (BUTD) with augmented data, and finds that it not only improves the overall performance on VQAv2, but also can withstand adversarial attack effectively, compared to the baseline model.
- Automatic seizure detection using Stockwell transform and boosting algorithm for long-term EEG56
A novel seizure detection method based on Stockwell transform is proposed for intracranial long-term EEG data which can achieve a satisfactory level of specificity and a very short delay time.
- 55
- Soft Expert Reward Learning for Vision-and-Language Navigation55
A Soft Expert Reward Learning (SERL) model is introduced to overcome the reward engineering designing and generalisation problems of the VLN task and the model outperforms the state-of-the-art methods on most of the evaluation metrics.
- Intelligent Home 3D: Automatic 3D-House Design From Linguistic Descriptions Only54
A House Plan Generative Model (HPGM) is proposed that first translates the language input to a structural graph representation and then predicts the layout of rooms with a Graph Conditioned Layout Prediction Network and generates the interior texture with a Language Conditioned Texture GAN.
- Skin Lesion Classification in Dermoscopy Images Using Synergic Deep Learning53
A synergic deep learning model is proposed, which not only uses dual deep convolutional neural networks (DCNNs) but also enables them to mutually learn from each other, which has a fully connected structure and predicts whether the pair of input images belong to the same class.
- Cross-depiction problem: Recognition and synthesis of photographs and artwork53
The main conclusions are (i) appearance-based recognition systems tend to be over-fitted to one depiction, (ii) models that explicitly encode spatial relations between parts are more robust, and (iii) recognition and non-photorealistic synthesis are related tasks.
- 52
- 52
- Convolutional Residual-Attention: A Deep Learning Approach for Precipitation Nowcasting51
To solve the problem of excessive loss of global information caused by the attention mechanism, the residual connection is introduced into the proposed model and achieves the state-of-the-art precipitation forecast accuracy on the radar echo dataset.
- HCVRD: A Benchmark for Large-Scale Human-Centered Visual Relationship Detection48
A webly-supervised approach to these problems is proposed and it is demonstrated that the proposed model provides a strong baseline on the authors' HCVRD dataset.
- 47
- 46
- Monitoring Depth of Anesthesia Based on Hybrid Features and Recurrent Neural Network45
A novel method based on Long Short-Term Memory (LSTM) and a sparse denoising autoencoder (SDAE) to combine the hybrid features of EEG to monitor the depth of anesthesia with higher prediction probability is proposed.
- V2C: Visual Voice Cloning43
This work designs a set of evaluation metrics, named MCD-DTW-SL, which help eval-uate the similarity between ground-truth speeches and the synthesised ones, and proposes a new task, V2C, which seeks to convert a para-graph of text to a speech with both desired voice speci-fied by a reference audio and desired emotion specified by areference video.
- 43
- Goal-Oriented Visual Question Generation via Intermediate Rewards41
A Deep Reinforcement Learning framework based on three new intermediate rewards, namely goal-achieved, progressive and informativeness that encourage the generation of succinct questions, which in turn uncover valuable information towards the overall goal is proposed.
- Image Captioning and Visual Question Answering Based on Attributes and Their Related External Knowledge.41
A visual question answering model is designed that combines an internal representation of the content of an image with information extracted from a general knowledge base to answer a broad range of image-based questions.
- 40
- Rethinking masked image modelling for medical image representation38
This work proposes and introduces two mutual comprehensive masking strategies, knowledge-driven masking (KDM), and sentence-driven masking (SDM), and substantially outperforms ImageNet pre-training, MIM-based pre-training, and medical image-report pre-training counterparts.
- 38
- 38
- 38
- Using Fractal and Local Binary Pattern Features for Classification of ECOG Motor Imagery Tasks Obtained from the Right Brain Hemisphere38
An algorithm for motor imagery (MI) classification of electrocorticogram (ECoG)-based BCI that employs multi-resolution fractal measures and local binary pattern (LBP) operators to form a combined feature for characterizing an ECoG epoch recording from the right hemisphere of the brain.
- Weakly-Supervised 3D Spatial Reasoning for Text-Based Visual Question Answering37
3D geometric information is introduced into the spatial reasoning process to capture the contextual knowledge of key objects step-by-step and achieves state-of-the-art performance on TextVQA and ST-VQ a datasets.
- 37
- Visual-Semantic Graph Matching for Visual Grounding37
This paper forms visual grounding as a graph matching problem to find node correspondences between a visual scene graph and a language scene graph, and learns unified contextual node representations of the two graphs by using a cross-modal graph convolutional network to reduce their discrepancy.
- 36
- Kernel Collaborative Representation-Based Automatic Seizure Detection in Intracranial EEG35
The seizure detection system based on KCR yields both a high sensitivity and a low false detection rate for long-term EEG, which is of great significance in the monitoring and diagnosis of epilepsy.
- 34
- 31
- Epileptic Seizure Detection with Log-Euclidean Gaussian Kernel-Based Sparse Representation31
A novel algorithm is developed to detect seizures employing log-Euclidean Gaussian kernel-based sparse representation (SR) in long-term EEG recordings which is more speedy and efficient than traditional seizure detection methods.
- 31
- An Improved Sparse Representation over Learned Dictionary Method for Seizure Detection31
A patient specific method is proposed for seizure detection in the long-term intracranial electroencephalogram (EEG) recordings based on sparse representation with online dictionary learning and elastic net constraint.
- 31
- ModaVerse: Efficiently Transforming Modalities with LLMs30
ModaVerse is introduced, a Multi-modal Large Language Model (MLLM) capable of comprehending and transforming content across various modalities in-cluding images, videos, and audio, and a novel Input/Output (I/O) alignment mechanism that operates directly at the level of natural language.
- VLN-PETL: Parameter-Efficient Transfer Learning for Vision-and-Language Navigation30
This work designs two PETL modules: Historical Interaction Booster and Cross-modal Interaction Booster and combines several existing PETL methods as the integrated VLN-PETL, where VLN-PETL achieves comparable or even better performance to full fine-tuning and outperforms other PETL methods with promising margins.
- Learning Graphs to Model Visual Objects across Different Depictive Styles30
This paper model visual classes using a graph with multiple labels on each node; weights on arcs and nodes indicate relative importance (salience) to the object description, and shows that this representation is able to improve upon Deformable Part Models for detection and Bag of Words models for classification.
- Missing Target-Relevant Information Prediction with World Model for Accurate Zero-Shot Composed Image Retrieval28
A novel prediction-based mapping network, named PrediCIR, is proposed, to adaptively predict the missing target visual content in reference images in the latent space before mapping for accurate ZS-CIR.
- 28
- A Study on Seizure Detection of EEG Signals Represented in 2D28
A deep learning-based model that modularizes the main steps of seizure detection and improves the adaptability of the model to different hardware resource constraints is proposed, in order to increase the convenience of the algorithm optimization or the replacement of each module.
- Confidence-aware Non-repetitive Multimodal Transformers for TextCaps28
A Confidence-aware Non-repetitive Multimodal Transformers (CNMT) that outperforms state-of-the-art models on TextCaps dataset, improving from 81.0 to 93.0 in CIDEr and addressing the issue of word redundancy in captions.
- Foreground-Background Imbalance Problem in Deep Object Detectors: A Review28
This paper analyzes the characteristics of the imbalance problem in different kinds of deep detectors, including one-stage and two-stage ones, and divides the existing solutions into two categories: sampling heuristics and non-sampling schemes, and reviews them in detail.
- Data-driven Meta-set Based Fine-Grained Visual Recognition28
This work proposes a data-driven meta-set based approach to deal with noisy web images for fine-grained recognition that can alleviate the harmful effects caused by out-of-distribution noise and properly exploit the in-dist distribution noisy samples for training.
- 28
- 27
- 26
- A low computation cost method for seizure prediction25
The presented seizure prediction algorithm extracts the Higuchi fractal dimension (HFD) of EEG signals as features to classify the patient's preictal or interictal state with Bayesian linear discriminant analysis (BLDA) as a classifier.
- SAME: Learning Generic Language-Guided Visual Navigation with State-Adaptive Mixture of Experts24
This paper investigates the core difficulties of sharing general knowledge and exploiting task-specific capabilities in learning navigation and proposes a novel State-Adaptive Mixture of Experts (SAME) model that effectively enables an agent to infer decisions based on different-granularity language and dynamic observations.
- Give Me Something to Eat: Referring Expression Comprehension with Commonsense Knowledge24
This paper collects a new referring expression dataset, called KB-Ref, containing $43$k expressions on 16k images, and presents an expression conditioned image and fact attention (ECIFA) network that extracts information from correlated image regions and commonsense knowledge facts.
- 24
- 23
- 23
- 22
- 22
- A Lightweight Bearing Fault Diagnosis Method Based on Multi-Channel Depthwise Separable Convolutional Neural Network22
The results show that the proposed model is superior to other classical deep learning models; thus, it has a smaller model size, higher accuracy, and less computation burden.
- 22
- 22
- Deep template matching for offline handwritten Chinese character recognition22
A novel method for learning siamese neural network which employ a special structure to predict the similarity between handwritten Chinese characters and template images and has a very promising generalization ability to the new classes that never appear in the training set.
- LLM as Copilot for Coarse-Grained Vision-and-Language Navigation21
VLN-Copilot is introduced, a framework enabling agents to actively seek assistance when encountering confusion, with the LLM serving as a copilot to facilitate navigation.
- DAM: Deliberation, Abandon and Memory Networks for Generating Detailed and Non-repetitive Responses in Visual Dialogue21
A novel generative decoding architecture to generate high-quality responses, which moves away from decoding the whole encoded semantics towards the design that advocates both transparency and flexibility.
- Communicative Learning with Natural Gestures for Embodied Navigation Agents with Human-in-the-Scene20
It is demonstrated that human gesture cues, even without predefined semantics, improve the object-goal navigation for an embodied agent, outperforming various state-of-the-art methods.
- 20
- What's to Know? Uncertainty as a Guide to Asking Goal-Oriented Questions20
A Bayesian model of the uncertainty in the implicit model maintained by the visual dialogue agent, and in the function used to select an appropriate output, enables a principled method for identifying when enough information has been acquired, and an action should be selected.
- Multi-Intersection Traffic Optimisation: A Benchmark Dataset and a Strong Baseline19
This work proposes a novel baseline model based on deep reinforcement learning with an encoder-decoder structure: an edge-weighted graph convolutional encoder to excavate multi-intersection relations; and an unified structure decoder to jointly model multiple junctions in a comprehensive manner, which significantly reduces the number of the model parameters.
- R-GAN: Exploring Human-like Way for Reasonable Text-to-Image Synthesis via Generative Adversarial Networks19
This work proposes R-GAN, which can generate reasonable images according to the given text in a human-like way, and introduces a coarse-to-fine shape generator based on a shape knowledge base.
- Diagnosis of Alzheimer's Disease Based on Deeply-Fused Nets19
Experiments show that when differentiating between patients with AD, mild cognitive impairment, and normal controls on a subset of the ADNI database without data leakage, the new architecture improves the accuracy by about 4 percentage points as compared to a single standard base network.
- 19
- 18
- 18
- 18
- Sparse representation-based EMD and BLDA for automatic seizure detection18
The results indicate that SR-EMD is more effective and efficient than the conventional Earth Mover’s Distance (EMD), and the good performance and fast speed of this algorithm make it suitable for the real-time seizure monitoring application.
- Image Captioning with an Intermediate Attributes Layer.18
This work shows that an intermediate image-to-attributes layer can dramatically improve captioning results over the current approach which directly connects an RNN to a CNN.
- Ground-Level Viewpoint Vision-and-Language Navigation in Continuous Environments17
This work represents the first attempt to highlight the generalization gap in VLN across varying heights of visual observation in realistic robot deployments, and transfers the connectivity graph from the HM3D and Gibson datasets as an extra resource to enhance spatial priors and a more comprehensive representation of real-world scenarios.
- An Adaptive Correlation Filtering Method for Text-Based Person Search17
Multi-modal Adaptive Correlation Filtering (MACF) method that adaptively learns the vital regions and keywords with a shared update strategy is proposed and a new mutually conditional gate to dynamically control the updating process of filters is introduced.
- 17
- VisIt-OSPRay: Toward an Exascale Volume Visualization System17
This work presents VisIt-OSPRay, a high-performance, scalable, hybrid-parallel rendering system in VisIt, using OSPRay and IceT, coupled with PIDX for scalable I/O.
- 17
- 17
- 16
- Proposal-free One-stage Referring Expression via Grid-Word Cross-Attention16
This paper presents a proposal-free one-stage (PFOS) model that is able to regress the region-of-interest from the image, based on a textual query, in an end-to-end manner and achieves the state- of-the-art performance on four referring expression datasets with higher efficiency, comparing to previous best one- stage and two-stage methods.
- 15
- Identity-Consistent Aggregation for Video Object Detection15
ClipVID is proposed, a VID model equipped with Identity-Consistent Aggregation layers specifically designed for mining fine-grained and identity-consistent temporal contexts and effectively reduces the redundancies through the set prediction strategy, making the ICA layers very efficient and further allowing us to design an architecture that makes parallel clip-wise predictions for the whole video clip.
- S3C: Semi-Supervised VQA Natural Language Explanation via Self-Critical Learning15
A new Semi-Supervised VQA-NLE via Self-Critical Learning (S3C), which evaluates the candidate explanations by answering rewards to improve the logical consistency between answers and rationales is proposed.
- 15
- 15
- Navigating Beyond Instructions: Vision-and-Language Navigation in Obstructed Environments14
A novel method called ObVLN (Obstructed VLN), which includes a curriculum training strategy and virtual graph construction to help agents effectively adapt to obstructed environments, is proposed, which shows that ObVLN not only maintains robust performance in unobstructed scenarios but also achieves a substantial performance advantage with unexpected obstructions.
- RadarLLM: Empowering Large Language Models to Understand Human Motion from Millimeter-Wave Point Cloud Sequence13
RadarLLM is presented, the first framework that leverages large language models (LLMs) for human motion understanding from radar signals, and introduces a radar-aware language model that establishes cross-modal alignment between radar and text in a shared embedding space.
- 12
- 12
- 12
- Memory-efficient Temporal Moment Localization in Long Videos11
Stochastic Bucket-wise Feature Sampling (SBFS), a stochastic sampling module that allows methods to process long videos at a constant memory footprint, and Locformer, a Transformer-based model that can process videos as long as 18 minutes are proposed.
- 11
- Prediction Method of Multiple Related Time Series Based on Generative Adversarial Networks11
A novel deep learning model Multiple Time Series Generative Adversarial Networks (MTSGAN) based on generative adversarial networks to solve multiple related time series prediction problems is proposed.
- MiniVLN: Efficient Vision-and-Language Navigation by Progressive Knowledge Distillation10
A two-stage knowledge distillation framework is introduced, producing a student model, MiniVLN, and showcasing the significant potential of distillation techniques in developing lightweight models.
- MeisterMorxrc at SemEval-2020 Task 9: Fine-Tune Bert and Multitask Learning for Sentiment Analysis of Code-Mixed Tweets10
The shared task of sentiment analysis of code-mixed tweets, which is a part of the SemEval-2020 competition, the team MeisterMorxrc preprocess datasets by replacing emoji and deleting uncommon characters and so on, and then fine-tune the Bidirectional Encoder Representation from Transformers(BERT) to perform the best.
- Transformer-Based Relational Inference Network for Complex Visual Relational Reasoning9
This article regards the complex referring expression comprehension (c-REF) task as the reasoning basis, in which c-REF seeks to localise a target object in an image guided by a complex query, and proposes a Transformer-based Relational Inference Network (Trans-RINet).
- Heritage image annotation via collective knowledge9
This work proposes to annotate images via collective knowledge, that is, it uncover relationships between the image and its neighbors by measuring similarities among metadata and conduct the metric learning to obtain the representations of image contents, and also generates semantic representations for images given collective semantic information from their neighbors.
- An Active Information Seeking Model for Goal-oriented Vision-and-Language Tasks.9
This work proposes an reinforcement-learning approach that maintains an distribution over its internal information, thus explicitly representing the ambiguity in what it knows, and needs to know, towards achieving its goal.
- Modelling Visual Objects Invariant to Depictive Style9
A model for visual object classes that is learnable and which is able to classify over a broad range of depictive styles is described, a graph in which simple shapes label region nodes is used.
- 8
- 8
- Kill Two Birds With One Stone: Weakly-Supervised Neural Network for Image Annotation and Tag Refinement8
This paper proposes to learn an image annotation model and refine the user-provided tags simultaneously in a weakly-supervised manner and indicates that the proposed model achieves the best performance compared to the state-of-the-art methods.
- What value high level concepts in vision to language problems8
A method of incorporating high-level concepts into the very successful CNN-RNN approach is proposed, and it is shown that it achieves a significant improvement on the state-of-the-art performance in both image captioning and visual question answering.
- Prime Shapes in Natural Images8
The most common of those found are familiar enough to be named: shapes such as triangles, squares and circles (more exactly, these shapes up to affine transformation).
- 8
- Robust Self-calibration from Single Image Using RANSAC8
First, it is shown how the vanishing point, vanishing line and foot-to-head plane homology can be used to obtain the calibration parameters and then the approach how to efficiently adopt RANSAC to estimate them is shown.
- Dataset, Challenge, and Evaluation for Tumor Segmentation Variability7
By exploring the two tasks with different metrics, the MMIS-2024 challenge aims to establish a global benchmark for multi-rater medical image segmentation, facilitating clinical AI deployments.
- T2VIndexer: A Generative Video Indexer for Efficient Text-Video Retrieval7
A model-based video indexer named T2VIndexer is introduced, which is a sequence-to-sequence generative model directly generating video identifiers and retrieving candidate videos with constant time complexity that consistently enhances the retrieval efficiency of current state-of-the-art models on four standard datasets.
- Applications and theoretical challenges in environmental emergency issues alerting system on IoT intelligence7
An IoT based emergency response system (IoT-ERS) solution has been proposed to safeguard people from emergencies based on real-time data and shows promising outcomes when compared to previous methods.
- 7
- RERERE: Remote Embodied Referring Expressions in Real indoor Environments.7
This work proposes the first Remote Embodied Referring Expression dataset of natural language references to remote objects in real images, which represents a complex challenge, but one that closely reflects one of the core visual problems in robotics.
- 7
- 6
- Screening key genes for intracranial aneurysm rupture using LASSO regression and the SVM-RFE algorithm6
The increase in IL-10 and ITGA5 may weaken the vascular wall by promoting inflammation in blood vessels and immune cells, which could have a harmful effect on the rupture of IAs.
- Script-to-Storyboard: A New Contextual Retrieval Dataset and Benchmark6
This work introduces the first contextual benchmark dataset Script-to-Storyboard (Sc2St) composed of storyboards to explicitly express story structures in the movie domain, and proposes the contextual retrieval task to facilitate movie story understanding.
- Visual-Semantic Decomposition and Partial Alignment for Document-based Zero-Shot Learning6
A novel network is proposed to extract multi-view semantic concepts from documents and images and align the matching rather than entire concepts and consistently outperform state-of-the-art methods under two document sources in three standard benchmarks for document-based zero-shot learning.
- 6
- 6
- Modular Graph Attention Network for Complex Visual Relational Reasoning6
This paper considers reasoning on complex referring expression comprehension (c-REF) task that seeks to localise the target objects in an image guided by complex queries, and proposes a novel Modular Graph Attention Network (MGA-Net), which mimics the human language understanding mechanism.
- Gold Seeker: Information Gain From Policy Distributions for Goal-Oriented Vision-and-Langauge Reasoning6
This work proposes a reinforcement-learning approach that maintains a distribution over its internal information, thus explicitly representing the ambiguity in what it knows, and needs to know, towards achieving its goal.
- Watch, Reason and Code6
A Watch-Reason-Code (WRC) model to synthesise programs that describe the process carried out in a set of video sequences and can achieve the state-of-the-art under a variety of settings is proposed.
- 6
- 5
- 5
- 5
- VLN-ChEnv: Vision-language Navigation in Changeable Environments5
This paper investigates 3 common scenarios where instructions and navigation scenes are imperfectly aligned: change of navigability, incorrect landmark references, and incorrect direction descriptions, and proposes an ImperfectVLN task and dataset for evaluating an agent's navigation performance under instruction and environment imperfectly matched conditions.
- 5
- 5
- Chop Chop BERT: Visual Question Answering by Chopping VisualBERT’s Heads5
A dynamic chopping module is designed that can automatically remove heads and layers of the VisualBERT at an instance level when dealing with different questions, and effectively reduce the parameters of the original model by 50%, while only damaging the accuracy by less than 1% on the VQA task.
- 5
- An Attribute-Based High-Level Image Representation for Scene Classification5
This paper proposes a novel high-level image representation which utilizes image attributes as features for scene classification and considerably outperforms other state-of-the-art methods.
- Development and validation of radiomics and deep transfer learning models to assess cognitive impairment in patients with cerebral small vessel disease4
These findings highlight that comprehensive data encompassing radiomics and DTL features showcase a robust predictive capability in distinguishing CSVD patients with cognitive impairment, offering insights for clinical applications despite limitations in sample size.
- 4
- Denoise-I2W: Mapping Images to Denoising Words for Accurate Zero-Shot Composed Image Retrieval4
This paper proposes a novel denoising image-to-word mapping approach, named Denoise-I2W, for mapping images into denoising pseudo-word tokens that, without intention-irrelevant visual information, enhance accurate ZS-CIR.
- SwitchGPT: Adapting Large Language Models for Non-Text Outputs4
A novel approach, \methodname, from a modality conversion perspective that evolves a text-based LLM into a multi-modal one, and results reveal that, with minimal training, LLMs can be conveniently adapted to comprehend requests for non-text responses, thus achieving higher flexibility in multi- modal scenarios.
- Image editing with varying intensities of processing4
The hypothesis is that the part of parameters that correspond to the intensity of processing of the network are implicitly embedded in the whole parameter set and can be explicitly extracted and utilized, even with limited supervision, in image editing with Varying Intensities of Processing.
- Utilising Prior Knowledge for Visual Navigation: Distil and Adapt4
This paper proposes to decompose the value function in the actor-critic reinforcement learning algorithm and incorporate the prior in the critic in a novel way that reduces the model complexity and improves model generalisation.
- Research on a Real-Time Monitoring System for Campus Woodland Fires via Deep Learning3
A G-YOLOv5n-CB forest fire detection algorithm based on the YOLOv5 algorithm is proposed and a set of real-time fire monitoring systems applicable to campus forest land with the aid of deep learning technology is developed.
- BotSCL: Heterophily-aware Social Bot Detection with Supervised Contrastive Learning3
BotSCL is presented, a heterophily-aware contrastive learning framework that can adaptively differentiate neighbor representations of heterophilic relations while assimilating the representations of homophilic neighbors and outperforms baseline approaches including the state-of-the-art bot detection approaches, partially heterophile GNNs and self-supervised contrast learning methods.
- 3
- 3
- 3
- Risk-neutral generative networks2
A generative approach to price options and extract risk-neutral densities from the market that significantly outperforms the extensive set of baseline models—including three parametric models and nine stochastic process models—in terms of accuracy and stability.
- 2
- Implement Referring Expression Comprehension by Extending Auto-focus Lens to Locked Vision Model2
A point-based two-stage framework for REC to quickly achieve localization by inserting a language-modulated auto-focus module into the locked vision model and reconstructing the raw annotations into soft masks at the feature point level as a metric of cross-modal correlation is proposed.
- 2
- 2
- 2
- 2
- 2
- Connecting Language and Vision to Actions2
This tutorial will comprehensively review existing state-of-the-art approaches to selected tasks such as image captioning, visual question answering (VQA) and visual dialog, presenting the key architectural building blocks and novel algorithms used to train models for these tasks.
- 2
- Auto-3D-house Design from Structured User Requirements1
This work carefully design a structured list that divides the requirements into three parts, which focus on the attributes of rooms, the outline of the building, and the style of decoration, respectively, and proposes an automatic house design framework, called auto-3D-house design (A3HD).
- An evidence-based analysis of machine learning prediction models for cognitive impairment in cerebral small vessel disease1
The findings indicate that ML models holds significant promise in forecasting the risk of cognitive impairment in patients with CSVD, however, future high-quality research that externally validates the algorithm through prospective studies with larger, more diverse cohorts is needed.
- Improving Online Source-Free Domain Adaptation for Object Detection by Unsupervised Data Acquisition1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- Toward Reasoning-Centric Video Object Segmentation via Multi-Modal Large Language Models–
RViSeg is presented, a reasoning-centric video object segmentation model that leverages the reasoning capability of Multi-modal Large Language Models (MLLM) to handle complex queries and introduces a novel Spatial Token Merge (STM) module that consolidates lengthy video tokens into compact region-level clusters, while preserving essential spatial details.
- –
- –
- –
- Enhanced Bayesian network classifier for intrusion detection: integrating semilazy learning and multiconditional entropy–
Experiments demonstrate that the EBNC outperforms alternative algorithms with respect to the 0-1 loss, bias, F1 score, as well as training and classification time, and highlight its efficiency and accuracy in intrusion detection.
- –
- –
- –
- –
- –
- Band Correlation‐Based Multichannel Multiscale Convolution Network for Intelligent Interference Recognition–
An innovative wireless communication interference identification model based on frequency band correlation is proposed, which uses LSTM to detect the frequency band correlation of interference signals and improve the accuracy of interference identification under low Jamming noise ratio (JNR).
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- Classification of COVID-19 in CT Scans Using Image Smoothing and Improved Deep Residual Network–
It is found that data smoothing can improve the quality of CT images of COVID-19 and improve the accuracy of the model and an improved ResNet structure is proposed to improve the model feature extraction and fusion by optimizing the structure of the input stem and downsampling parts.
- Consistent-Resolution Network for 3D Hand Shape Estimation from a Single RGB Image–
Through extensive experiments on a large real-world dataset FreiHAND, it is shown that the proposed method can predict precise and suitable 3D hand shape from a monocular view.
- Tracking via Enhanced Context-Aware Correlation Filter–
Based on correlation filter model and context clues, more discriminant information is provided during the tracking, to solve the interference such as occlusion and in-plane rotation and establishes an optimized target regression model.
- –
- –
- –
Publication data from OpenAlex, with missing venues and authors filled in from Crossref; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-12. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.
Report an error
Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.