Is this you? Claim this profile to correct it, add a bio and choose the work people see first.
Claim this profileWorks255 from public data
- SCA-CNN: Spatial and Channel-Wise Attention in Convolutional Networks for Image Captioning1,991
This paper introduces a novel convolutional neural network dubbed SCA-CNN that incorporates Spatial and Channel-wise Attentions in a CNN that significantly outperforms state-of-the-art visual attention-based image captioning methods.
- Attentional Factorization Machines: Learning the Weight of Feature Interactions via Attention Networks1,030
A novel model named Attentional Factorization Machine (AFM), which learns the importance of each feature interaction from data via a neural attention network, which consistently outperforms the state-of-the-art deep learning methods Wide&Deep and DeepCross with a much simpler structure and fewer model parameters.
- Video Question Answering via Gradually Refined Attention over Appearance and Motion840
This paper proposes an end-to-end model which gradually refines its attention over the appearance and motion features of the video using the question as guidance and demonstrates the effectiveness of the model by analyzing the refined attention weights during the question answering procedure.
- 490
- Counterfactual Samples Synthesizing for Robust Visual Question Answering341
A model-agnostic Counterfactual Samples Synthesizing (CSS) training scheme that significantly improves both visual-explainable and question-sensitive abilities of VQA models and, in return, the performance of these models is further boosted.
- Zero-Shot Visual Recognition Using Semantics-Preserving Adversarial Embedding Networks314
Through adversarial learning of the two subspaces, SP-AEN can transfer the semantics from the reconstructive subspace to the discriminative one, accomplishing the improved zero-shot recognition of unseen classes.
- 314
- Boundary Proposal Network for Two-stage Natural Language Video Localization196
A novel Boundary Proposal Network (BPNet) is proposed, a universal two-stage framework that gets rid of the issues mentioned above and outperforms the state-of-the-art NLVL methods.
- 193
- Rethinking the Bottom-Up Framework for Query-Based Video Localization189
It is argued that the performance of bottom-up framework is severely underestimated by current unreasonable designs, including both the backbone and head network, and designed a novel top-up model: Graph-FPN with Dense Predictions (GDP).
- An artificial intelligence based data-driven approach for design ideation170
An integrated approach for enhancing design ideation by applying artificial intelligence and data mining techniques, which consists of two models, a semantic ideation network and a visual concepts combination model, which provide inspiration semantically and visually based on computational creativity theory.
- Counterfactual Critic Multi-Agent Training for Scene Graph Generation167
CMAT is a multi-agent policy gradient method that frames objects into cooperative agents, and then directly maximizes a graph-level metric as the reward, and uses a counterfactual baseline that disentangles the agent-specific reward by fixing the predictions of other agents.
- Scene Dynamics: Counterfactual Critic Multi-Agent Training for Scene Graph Generation.167
CMAT is a multi-agent policy gradient method that frames objects into cooperative agents, and then directly maximizes a graph-level metric as the reward, and uses a counterfactual baseline that disentangles the agent-specific reward by fixing the predictions of other agents.
- DEBUG: A Dense Bottom-Up Grounding Approach for Natural Language Video Localization156
This paper proposes a novel dense bottom-up framework: DEnse Bottom-Up Grounding (DEBUG), which regards all frames falling in the ground truth segment as foreground, and each foreground frame regresses the unique distances from its location to bi-directional ground truth boundaries.
- Hierarchical Fashion Graph Network for Personalized Outfit Recommendation141
A new framework, Hierarchical Fashion Graph Network (HFGN), is developed, to model relationships among users, items, and outfits simultaneously, and achieves significant improvements over the state-of-the-art compatibility matching models like NGNN and outfit recommenders like FHN.
- HealthGPT: A Medical Large Vision-Language Model for Unifying Comprehension and Generation via Heterogeneous Knowledge Adaptation138
This work presents HealthGPT, a powerful Medical Large Vision-Language Model (Med-LVLM) that integrates medical visual comprehension and generation capabilities within a unified autoregressive paradigm through a novel heterogeneous low-rank adaptation technique and a tailored hierarchical visual perception approach.
- Ref-NMS: Breaking Proposal Bottlenecks in Two-Stage Referring Expression Grounding126
Ref-NMS is proposed, which is the first method to yield expression-aware proposals at the first stage, and introduces a lightweight module to predict a score for aligning each box with a critical object, resulting in a significantly improved grounding performance.
- Video Question Answering via Attribute-Augmented Attention Network Learning118
This paper proposes the attribute-augmented attention network learning framework that enables the joint frame-level attribute detection and unified video representation learning for video question answering and incorporates the multi-step reasoning process for the proposed attention network to further improve the performance.
- The Devil is in the Labels: Noisy Label Correction for Robust Scene Graph Generation113
This paper proposes a novel model-agnostic NoIsy label CorrEction strategy for SGG: NICE, which can not only detect noisy samples but also reassign more high-quality predicate labels to them.
- Management of male erectile dysfunction: From the past to the future107
With the rapid development of technologies, some new methods, such as low-intensity extracorporeal shock wave and stem cell injection therapy can even repair the organic damage of the corpora cavernosa and these are important directions for the treatment of male erectile dysfunction in the future.
- 107
- CATR: Combinatorial-Dependence Audio-Queried Transformer for Audio-Visual Video Segmentation105
A decoupled audio-video transformer that combines audio and video features from their respective temporal and spatial dimensions, capturing their combined dependence is proposed, and a block is designed, which, when stacked, enables capturing audio-visual fine-grained combinatorial-dependence in a memory-efficient manner.
- De-Biased Court’s View Generation with Causality101
A novel Attentional and Counterfactual based Natural Language Generation (AC-NLG) method, consisting of an attentional encoder and a pair of innovative counterfactual decoders, to eliminate the confounding bias in data and generate judgment-discriminative court’s views by incorporating with a synergistic judgment predictive model.
- Shapley Counterfactual Credits for Multi-Agent Reinforcement Learning98
Sh Shapley Counterfactual Credit Assignment is proposed, a novel method for explicit credit assignment which accounts for the coalition of agents and outperforms existing cooperative MARL algorithms significantly and achieves the state-of-the-art.
- 96
- Video Relation Detection with Spatio-Temporal Graph93
By combining the model (VRD-GCN) and the proposed association method, the framework for video relation detection achieves the best performance in the latest benchmarks and a series of ablation studies demonstrate the method's effectiveness.
- Human-like Controllable Image Captioning with Verb-specific Semantic Roles86
A new control signal for CIC is proposed: Verb-specific Semantic Roles (VSR), which consists of a verb and some semantic roles, which represents a targeted activity and the roles of entities involved in this activity.
- 84
- Exploiting temporal stability and low-rank structure for motion capture data refinement79
An efficient optimization method derived from the augmented Lagrange multiplier algorithm is presented to solve the mocap data refinement problem and a trust data detection method is introduced to improve the degree of automation for processing the entire set of the data and boost the performance.
- Ferroptosis: the emerging player in remodeling triple-negative breast cancer66
The contribution of ferroptosis in TNBC progression, diagnosis, and treatment is explored to provide novel perspectives and therapeutic strategies for TNBC management.
- Learning Hybrid Behavior Patterns for Multimedia Recommendation65
A Hybrid Clustering Graph Convolutional Network (HCGCN) is proposed for multimedia recommendation that performs high-order graph convolutions inside user- item clusters and item-item clusters to capture various user behavior patterns and designs corresponding clustering losses to enhance user-item preference feedback and multimodal representation learning constraint to adjust the modality importance.
- Consensus Graph Representation Learning for Better Grounded Image Captioning64
The Consensus Rraph Representation Learning framework (CGRL) is proposed for GIC that incorporates a consensus representation into the grounded captioning pipeline that can capture both the correct linguistic characteristics and visual relevance, and then grounding appropriate image regions further.
- Multi-interaction Network with Object Relation for Video Question Answering60
A new attention mechanism called multi-interaction, which can capture both element-wise and segment-wise sequence interactions, simultaneously, simultaneously is proposed, which achieves the new state-of-the-art performance.
- 56
- 55
- EduPlanner: LLM-Based Multiagent Systems for Customized and Intelligent Instructional Design53
EduPlanner is developed, an LLM-based multiagent system comprising an evaluator agent, an optimizer agent, and a question analyst, working in adversarial collaboration to generate customized and intelligent instructional design for curriculum and learning activities.
- Mining Spatial-Temporal Patterns and Structural Sparsity for Human Motion Data Denoising53
This paper proposes a data-driven-based robust human motion denoising approach by mining the spatial-temporal patterns and the structural sparsity embedded in motion data by replacing the regularly used entire pose model with a much fine-grained partlet model as feature representation to exploit the abundant local body part posture and movement similarities.
- Counterfactual Samples Synthesizing and Training for Robust Visual Question Answering52
A novel model-agnostic Counterfactual Samples Synthesizing and Training (CSST) strategy, which significantly improves both visual-explainable and question-sensitive abilities of VQA models.
- Text-guided multimodal depression detection via cross-modal feature reconstruction and decomposition51
- Unified fair federated learning for digital healthcare51
This work proposes Federated Learning with Unified Fairness Objective (FedUFO), a unified framework consolidating diverse fairness levels within FL that ensures consistent performance across all subpopulations and enhances the overall efficacy of FL in healthcare and other domains while maintaining accuracy levels comparable with those of existing methods.
- Compositional Feature Augmentation for Unbiased Scene Graph Generation50
A novel Compositional Feature Augmentation (CFA) strategy is proposed, which is the first unbiased SGG work to mitigate the bias issue from the perspective of increasing the diversity of triplet features and can be seamlessly incorporated into various SGG frameworks.
- Multi-Turn Video Question Answering via Multi-Stream Hierarchical Attention Context Network50
This paper proposes the hierarchical attention context network for context-aware question understanding by modeling the hierarchically sequential conversation context structure and develops the multi-stream spatio-temporal attention network for learning the joint representation of the dynamic video contents and context- aware question embedding.
- Natural Language Video Localization with Learnable Moment Proposals49
It is argued that the performance of propose-and-rank models are underestimated due to the predefined manners: 1) Hand-designed rules are hard to guarantee the complete coverage of targeted segments and 2) Densely sampled candidate moments cause redundant computation and degrade theperformance of ranking process.
- 48
- Bidirectional Self-Training with Multiple Anisotropic Prototypes for Domain Adaptive Semantic Segmentation45
It is argued that the latent space contains more information to be exploited thus taking one step further to capitalize on it and proposing to represent each category with multiple and anisotropic prototypes via Gaussian Mixture Model, in order to fit the de facto distribution of source domain and estimate the likelihood of target samples based on the probability density.
- Active Learning for Point Cloud Semantic Segmentation via Spatial-Structural Diversity Reasoning44
A new 3D region-based active learning method that groups the original point clouds into superpoints and incrementally selects the most informative and representative ones for label acquisition, achieved via a graph reasoning network that considers both the spatial and structural diversities of superpoints.
- Classification-Then-Grounding: Reformulating Video Scene Graphs as Temporal Bipartite Graphs41
This paper proposes a new classification-then-grounding framework for VidSGG, which can avoid all the three overlooked drawbacks, and reformulates the video scene graphs as temporal bipartite graphs, where the entities and predicates are two types of nodes with time slots, and the edges denote different semantic roles between these nodes.
- Sparse motion bases selection for human motion denoising41
A data-driven based human motion denoising method that sparsely selects the most correlated subset of motion bases for clean motion reconstruction that outperforms its competitors and can be more easily applied to real-world applications.
- NICEST: Noisy Label Correction and Training for Robust Scene Graph Generation40
This article proposes a novel NoIsy label CorrEction and Sample Training strategy for SGG: NICEST, which rules out these noisy label issues by generating high-quality samples and designing an effective training strategy, and proposes a new benchmark, VG-OOD, by reorganizing the prevalent VG dataset.
- Predicting missing markers in human motion capture usingl1‐sparse representation39
A novel method which is named l1‐sparse representation of missing markers prediction (L1‐SRMMP) is proposed, which casts the predicting missing markers as finding a sparse representation of the observable data of the incomplete pose, and then it is used to predict the missing data.
- An Efficient Keyframe Extraction from Motion Capture Data38
The experiments demonstrate that the proposed keyframe extraction method can not only compress and summarize the motion capture data efficiently, but also keep the consistency of keyframe collection between similar human motion sequences, which is of great benefit to further motion data retrieval or editing.
- 36
- Compositional Prompt Tuning with Motion Cues for Open-vocabulary Video Relation Detection35
This paper presents Relation Prompt (RePro) for Open-vocabulary Video Visual Relation Detection (Open-VidVRD), where conventional prompt tuning is easily biased to certain subject-object combinations and motion patterns.
- Label Semantic Knowledge Distillation for Unbiased Scene Graph Generation34
A novel model-agnostic Label Semantic Knowledge Distillation (LS-KD) is proposed for unbiased SGG, which dynamically generates a “soft” label for each subject-object instance by fusing a predicted Label Semantic Distribution (LSD) with its original one-hot target label.
- Improving Weakly Supervised Object Localization via Causal Intervention33
The proposed method, dubbed CI-CAM, explores the causalities among image features, contexts, and categories to eliminate the biased object-context entanglement in the class activation maps thus improving the accuracy of object localization.
- Multichannel Attention Refinement for Video Question Answering33
Appearance, motion, and audio features are extracted from the video, and question-guided attentions are refined to generate the expressive clues that support the correct answer in VideoQA.
- EyecareGPT: Boosting Comprehensive Ophthalmology Understanding with Tailored Dataset, Benchmark and Model30
The Eyecare Kit is proposed, which systematically tackles three key challenges of intelligent ophthalmic diagnosis with the tailored dataset, benchmark and model, and develops the EyecareGPT, optimized for fine-grained ophthalmic visual understanding thoroughly.
- 29
- Tell and guess: cooperative learning for natural image caption generation with hierarchical refined attention26
This paper designed the “Tell and Guess” Cooperative Learning model with a Hierarchical Refined Attention mechanism (CL-HRA) that bidirectionally improves the performance to generate more informative captions.
- 26
- 25
- 24
- 24
- 23
- 22
- 22
- 22
- LSTM-in-LSTM for generating long descriptions of images22
This model outperforms state-of-theart methods on several benchmark datasets when used to generate long rich fine-grained descriptions of given images in terms of four different metrics (BLEU, CIDEr, ROUGE-L, and METEOR).
- 21
- 21
- 21
- Hierarchical Attention Based Spatial-Temporal Graph-to-Sequence Learning for Grounded Video Description21
Two ways to construct a sequence graph that captures spatial-temporal correlations among different objects in each frame are exploited and a novel graph topology refinement technique is presented to discover optimal underlying graph structure.
- Video Question Answering via Knowledge-based Progressive Spatial-Temporal Attention Network21
A knowledge-based progressive spatial-temporal attention network is proposed to tackle the problem of video question answering by taking the spatial and temporal dimension of video content into account and employing an external knowledge base to improve the answering ability of the network.
- Attentional Image Retweet Modeling via Multi-Faceted Ranking Network Learning20
A novel attentional multi-faceted ranking network learning framework with multi-modal neural networks is developed for the proposed heterogenous IRM network to learn the joint image tweet representations and user preference representations for prediction task.
- 20
- 19
- Video question answering via grounded cross-attention network learning18
A novel hierarchical cross-attention mechanism of mutual attention learning for video question answering, named as GCANet is proposed, which helps to learn the joint question-video representation based on both rough representation and grounded representation of video for video questioning.
- 18
- 17
- Abstractive meeting summarization by hierarchical adaptive segmental network learning with multiple revising steps17
This paper proposes the hierarchical neural encoder based on adaptive recurrent networks to learn the semantic representation of meeting conversation with adaptive conversation segmentation with a multi-step revising mechanism to refine the learned semantic representation.
- 17
- 17
- 15
- 15
- Distributionally Generative Augmentation for Fair Facial Attribute Classification14
This work proposes a novel, generation-based two-stage framework to train a fair FAC model on biased data without additional annotation, and enhances interpretability by explicitly showing the spurious attributes in image space.
- 14
- 14
- 14
- 14
- 13
- 13
- 13
- 13
- 13
- 13
- Perspective‐aware cartoon clips synthesis13
In order to construct the cartoons smoothly, the character's edge distance and motion direction distance are demonstrated to be the factors affecting the human perception in similarity evaluation, and utilized in cartoon clips synthesis.
- 11
- Disambiguating named entities with deep supervised learning via crowd labels11
A crowd model is devised to elicit the underlying features (crowd features) from crowd labels that indicate a matching candidate for each mention, and then use the crowd features to fine-tune a dynamic convolutional neural network (DCNN).
- 11
- Adaptive control in cartoon data reusing11
A novel approach, which reuses Traditional Chinese Cartoon to create new animations, utilizes image processing, computer vision, and machine learning in cartoon creation and the experiment results demonstrate that the animation's quality can be effectively improved by the fusion of these techniques.
- 10
- Unified Normalization for Accelerating and Stabilizing Transformers10
Unified Normalization is proposed, which can speed up the inference by being fused with other linear operations and achieve comparable performance on par with LN, and strives to boost performance by calibrating the activation and gradient statistics with a tailored fluctuation smoothing strategy.
- 10
- 10
- 9
- 9
- 9
- 9
- 8
- 8
- 8
- 8
- Weak Supervision Enhanced Generative Network for Question Generation8
The Weak Supervision Enhanced Generative Network (WeGen) is proposed which automatically discovers relevant features of the passage given the answer span in a weakly supervised manner to improve the quality of generated questions.
- 8
- Hierarchical Temporal Fusion of Multi-grained Attention Features for Video Question Answering8
This work proposes the multi-granularity temporal attention network that enables to search for the specific frames in a video that are holistically and locally related to the answer.
- 8
- VL-NMS: Breaking Proposal Bottlenecks in Two-stage Visual-language Matching7
VL-NMS is proposed, which is the first method to yield query-aware proposals at the first stage of a two-stage matching method and validated on three multimodal matching tasks, namely referring expression grounding, phrase grounding, and image-text matching.
- 7
- Video Dialog via Multi-Grained Convolutional Self-Attention Context Networks7
A novel approach for video dialog called multi-grained convolutional self-attention context network, which combines video information with dialog history and can achieve higher time efficiency and the extensive experiments also show the effectiveness of the method.
- 7
- 7
- 7
- 6
- 6
- Photo Stream Question Answer6
This paper presents a new visual question answering (VQA) task -- Photo Stream QA, which aims to answer the open-ended questions about a narrative photo stream, and proposes an end-to-end baseline (E-TAA), which provides promising results outperforming all the other baseline methods.
- 6
- 6
- 6
- 6
- 6
- 6
- 5
- 5
- 5
- 5
- 5
- 5
- 5
- 5
- Video Dialog via Progressive Inference and Cross-Transformer5
This paper introduces a novel progressive inference mechanism for video dialog, which progressively updates query information based on dialog history and video content until the agent think the information is sufficient and unambiguous.
- 5
- 5
- 5
- Robust Modality-Incomplete Anomaly Detection: A Modality-Instructive Framework with Benchmark4
RADAR, a robust two-stage Robust modAlity-instructive fusing & Detecting frAmewoRk, is proposed, which integrates a Modality-Incomplete Instruction mechanism-guiding the multimodal Transformer to focus more on available modal info, and a Double-Pseudo Hybrid Module to highlight unique modality combinations and reduce overfitting.
- 4
- 4
- 4
- 4
- 4
- 4
- 4
- 4
- 4
- 4
- 4
- Adaptive and compact shape descriptor by progressive feature combination and selection with boosting4
- 4
- 4
- 3
- 3
- 3
- 3
- 3
- 3
- 3
- 3
- FedAA: Using Non-sensitive Modalities to Improve Federated Learning while Preserving Image Privacy3
Experiments show that FedAA significantly outperforms recent non-IID federated learning algorithms while preserving image privac, by sharing information from non-sensitive modalities.
- 3
- 3
- Training Encrypted Models with Privacy-preserved Data on Blockchain3
Based on FL, premodel sent a novel and decentralized approach to training encrypted models with privacy-preserved data on Blockchain and demonstrates that the approach is practical in real-world applications.
- 3
- Learning Max-Margin GeoSocial Multimedia Network Representations for Point-of-Interest Suggestion3
This paper proposes a novel max-margin metric geosocial multimedia network representation learning framework by exploiting users' check-in behavior and their social relations and develops a random-walk based learning method with max- margin metric network embedding.
- ENCORE3
A novel neural framework, named ENCORE (External Neural COnstraints REgularized distant supervision), which allows an integration of other information for standard DS through regularizations under multiple external neural networks.
- 3
- A multimodal vision-language model for comprehensive dental diagnosis and enhanced clinical practice2
DentVLM is a dental vision-language model developed to support dental diagnosis across seven oral imaging modalities and 36 tasks that surpasses junior readers, matches intermediate general practitioners and approaches senior specialists, and reduces diagnostic time by 15.0-37.0% in collaborative clinical workflows.
- 2
- 2
- 2
- 2
- 2
- 2
- 2
- 2
- Video question answering via multi-granularity temporal attention network learning2
This work proposes the multi-granularity temporal attention network (MGTA-Net) that enables to search for the specific frames in a video that are holistically and locally related to the answer.
- 2
- 2
- 2
- 2
- 2
- 2
- 2
- 2
- 1
- 1
- MAPTrans: mutual attention transformer with dynamic meta-path pruning for drug repositioning1
MAPTrans, which dynamically optimizes the representation of disease and drug with a multi-level meta-path aggregation strategy, and a mutual attention mechanism Transformer architecture with a cross-view interaction that fuses the information of drugs and diseases in a multi-view space.
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- –
- –
- –
- –
- –
- –
- –
- –
- Generalized Visual Relation Detection With Diffusion Models–
Benefiting from the diffusion-based generative process, the Diff-VRD is able to generate visual relations beyond the pre-defined category labels of datasets, and is able to generate visual relations beyond the pre-defined category labels of datasets.
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- Integrating Side Information for Boosting Machine Comprehension–
A novel attention-based architecture is presented which could feed the side information representations into word level embeddings to explore the comprehension performance and show consistent improvements of the model over various baselines.
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
Publication data from OpenAlex, with missing venues and authors filled in from Crossref; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-13. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.
Report an error
Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.