Is this you? Claim this profile to correct it, add a bio and choose the work people see first.
Claim this profileAcademic lineage
Possible advisorsa guess from early papers, not confirmed
- Tat‐Seng ChuaSuggested from co-authorship
Is this you? Claim this profile to confirm or dismiss it.
Works538 from public data
- NUS-WIDE3,409
The benchmark results indicate that it is possible to learn effective models from sufficiently large image dataset to facilitate general image retrieval and four research issues on web image annotation and retrieval are identified.
- MMGCN951
A Multi-modal Graph Convolution Network (MMGCN) framework built upon the message-passing idea of graph neural networks, which can yield modal-specific representations of users and micro-videos to better capture user preferences is designed.
- Revisiting Graph Based Collaborative Filtering: A Linear Residual Graph Convolutional Network Approach676
This paper revisits GCN based CF models from two aspects and proposes a residual network structure that is specifically designed for CF with user-item interaction modeling, which alleviates the over smoothing problem in graph convolution aggregation operation with sparse user- item interaction data.
- A Neural Influence Diffusion Model for Social Recommendation589
A deep influence propagation model is proposed to stimulate how users are influenced by the recursive social diffusion process for social recommendation, with more than 13% performance improvements over the best baselines for top-10 recommendation on the two datasets.
- Crowded Scene Analysis: A Survey488
The background knowledge and the available features related to crowded scenes are provided and existing models, popular algorithms, evaluation protocols, and system performance are provided corresponding to different aspects of the crowded scene analysis.
- 470
- 458
- Deep Representation Learning With Part Loss for Person Re-Identification439
Experimental results on three person ReID datasets, i.e., Market1501, CUHK03, and VIPeR, show that the proposed deep representation learning procedure named part loss network outperforms existing deep representations.
- DiffNet++: A Neural Influence and Interest Diffusion Network for Social Recommendation372
DiffNet++ is proposed, an improved algorithm of DiffNet that models the neural influence diffusion and interest diffusion in a unified framework and reformulating the social recommendation as a heterogeneous graph with social network and interest network as input.
- Estimation-Action-Reflection: Towards Deep Interaction Between Conversational and Recommender Systems353
This work proposes a new CRS framework named Estimation"Action" Reflection, or EAR, which consists of three stages to better converse with users, and conducts extensive experiments on two datasets from Yelp and LastFM to demonstrate significant improvements over the state-of-the-art method CRM.
- Deep Item-based Collaborative Filtering for Top-N Recommendation339
This article proposes a more expressive ICF solution by accounting for the nonlinear and higher-order relationships among items, and treats this solution as a deep variant of ICF, thus term it as DeepICF.
- Multiple feature hashing for real-time large scale near-duplicate video retrieval333
This paper presents a novel approach - Multiple Feature Hashing (MFH) to tackle both the accuracy and the scalability issues of NDVR and shows that the proposed method outperforms the state-of-the-art techniques in both accuracy and efficiency.
- 305
- Attentive Group Recommendation301
The AGREE model not only improves the group recommendation performance but also enhances the recommendation for users, especially for cold-start users that have no historical interactions individually.
- 289
- 281
- Self-Supervised Video Hashing With Hierarchical Binary Auto-Encoder276
This paper proposes a novel unsupervised video hashing framework dubbed SSVH, which is able to capture the temporal nature of videos in an end-to-end learning to hash fashion, and designs a hierarchical binary auto-encoder to model the temporal dependencies in videos with multiple granularities.
- Image annotation by k NN-sparse graph-based label propagation over noisily tagged web images269
This article exploits the problem of annotating a large-scale image corpus by label propagation over noisily tagged web images by proposing a novel kNN-sparse graph-based semi-supervised learning approach for harnessing the labeled and unlabeled data simultaneously.
- Point-of-Interest Recommendations260
This work proposes to learn a set of potential locations that each individual's friends have checked-in before and this individual is most interested in to improve POI recommendation accuracy and address cold-start problem.
- 244
- 230
- 227
- Learning to Compose and Reason with Language Tree Structures for Visual Grounding220
A natural language grounding model that can automatically compose a binary tree structure for parsing the language and then perform visual reasoning along the tree in a bottom-up fashion and achieves the state-of-the-art performance with more explainable reasoning is proposed.
- 217
- Inferring semantic concepts from community-contributed images and noisy tags212
This paper proposes a novel sparse graph-based semi-supervised learning approach for harnessing the labeled and unlabeled data simultaneously, and constructs an informative compact concept space with small semantic gap to infer the semantic concepts in this space to bridge the semantic gap.
- Spectral-Spatial Constraint Hyperspectral Image Classification209
This paper proposes a hyperspectral image classification method to address both the pixel spectral and spatial constraints, in which the relationship among pixels is formulated in a hypergraph structure, which is conducted on the combinational hypergraph.
- A Review-aware Graph Contrastive Learning Framework for Recommendation192
A novel Review-aware Graph Contrastive Learning (RGCL) framework for review-based recommendation, where each edge feature is composed of both the user-item rating and the corresponding review semantics, and two additional contrastive learning tasks are designed to provide self-supervised signals for the two components in recommendation process.
- Learning Fair Representations for Recommendation: A Graph-based Perspective189
This paper proposes a novel graph based technique for ensuring fairness of any recommendation models, and extensive experimental results clearly show the effectiveness of the proposed model for fair recommendation.
- 186
- 164
- Harvesting visual concepts for image search with complex queries160
A scheme to enhance web image reranking for complex queries by fully exploring the information from simple visual concepts is presented and two applications are introduced: photo-based question answering and textual news visualization.
- 150
- 148
- 148
- 144
- Creating Something From Nothing: Unsupervised Knowledge Distillation for Cross-Modal Hashing139
This paper proposes a novel approach that enables guiding a supervised method using outputs produced by an unsupervised method, and makes use of teacher-student optimization for propagating knowledge.
- Image Classification and Retrieval are ONE138
It is demonstrated that the essentials of image classification and retrieval are the same, since both tasks could be tackled by measuring the similarity between images, and a unified algorithm is proposed, which only involves manual object definition, regional description and nearest-neighbor search.
- 135
- Enhanced Graph Learning for Collaborative Filtering via Mutual Information Maximization133
It is argued that node embedding learning and graph structure learning can mutually enhance each other in CF, as updated node embeddings are learned from previous graph structure, and vice versa.
- Knowledge-aware Multimodal Dialogue Systems132
A Knowledge-aware Multimodal Dialogue model is presented to address the limitation of text-based dialogue systems and significantly outperforms state-of-the-art methods, demonstrating the efficacy of modeling visual modality and domain knowledge for dialogue systems.
- Interleaved Structured Sparse Convolutional Neural Networks130
This paper presents a modularized building block, IGC-V2: interleaved structured sparse convolutions, which generalizes interleaves group convolutions to the product of more structured sparse kernels, further eliminating the redundancy.
- Generative-Contrastive Graph Learning for Recommendation128
A novel Variational Graph Generative-Contrastive Learning (VGCL) framework for recommendation that leverages variational graph reconstruction to estimate a Gaussian distribution of each node, then generates multiple contrastive views through multiple samplings from the estimated distributions, which builds a bridge between generative and contrastive learning.
- Joint Item Recommendation and Attribute Inference124
An Adaptive Graph Convolutional Network approach for joint item recommendation and attribute inference is proposed, which could adaptively adjust the graph embedding learning parameters by incorporating both the given attributes and the estimated attribute values, in order to provide weakly supervised information to refine the two tasks.
- 121
- Beyond Doctors120
An adaptive multimodal multi-task learning model is proposed to co-regularize the modality agreement, temporal progression and discriminative capabilities of different modalities of Alzheimer's disease and is theoretically show that it is a linear system.
- W2Go120
A novel Automatic Landmark Ranking (ALR) method is proposed by utilizing the tag and geo-tag information of photos in Flickr and user knowledge from Yahoo Travel Guide, which can automatically recognize and rank the landmarks for travellers.
- 117
- From Static to Dynamic: Adapting Landmark-Aware Image Models for Facial Expression Recognition in Videos116
A novel Static-to-Dynamic model (S2D) that leverages existing SFER knowledge and dynamic information implicitly encoded in extracted facial landmark-aware features, thereby significantly improving DFER performance and achieving a new state of the art.
- 115
- 114
- 112
- Dynamic captioning112
A video accessibility enhancement scheme with a Dynamic Captioning approach, which explores a rich set of technologies including face detection and recognition, visual saliency analysis, text-speech alignment, etc, to help hearing impaired audience better recognize the speaking characters.
- Enhancing Micro-video Understanding by Harnessing External Sounds110
A deep transfer model is developed which can jointly enhance the concept-level representation of micro-videos and the venue category prediction and alleviate the sparsity problem of unpopular categories by leveraging the external acoustic knowledge.
- Deep Adversarial Graph Attention Convolution Network for Text-Based Person Search109
A novel deep adversarial graph attention convolution network (A-GANet) for text-based person search that learns an effective joint textual-visual latent feature space in adversarial learning manner, bridging modality gap and facilitating pedestrian matching.
- Differentiable Cross-modal Hashing via Multimodal Transformers108
A differentiable cross-modal hashing method is proposed that utilizes the multimodal transformer as the backbone to capture the location information in an image when encoding the visual content and this method outperforms many existing solutions.
- 108
- 107
- 106
- Beyond search106
A novel scheme that is able to summarize the content of video search results by mining and threading “key” shots, such that users can get an overview of main content of these videos at a glance is presented.
- Image clustering based on sparse patch alignment framework105
This paper proposes a novel algorithm, named sparse patch alignment framework, for the embedding of data lying in multiple manifolds, and proposes an optimization strategy for constructing local patches, which adopt sparse representation to select a few neighbors of each data point that span a low-dimensional affine subspace passing near that point.
- 104
- 104
- 103
- 101
- 97
- Play and Rewind97
This paper proposes a novel unsupervised video hashing framework called Self-Supervised Temporal Hashing (SSTH) that is able to capture the temporal nature of videos in an end- to-end learning-to-hash fashion and develops a backpropagation rule that tackles the non-differentiability of BLSTM.
- Dual Learning for Explainable Recommendation: Towards Unifying User Preference Prediction and Review Generation95
A unified dual framework of how to inject the probabilistic duality of the two tasks in the training stage is designed, and a transfer learning based model for preference prediction and review generation is proposed.
- 94
- Enhancing Person Re-identification in a Self-Trained Subspace87
A self-trained subspace learning paradigm for person re-ID is proposed that effectively utilizes both labeled and unlabeled data to learn a discriminative subspace where person images across disjoint camera views can be easily matched.
- Cross-Entropy Adversarial View Adaptation for Person Re-identification86
This work learns view-invariant subspace for person re-ID, and its corresponding similarity metric using an adversarial view adaptation approach, and achieves notably improved performance in comparison with the state-of-the-arts on benchmark datasets.
- Real-World Person Re-Identification via Degradation Invariance Learning84
By introducing a self-supervised disentangled representation learning strategy, this framework is able to simultaneously extract identity-related robust features and remove real-world degradations without extra supervision and is ableTo achieve state-of-the-art performance on several Re-ID benchmarks.
- Beyond Object Proposals: Random Crop Pooling for Multi-Label Image Recognition84
This paper proposes an object-proposal-free framework for multi-label image recognition: random crop pooling (RCP), which performs stochastic scaling and cropping over images before feeding them to a standard convolutional neural network, which works quite well with a max-pooling operation for recognizing the complex contents of multi- label images.
- MEGCF: Multimodal Entity Graph Collaborative Filtering for Personalized Recommendation83
A novel model, multimodal entity graph collaborative filtering, short for MEGCF, is developed that captures the semantic correlation between interactions and the features obtained from MFE, thus making a better match between MFE and UIM.
- Multicolumn Bidirectional Long Short-Term Memory for Mobile Devices-Based Human Activity Recognition83
- Multimodal Dialog System81
A Multimodal diAloG system with adaptIve deCoders, MAGIC, which outperforms the existing methods and achieves the state-of-the-art performance over a benchmark dataset.
- 81
- 80
- 80
- Semi-supervised kernel density estimation for video annotation79
An improved algorithm named semi-supervised adaptive kernel density estimation (SSAKDE) which employs local adaptive kernels rather than a fixed kernel, such that broader kernels can be applied in the regions with low density, and more accurate density estimates can be obtained.
- Multi-Stage Vision Token Dropping: Towards Efficient Multimodal Large Language Model78
Multi-stage Token Dropping (MustDrop) is proposed to measure the importance of each token from the whole lifecycle, including the vision encoding stage, prefilling stage, and decoding stage, and can more precisely recognize the important and redundant tokens, thus achieving an optimal balance between performance and efficiency.
- Contrastive Video Question Answering via Video Graph Transformer78
With superior video encoding and QA solution, it is shown that CoVGT can achieve much better performances than previous arts on video reasoning tasks and can also benefit from cross-modal pretraining, yet with orders of magnitude smaller data.
- Fast Matrix Factorization With Nonuniform Weights on Missing Data77
A fast learning method, for which the time complexity is determined by the number of observed entries in the data matrix rather than the matrix size, and the key idea is truncated singular value decomposition on the weight matrix to get a more compact representation of the weights.
- Improving Recommendation Fairness via Data Augmentation75
This paper designs a bi-level optimization target, with the inner optimization generates better fake data to augment training data with the authors' hypotheses, and the outer one updates the recommendation model parameters based on the augmented training data.
- Image Categorization by Learning a Propagated Graphlet Path74
This paper proposes to learn object-shaped and directional receptive fields for image categorization, and derives a manifold propagation to calculate the postembedding graphlets by leveraging visual saliency maps.
- 72
- Gated CNN: Integrating multi-scale feature layers for object detection71
The proposed G-CNN employs a detector with two branches to predict the locations and categories of objects, respectively, as well as an inter-class loss to help detectors learn discrepant information among categories so that the learned detectors could better differentiate similar objects of different categories.
- Double Correction Framework for Denoising Recommendation70
A Double Correction Framework for Denoising Recommendation (DCF) is proposed, which contains two correction components from views of more precise sample dropping and avoiding more sparse data, and derives a lower bound for the loss through concentration inequality to identify and reuse hard samples.
- Detecting Densely Distributed Graph Patterns for Fine-Grained Image Categorization70
A dense graph mining algorithm is developed to discover graphlets representative to each super-/sub-category, and the discovered graphlets from each sub-category accurately capture those tiny discriminative object components, e.g., bird claws, heads, and bodies.
- 70
- Gloss-driven Conditional Diffusion Models for Sign Language Production69
A Gloss-driven Conditional Diffusion Model for SLP, in which the sign gloss sequence is encoded by a Transformer-based encoder and input into the diffusion model as a semantic prior condition, achieves competitiveness in both quantitative performance and qualitative visualization.
- Gloss Semantic-Enhanced Network with Online Back-Translation for Sign Language Production68
A Gloss semantic-Enhanced Network is proposed with Online Back-Translation (GEN-OBT) for G2P in the SLP task, which outperforms the state-of-the-art models and designs a CTC-based reverse decoder to convert the generated poses backward into glosses, which guarantees the semantic consistency during the processes of gloss- to-pose and pose-to-gloss.
- 67
- Video accessibility enhancement for hearing-impaired users67
A scheme to enhance video accessibility using a Dynamic Captioning approach, which explores a rich set of technologies including face detection and recognition, visual saliency analysis, text-speech alignment, etc, to help the hearing-impaired audience better recognize the speaking characters.
- Real-Time Semantic Segmentation via Spatial-Detail Guided Context Propagation62
This work proposes the spatial-detail guided context propagation network (SGCPNet), which uses the spatial details of shallow layers to guide the propagation of the low-resolution global contexts, in which the lost spatial information can be effectively reconstructed and improve the model efficiency.
- 62
- 62
- FCL-GAN: A Lightweight and Real-Time Baseline for Unsupervised Blind Image Deblurring59
A lightweight and real-time unsupervised BID baseline, termed Frequency-domain Contrastive Loss Constrained Lightweight CycleGAN (shortly, FCL-GAN), with attractive properties, i.e., no image domain limitation, no image resolution limitation, 25x lighter than Sota, and 5x faster than SOTA is proposed.
- Adaptive Data-Free Quantization58
An Adaptive Data-Free Quantization (AdaDFQ) method is proposed, which revisits DFQ from a zero-sum game perspective upon the sample adaptability between two players — a generator and a quantized network, and reveals the largest adaptability is NOT the best for sample generation to benefit Q's generalization.
- 57
- Joint Multi-Grained Popularity-Aware Graph Convolution Collaborative Filtering for Recommendation57
This work develops a Joint Multigrained Popularity-aware Graph Convolution Collaborative Filtering model, short for JMP-GCF, which uses a popularity-aware embedding generation to construct multigrained popularity features and uses the idea of joint learning to capture the signals within and between different granularities of popularity features that are relevant for modeling user preferences.
- Memory-Augmented Relation Network for Few-Shot Learning57
This work investigates a new metric-learning method to explicitly exploit its relationships with the others in the working context, and formulate the distance metric as a learnable relation module which learns to compare for similarity measurement, and equip theWorking context with memory slots, both contributing to generality.
- 56
- Graph Bottlenecked Social Recommendation55
This paper proposes a novel Graph Bottlenecked Social Recommendation (GBSR) framework, a model-agnostic social denoising framework that aims to maximize the mutual information between the denoised social graph and recommendation labels, meanwhile minimizing it between the denoised social graph and the original one.
- 55
- Multimedia Question Answering53
QA's evolution from text to multimedia and the challenges in the field are identified by surveying recent research in multimedia question answering.
- Multimodality Invariant Learning for Multimedia-Based New Item Recommendation52
This paper argues that users' inherent content preference is stable and better kept invariant to arbitrary modality missing environments, and proposes a novel Multimodality Invariant Learning reCommendation (a.k.a MILK) framework, which designs multi-modal heterogeneous environments with cyclic mixup to augment training data, in order to mimic any modality missing for invariant user preference learning.
- 52
- 52
- 51
- Movie Question Answering: Remembering the Textual Cues for Layered Visual Contents51
A Layered Memory Network (LMN) that represents frame-level and clip-level movie content by the Static Word Memory module and the Dynamic Subtitle Memory module is put forward, which achieves the state-of-the-art performance on the online evaluation task of 'Video+Subtitles'.
- Event driven summarization for web videos51
This paper presents a novel solution by mining and threading "key" shots, which can provide an overview of main contents of videos at a glance, by summarizing a large set of diverse videos by utilizing the way of dynamic video skimming.
- 50
- Towards Multimodal Emotional Support Conversation Systems49
This work introduces the Multimodal Emotional Support Conversation dataset, a first-of-its-kind resource enriched with comprehensive annotations across text, audio, and video modalities, and proposes a general Sequential Multimodal Emotional Support framework (SMES) grounded in Therapeutic Skills Theory.
- Fine-Grained Fashion Similarity Prediction by Attribute-Specific Embedding Learning49
Extensive experiments on three fashion-related datasets, i.e., FashionAI, DARN, and DeepFashion, show the effectiveness of ASEN for fine-grained fashion similarity prediction and its potential for fashion reranking.
- 49
- A Coarse-to-Fine Multi-stream Hybrid Deraining Network for Single Image Deraining49
A new coarse-to-fine single image deraining framework termed Multi-stream Hybrid Deraining Network (shortly, MH-DerainNet) is proposed, which combines the SSIM loss and perceptual loss to preserve the per-pixel similarity as well as preserving the global structures so that the deraining result is more accurate.
- 49
- 48
- 46
- Learning Hierarchical Visual Transformation for Domain Generalizable Visual Matching and Recognition45
An effective Hierarchical Visual Transformation network is proposed to be extended as EHVT, which helps the model to effectively learn domain-invariant representation and narrow the domain gap in various visual matching and recognition tasks, such as stereo matching, pedestrian retrieval, and image classification.
- 45
- 45
- 45
- 44
- Fair Representation Learning for Recommendation: A Mutual Information Perspective43
This paper re-define recommendation fairness with a novel two-fold mutual information objective, and proposes novel mutual information (upper/lower) bounds with contrastive information estimation for model optimization.
- 43
- Privileged Graph Distillation for Cold Start Recommendation42
A privileged graph distillation model (PGD) that can learn better embeddings of each entity by injecting complex higher-order relationships from the constructed heterogeneous graph structure for warm users and items with privileged CF links is proposed.
- Large-Scale Aerial Image Categorization Using a Multitask Topological Codebook42
This work proposes an efficient aerial image categorization algorithm that focuses on learning a discriminative topological codebook of aerial images under a multitask learning framework and is competitive to several existing recognition models.
- 41
- 41
- 41
- 41
- Image quality assessment based on matching pursuit41
This paper decomposes the reference image using matching pursuit with a separable 2D Gabor dictionary, thus obtaining structural information, and develops a characterization for this information and its importance and discusses the relationship between the structural distortion intensity and the subjective quality measurement on the energy scale.
- Generative Metric Learning for Adversarially Robust Open-world Person Re-Identification40
This work improves the robustness of open-world re-ID models by proposing a generative metric learning approach to generate adversarial examples that are regularized to produce robust distance metric.
- 40
- 39
- 39
- 39
- GoRec: A Generative Cold-start Recommendation Framework38
A Generative cold-start Recommendation (GoRec) framework for multimedia-based new item recommendation that first estimates the underlying distribution of each warm item conditioned on the multimedia content representation and proposes a uniformity-enhanced optimization objective to ensure the latent space of CVAE is more distinguishable and informative.
- Automatic Depression Detection via Learning and Fusing Features From Visual Cues38
This article proposes a novel ADD method via learning and fusing features from visual cues, which achieves the state-of-the-art performance on the Distress Analysis Interview Corpus Wizard- of-Oz dataset compared with other visual-feature-based methods, showing its effectiveness.
- Revisiting Local Descriptor for Improved Few-Shot Classification38
A new method, named DCAP, is presented, in which it is shown that the reliance on sophisticated classifiers is not necessary, and a simple classifier applied directly to improved feature embeddings can instead outperform most of the leading methods in the literature.
- Augmented Collaborative Filtering for Sparseness Reduction in Personalized POI Recommendation38
This article aims to exploit geotagged photos from online photo-sharing sites for the purpose of personalized Point-of-Interest (POI) recommendation by augmenting current collaborative filtering algorithms along from multiple perspectives and applying the matrix factorization algorithm to integrate the disparate sources of preference and relationship information.
- 38
- Cross-Domain Collaborative Learning in Social Multimedia38
This work proposes a generic Cross-Domain Collaborative Learning (CDCL) framework based on non-parametric Bayesian dictionary learning model for cross-domain data analysis that can effectively explore the virtues of different information sources to complement and enhance each other forCross- domain data analysis.
- 37
- Average User-Side Counterfactual Fairness for Collaborative Filtering37
This work shows that removing causal effects of sensitive attributes is equal to average counterfactual fairness in CF, and is one of the first few attempts to achieve counterfactual fairness from the causal effect estimation perspective in CF.
- 37
- Learning to Transfer Graph Embeddings for Inductive Graph based Recommendation37
This paper proposes an inductive Graph based Transfer learning framework for personalized video highlight Recommendation (TransGRec), which is composed of a graph neural network followed by an item embedding transfer network that exploits the higher-order proximity between users and segments to alleviate the user cold-start problem.
- Joint Subspace Recovery and Enhanced Locality Driven Robust Flexible Discriminative Dictionary Learning37
The RFDDL mainly improves the data representation and classification abilities by enhancing the robust property to sparse errors and encoding the locality, reconstruction error, and label consistency more accurately.
- From text question-answering to multimedia QA on web-scale media resources37
The aim of this paper is to extend the text-based QA research to multimedia QA to tackle a range of factoid, definition and "how-to" QA in a common framework and to find multimedia answers from Web-scale media resources such as Flicker and YouTube.
- LipFormer: Learning to Lipread Unseen Speakers Based on Visual-Landmark Transformers36
This paper proposes to use multi-modal features, i.e., visual and landmark, to describe the lip motion while being irrespective to speaker characteristics, and proposes a sentence-level framework, dubbed LipFormer, based on visual-landmark transformer architecture.
- 36
- 36
- SGINet: Toward Sufficient Interaction Between Single Image Deraining and Semantic Segmentation35
A Semantic Guided Interactive Network (SGINet), which considers the sufficient interaction between SID and semantic segmentation using a three-stage deraining manner, i.e., coarse deraining, semantic information extraction, and semantics guided deraining.
- On fusing the latent deep CNN feature for image classification35
This paper proposes a method to fuse the latent features extracted from the middle layers in a CNN to train a more robust classifier, and uses the late fusion strategy to combine the prediction of these classifiers.
- 35
- 34
- Adversarial Video Moment Retrieval by Jointly Modeling Ranking and Localization34
The key of the solution is to formulate the video moment retrieval task as an adversarial learning problem with two tightly connected components, which is able to jointly optimize the performance of both video moment ranking and video moment localization.
- Video Retrieval with Similarity-Preserving Deep Temporal Hashing33
This article proposes an end-to-end video retrieval framework called the Similarity-Preserving Deep Temporal Hashing (SPDTH) network, which unifies video temporal modeling and learning to hash into one step to allow for maximum retention of information.
- Weakly Supervised Multilabel Clustering and its Applications in Computer Vision33
This paper proposes weakly supervised multilabel clustering, which allows assigning multiple labels to a bag and a deterministic annealing strategy is developed to optimize the nonconvex objective function.
- A relaxed ranking-based factor model for recommender system from implicit feedback33
A Relaxed Ranking-based Factor Model, RRFM, is proposed to relax pairwise ranking into a SVM-like task, where positive and negative feedbacks are separated by the soft boundaries, and their non-separate property is employed to capture the characteristic of unobserved data.
- 33
- 32
- 32
- 32
- Discrete to Continuous: Generating Smooth Transition Poses from Sign Language Observations31
This work proposes a novel framework, Sign-D2C, that employs a conditional diffusion model to synthesize contextually smooth transition frames, enabling the seamless construction of continuous sign language sequences.
- Read, Diagnose and Chat: Towards Explainable and Interactive LLMs-Augmented Depression Detection in Social Media31
A new depression detection system based on LLMs that is both interpretable and interactive that outperforms traditional methods across various settings and is demonstrated through case studies.
- 31
- Learning User Attributes via Mobile Social Multimedia Analytics31
A unified model is proposed to jointly regularize the source consistency and graph-constrained relatedness among tasks and is able to learn the attribute-specific and attribute-sharing features via graph-guided fused lasso penalty.
- Dimensionality reduction on Anchorgraph with an efficient Locality Preserving Projection31
An improved dimensionality reduction algorithm called Anchorgraph-based Locality Preserving Projection (AgLPP) is proposed, trying to cope with the limitations via a novel estimation of the relationship between data points.
- Domain Generalization for Face Anti-Spoofing via Content-Aware Composite Prompt Engineering30
This work proposes a novel Content-aware Composite Prompt Engineering (CCPE), a novel Content-aware Composite Prompt Engineering (CCPE) that generates instance-wise composite prompts, including both fixed template and learnable prompts, including both fixed template and learnable prompts.
- 30
- Image detail enhancement with spatially guided filters30
A spatially guided map is constructed which exploits the spatial influence of image details based on the edge response of an image and is integrated into two state-of-the-art image filters and applies the improved filters to the task of image detail enhancement.
- 29
- 29
- 29
- In-video product annotation with web information mining29
This article presents a novel solution to the annotation of specific products in videos by mining information from the Web by simultaneously leveraging Amazon and Google image search engine.
- Exploring Transferability of Multimodal Adversarial Samples for Vision-Language Pre-Training Models With Contrastive Learning28
A novel gradient-based multimodal adversarial attack method, underpinned by contrastive learning, to improve the transferability of multimodal adversarial samples in VLP models, concurrently generates adversarial texts and images within imperceptive perturbation.
- A Topic-Attentive Transformer-based Model For Multimodal Depression Detection28
Large pre-trained models are introduced, and the fine-tuning strategy is adopted based on the small-scale ADD training data, and a two-branch architecture with a late-fusion strategy for building the TOAT model, in which the textual and audio data are encoded independently.
- Online Filter Clustering and Pruning for Efficient Convnets28
An extra cluster loss term is added in the loss function which can force filters in each cluster to be similar online online and fine-tune the pruned network to compensate the loss.
- Doubly Abductive Counterfactual Inference for Text-Based Image Editing27
A Doubly Abductive Counterfactual inference framework that can support a wide spectrum of user editing intents, including addition, removal, manipulation, replacement, style transfer, and facial change, which are extensively validated in both qualitative and quantitative evaluations.
- Robust Attention Deraining Network for Synchronous Rain Streaks and Raindrops Removal27
This paper proposes a new and universal SID model with novel modules, termed Robust Attention Deraining Network (RadNet), with strong robustness and generalization ability, and demonstrates that this RadNet could outperform other state-of-the-art SID methods.
- Early-Learning regularized Contrastive Learning for Cross-Modal Retrieval with Noisy Labels27
The proposed Early Learning regularized Contrastive Learning method for Cross Modal Retrieval with Noisy Labels (ELRCMR) is evaluated, and the result shows it could solve the cluster drift in conventional solutions and achieve promising performance on widely used benchmark datasets.
- CRNet: Unsupervised Color Retention Network for Blind Motion Deblurring27
A novel unsupervised color retention network termed CRNet is proposed to perform blind motion deblurring and a new module called pyramid global blur feature perception (PGBFP) is proposed, to quantitatively prove the effectiveness of the network in color retention.
- Single-shot Semantic Image Inpainting with Densely Connected Generative Networks27
A new end-to-end framework named Single-shot Densely Connected Generative Network (SSDCGN), which generates visually realistic and semantically distinct pixels for the missing content by a battery of symmetric encoder-decoder groups.
- Computational Social Indicators27
A novel graph-based multi-channel ranking scheme for social indicator computation by exploring the rich multi- Channel Web data and using a unified model to learn the cluster-wise common spaces, perform ranking separately upon each space, and fuse these rankings to produce the final one.
- 27
- 27
- 27
- 26
- Unlearnable Examples Give a False Sense of Security: Piercing through Unexploitable Data with Learnable Examples26
LE is formally defined by introducinglearnable unauthorized examples (LEs) which are UEs with their protection removed, which is the first generalizable countermeasure to UEs across supervised learning and unsupervised learning.
- Multi-scale Spatial Representation Learning via Recursive Hermite Polynomial Networks26
Recursive Hermite Polynomial Networks are proposed, which recursively constructs sub-scale representations to avoid the artifacts caused by naively applying the dilation convolution, and reveal its superiority over state-of-the-art alternatives on a variety of image recognition tasks.
- Visual Classification of Furniture Styles26
This article constructs a novel dataset of furniture styles that contains 16 common style categories and implements three strategies with respect to two categories of classification, that is, handcrafted classification and learning-based classification and the convolutional neural network.
- Exploring large scale data for multimedia QA26
An initial study on exploring large scale data for MMQA, which constructs a web video dataset and discusses its query strategy, statistics, feature description and groundtruth, and conducts experiments to answer definition event questions.
- Sign-IDD: Iconicity Disentangled Diffusion for Sign Language Production25
This work proposes a pioneering iconicity disentangled diffusion framework, termed Sign-IDD, specifically designed for SLP, which incorporates a novel Iconicity Disentanglement module to bridge the gap between relative positions among joints.
- Embedded Heterogeneous Attention Transformer for Cross-Lingual Image Captioning25
The proposed Embedded Heterogeneous Attention Transformer (EHAT) framework effectively addresses the challenges of cross-lingual image captioning, paving the way for improved multilingual image analysis and understanding.
- 25
- LCSNet: End-to-end Lipreading with Channel-aware Feature Selection25
An end-to-end deep neural network-based lipreading model (LCSNet) that extracts the short-term spatio-temporal features and the motion trajectory features from the lip region in the video clips and achieves improvement of 52% and 47% compared to LipNet.
- Advance on large scale near-duplicate video retrieval25
A comprehensive survey and an updated review of the advance on large-scale NDVR to supply guidance for researchers and present the development trends and research directions of this topic.
- Multimodal Counterfactual Learning Network for Multimedia-based Recommendation24
This work proposes a novel model called Multimodal Counterfactual Learning Network (MCLN), in which user-uninteracted items' multimodal content is additionally exploited to further purify the representation of user preference-relevant multimodAL content that better matches the user's interests, yielding state-of-the-art performance.
- 24
- Deep Neighborhood Component Analysis for Visual Similarity Modeling24
A novel end-to-end approach for visual similarity modeling, called deep neighborhood component analysis, is proposed, which discriminatively trains deep neural networks to jointly learn visual features and similarities.
- 24
- 24
- Align Is Not Enough: Multimodal Universal Jailbreak Attack Against Multimodal Large Language Models23
This work proposes a unified multimodal universal jailbreak attack framework that leverages iterative image-text interactions and transfer-based strategy to generate a universal adversarial suffix and image, and validates that multimodal universal jailbreak attacks can bring higher-quality undesirable generations across different MLLMs.
- Erasing-based Attention Learning for Visual Question Answering23
This paper proposes to enforce the correlation between the attention and the non-attention parts as a constraint for attention learning, and adopts an attention-guided erasing scheme and associates a typical classification loss with the above distance constraint to learn a more discriminative attention map for answer prediction.
- A Spatial-Temporal Probabilistic Matrix Factorization Model for Point-of-Interest Recommendation23
A new Spatial-Temporal Probabilistic Matrix Factorization (STPMF) model is proposed that models a user’s preference for POI as the combination of his geographical preference and other general interest in POI and captures the temporal dynamics of user”s interest as well by modeling checkin data in a unique way.
- 23
- Directional projection based image fusion quality metric23
Experimental results demonstrate that the proposed metric outperforms other metrics while it is consistent with the subjective evaluation of image fusion.
- Unveiling Uncertainty: A Deep Dive into Calibration and Performance of Multimodal Large Language Models22
It is revealed that MLLMs tend to give answers rather than admit uncertainty, but this self-assessment improves with proper prompt adjustments, and techniques such as temperature scaling and iterative prompt optimization are proposed to calibrate MLLMs.
- 22
- 22
- Category-Level Articulated Object 9D Pose Estimation via Reinforcement Learning22
A novel framework called Articulated object 9D Pose Estimation via Reinforcement Learning (ArtPERL), which formulates the category-level ArtOPE as a reinforcement learning problem, and introduces a joint-centric pose modeling strategy that estimates 9D pose by fitting joint states via reinforced agent training.
- Topic-enhanced Graph Neural Networks for Extraction-based Explainable Recommendation22
This paper focuses on the explicit and implicit analysis of review information simultaneously and proposes novel a Topic-enhanced Graph Neural Networks (TGNN) to fully explore review information for better explainable recommendations.
- 22
- 22
- A Joint Neural Model for User Behavior Prediction on Social Networking Platforms22
A neural joint behavior prediction model named Neural Joint Behavior Prediction Model (NJBP) is proposed to mutually enhance the prediction performance of these two tasks on social networking platforms and designs a fusion layer in the neural network to model the positive correlation of users’ two kinds of behaviors.
- 21
- 21
- Multimodal Feature Extraction and Fusion for Emotional Reaction Intensity Estimation and Expression Classification in Videos with Transformers21
This paper proposes a streamlined approach that handles the challenges of classification effectively for the Expression Classification Challenge and achieves outstanding results on the Emotional Reaction Intensity (ERI) Estimation Challenge, surpassing the baseline method by an impressive 84% increase, as measured by the Pearson Coefficient, on the validation dataset.
- Set2setRank21
This paper explores the unique characteristics of the implicit feedback and proposes Set2setRank framework for recommendation, a model-agnostic framework that can be easily applied to most recommendation prediction approaches, and is time efficient in practice.
- Random walk based distributed representation learning and prediction on Social Networking Services21
A random walk based distributed representation learning model is proposed to jointly predict these behaviors on SNSs that combines the two behaviors, and adopts a random walk to capture higher-order relationships between users and items.
- Biologically Inspired Media Quality Modeling21
This paper proposes a biologically inspired quality model, focusing on interpreting how humans perceive visually and semantically important regions in an image (or a video clip) and shows that this model outperforms many of its competitors remarkably.
- 21
- Movie2Comics21
A scheme that is able to automatically turn a movie to comics with two principles: optimizing the information reservation of movie and generating outputs following the rules and styles of comics is proposed.
- Few-Shot Learner Parameterization by Diffusion Time-Steps20
Time-step Few-shot (TiF) learner significantly outperforms OpenCLIP and its adapters on a variety of fine-grained and customized few-shot learning tasks.
- Robust Low-Rank Convolution Network for Image Denoising20
This paper proposes a new encoder-decoder based image restoration network, termed Robust Low-Rank Convolution Network with Feature Denoising (LRCnet), and designs a residual dense connection with a channel attention to connect multilevel feature effectively to obtain more useful information to enhance the data representation.
- 20
- Visual summarization of image collections by fast RANSAC20
A novel approach to select a summary set of images from a large image collection by improved Random Sample Consensus (RANSAC) and Affinity Propagation (AP) clustering that significantly outperforms other methods.
- Video recommendation over multiple information sources20
A universal scheme to integrate rich information for personalized video recommendation based on multiple ranking lists based on a multi-task rank aggregation approach to generate a final result for video recommendation is proposed.
- ViewFocus20
A novel system to explore places of interests based on the large amount of photos that are placed on Google Maps, named ViewFocus, estimates the view directions of photos via robust object matching and camera reconstruction techniques, and geo-registers the directions on the map.
- 19
- 19
- RGCF: Refined graph convolution collaborative filtering with concise and expressive embedding19
A novel GCN-based recommendation model, named Refined Graph convolution Collaborative Filtering (RGCF), where a refined graph convolution structure is designed to match non-semantic ID inputs, and a new fine-tuned symmetric normalization is proposed to mine node popularity characteristics and further incorporate the popularity features into the embedding learning process.
- Densely Connected Attention Flow for Visual Question Answering19
A novel DCAF (Densely Connected Attention Flow) framework for modeling dense interactions densely connects all pairwise layers of the network via Attention Connectors, capturing fine-grained interplay between image and question across all hierarchical levels.
- Cross-Modality Feature Learning via Convolutional Autoencoder19
Compared to the conventional solutions relying on hand-crafted features, the proposed MUCAE approach encodes features from image pixels and text characters directly and produces more representative and robust features.
- Knowledge-aware Multimodal Fashion Chatbot19
A multimodal domain knowledge enriched fashion chatbot forms a taxonomy-based learning module to capture the fine-grained semantics in images and leverages an end-to-end neural conversational model to generate responses based on the conversation history, visual semantics, and domain knowledge.
- 19
- Lace fabric image retrieval based on multi-scale and rotation invariant LBP19
This paper proposes an effective Multi-scale and Rotation Invariant Local Binary Pattern (MRI-LBP) feature, based on the thorough analysis on the characteristics of this kind of image, which is used in the lace fabric image retrieval task.
- Invariance Matters: Empowering Social Recommendation via Graph Invariant Learning18
This paper approaches the social denoising problem from the perspective of graph invariant learning and proposes a novel approach, Social Graph Invariant Learning (SGIL), which aims to uncover stable user preferences within the input social graph, thereby enhancing the robustness of graph-based social recommendation systems.
- Inner Knowledge-based Img2Doc Scheme for Visual Question Answering18
An inner knowledge-based Img2Doc algorithm for VQA is presented and it is demonstrated that the proposed method achieves superior performance on multiple benchmark datasets.
- 18
- Advertising object in web videos18
This paper presents a novel video advertising system called VideoAder that leverages the well organized media information from the video corpus for embedding visual content relevant ads into a set of precisely located insertion position.
- On Video Recommendation over Social Network18
A novel approach to improve the accuracy of video recommendation by calculating a recommendation score for each video candidate that composes of two parts: the interest degree of this video by the user's friends, and the relationship strengths between the user and his friends.
- Static for Dynamic: Towards a Deeper Understanding of Dynamic Facial Expressions Using Static Expression Data17
This work proposes Static-for-Dynamic (S4D), a unified dual-modal learning framework that integrates SFER data as a complementary resource for DFER, and proposes an innovative Mixture of Adapter Experts (MoAE) module that facilitates task-specific knowledge acquisition while effectively extracting shared knowledge from both static and dynamic expression data.
- Group Multi-View Transformer for 3D Shape Analysis With Spatial Encoding17
A compression method based on knowledge distillation is introduced for 3D shape recognition methods, which largely reduces the number of parameters while preserving model performance as much as possible and proposes two compressed versions, namely GMViT-simple and GMViT-mini.
- Path-Specific Causal Reasoning for Fairness-aware Cognitive Diagnosis17
This work argues that sensitive attributes of students can also provide useful information, and only the shortcuts directly related to the sensitive information should be eliminated from the diagnosis process, and employs causal reasoning and design a novel Path-Specific Causal Reasoning Framework (PSCRF) to achieve this goal.
- Dual Context-Aware Refinement Network for Person Search17
A novel dual context-aware refinement network (DCRNet) for person search is proposed, which jointly explores two kinds of contexts including intra-instance context and inter- instance context to learn discriminative representation.
- 17
- 17
- 17
- Mediapedia: Mining Web Knowledge to Construct Multimedia Encyclopedia17
A dynamic multimedia encyclopedia that takes advantage of, and in fact is built from the text and image resources on the Web, and is flexible and extensible such that it can easily incorporate new kinds of mediums such as video and languages into the framework.
- Linguistics-Vision Monotonic Consistent Network for Sign Language Production16
A Transformer-based Linguistics-Vision Monotonic Consistent Network (LVMCN) is proposed for SLP, which constrains fine-grained cross-modal monotonic alignment and coarse-grained multimodal semantic consistency in language-visual cues through Cross-modal Semantic Aligner (CSA) and Multimodal Semantic Comparator (MSC).
- DSIS-DPR:Structured Instance Segmentation and Diffusion Prior Refinement for Dental Anatomy Learning16
- Decoupled Cross-Scale Cross-View Interaction for Stereo Image Enhancement in the Dark16
A novel LLSIE model named Decoupled Cross-Scale Cross-View Interaction Network (DCI-Net) is proposed, designed to promote sufficient dual-view information exchange and achieves better illumination adjustment and detail recovery, and achieves SOTA performance compared to other related methods.
- Deep Adversarial Inconsistent Cognitive Sampling for Multiview Progressive Subspace Clustering16
A novel deep adversarial inconsistent cognitive sampling (DAICS) method for multiview progressive subspace clustering and demonstrates the superiority of DAICS over state-of-the-art methods.
- Deep Spatial Pyramid Features Collaborative Reconstruction for Partial Person ReID16
This paper proposes a novel spatial pyramid feature construction approach (DCR) for partial person ReID, which can effectively and efficiently tackle the occlusion in arbitrary sizes and outperforms all the competitors with a large margin.
- Model Inversion Attacks Through Target-Specific Conditional Diffusion Models15
This work introduces a novel target-specific conditional diffusion model (CDM) to purposely approximate the target classifier's private data distribution and achieve a superior accuracy-fidelity balance, and proposes an improved max-margin loss that replaces the hard max with top-k maxes.
- 15
- Efficient and self-adaptive rationale knowledge base for visual commonsense reasoning15
This paper proposes a rationale knowledge base (RKB) incorporating the convolution fusion mechanism to import the VCR-related knowledge and demonstrates the effectiveness of the proposed method with respect to the three sub-tasks.
- 15
- Perceptually Guided Photo Retargeting15
Perceptually guided photo retargeting, which shrinks a photo by simulating a human's process of sequentially perceiving visually/semantically important regions in a photo, is proposed, which outperforms its competitors significantly and AGP is more indicative of photo esthetics than conventional saliency maps.
- Multimedia encyclopedia construction by mining web knowledge15
A dynamic multimedia encyclopedia that takes advantage of, and in fact is built from the text and image resources on the Web, and is flexible and extensible such that it can easily incorporate new kinds of mediums such as video and languages into the framework.
- VAEmo: Efficient Representation Learning for Visual-Audio Emotion With Knowledge Injection14
VAEmo is proposed, an efficient two-stage framework for emotion-centric joint VA representation learning with external knowledge injection that achieves state-of-the-art performance with a compact design, highlighting the benefit of unified cross-modal encoding and emotion-aware semantic guidance for efficient, generalizable VA emotion representations.
- Diversifying Inference Path Selection: Moving-Mobile-Network for Landmark Recognition14
This paper proposes a novel Moving-Mobile-Network, named M2Net, for landmark recognition, equipped each landmark image with located geographic information, and intuitively finds that M2 net can essentially promote the diversity of the inference path (selected blocks subset) selection, so as to enhance the recognition accuracy.
- Image Classification via fusing the latent deep CNN feature14
This paper proposes a method to fuse the latent features extracted from the middle layers in a CNN to train a more robust classifier and shows that the combined classifier obtains good performance compared with the conventional CNN.
- Towards efficient support relation extraction from RGBD images14
Experimental results show that the proposed RGBD image segmentation method based on support relation abstraction produces more accurate results than segmentation methods based on ground-truth support relations.
- Generalized Kullback–Leibler Divergence Loss13
New state-of-the-art adversarial robustness on the public leaderboard — RobustBench and competitive knowledge distillation performance across CIFAR/ImageNet models and CLIP models are achieved, demonstrating the substantial practical merits.
- 13
- FRC-Net: A Simple Yet Effective Architecture for Low-Light Image Enhancement13
This paper presents a novel full-resolution representation strategy to replace all feature scaling operations, which can prevent the information degradation by making the intermediary features keep the original resolution.
- Crowdsourcing Truth Inference via Reliability-Driven Multi-View Graph Embedding13
A Reliability-driven Multi-view Graph Embedding framework for Truth inference (TiReMGE), which explores multiple crowdsourced relationships by organically integrating worker reliabilities into a graph space that is constructed from crowdsourced triples and significantly outperforms the nine state-of-the-art baselines.
- 13
- 13
- View-based 3D object retrieval by bipartite graph matching13
In the proposed method, representative views are first selected by using view clustering for each object, and the corresponding weights are given based on the cluster results.
- SwimVG: Step-Wise Multimodal Fusion and Adaption for Visual Grounding12
Swip and CIA are both parameter-efficient paradigms, and they fuse the cross-modal features from shallow to deep layers gradually, and they fuse the cross-modal features from shallow to deep layers gradually.
- Text Proxy: Decomposing Retrieval from a 1-to-N Relationship into N 1-to-1 Relationships for Text-Video Retrieval12
Text-Video-ProxyNet (TV-ProxyNet), a novel framework designed to decompose the conventional 1-to-N relationship of TVR into N distinct 1-to-1 relationships, achieves state-of-the-art performance on MSRVTT and ActivityNet Captions, and a 2.0% improvement on DiDeMo compared to existing methods, validating the approach's ability to enhance semantic mapping and reduce error propensity.
- OCR-oriented Master Object for Text Image Captioning12
This paper proposes a novel text image captioning method by purifying the OCR-oriented scene graph with themaster object, which considers the master object as a proxy to connect OCR tokens and other regions in the image.
- 12
- 12
- eHeritage of shadow puppetry12
The sparsity optimization over simplexes formulation to automatically assemble weighted instances of different atomic actions into a smooth shadow puppetry animation sequence is proposed and extensive experimental results on the creation of puppetry characters and puppetry plays well demonstrate the effectiveness of the proposed system.
- EduStudio: towards a unified library for student cognitive modeling11
A unified PyTorch-based library EduStudio is developed, which unifies Cognitive Diagnosis and Knowledge Tracing for student cognitive modeling and provides eco-services of EduStudio, such as the repository that collects resources about student cognitive modeling and the leaderboard that demonstrates comparison among models.
- Revisiting Audio-Visual Segmentation with Vision-Centric Transformer11
A new Vision-Centric Transformer (VCT) framework is proposed that leverages vision-derived queries to iteratively fetch corresponding audio and visual information, enabling queries to better distinguish between different sounding objects from mixed audio and accurately delineate their contours.
- DAT: Dialogue-Aware Transformer with Modality-Group Fusion for Human Engagement Estimation11
This paper proposes a Dialogue-Aware Transformer framework with Modality-Group Fusion (MGF), which relies solely on audio-visual input and is language-independent, for estimating human engagement in conversations, and employs a modality-group fusion strategy that independently fuses audio and visual features within each modality for each person before inferring the entire audio-visual content.
- CropCap: Embedding Visual Cross-Partition Dependency for Image Captioning11
This paper proposes a novel image captioner embedding visual cross-partition dependency, dubbed CropCap, where the visual sequence generated from the Swin Transformer-based pre-embedding network is fed into the proposed cross- partition dependency module to refinedly model the interaction between partial representations on both the time step and embedded dimension.
- Iterative Adversarial Attack on Image-Guided Story Ending Generation11
An iterative adversarial attack method that fuses image and text modality attacks, allowing for an attack search for adversarial text and image in a more effective iterative way, indicating the potential for improving the adversarial robustness of multimodal text generation models, such as multimodal machine translation, multimodal question answering, etc.
- Towards Feature Distribution Alignment and Diversity Enhancement for Data-Free Quantization11
A new and effective data-free quantization method termed ClusterQ, which utilizes the feature distribution alignment for synthetic data generation and incorporates the diversity enhancement to solve class-wise mode collapse, is proposed.
- Personalized Multimedia Item and Key Frame Recommendation11
This paper designs a model to discern both the collaborative and visual dimensions of users, and model how users make decisive item preferences from these two spaces, and argues that the key challenge of this problem lies in discovering users' visual profiles for key frame recommendation.
- 11
- Learning content–social influential features for influence analysis11
A novel method to deeply learn the unified feature representations for both user pair and content, where the homogeneous feature similarity can be used to estimate the propagation probability between users with given content.
- Learning Cooking Techniques from YouTube11
This paper explores how to leverage YouTube video collections as a source to automatically mine videos of basic cooking techniques and leverages the trajectory bag of words model to represent human motion.
- CARE Transformer: Mobile-Friendly Linear Visual Transformer via Decoupled Dual Interaction10
A new deCoupled duAl-interactive lineaR attEntion (CARE) mechanism is proposed, revealing that features’ decoupling and interaction can fully unleash the power of linear attention.
- Intermediary-Generated Bridge Network for RGB-D Cross-Modal Re-Identification10
A novel approach is proposed that constructs an intermediary mode through the constraints of self-supervised intermediary learning, which is freedom from modal prior knowledge and additional module parameters, and designs a bridge network to fully mine the intermediary role of generated modality through carrying out multi-modal integration and decomposition.
- Multimodal Large Language Models with Adaptive Preference Optimization for Sequential Recommendation9
A Multimodal LLM framework that integrates Hardness-aware and Noise-regularized preference optimization for Recommendation (HaNoRec) is proposed, which dynamically adjusts optimization weights based on both the estimated hardness of each training sample and the policy model's real-time responsiveness, prioritizing harder examples.
- Boundary Discretization and Reliable Classification Network for Temporal Action Detection9
A novel Boundary Discretization and Reliable Classification Network (BDRC-Net) is proposed that addresses the issues above by introducing boundary discretization and reliable classification modules and achieves competitive detection performance.
- Fair Personalized Learner Modeling Without Sensitive Attributes9
A novel fairness objective tailored for personalized learner modeling is introduced and a max-min strategy is designed to facilitate both sensitive information inference and fair CD modeling to achieve fair personalized learner modeling without relying on any sensitive attribute input.
- Gt-Mean Loss: a Simple Yet Effective Solution for Brightness Mismatch in Low-Light Image Enhancement9
The GT-mean loss is proposed, a simple yet effective loss function directly modeling the mean values of images from a probabilistic perspective that results in consistent performance improvements across various methods and datasets.
- Listening to the Unspoken: Exploring '365' Aspects of Multimodal Interview Performance Assessment9
A novel and comprehensive framework that explores ''365'' aspects of interview performance by integrating three modalities, six responses per candidate, and five key evaluation dimensions, secured first place in the AVI Challenge 2025, demonstrating its effectiveness and robustness in advancing automated and multimodal interview performance assessment.
- Few-Shot Partial Multi-View Learning9
A new task is proposed, called few-shot partial multi-view learning, which focuses on overcoming the negative impact of the view-missing issue in the low-data regime and proposes a new unified Gaussian dense-anchoring method.
- 9
- Joint Sketch-Attribute Learning for Fine-Grained Face Synthesis9
A hybrid generator is proposed to learn a unified embedding of shape from sketches and appearance from attributes for synthesizing images with conditional GANs and an attribute modulation module is proposed, which transfers user-preferred attributes to reinforce sketch representation with appearance details.
- 9
- Texture-adaptive hole-filling algorithm in raster-order for three-dimensional video applications9
A texture-adaptive hole-filling algorithm for post-processing of rendered image on 3D video applications that outperforms other benchmarks on perceptual quality with considerable savings of computational cost.
- 8
- Geometric-Augmented Self-Distillation for Graph-Based Recommendation8
Geometric-augmented graph learning with hyperbolic and Euclidean geometries is investigated to delve into local structural and hierarchical knowledge from the interaction graph to verify the effectiveness and efficiency of knowledge transfer by self-distillation to aggregate knowledge adaptively for personalized recommendation.
- 8
- 8
- 8
- Traits Run Deep: Enhancing Personality Assessment via Psychology-Guided LLM Representations and Multimodal Apparent Behaviors8
This work proposes a novel personality assessment framework called Traits Run Deep that employs psychology-informed prompts to elicit high-level personality-relevant semantic representations and is the first to apply personality-specific prompts to guide large language models (LLMs) in extracting personality-aware semantics for improved representation quality.
- 8
- 8
- Unified Representation Learning for Discrete Attribute Enhanced Completely Cold-Start Recommendation8
- Clarity ChatGPT: An Interactive and Adaptive Processing System for Image Restoration and Enhancement8
The case studies demonstrate that Clarity ChatGPT effectively improves the generalization and interaction capabilities in the IRE, and also fills the gap in the low-level domain of the existing vision-language model.
- Bi-direction Context Propagation Network for Real-time Semantic Segmentation.8
This work proposes the Bi-direction Context Propagation Network (BCPNet), a network that builds bi-directional paths in its network architecture, allowing the backward context propagation and the forward spatial detail propagation simultaneously.
- Dual Hierarchical Temporal Convolutional Network with QA-Aware Dynamic Normalization for Video Story Question Answering8
A novel framework named Dual Hierarchical Temporal Convolutional Network (DHTCN) is proposed to address the aforementioned defects together and obtains state-of-the-art results on the both datasets.
- 8
- Aircraft type recognition based on convex hull features and SVM8
A novel aircraft type recognition algorithm based on the aircrafts' convex hull features and Support Vector Machine (SVM) which can successfully eliminate the need to segment the object region from the complex background.
- Streaming Drag-Oriented Interactive Video Manipulation: Drag Anything, Anytime!7
A training-free approach to streaming, fine-grained control over the outputs of autoregressive video diffusion models, allowing the model to fully exploit contextual information while mitigating its interference via selectively propagating visual cues along generation.
- Dual-View Alignment Learning With Hierarchical-Prompt for Class-Imbalance Multi-Label Image Classification7
This work proposes a novel method termed Dual-View Alignment Learning with Hierarchical Prompt (HP-DVAL), which leverages multi-modal knowledge from vision-language pretrained (VLP) models to mitigate the class-imbalance problem in multi-label settings and introduces a hierarchical prompt-tuning strategy.
- Dual Video Summarization: From Frames to Captions7
A dual video summarization framework that combines a cross-modal video summarizer to select the most semantically consistent frames according to the pseudo score label and a ranking scheme of video representation from two aspects: feature-oriented and sequence-oriented.
- Unsupervised Color Retention Network and New Quantization Metric for Blind Motion Deblurring7
A general unsupervised color retention network termed CRNet for blind motion deblurring is proposed, which can be easily extended to other tasks suffering from chromatic aberration and new concepts of blur offset estimation and adaptive blur correction are introduced, so that more detailed information can be retained to improve thedeblurring task.
- 7
- Eigenvector-Based Distance Metric Learning for Image Classification and Retrieval7
This work proposes a novel distance metric learning approach by introducing base vectors explicitly learned from training data that can reflect the intrinsic information of the input features and can extend to any kernel space without changing the objective function.
- Spatially guided local Laplacian filter for nature image detail enhancement7
This paper proposes a spatially guided LLF by extending the single-value key parameter into a multi-value matrix that dynamically assigns filtering strengths according to image contents, and proposes a simple but effective technique, which only approximates the richness of image details instead of specific contents.
- Biomedical Relation Extraction via Adaptive Document-Relation Cross-Mapping and Concept Unique Identifier6
This work proposes a document-level Bio-RE framework via LLM Adaptive Document-Relation Cross-Mapping (ADRCM) Fine-Tuning and Concept Unique Identifier (CUI) Retrieval-Augmented Generation (RAG), and proposes ADRCM fine-tuning, a novel fine-tuning recipe that establishes mappings across different documents and relations, enhancing the model’s contextual understanding and cross-sentence inference capabilities.
- 6
- 6
- SignAligner: Harmonizing Complementary Pose Modalities for Coherent Sign Language Generation6
This work introduces PHOENIX14T+, an extended version of the widely-used RWTH-PHOENIX-Weather 2014T dataset, featuring three new sign representations: Pose, Hamer and Smplerx, and proposes a novel method, SignAligner, for realistic sign language generation, consisting of three stages: text-driven pose modalities co-generation, online collaborative correction of multimodality, and realistic sign video synthesis.
- End-to-End Explainable Fake News Detection Via Evidence-Claim Variational Causal Inference6
This model includes an evidence-claim variational causal inference component, which not only utilizes explanation content to improve fake news detection but also employs a variational approach to address the distributional bias between the ground truth explanation in the training set and the prediction explanation in the test set.
- Math Word Problem Generation via Disentangled Memory Retrieval6
This work introduces a simple memory retrieval module to search related training MWPs, which are used to augment the generation and proposes a disentangled memory retrieval module based on the simple memory retrieval module.
- 6
- Pseudo Content Hallucination for Unpaired Image Captioning6
PCH is proposed to help the captioning model enlarge the perception of the ob- jects and capture the relations between the objects to enable the capture of additional semantics from the image, ultimately enhancing the sentence generation ability.
- 6
- 6
- Viewport Prediction for Volumetric Video Streaming by Exploring Video Saliency and Trajectory Information6
A novel approach, named Saliency and Trajectory Viewport Prediction (STVP), which aims to improve the precision of viewport prediction in volumetric video streaming and introduces a novel sampling method, Uniform Random Sampling (URS), to reduce computational complexity while still preserving video features in an efficient manner.
- Correction to: On fusing the latent deep CNN feature for image classification6
The original version of this articleUnfortunately, the spelling of Guangcan Liu’s name was incorrect.
- 6
- A short survey on small reasoning models: training, inference, applications, and research directions5
A timely and comprehensive survey of recently published research focused on Small Reasoning Models, which serves as an essential reference for researchers and practitioners interested in leveraging or developing SRMs to unlock advanced reasoning functionalities with improved efficiency.
- Seeing through the noisy dark: toward real-world low-light image enhancement and denoising5
A novel low-light enhancement and denoising network (RLED-Net) for real-world low-light images in the sRGB color space, using a differentiable latent subspace reconstruction block (LSRB) to embed real-world images into low-rank subspaces to suppress noise and reduce its impact during enhancement.
- I 3 -MRec: Invariant Learning with Information Bottleneck for Incomplete Modality Recommendation5
A novel method called I 3-MRec, which uses Invairant learning with Information bottleneck principle for Incomplete Modality Recommendation to achieve robust performance in missing modality scenarios, and a missing-aware fusion module is developed to explicitly simulate modality-missing scenarios.
- InterMind: Doctor-Patient-Family Interactive Depression Assessment Empowered by Large Language Models5
To enhance LLMs' performance in psychological counseling and diagnostic interpretability, this work integrates retrieval-augmented generation (RAG) and chain-of-thoughts (CoT) techniques for data augmentation, which mitigates the hallucination issue of LLMs in specific scenarios after instruction fine-tuning.
- Moderating the Generalization of Score-Based Generative Model5
The first Moderated Score-based Generative Model (MSGM) is proposed, which introduces a novel score adjustment strategy that redirects the score function away from undesirable data during the continuous-time stochastic differential equation process.
- Asymmetric Deformable Spatio-temporal Framework for Infrared Object Tracking5
A novel Asymmetric Deformable Spatio-Temporal Framework (ADSF) is proposed to fully exploit collaborative shape and temporal clues in terms of the objects to fully exploit collaborative shape and temporal clues in terms of the objects.
- 5
- Self-Supervised Cross Domain Social Recommendation5
This paper proposes a novel Self-supervised Cross Domain Social Recommendation (SCDSR), aiming at cold-start social users, which innovatively integrating information domain and social domain into a heterogeneous graph and builds higher-order connections between cold- start social users and items via very limited bridge users.
- Stereo Image Rain Removal via Dual-View Mutual Attention5
A new StereoIRR method, which incorporates a new Dual-view Mutual Attention (DMA) mechanism which generates mutual attention maps by taking left and right views as key information for each other to facilitate cross-view feature fusion and outperforms other related monocular and stereo image rain removal methods on several datasets.
- The Balanced Loss Curriculum Learning5
This paper proposes a novel CL framework, named Balance Loss Curriculum Learning (BLCL), to reveal the comprehensive difficulty level of an example and improve the curriculum process based on the deep architectures.
- 5
- Quality Matters: Assessing cQA Pair Quality via Transductive Multi-View Learning5
A transductive multi-view learning model is presented, designed to find a latent common space by unifying and preserving information from various views, including question, answer, QA relevance, asker, and answerer, to enhance the representation ability of the common space.
- 5
- 5
- Searching for Recent Celebrity Images in Microblog Platform5
The proposed method is based on the idea of refining the initial text-based search results by utilizing multimedia plus social information and employs a multi-modal graph based learning method to properly rank the obtained tweets by integrating social and visual information.
- Generalizable Engagement Estimation in Conversation via Domain Prompting and Parallel Attention4
DAPA (Domain-Adaptive Parallel Attention), a novel framework for generalizable conversational engagement modeling, introduces a Domain Prompting mechanism by prepending learnable domain-specific vectors to the input, explicitly conditioning the model on the data's origin to facilitate domain-aware adaptation while preserving generalizable engagement representations.
- Video Flow as Time Series: Discovering Temporal Consistency and Variability for VideoQA4
The T3T is introduced, a novel architecture that models time consistency and time variability in temporal modeling and underscores the importance of a nuanced approach to temporal modeling in improving the accuracy and depth of video-based question answering.
- 4
- Grid Jigsaw Representation with CLIP: a new perspective on image clustering4
Inspired by human jigsaw puzzle processing, the traditional jigsaw learning is modified to gain a more sequential and incremental understanding of image structure, and the pretrain-based Grid Jigsaw Representation (pGJR) is proposed.
- Text2Lip: Progressive Lip-Synced Talking Face Generation from Text via Viseme-Guided Rendering4
Text2Lip is a viseme-centric framework that constructs an interpretable phonetic-visual bridge by embedding textual input into structured viseme sequences by embedding textual input into structured viseme sequences to establish a new paradigm for controllable and flexible talking face generation.
- Deep sub-ensembles meets quantile regression: uncertainty-aware imputation for time series4
Quantile Sub-Ensembles is proposed, a novel method that estimates uncertainty with ensembles of quantile-regression-based task networks and incorporate Quantile Sub-Ensembles into a non-generative time series imputation method that not only produces accurate and reliable imputations, but also remains computationally efficient due to its non-generative framework.
- Gradient-Aware Logit Adjustment Loss for Long-Tailed Classifier4
The Gradient-Aware Logit Adjustment (GALA) loss is proposed, which adjusts the logits based on accumulated gradients to balance the optimization process and finds that most of the solutions to long-tailed problems are still biased towards head classes in the end.
- 4
- 4
- CITE: Compact Interactive TransformEr for Multilingual Image Captioning4
Extensive experiments on the COCO-EN-CN dataset show that the single CITE model with more parameter-efficient can maintain the competitive performance or even better than the monolingual captioning models.
- 4
- 4
- 4
- Image matching by fast random sample consensus4
An algorithm called fast Random Sample Consensus (abbr. F-RANSAC) is presented which is able to improve computation efficiency and robustness and slightly improves accuracy on affine covariant region datasets.
- Videoader4
A novel video advertising system called VideoAder that leverages the rich information from the video corpus for embedding visual content relevant ads and their potential embedding positions in the video stream is presented.
- Estimating Poses of World’s Photos with Geographic Metadata4
A novel method to estimate the view directions of world's photos well is presented, and the pose referenced on Google Maps using the geographic Metadata of photos is obtained using normalized 8-point algorithm.
- 4
- 3
- When low-light meets flares: Towards Synchronous Flare Removal and Brightness Enhancement3
A novel Fourier transform-based deep FLLIE network termed Synchronous Flare Removal and Brightness Enhancement (SFRBE) is developed and a Residual Directional Fourier Block (RDFB) is introduced that learns in the frequency domain to extract accurate global information and capture detailed features from multiple directions.
- A Unified Generative Hashing for Cross-Modal Retrieval3
This work proposes a unified generative cross-modal hashing method that leverages a single Uniform Mixture-of-Expert Decoder (UMoED) for both image and text modalities and achieves the state-of-the-art performance in cross-modal hashing retrieval.
- 3
- Controllable Relation Disentanglement for Few-Shot Class-Incremental Learning3
This paper proposes a simple yet effective approach, dubbed ConTrollable Relation-disentangLed Few-Shot Class-Incremental Learning (CTRL-FSCIL), to address FSCIL from a new perspective: enhancing FSCIL via disentangling spurious relations between categories.
- Joint Adversarial Purification: Mitigating the Threat of Multimodal Adversarial Examples3
JAP is established, a novel defense framework that synergistically eliminates adversarial perturbations across modalities through cross-modal interaction that establishes a new paradigm for securing VLP models against multimodal adversarial attacks.
- 3
- Rebalancing Contrastive Alignment with Bottlenecked Semantic Increments in Text-Video Retrieval3
GARE, a Gap-Aware Retrieval framework that introduces a learnable, pair-specific increment between text and video, redistributing gradients to relieve optimization tension and absorb noise, and validating the effectiveness of gap-aware tension mitigation.
- 3
- 3
- 3
- 3
- 3
- 3
- Vibration-Based Uncertainty Estimation for Learning from Limited Supervision3
A novel approach is presented that measures the uncertainty from the vibration of sequential data, so that deep neural networks can make use of the results for learning from limited supervision.
- NASTER: Non-local Attentional Scene Text Recognizer3
A new framework for Scene Text Recognition (STR), named Non-Local Attentional Scene text Recognizer (NASTER) is proposed, which uses ResNet with Global Context Block (GC block) to extract global visual features.
- WFN-PSC3
Considering that image dehazing is a low-level task in the computer vision, the results show that the proposed method can be a good pre-processing to assist the high-level computer vision task.
- 3
- Explainable Social Contextual Image Recommendation with Hierarchical Attention.3
An explainable social contextual image recommendation model is developed to simultaneously explain and predict users' preferences to images, and explain the underlying reasons for each user's behavior with the learned hierarchial attention scores.
- Efficient Graph Based Multi-view Learning3
A method named multi-anchor-graph learning (MAGL), which aims to utilize anchor graphs for the adjacency estimation, which can not only sufficiently explore the complementation of multiple graphs built upon different views but also keep an acceptable time complexity.
- Learning from social media network3
This special issue is organized with the purpose of introducing novel research work on learning from social media network, i.e., how to use learning technology to facilitate the analysis of social medianetwork and further make application under the scenario of Web 2.0.
- 3
- 3
- 2
- Visual Token Compression Enhances Robustness of MLLMs2
This paper shows for the first time that visual token pruning enhances the robustness of Multimodal Large Language Models (MLLMs), mitigating vulnerabilities such as jailbreak attacks and hallucinations, and maintains strong results on general datasets like MME.
- Benchmarking Trustworthiness in Multimodal LLMs for Video Understanding2
Trust-videoLLMs provides a publicly available, extensible toolkit for standardized trustworthiness assessments, addressing the critical gap between accuracy-focused benchmarks and demands for robustness, safety, fairness, and privacy.
- Taming "Zombie" Agents: A Markov State-Aware Framework for Resilient Multi-Agent Evolution2
Extensive experiments on general reasoning, domain-specific, and hallucination challenge tasks show that the proposed AgentRevive method consistently outperforms strong baselines and significantly reduces token consumption through state-aware agent scheduling.
- 2
- 2
- 2
- 2
- RAGG: Retrieval-Augmented Grasp Generation Model2
This work develops RAGG as a two-stage approach, encompassing retrieval-guided generation and structurally stable grasp refinement, and introduces a Progressive Refinement Network (PRN) with Kolmogorov-Arnold Network (KAN) layers to refine the generated coarse grasp.
- 2
- 2
- Exploring Robust Face-Voice Matching in Multilingual Environments2
This paper presents Team Xaiofei's innovative approach to exploring Face-Voice Association in Multilingual Environments (FAME) at ACM Multimedia 2024 by building upon Fusion and Orthogonal Projection, introducing four key components: a dual-branch structure, dynamic sample pair weighting, robust data augmentation, and score polarization strategy.
- 2
- 2
- One-Bit Supervision for Image Classification: Problem, Solution, and Beyond2
Theoretical analysis shows that one-bit annotation is more efficient than full-bit annotation in most cases and gives the conditions of combining the authors' approach with active learning, and further integrates the one-bit supervision framework into the self-supervised learning algorithm, which yields an even more efficient training schedule.
- 2
- 2
- Introduction to the Special Issue on the Cross-Media Analysis for Visual Question Answering2
This special issue focuses on new theories and algorithms for visual question answering via cross-media analysis, as well as their applications in the practice of human-computer interaction, multimedia search, visual description for the blind, incident reports for surveillance, seeing chat bots, or even robotic intelligence.
- Enhancing Low-Light Images with JPEG Artifact Based on Image Decomposition2
This work decomposes an input image into an illumination layer and a reflectance layer, which decouple the issues of low lightness and JPEG artifacts, which can be dealt with separately with the off-the-shelf enhancing and deblocking techniques.
- Generating Chinese Poems from Images Based on Neural Network2
A generative model based on deep recurrent framework that describes images in the form of poems that generates each line of the poem incrementally according to the extracted semantic information from the images by a recurrent neural network is presented.
- 2
- Retrieving Images by Multiple Samples via Fusing Deep Features2
A novel image retrieval paradigm that users could input two images as query to search the images that include the content of the two input images-synchronously by proposing the FWC Feature weighting by Clustering, a novel algorithm to weight the two query features.
- Multifold Concept Relationships Metrics2
This paper first defines five specific relationships between concepts and leverage them to annotate the images collected from commercial search engine and extract some conceptual features in textual and visual domain to train the concept model.
- Advances in Multimedia Modeling2
This talk covers perspectives on adaptive video streaming and how such techniques are essential for systems with heterogeneous devices and mechanisms that use Quality of Experience (QoE) metrics to enhance its performance for the users will be shown.
- Marginalized multi-layer multi-instance kernel for video concept detection2
This paper proposes a novel kernel, termed marginalized multi-layer multi-instance (MarMLMI) kernel for video concept detection, which exploits the hierarchical structure of video, i.e., both themulti-layer structure and the multi- instance relationship.
- iComics2
This demonstration presents a system, named iComics, for automatic conversion of movie into comics, which designs three components to realize the system: script-face mapping, key-scene extraction, and cartoonization.
- 2
- Post-hoc Provider Fairness Adaptation via Hierarchical Exposure Alignment1
Post-hoc Fairness Adaptation (PFA), a lightweight framework that equips a frozen recommender with a fairness adapter, enabling flexible fairness control without retraining the backbone model, is proposed, which explicitly balances inter- and intra-group provider exposure disparities, enabling flexible adaptation to diverse fairness requirements.
- MURE: Hierarchical Multi-Resolution Encoding via Vision-Language Models for Visual Document Retrieval1
MURE is developed, a novel framework that employs VLMs as a hierarchical multi-resolution encoder, integrates resolution-level Matryoshka representation learning (RMRL) for effective feature fusion, and applies a semantic-aware hierarchical clustering mechanism for visual token compression.
- 1
- AMATA: Adaptive Multi-Agent Trajectory Alignment for Knowledge-Intensive Question Answering1
AMATA is an Adaptive Multi-Agent Trajectory Alignment framework that dynamically integrates external knowledge to improve response interpretability and factual grounding, and formalizes multi-agent collaboration with external tools as a trajectory preference alignment problem, incorporating question-aware agent customization and inter-agent preference harmonization.
- 1
- 1
- An Efficient Hybrid Cascade Tracker with Spiking Neural Networks for Event Domain Tracking1
This work proposes an Efficient Hybrid Cascade Tracker with SNN for object tracking in the event domain that outperforms most of the existing state-of-the-art trackers by a significant margin and also achieves a great advantage in terms of energy consumption.
- CLAIP-Emo: Parameter-Efficient Adaptation of Language-Supervised Models for In-the-Wild Audiovisual Emotion Recognition1
CLAIP-Emo is presented, a modular framework that reframes in-the-wild AVER as a parameter-efficient adaptation of language-supervised foundation models (CLIP/CLAP), suggesting that parameter-efficient adaptation of language-supervised foundation models provides a scalable alternative to domain-specific pre-training for real-world AVER.
- Selective Mixup for Debiasing Question Selection in Computerized Adaptive Testing1
A debiasing framework consisting of two key modules: Cross-Attribute Examinee Retrieval and Selective Mixup-based Regularization that substantially improves the generalization ability of question selection.
- 1
- ViewGauss: A Head Movement Dataset for 6DoF Gaussian Splatting Video Viewing1
ViewGauss is introduced-the first publicly available dataset that captures full 6DoF head movement during the viewing of 4D Gaussian splatting videos, and a preliminary viewpoint prediction experiment using the iTransformer model shows that head orientation patterns in 4D Gaussian splatting video scenes are not only temporally coherent but also learnable.
- 1
- RATE: Robust Adversarial Training and Temperature-scaled Ensemble Framework for Trustworthy Misinformation Detection1
This work presents a novel temperature-scaled ensemble frame with robust adversarial training(RATE) for calibrating misinformation detection models, offering a universal solution for constructing reliable misinformation detection systems.
- 1
- 1
- DFGAP: Towards Depth-Free Cross-Category GAParts Perception via Uncertainty-Quantified Modeling1
This paper proposes DFGAP, a novel depth-free framework for RGB-based GAParts segmentation and pose estimation, and independently model the ill-pose problems from the absence of depth for GAPart segmentation and pose estimation, by clearly quantifying the pixel-wise segmentation probability and relative depth.
- Optimizing low-rank adaptation with decomposed matrices and adaptive rank allocation1
This paper proposed two novel PEFT strategies to improve the capability of LoRA in single-task and multi-task scenarios and conducted extensive experiments to show the effectiveness of the proposed method.
- 1
- Special Issue on Conversational Information Seeking1
An overview of ACM TWEB’s Special Issue on Conversational Information Seeking highlights both research and practical applications in this field and discusses the future potential of conversational information seeking technology.
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- Deep Graph Laplacian Hashing for Image Retrieval1
This paper designs a novel Deep Graph Laplacian Hashing method (DGLH) method to simultaneously learn robust image features and hash functions in an unsupervised manner and demonstrates that DGLH can outperform the state-of-the-art un supervised hashing methods in image retrieval tasks.
- 1
- Mental Visual Indexing1
MVI is a novel paradigm for fast video browsing dubbed Mental Visual Indexing that is underpinned by a sequence model in terms of Recurrent Neural Network (RNN), which is trained by automatically generated shot sequences from a rigorous Bayesian framework, which simulates user feedback process.
- 1
- An Intention-Aware Interactive System for Mobile Video Browsing1
An RNN-based \textit{intention-aware} system to overcome limitations in video browsing, where the user only needs to select one of the displayed shots that is most visually similar to her mental target and then the user's choice will further tailor the search to the target.
- 1
- 1
- A Conditional Adaptive Filtering Algorithm and Its Application in Echo Cancellation1
This paper proposes a conditional adaptive filtering algorithm that need not require the adaptive filter converge to reach the optimal value and the filter updates its coefficients only when the condition is satisfied.
- Research on Roads Automatic Extraction from Low Resolution Remote Sensing Image1
This paper proposes an automatic main-road extraction method, which is based on line segment perceptual grouping and dynamic programming, and shows robustness and good performance in remote sensing images such as Landsat7,SPOT and SAR.
- From disagreement to insight: multi-agent collaborative reasoning for multimodal emotional support conversations–
MiAgent, a novel Multi-Agent Collaborative Reasoning framework for multimodal emotional support conversations, introduces an affective reasoning agent that analyzes cross-modal disagreement reasoning and a supporter agent is trained via the distillation-based reasoning to generate empathetic and effective responses.
- –
- LLM-Driven Aspect-Based Semantic Alignment for Review-Based Recommendation–
LLM-ASAR is proposed, a Large Language Model-driven framework for Aspect-based Semantic Alignment in Review-based Recommendation that explicitly aligns aspect-level review semantics with user–item interaction patterns and achieves significant accuracy improvements over state-of-the-art baselines.
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- Capturing Motif Topological Diversity via Geometry-Adaptive Riemannian Molecular Representation Learning–
A geometry-adaptive Riemannian framework for molecular representation learning, which explicitly models motifs as the basic units and learns their embeddings across multiple constant-curvature spaces, offering a general and scalable framework for scientific molecular modeling.
- –
- –
- Fine-grained Text-Video Retrieval with Patch-level Temporal Difference and Aggregation–
TRFG is proposed, a novel framework for text-video retrieval that addresses the challenges of modeling Temporal Reasoning and Fine-Grained cross-modal alignment and achieves state-of-the-art performance across multiple backbones and retrieval tasks.
- –
- –
- –
- Molecular ground-state conformation prediction based on the Mamba state space model–
An approach of Mamba-driven multi-perspective structural understanding (MPSU-Mamba) is proposed to localize molecular ground-state conformation and indicates that MPSU-Mamba significantly outperforms existing methods.
- –
- –
- Differential Mental Disorder Detection with Psychology-Inspired Multimodal Stimuli–
This work proposes a paradigm-aware multimodal framework that leverages inter-disorder differences prior knowledge as prompt-guided semantic descriptions to capture task-specific affective and interaction contexts for multimodal representation learning in the new differential mental disorder detection task.
- ZeroRel: Relational Reasoning via Graph-guided Large Language Models–
A novel self-supervised framework for relational reasoning over RDBs that treats context sparsity as a controllable curriculum variable and leverages it to induce a progressive shift from semantic-dominant inference to structure-aware relational reasoning.
- –
- 1078 - Understanding and Benchmarking the Trustworthiness in Multimodal LLMs for Video Understanding–
- Infrared Object Tracking via Complementary Dual-domain Interaction with Target-guided Frequency Transformation–
A novel tracker is proposed that introduces a Target-guided Frequency Transformation Module (TFTM) and a Dual-domain Interactive Fusion Network (DIFN) that extracts multi-frequency representations across scales and orientations, guided by an adaptive mask strategy to suppress background interference.
- –
- –
- –
- Learning Speaker-Invariant Visual Features for Lipreading–
SIFLip is a speaker-invariant visual feature learning framework that disentangles speaker-specific attributes using two complementary disentanglement modules (Implicit Disentanglement and Explicit Disentanglement) to improve generalization.
- –
- MGCA-Net: Multi-Grained Category-Aware Network for Open-Vocabulary Temporal Action Localization–
A Multi-Grained Category-Aware Network (MGCA-Net) comprising a localizer, an action presence predictor, a base classifier, and a coarse-to-fine classifier achieves multi-grained category modeling, effectively improving localization performance.
- Synthesizing Multi-Person and Rare Pose Images for Human Pose Estimation–
This paper designs a controllable pose generator named PoseFactory and introduces a multi-person image generator named MultipGenerator, which is conditioned on multiple human poses and textual descriptions of complex scenes and demonstrates its superior performance both quantitatively and qualitatively.
- –
- Disentangling Foreground and Background for vision-Language Navigation via Online Augmentation–
A consensus-driven online augmentation strategy that first leverage semantically-enhanced landmark identification to disentangle foreground and background as candidate augmented features and encourages the agent to consolidate two-stage voting results on feature preferences according to diverse instructions and navigational locations.
- –
- MCHM25: Multimedia Computing for Health and Medicine–
The 2nd International Workshop on Multimedia Computing for Health and Medicine (MCHM'25), held in conjunction with ACM Multimedia 2025, focuses on advanced multimedia computing techniques, including mobile and hardware solutions, for tackling real-world problems in healthcare.
- MetaPipe: Predicting Metaphoric Associations and Turning Your Metaphoric Imagination into Reality–
MetaPipe is introduced, a mobile-based design tool that assists users in crafting high-quality visual metaphors by leveraging three interactive modules and significantly enhances the visual metaphor design process, enabling the creation of high-quality and creative images.
- Mamba-driven multi-perspective structural understanding for molecular ground-state conformation prediction–
An approach of Mamba-driven multi-perspective structural understanding (MPSU-Mamba) is proposed to localize molecular ground-state conformation and indicates that MPSU-Mamba significantly outperforms existing methods on QM9 and Molecule3D datasets.
- –
- –
- Seeing is Believing? Enhancing Vision-Language Navigation using Visual Perturbations–
This work preliminarily investigates whether advanced VLN models genuinely comprehend the visual content of their environments by introducing varying levels of visual perturbations, and presents a versatile Multi-Branch Architecture (MBA) designed to delve into the impact of both the branch quantity and visual quality.
- –
- Dual-head prediction and reconstruction with coarse-to-fine masks for visual reinforcement learning–
Experimental results on both multiple continuous control and discrete control tasks demonstrate that the DPRM algorithm significantly enhances performance, leading to higher reward accumulation and faster convergence.
- –
- Dual-View Alignment Learning with Hierarchical-Prompt for Class-Imbalance Multi-Label Classification–
- –
- –
- Causality guided co-attention network for visual question answering–
A new causality inspired co-attention network is proposed, which utilizes causal graphs to guide the self-attention mechanism learning cross-modal representations of the IOFs and SOFs with question features and a dual output multi-modal feature fusion method to promote the model performance.
- –
- Talking-DiSSM: Enhancing Temporal Consistency in Talking Face Video Generation with Bidirectional SSMs–
Talking-DiSSM is proposed, an end-to-end method for generating audio-driven talking face videos using State-Space Models (SSMs) that generates temporally consistent, high-quality, and identity-preserving talking face videos synchronized with the driving audio, achieving state-of-the-art results compared to existing models.
- –
- –
- SDI-Net: Toward Sufficient Dual-View Interaction for Low-light Stereo Image Enhancement–
This work proposes a model called SDI-Net, aiming to fully exploit the correlations between the binocular views via the attention mechanism, and designs a module named Cross-View Sufficient Interaction Module (CSIM), aiming to fully exploit the correlations between the binocular views via the attention mechanism.
- Dual-Stream Keyframe Enhancement for Video Question Answering–
A dual-stream keyframe selection strategy by fusing the differentiable and non-differentiable results together to achieve a unified approach and significantly improves the performance of VideoQA by effectively incorporating temporal components and enhancing the keyframe ranking process.
- –
- –
- Low-Light Image Enhancement Based on Mutual Guidance Between Enhancing Strength and Image Appearance–
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- Regularizing flat latent variables with hierarchical structures–
This paper uses the stratified relationships in topic hierarchies to regularize the flat topics and proposes two theoretically sound and practical inference methods to solve the model.
- Gaze Shifting Kernel: Engineering Perceptually- Aware Features for Scene Categorization–
A novel gaze shifting kernel for scene image categorization is proposed, focusing on discovering the mechanism of humans perceiving visually/semantically salient regions in a scene.
- Visual Understanding with RGB-D Sensors–
This special issue is specifically dedicated to new algorithms and applications based on the Kinect sensors, and serves as a convincing forum for researchers and practitioners to disseminate their latest research on visual understanding with RGB-D sensors.
- Evaluation on the Impact of Image Quality on Image Retrieval–
A baseline system with the classic Bag-of-Visual-Words model and the inverted index structure is built and some conclusions are drawn from the evaluation results.
- –
- –
- –
- Video Reference: A Video Question Answering Engine–
The Video Reference system automatically extracts videos from YouTube as a video reference responding to a textual question, and shows results on real questions sampled from Yahoo! Answering.
- –
- –
- Remote sensing image retrieval based on multiple instance learning–
Relevance feedback(RF) based on multiple instance learning(MIL) was applied to the remote sensing image retrieval and yields better performance than traditional approaches.
- A QUALITY METRIC BASED ON COLOR SIMILARITY FOR IMAGE FUSION–
A new objective, non-reference quality metric for evaluating the performance of pixel-level image fusion, designed on a likelihood function for color cue that measures the similarity of color- or grey-level distributions is proposed.
- –
Publication data from OpenAlex, with missing venues and authors filled in from Crossref; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-14. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.
Report an error
Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.