Is this you? Claim this profile to correct it, add a bio and choose the work people see first.
Claim this profileAcademic lineage
Possible advisorsa guess from early papers, not confirmed
- Hong-Jiang ZhangSuggested from co-authorship
Is this you? Claim this profile to confirm or dismiss it.
Works275 from public data
- Heterogeneous Network Embedding via Deep Architectures600
It is demonstrated that the rich content and linkage information in a heterogeneous network can be captured by a multi-resolution deep embedding function, so that similarities among cross-modal data can be measured directly in a common embedding space.
- Task Agnostic Meta-Learning for Few-Shot Learning546
An entropy-based approach that meta-learns an unbiased initial model with the largest uncertainty over the output labels by preventing it from over-performing in classification tasks, which outperforms compared meta-learning algorithms in both few-shot classification and reinforcement learning tasks.
- Stock Price Prediction via Discovering Multi-Frequency Trading Patterns524
A novel State Frequency Memory (SFM) recurrent network is proposed to capture the multi-frequency trading patterns from past market data to make long and short term predictions over time.
- Differential Recurrent Neural Networks for Action Recognition516
This study proposes a differential gating scheme for the LSTM neural network, which emphasizes on the change in information gain caused by the salient motions between the successive frames, and thus the model is termed as differential Recurrent Neural Network (dRNN).
- Correlative multi-label video annotation504
A third paradigm is proposed which simultaneously classifies concepts and models correlations between them in a single step by using a novel Correlative Multi-Label (CML) framework and is compared with the state-of-the-art approaches in the first and second paradigms on the widely used TRECVID data set.
- 458
- 376
- Loss-Sensitive Generative Adversarial Networks on Lipschitz Densities366
The Lipschitz regularization theory and algorithms for a novel Loss-Sensitive Generative Adversarial Network (LS-GAN) are presented, yielding a regularized model that can better generalize to produce new data from a reasonable number of training examples than the classic GAN.
- Contrastive Learning with Stronger Augmentations336
A general framework called Contrastive Learning with Stronger Augmentations (CLSA) is proposed to complement current contrastive learning approaches and the distribution divergence between the weakly and strongly augmented images over the representation bank is adopted to supervise the retrieval of strongly augmented queries from a pool of instances.
- 308
- Image annotation by k NN-sparse graph-based label propagation over noisily tagged web images269
This article exploits the problem of annotating a large-scale image corpus by label propagation over noisily tagged web images by proposing a novel kNN-sparse graph-based semi-supervised learning approach for harnessing the labeled and unlabeled data simultaneously.
- 262
- 248
- Cross-Domain Contrastive Learning for Unsupervised Domain Adaptation238
This work builds upon contrastive self-supervised learning to align features so as to reduce the domain discrepancy between training and testing sets, and introduces a simple yet effective framework CDCL, for domain alignment.
- 235
- Causal Attention for Vision-Language Tasks228
CATT improves various popular attention-based vision-language models by considerable margins and has great potential in large-scale pre-training, e.g., it can promote the lighter LXMERT, which uses fewer data and less computational power, comparable to the heavier UNITER.
- AET vs. AED: Unsupervised Representation Learning by Auto-Encoding Transformations Rather Than Data220
The experiments show that AET greatly improves over existing unsupervised approaches, setting new state-of-the-art performances being greatly closer to the upper bounds by their fully supervised counterparts on CIFAR-10, ImageNet and Places datasets.
- Weakly-Shared Deep Transfer Networks for Heterogeneous-Domain Knowledge Propagation214
This paper develops a novel deep network structure, capable of transferring labeling information across heterogeneous domains, especially from text domain to image domain, and presents a novel architecture of DTNs to translate cross-domain information from text to image.
- Hierarchical Long Short-Term Concurrent Memory for Human Interaction Recognition213
A novel Hierarchical Long Short-Term Concurrent Memory (H-LSTCM) is proposed to model the long-term inter-related dynamics among a group of persons for recognizing human interactions by comparing against baseline and state-of-the-art methods.
- Inferring semantic concepts from community-contributed images and noisy tags212
This paper proposes a novel sparse graph-based semi-supervised learning approach for harnessing the labeled and unlabeled data simultaneously, and constructs an informative compact concept space with small semantic gap to infer the semantic concepts in this space to bridge the semantic gap.
- 206
- Multitask AET with Orthogonal Tangent Regularity for Dark Object Detection188
This work proposes a novel multitask auto encoding transformation (MAET) model which is able to explore the intrinsic pattern behind illumination translation and achieves the object detection task by decoding the bounding box coordinates and classes.
- AdCo: Adversarial Contrast for Efficient Learning of Unsupervised Representations from Self-Trained Negative Adversaries175
The proposed Adversarial Contrastive (AdCo) model not only achieves superior performances, but also can be pre-trained more efficiently with much shorter GPU time and fewer epochs.
- CTCNet: A CNN-Transformer Cooperation Network for Face Image Super-Resolution164
This work proposes an efficient CNN-Transformer Cooperation Network (CTCNet) for face super-resolution tasks, which uses the multi-scale connected encoder-decoder architecture as the backbone and devise a novel Local-Global Feature Cooperation Module (LGCM) to promote the consistency of local facial detail and global facial structure restoration simultaneously.
- 164
- 160
- 156
- PC-DARTS: Partial Channel Connections for Memory-Efficient Differentiable Architecture Search.150
This paper presents a novel approach, namely, Partially-Connected DARTS, by sampling a small part of super-network to reduce the redundancy in exploring the network space, thereby performing a more efficient search without comprising the performance.
- Generalized Deep Transfer Networks for Knowledge Propagation in Heterogeneous Domains144
A novel generalized deep transfer networks (DTNs) capable of transferring label information across heterogeneous domains, textual domain to visual domain, and to share the labels between two domains are proposed, able to generate domain-specific and shared interdomain features.
- 144
- Knowledge-Enriched Distributional Model Inversion Attacks138
A novel inversion-specific GAN is presented that can better distill knowledge useful for performing attacks on private models from public data, and generalize better to a variety of datasets and models.
- Interleaved Structured Sparse Convolutional Neural Networks130
This paper presents a modularized building block, IGC-V2: interleaved structured sparse convolutions, which generalizes interleaves group convolutions to the product of more structured sparse kernels, further eliminating the redundancy.
- 130
- 128
- 123
- An efficient sparse metric learning in high-dimensional space via l 1 -penalized log-determinant regularization119
An efficient sparse metric learning algorithm in high dimensional space via an l1-penalized log-determinant regularization that can be efficiently optimized in a block coordinate descent fashion which is much faster than the standard semi-definite programming which has been widely adopted in many other advanced distance learning algorithms.
- Towards semantic knowledge propagation from text corpus to web images118
A mathematical model for the functional relationships between text and image features is developed so as to indirectly transfer semantic knowledge through feature transformations, which is accomplished by mapping instances from different domains into a common space of unspecific topics.
- An Adversarial Approach to Hard Triplet Generation107
This work proposes an adversarial network for Hard Triplet Generation (HTG) to optimize the network ability in distinguishing similar examples of different categories as well as grouping varied examples of the same categories.
- HiEve: A Large-Scale Benchmark for Human-Centric Video Analysis in Complex Events103
A new large-scale dataset with comprehensive annotations, named HiEve, for the understanding of human motions, poses, and actions in a variety of realistic events, especially in crowd and complex events, and it contains a record number of poses, the largest number of action instances, and one of the largest numbers of trajectories lasting for longer time.
- Mining collective intelligence in diverse groups102
A probabilistic model is proposed to jointly assess the reliability of sources and find the true data, and reveals the latent group structure among dependent sources, and aggregate the information at the group level rather than from individual sources directly.
- Boundary Perception Guidance: A Scribble-Supervised Semantic Segmentation Approach99
This work presents a novel Boundary Perception Guidance approach, which consists of two basic components, i.e. prediction refinement and boundary regression, and employs class-agnostic edge maps for supervision to effectively guide the segmentation network in localizing the boundaries between different semantic regions.
- OTAvatar: One-Shot Talking Face Avatar with Controllable Tri-Plane Rendering95
This paper proposes One-shot Talking face Avatar (OTAvatar), which constructs face avatars by a generalized controllable tri-plane rendering solution so that each personalized avatar can be constructed from only one portrait as the reference.
- Deep Ordinal Hashing With Spatial Attention95
A novel deep ordinal hashing (DOH) method is proposed, which learns ordinal representations to generate ranking-based hash codes by leveraging the ranking structure of feature space from both local and global views.
- 93
- 92
- Cross-Receptive Focused Inference Network for Lightweight Image Super-Resolution91
A novel Cross-receptive Field Guided Transformer is designed to enable the selection of contextual information required for reconstruction by using a modulated convolutional kernel that understands the current semantic information and exploits the information interaction within different self-attention.
- A Bayesian Federated Learning Framework With Online Laplace Approximation87
A novel FL framework that uses online Laplace approximation to approximate posteriors on both the client and server side is proposed and achieves state-of-the-art results on several benchmarks, clearly demonstrating the advantages of the proposed method.
- 87
- 85
- Global Versus Localized Generative Adversarial Nets83
The LGAN can be applied to train a robust classifier that prefers locally consistent classification decisions on the manifold, and the resultant regularizer is closely related with the Laplace-Beltrami operator.
- 83
- 81
- 81
- 80
- 79
- 79
- Semi-supervised kernel density estimation for video annotation79
An improved algorithm named semi-supervised adaptive kernel density estimation (SSAKDE) which employs local adaptive kernels rather than a fixed kernel, such that broader kernels can be applied in the regions with low density, and more accurate density estimates can be obtained.
- Correlative Linear Neighborhood Propagation for Video Annotation77
This paper proposes a novel method named correlative linear neighborhood propagation to improve annotation performance and demonstrates its effectiveness and efficiency on the Text REtrieval Conference VIDeo retrieval evaluation data set.
- 75
- Supervised Quantization for Similarity Search74
This paper presents a compact coding approach, supervised quantization, that simultaneously learns feature selection that linearly transforms the database points into a low-dimensional discriminative subspace, and quantizes the data points in the transformed space.
- 73
- LatentAvatar: Learning Latent Expression Code for Expressive Neural Head Avatar71
Experimental results show that the LatentAvatar is able to capture challenging expressions and the subtle movement of teeth and even eyeballs, which outperforms previous state-of-the-art solutions in both quantitative and qualitative comparisons.
- 69
- On clustering heterogeneous social media objects with outlier links68
A probability measure is designed on the social media networks which output a configuration of clusters that are consistent with both content and link structure and the advantage of the method is shown over other state-of-the-art algorithms.
- Correlative multilabel video annotation with temporal kernels58
This article proposes another paradigm of the video annotation method that simultaneously annotates the concepts as well as model correlations between them in one step by the proposed Correlative Multilabel (CML) method, which benefits from the compensation of complementary information between different labels.
- Hierarchical Deep CNN Feature Set-Based Representation Learning for Robust Cross-Resolution Face Recognition57
A feature set-based representation learning (FSRL) scheme to collaboratively represent the hierarchical features for more accurate recognition and the superiority of the presented algorithm to the other competitive CRFR approaches is verified.
- Large-Scale Bisample Learning on ID Versus Spot Face Recognition57
To tackle the bisample problem with only two samples for each class, a classification–verification–classification training strategy is proposed to progressively enhance the IvS performance, and a dominant prototype softmax is incorporated to make the deep learning scalable on large-scale classes.
- Joint Intermodal and Intramodal Label Transfers for Extremely Rare or Unseen Classes57
A model for revealing the functional relationships between text and image features as to directly transfer intermodal and intramodal labels to annotate the images and evaluates the effectiveness on an image classification task and shows the effectiveness with respect to the other compared algorithms.
- Learning semantic distance from community-tagged media collection57
A semi-definite programming is formulated to encode the above two aspects of criteria to learn the distance metric and it is shown such an optimization problem can be efficiently solved with a closed-form solution.
- Recommending Flickr groups with social topic model56
This paper presents a probabilistic latent topic model to model Flickr groups in an integrated framework, expecting to jointly discover the latent interests for users and groups and simultaneously learn the recommendation function.
- 55
- High-Fidelity Clothed Avatar Reconstruction from a Single Image54
A coarse-to-fine way to realize a high-fidelity clothed avatar reconstruction (CAR) from a single image using an implicit model to learn the general shape in the canonical space of a person in a learning-based way and a hyper-network to generate a good initialization.
- MSRA-USTC-SJTU AT TRECVID 2007: HIGH-LEVEL FEATURE EXTRACTION AND SEARCH54
This paper describes the MSRA-USTC-SJTU experiments for TRECVID 2007 and investigates the benefit of unlabeled data by semi-supervised learning, and the multi-layer (ML) multi-instance (MI) relation embedded in video by MLMI kernel, as well as the correlations between concepts by correlative multi-label learning.
- An End-to-End Foreground-Aware Network for Person Re-Identification53
This work proposes an end-to-end foreground-aware network to discriminate against the foreground from the background by learning a soft mask for person re-identification and demonstrates the effectiveness of this approach on three challenging datasets.
- POTTER: Pooling Attention Transformer for Efficient Human Mesh Recovery52
A pure transformer architecture named POoling aTention TransformER (POTTER) for the HMR task from single images, which outperforms the SOTA method METRO by only requiring 7% of total parameters and 14% of the Multiply-Accumulate Operations on the Human3.6M and 3DPW datasets.
- 52
- Learning Generalized Transformation Equivariant Representations Via AutoEncoding Transformations51
The proposed models outperform the state-of-the-art models in both unsupervised and (semi-)supervised tasks and are shown to surpass the fully supervised representation pretrained on ImageNet when they are fine-tuned for the object detection task.
- Sharp Attention Network via Adaptive Sampling for Person Re-Identification51
Extensive experimental evaluations demonstrate the superiority of this new sharp attention model for person re-ID over other related existing, published state-of-the-art works on three challenging benchmarks, including CUHK03, Market-1501, and DukeMTMC-reID.
- State-frequency memory recurrent neural networks51
The State-Frequency Memory is proposed, a novel recurrent architecture that allows to separate dynamic patterns across different frequency components and their impacts on modeling the temporal contexts of input sequences by jointly decomposing memorized dynamics into state-frequency components.
- 51
- Structure-sensitive manifold ranking for video concept detection51
This paper proposes a novel graph-based learning approach, named Structure-Sensitive Manifold Ranking (SSMR), based on a structure-sensitive similarity measure, which takes local distribution differences into account to more accurately measure pairwise similarity.
- AIParsing: Anchor-Free Instance-Level Human Parsing50
An instance-level human parsing network which is anchor-free and solvable on a pixel level, which achieves the best global-level and instance- level performance over state-of-the-art one-stage top-down alternatives.
- 50
- 50
- GraphTER: Unsupervised Learning of Graph Transformation Equivariant Representations via Auto-Encoding Node-Wise Transformations49
This work proposes a novel unsupervised learning of Graph Transformation Equivariant Representations (GraphTER), aiming to capture intrinsic patterns of graph structure under both global and local transformations.
- 49
- 48
- Efficient Image Super-Resolution With Feature Interaction Weighted Hybrid Network46
This work proposes a Feature Interaction Weighted Hybrid Network (FIWHN), which comprises a series of Wide-residual Distillation Interaction Block (WDIB) as the backbone, and incorporates a Transformer and explores a novel architecture to combine CNN and Transformer.
- AVT: Unsupervised Learning of Transformation Equivariant Representations by Autoencoding Variational Transformations46
This work presents a novel principled method by Autoencoding Variational Transformations (AVT) to train the networks by maximizing the mutual information between the transformations and representations, and demonstrates the proposed AVT model sets a new record for the performances on unsupervised tasks.
- Automated Pulmonary Nodule Detection: High Sensitivity with Few Candidates45
A pulmonary detection framework that can achieve high sensitivity with few candidates is proposed and a novel Attention 3D CNN (Attention 3D-CNN) which efficiently utilizes contextual information is proposed to further remove the overwhelming majority of false positives.
- Online multi-label active annotation45
This paper proposes a scalable framework for annotation-based video search, as well as a novel approach to enable large-scale semantic concept annotation, that is, online multi-label active learning, scalable to both the video sample dimension and concept label dimension.
- 44
- 42
- Typicality ranking via semi-supervised multiple-instance learning42
A novel typicality ranking scheme for categorizing natural scenes through a two-stage semi-supervised multiple-instance learning method that only needs image-level labels instead of region-level ones in the training stage and is fully automated and no human feedback is required.
- 41
- 40
- 38
- Schedule On the Fly: Diffusion Time Prediction for Faster and Better Image Generation35
The Time Prediction Diffusion Model (TPDM) is introduced, which employs a plug-and-play Time Prediction Module (TPM) that predicts the next noise level based on current latent features at each denoising step, and trains the TPM using reinforcement learning to maximize a reward that encourages high final image quality while penalizing excessive denoising steps.
- Progressive Unsupervised Person Re-Identification by Tracklet Association With Spatio-Temporal Regularization35
Experimental results show that with the spatio-temporal constraint in the training phase, the proposed approach outperforms the state-of-the-art unsupervised methods by notable margins on DukeMTMC-reID, and achieves competitive performance to fully supervised methods on both DukeMT MC-re ID and Campus4K datasets.
- Concurrence-Aware Long Short-Term Sub-Memories for Person-Person Action Recognition35
A novel Concurrent Long Short-Term Memories (Co-LSTM) is proposed to model the long-term inter-related dynamics between two interacting people on the bonding boxes covering people and is shown to have superior performance compared with the state-of-the-arts methods.
- 35
- 35
- Prior-knowledge and attention based meta-learning for few-shot learning34
A novel paradigm of meta- learning approach with three developments to introduce attention mechanism and prior-knowledge for meta-learning is presented, which alleviates the meta-learner's few-shot cognition burden.
- DCCL: A Benchmark for Cervical Cytology Analysis34
This paper introduces the largest cervical cytology dataset, called Deep Cervical Cytological Lesions (referred to as DCCL), which can serve as a valuable resource and platform for researchers to develop new algorithms and pipelines for advanced cervical cancer diagnosis and prevention.
- EnAET: Self-Trained Ensemble AutoEncoding Transformations for Semi-Supervised Learning.34
This study trains an Ensemble of Auto-Encoding Transformations (EnAET) to learn from both labeled and unlabeled data based on the embedded representations by decoding both spatial and non-spatial transformations under a rich family of transformations.
- Temporal Order-Preserving Dynamic Quantization for Human Action Recognition from Multimodal Sensor Streams33
This work proposes a temporal order-preserving dynamic quantization method to extract the most discriminative patterns of the action sequence and presents a multimodal feature fusion method that can be derived in this dynamic quantification framework to exploit different discrim inative capability of features from multiple modalities.
- 32
- Online community detection in social sensing32
This paper designs online algorithms for community detection from the location data collected from such social sensing applications in real time using a robust and efficiently updateable model with the use of Gibbs sampling, and shows its effectiveness and efficiency forsocial sensing applications.
- Flow Generator Matching30
This paper presents a novel solution with theoretical guarantees in the form of Flow Generator Matching (FGM), an innovative approach designed to accelerate the sampling of flow-matching models into a one-step generation, while maintaining the original performance.
- CaCo: Both Positive and Negative Samples are Directly Learnable via Cooperative-Adversarial Contrastive Learning30
It is shown that the positive and negative samples can be cooperatively and adversarially learned by minimizing and maximizing the contrastive loss, respectively.
- 30
- Cross Paradigm Representation and Alignment Transformer for Image Deraining28
This work proposes a novel Cross Paradigm Representation and Alignment Transformer, which aligns and interacts with features in a two-stage progressive manner, enabling adaptive guidance and complementarity, and introduces the Adaptive Alignment Frequency Module (AAFM), which reduces the information gap within and between paradigms.
- 28
- 28
- 28
- Generalized Loss-Sensitive Adversarial Learning with Manifold Margins27
A pullback operator to map samples back to their data manifold, and a manifold margin is defined as the distance between the pullback representations to distinguish between real and fake samples and learn the optimal generators is defined.
- 27
- 27
- 27
- OmniMotionGPT: Animal Motion Generation with Limited Data26
This work designs a model architecture that imitates Generative Pretraining Transformer (GPT), utilizing prior knowledge learned from human data to the animal domain, and introduces AnimalML3D, the first text-animal motion dataset with 1240 animation sequences spanning 36 different ani-mal identities.
- Learning Label Preserving Binary Codes for Multimedia Retrieval26
This article presents a novel multimedia hashing framework, called Label Preserving Multimedia Hashing (LPMH), which is competitive with state-of-the-art methods in both speed and accuracy for multimedia similarity search.
- 26
- 25
- CLARE: A Joint Approach to Label Classification and Tag Recommendation25
A principled way to capture the relations between labels and tags is provided, and a novel framework CLARE is proposed, which fuses data CLAssification and tag REcommendation into a coherent model.
- 24
- Self-Guidance: Boosting Flow and Diffusion Generation on Their Own23
Self-Guidance (SG), which can significantly improve the quality of the generated image by suppressing the generation of low-quality samples, and an efficient variant of SG, named SG-prev, which reuses the output from the immediately previous diffusion step to avoid additional forward passes of the diffusion network.
- Video annotation based on temporally consistent Gaussian random field23
A novel method for automatically annotating video semantics, called temporally consistent Gaussian random field (TCGRF), which adapts the temporal consistency property of video data into graph-based semi-supervised learning to improve the annotation results.
- 22
- 22
- Transfer Learning of Distance Metrics by Cross-Domain Metric Sampling across Heterogeneous Spaces22
This paper focuses on the problem of transfer learning as a way to close the semantic gap between different domains, and shows how to use correspondence information between two domains in order to set up distance functions for the semantically more challenging domain.
- 22
- FeatER: An Efficient Network for Human Reconstruction via Feature Map-Based TransformER21
This work proposes FeatER, a novel transformer design that preserves the inherent structure of feature map representations when modeling attention while reducing memory and computational costs in a set of human reconstruction tasks.
- A Unified Multi-Scenario Attacking Network for Visual Object Tracking21
A Unified and Effective Network, named UEN, to attack visual object tracking models, which is able to attack many state-of-the-art trackers effectively and outperform the introduced baseline in terms of attacking ability and attacking efficiency.
- 21
- Improved Techniques for Model Inversion Attacks20
A variety of new techniques that can significantly boost the performance of MI attacks against deep neural networks (DNNs) are presented and a proposal to model private data distribution in order to better reconstruct representative data points is proposed.
- Integrated graph-based semi-supervised multiple/single instance learning framework for image annotation20
This paper proposes an integrated graph-based semi-supervised learning framework to utilize these two types of representations simultaneously, and explores an effective and computationally efficient strategy to convert the multiple-instance representation into a single-instance one.
- Multi-Condition Latent Diffusion Network for Scene-Aware Neural Human Motion Prediction19
A Multi-Condition Latent Diffusion network (MCLD) is proposed that reformulates the human motion prediction task as a multi-condition joint inference problem based on the given historical 3D body motion and the current 3D scene contexts, and achieves significant improvements over the state-of-the-art methods on both realistic and diverse predictions.
- 19
- MSRA atT TRECVID 2008: High-Level Feature Extraction and Automatic Search.19
This paper describes the MSRA experiments for TRECVID 2008, which representatively investigated the benefit of global and local low-level features by a variety of learning-based methods, including supervised and semi-supervised learning algorithms.
- Self-similarity Driven Scale-invariant Learning for Weakly Supervised Person Search18
A novel one-step framework, named Self-similarity driven Scale-invariant Learning (SSL), based on the self-similarity prior that it shows the same statistical properties of an image at different scales is proposed, which can solve scale variation problem effectively and perform favorably against state-of-the-art methods.
- Video semantic analysis based on structure-sensitive anisotropic manifold ranking18
A novel graph-based SSL framework, named structure-sensitive anisotropic manifold ranking (SSAniMR), is deduced from PDE-based anisotrop diffusion, which significantly outperforms existing graph- based SSL methods and is effective for video semantic annotation.
- Refining video annotation by exploiting pairwise concurrent relation18
This paper proposes to refine video annotation by leveraging the pairwise concurrent relation among video concepts, explicitly modeled by a concurrent matrix and then a propagation strategy is adopted to refine the annotations.
- 18
- UltrAvatar: A Realistic Animatable 3D Avatar Diffusion Model with Authenticity Guided Textures17
A novel 3D avatar generation approach termed UltrAvatar with enhanced fidelity of geometry, and superior quality of physically based rendering (PBR)textures without unwanted lighting is proposed, outperforming the state-of-the-art methods by a large margin.
- Monocular 3D Object Detection with Bounding Box Denoising in 3D by Perceiver17
A stage-wise approach, which combines the information flow from 2D-to-3D (3D bounding box proposal generation with a single 2D image) and 3D-to-2D (proposal verification by denoising with 3D-to-2D contexts) in a top-down manner, and shows that MonoXiver consistently achieves improvement with limited computation overhead.
- Previewer for Multi-Scale Object Detector17
A novel light-weight previewer block is proposed, which previews the objectness probability for the potential regression region of each prior box, using the stronger features with larger receptive fields and more contextual information for better predictions.
- 17
- A Temporal Order Modeling Approach to Human Action Recognition from Multimodal Sensor Data17
A novel temporal order modeling approach to human action recognition is proposed that explores subspace projections to extract the latent temporal patterns from different human action sequences and introduces a sequential optimization algorithm to learn the optimized projections that preserve the pairwise label similarity of the action sequences.
- 17
- 17
- 16
- Very Short-Term Solar Power Forecasting Using a Frequency Incorporated Deep Learning Model15
The results show that although the SFM model with uniform frequency selection does not significantly improve upon the L STM model, the proper selection of frequencies yields overall better performances than the LSTM and 27% better than the persistent forecasts for forecast horizons up to one minute.
- 15
- 15
- 15
- INTERACTIVE VIDEO ANNOTATION BY MULTI-CONCEPT MULTI-MODALITY ACTIVE LEARNING15
A multi-concept multi-modality active learning method for video annotation in which multiple concepts and multiple modalities can be simultaneously taken into consideration is proposed.
- POST: POlicy-Based Switch Tracking14
The proposed POST tracker consists of multiple weak but complementary experts (trackers) and adaptively assigns one suitable expert for tracking in each frame and maintains the performance merit of multiple diverse models while favorably ensuring the tracking efficiency.
- 14
- Deep Learning Architecture with Dynamically Programmed Layers for Brain Connectome Prediction13
A dynamically programmed layer that is critical in determining the alignment between the neuronal activations of pair-wise combinations of neurons is introduced that exploits the spatial and temporal nature of the neuronal activation data.
- 13
- 13
- 13
- 13
- 12
- Taxi Utilization Rate Maximization by Dynamic Demand Prediction: A Case Study in the City of Chicago12
A real-time taxi trip optimization strategy with dynamic demand prediction was designed using long short-term memory (LSTM) architecture to maximize the taxi utilization rate and promising results have been shown by implementing the strategy.
- Learning to Adaptively Scale Recurrent Neural Networks12
Adaptively Scaled Recurrent Neural Networks (ASRNN) are proposed, a simple but efficient way to handle multiscale RNNs that can efficiently adapt scales based on different sequence contexts and yield better performances than baselines without dynamical scaling abilities.
- Efficient semantic annotation method for indexing large personal video database12
A novel annotation framework based on active learning and semi-supervised ensemble method, which is specially designed for personal video database, shows that the proposed method performs superior to both existing semi- supervised learning algorithms and the general active learning algorithms in terms of annotation accuracy and performance stability.
- Microsoft Research Asia TRECVID 2006 High-Level Feature Extraction and Rushes Exploitation.12
The MSRA experiments for TRECVID 2006 were described, including details of the approaches and performance analyses for high-level feature extraction task and rushes exploitation task, and the approaches to concept modeling were validated.
- Zero-shot High-fidelity and Pose-controllable Character Animation11
PoseAnimate is proposed, a novel zero-shot I2V framework for character animation that outperforms the state-of-the-art training-based methods in terms of character consistency and detail fidelity and maintains a high level of temporal coherence throughout the generated animations.
- MorphGANFormer: Transformer-based Face Morphing and De-Morphing11
This paper proposes to develop a transformer-based alternative to face morphing and demonstrates its superiority to StyleGAN-based methods, and introduces a bipartite structure to exploit long-range interactions in face images for iterative propagation of information from latent variables to salient facial features.
- Self-Supervised Graph Representation Learning via Topology Transformations11
The Topology Transformation Equivariant Representation learning is presented, a general paradigm of self-supervised learning for node representations of graph data to enable the wide applicability of Graph Convolutional Neural Networks (GCNNs).
- Multi-level Similarity Perception Network for Person Re-identification11
A novel deep Siamese architecture based on a convolutional neural network and multi-level similarity perception for the person re-identification (re-ID) problem and final feature embedding by simultaneously encoding original global features and discriminative local features is proposed.
- 11
- 11
- 11
- 10
- 10
- Rethink and Redesign Meta learning10
This paper rethink the meta-learning algorithm and presents a novel paradigm of meta-learning approach with three developments to introduce attention mechanism and past knowledge step by step to improve the performance of meta-learning, and the TOF problem can be reduced.
- Temporal Order-based First-Take-All Hashing for Fast Attention-Deficit-Hyperactive-Disorder Detection10
A novel First-Take-All (FTA) hashing framework to investigate the problem of fast ADHD subjects detection through the fMRI time-series of neuron activities and demonstrates the FTA can outperform the state-of-the-art baselines using only neural activity time series without any phenotypic information.
- 10
- 10
- Multi-Constraint Transferable Generative Adversarial Networks for Cross-Modal Brain Image Synthesis9
A novel Brain Generative Adversarial Network (BrainGAN) that explores GANs with multi-constraint and transferable property for cross-modal brain image synthesis, and shows how BrainGAN can learn meaningful tissue representations with rich variability of brain images.
- Adversarial Pretraining of Self-Supervised Deep Networks: Past, Present and Future9
This paper reviews several representative adversarial pretraining models based on Contrastive Learning and Masked Image Modeling, respectively, two popular self-supervised pretraining methods in literature.
- Online Multi-Label Active Learning for Large-Scale Multimedia Annotation9
This paper proposes a scalable framework for annotation-based video search, as well as a novel approach to enable large-scale semantic concept annotation, that is, online multi-label active learning, scalable to both the video sample dimension and concept label dimension.
- 9
- 9
- Self-supervised Multi-view Learning via Auto-encoding 3D Transformations8
This work proposes a novel self-supervised framework to learn Multi-View Transformation Equivariant Representations (MV-TER), exploring the equivariant transformations of a 3D object and its projected multiple views that it derive.
- 8
- State-Driven Dynamic Sensor Selection and Prediction with State-Stacked Sparseness8
A novel dynamic prediction model is developed that uses the notion of state-stacked sparseness to select a subset of the most critical sensors as a function of evolving system state.
- 8
- 8
- S2AFormer: Strip Self-Attention for Efficient Vision Transformer7
S2AFormer is introduced, an efficient Vision Transformer architecture built around a novel Strip Self-Attention (SSA) mechanism that delivers substantial accuracy improvements together with superior inference speed and throughput across both GPU and non-GPU platforms.
- 7
- 7
- Deep Semantic Multimodal Hashing Network for Scalable Multimedia Retrieval.7
A unified deep multimodal hashing framework is proposed to learn compact and high-quality hash codes by exploiting the feature representation learning, inter-modality similarity preserving learning, semantic label preserving learning and hash functions learning with bit balanced constraint simultaneously.
- Latent Dirichlet Truth Discovery: Separating Trustworthy and Untrustworthy Components in Data Sources7
A latent Dirichlet truth (LDT) discovery model is proposed that defines a random field over all the possible configurations of the trustworthy degrees of sources and facts, and the most probable configuration is inferred by a maximum a posteriori criterion over the observed claims.
- Kernel-Based Linear Neighborhood Propagation for Semantic Video Annotation7
This approach combines the consistency assumption and the Local Linear Embedding (LLE) method in a nonlinear kernel-mapped space, which improves a recently proposed method Local Neighborhood Propagation (LNP) by tackling the limitation of its local linear assumption on the distribution of semantics.
- Jointly Understand Your Command and Intention: Reciprocal Co-Evolution Between Scene-Aware 3D Human Motion Synthesis and Analysis6
This paper proposes a cascaded generation strategy that factorizes text-driven scene-specific human motion generation into three stages: goal inferring, path planning, and pose synthesizing, and jointly improves realistic human motion synthesis and robust human motion analysis in 3D scenes.
- Enhancing semantic audio-visual representation learning with supervised multi-scale attention6
This paper proposes a novel supervised multi-scale attention for enhancing semantic audio-visual representation learning from multimedia data, utilizing the audio-visual attention mechanism to combine multi-scale features.
- 6
- 6
- Mixture Factorized Ornstein-Uhlenbeck Processes for Time-Series Forecasting6
A Mixture Factorized OU process (MFOUP) is developed that is able to capture the changing states of multiple mixed hidden factors, from which it can infer their roles in driving the movements of time series.
- 6
- 6
- Multimedia Information Networks in Social Media6
This chapter reviews different categories of recent systems related to multimedia information networks, summarizes the popular inference methods used in recent works, and discusses the applications related toimedia information networks.
- C-Evolve: Consensus-based Evolution for Prompt Groups5
This paper introduces Consensus-Evolve, an evolutionary algorithm that discovers a group of prompts whose aggregated outputs after majority voting achieve optimal performance, and achieves state-of-the-art performance across a wide range of tasks.
- 5
- Guest Editorial Introduction to the Special Section on Representation Learning for Visual Content Understanding5
This special section focuses on data-effective representation learning methods for visual content understanding using the ResNet method for image classification, the DeepFace method for face recognition, and the feature pyramid networks (FPNs) method for object detection.
- 5
- 5
- 5
- Ontological Random Forests for Image Classification5
The authors propose an ontological random forest algorithm where the of decision trees are determined by semantic relations among categories and hierarchical features are automatically learned by multiple-instance learning to capture visual dissimilarities at different concept levels.
- 5
- Guest Editorial: Deep Learning for Multimedia Computing5
The twenty papers in this special section aim at providing a forum to present recent advancements in deep learning research that directly concerns the multimedia community.
- Enhancing Multi-lingual Information Extraction via Cross-Media Inference and Fusion5
A new information fusion approach to integrate facts extracted from cross-media objects (videos and texts) into a coherent common representation including multi-level knowledge (concepts, relations and events) is described.
- 5
- Openstory++: A Large-scale Dataset and Benchmark for Instance-aware Open-domain Visual Storytelling4
This work introduces Openstory++, a large-scale dataset combining additional instance-level annotations with both images and text, and presents Cohere-Bench, a pioneering benchmark framework for evaluating the image generation tasks when long multimodal context is provided.
- AdPE: Adversarial Positional Embeddings for Pretraining Vision Transformers via MAE+4
An Adversarial Positional Embedding (AdPE) approach that distorts the local visual structures by perturbing the position encodings so that the learned transformer cannot simply use the locally correlated patches to predict the missing ones.
- 4
- MSAFusionNet: Multiple Subspace Attention Based Deep Multi-modal Fusion Network4
This paper addresses the task of end-to-end segmentation based on multi-modal data and proposes a novel deep learning framework, multiple subspace attention-based deep multi- modal fusion network (referred to as MSAFusionNet hereon-forth).
- Image generation via latent space learning using improved combination4
A two-stage combining method of AE and GAN under unsupervised and supervised conditions respectively, each stage performed to improve the effect of modeling latent distribution, with loop optimization of the network parameters during training allowing the framework to map an input noise to a high quality image.
- 4
- LEGO-MM: LEarning Structured Model by Probabilistic loGic Ontology Tree for MultiMedia4
A new framework is proposed, termed LEarning Structured Model by Probabilistic loGic Ontology Tree for MultiM edia (LEGO 1 -MM), which can seamlessly integrate both the new target training examples and the existing primitive concept models to infer the more complex concept models.
- 4
- 4
- Learning Dual-Domain Multi-Scale Representations for Single Image Deraining3
This work proposes a novel Dual-Domain Multi-Scale Representation Network (DMSR), which exploits joint multi-scale representations from both external and internal domains in parallel while leveraging the strengths of both spatial and frequency domains to capture more comprehensive properties.
- EigenActor: Variant Body-Object Interaction Generation Evolved from Invariant Action Basis Reasoning3
- 3
- 3
- 3
- 3
- 3
- Semantic Image Retrieval with Feature Space Rankings3
This work proposes a novel hash learning method that encodes features’ relative ordering instead of quantizing their numeric values in a set of low-dimensional ranking subspaces, and formulate the ranking-based hash learning problem as the optimization of a continuous probabilistic error function using softmax approximation.
- Guest Editorial: Big Media Data: Understanding, Search, and Mining3
This editorial, as the fourth part of this special issue, introduces four papers on fine-grained image classification, object retrieval, and applications of social media.
- 3
- 3
- 3
- SCASeg: Strip Cross-Attention for Efficient Semantic Segmentation2
Strip Cross-Attention (SCASeg), an innovative decoder head specifically designed for semantic segmentation, is proposed and a Cross-Layer Block (CLB) that integrates hierarchical feature maps from various encoder and decoder stages to form a unified representation for Keys and Values is introduced.
- Lightweight High-Resolution Subject Matting in the Real World2
Considering efficient inference of mobile depolyment framework, a symmetric pixel shuffle module and a lightweight module TRSU are designed and the proposed PSUNet has the best objective performance on the high-resolution benchmark dataset.
- 2
- Large-scale supervised similarity learning in networks2
A factorized similarity learning (FSL) is proposed to integrate the link, node content, and user supervision into a uniform framework by using matrix factorization, and the final similarities are approximated by the span of low-rank matrices.
- 2
- 2
- 2
- 2
- Equilibrated Diffusion: Frequency-aware Textual Embedding for Equilibrated Image Customization1
Equilibrated Diffusion is proposed, an innovative method that achieves equilibrated image customization by decoupling entangled concept attributes from a frequency-aware perspective, thus harmonizing textual and visual consistency.
- Exploring the Robustness of Human Parsers Toward Common Corruptions1
This paper proposes a novel heterogeneous augmentation-enhanced mechanism to bolster robustness under commonly corrupted conditions and demonstrates good universality which can improve the robustness of the human parsing models and even the semantic segmentation models when facing various image common corruptions.
- Planning Paths through Occlusions in Urban Environments1
This paper uses an inpainting model to fill in a sparse, occluded, semantic lidar point cloud and plans dynamically feasible paths for a vehicle to traverse through the open and inpainted spaces to focus on turns and intersections where occlusions significantly impact navigability.
- Rethinking the Combined and Individual Orders of Derivative of States for Differential Recurrent Neural Networks1
The proposed model progressively builds up the ability of the LSTM gates to detect salient dynamical patterns in deeper stacked layers modeling higher orders of DoS; thus, the proposed L STM model is termed deep differential Recurrent Neural Network (d2RNN).
- 1
- 1
- Primal-Dual Group Convolutions for Deep Neural Networks1
This paper presents a simple and modularized neural network architecture, named primal-dual group convolutional neural networks (PDGCNets), and demonstrates that these networks are more efficient in using parameters and computation complexity with similar or higher accuracy.
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- POLYCRYSTALLINE SILICON THIN FILM OBTAINED BY ALUMINUM INDUCED CRYSTALLIZATION1
It is verified that aluminum can cause lateral crystallization of a-Si film and the resultant poly-Si films show preferred crystal orientation.
- –
- –
- –
- –
- –
- –
- –
- Multimedia Content Understanding by Learning from Very Few Examples–
This tutorial will present serval parallel efforts on building deep learning models with very few supervision information, with or without unsupervised data available, and discuss several paradigms of learning-to-learn approaches.
- –
- –
- –
- Web-Scale Multimedia Information Networks This paper proposes a unified structured representation called MINets to combine information in the multimedia content network, cross-media links, and the associated ontological network.–
A unified STRUCTURED representation called multimedia information networks (MINets), which in- corporates ontology and cross-media links, covering both content and context knowledge is introduced, which makes it possible to increase the portability of information between different resources to increase information utilization levels.
- –
- A Unifying Multi-Label Temporal Kernel Machine with Its Application to Video Annotation–
This paper proposes a new paradigm of the video annotation method that simultaneously annotates the concepts as well as model correlations between them in one step by the proposed Correlative Multi-Label (CML) method and extends the proposed method to exploit the rich temporal information in the videos.
- –
Publication data from OpenAlex, with missing venues and authors filled in from Crossref; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-14. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.
Report an error
Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.