Is this you? Claim this profile to correct it, add a bio and choose the work people see first.
Claim this profileWorks117 from public data
- UNITER: UNiversal Image-TExt Representation Learning2,731
UNITER, a UNiversal Image-TExt Representation, learned through large-scale pre-training over four image-text datasets is introduced, which can power heterogeneous downstream V+L tasks with joint multimodal embeddings.
- Less is More: CLIPBERT for Video-and-Language Learning via Sparse Sampling836
Experiments on text-to-video retrieval and video question answering on six datasets demonstrate that CLIPBERT outperforms (or is on par with) existing methods that exploit full-length videos, suggesting that end- to-end learning with just a few sparsely sampled clips is often more accurate than using densely extracted offline features from full- length videos, proving the proverbial less-is-more principle.
- Emu3: Next-Token Prediction is All You Need811
Emmu3 is introduced, a new suite of state-of-the-art multimodal models trained solely with next-token prediction that outperforms several well-established task-specific models in both generation and perception tasks, surpassing flagship models such as SDXL and LLaVA-1.6.
- HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-training573
HELP, a novel framework for large-scale video+language omni-representation learning that achieves new state of the art on multiple benchmarks over Text-based Video/Video-moment Retrieval, Video Question Answering (QA), Video-and-language Inference and Video Captioning tasks across different domains is presented.
- Large-Scale Adversarial Training for Vision-and-Language Representation Learning561
To enable large-scale training, VILLA adopts the "free" adversarial training strategy, and combines it with KL-divergence-based regularization to promote higher invariance in the embedding space.
- Multispectral Deep Neural Networks for Pedestrian Detection542
This work carefully design four ConvNet fusion architectures that integrate two-branch ConvNets on different DNNs stages, all of which yield better performance compared with the baseline detector, and finds that ConvNet-based pedestrian detectors trained by color or thermal images separately provide complementary information in discriminating human instances.
- Relation-Aware Graph Attention Network for Visual Question Answering400
A Relation-aware Graph Attention Network (ReGAT), which encodes each image into a graph and models multi-type inter-object relations via a graph attention mechanism, to learn question-adaptive relation representations.
- StoryGAN: A Sequential Conditional GAN for Story Visualization312
A new story-to-image-sequence generation model, StoryGAN, based on the sequential conditional GAN framework is proposed, which outperformed state-of-the-art models in image quality, contextual consistency metrics, and human evaluation.
- 240
- TVT: Transferable Vision Transformer for Unsupervised Domain Adaptation188
This paper first comprehensively investigates the performance of ViT on a variety of domain adaptation tasks, and proposes an unified framework, namely Transferable Vision Transformer (TVT), to fully exploit the transferability ofViT for domain adaptation.
- Emu: Generative Pretraining in Multimodality169
Emu, a Transformer-based multimodal foundation model, is presented, which can seamlessly generate images and texts in multi-modality context through a one-model-for-all autoregressive training process and demonstrates superb performance compared to state-of-the-art large multimodAL models.
- Behind the Scene: Revealing the Secrets of Pre-trained Vision-and-Language Models144
VALUE (Vision-And-Language Understanding Evaluation), a set of meticulously designed probing tasks generalizable to standard pre-trained V+L models, aiming to decipher the inner workings of multimodal pre-training.
- Language-Based Image Editing with Recurrent Attentive Models143
A generic modeling framework for two subtasks of LBIE: language-based image segmentation and image colorization using recurrent attentive models to fuse image and language features is proposed.
- Wasserstein Contrastive Representation Distillation121
Wasserstein Contrastive Representation Distillation (WCoRD) is proposed, which leverages both primal and dual forms of Wasserstein distance for KD, yielding a contrastive learning objective that maximizes the lower bound of mutual information between the teacher and the student networks.
- Insulator Faults Detection in Aerial Images from High-Voltage Transmission Lines Based on Deep Learning Model119
The experimental results and analysis demonstrate that the proposed CSPD-YOLO model performs better in insulator fault detection from high-voltage transmission lines with a complex background.
- UC2: Universal Cross-lingual Cross-modal Vision-and-Language Pre-training114
UC2, the first machine translation-augmented framework for cross-lingual cross-modal representation learning, is introduced, to tackle the scarcity problem of multilingual captions for image datasets and facilitate the learning of a joint embedding space of images and all languages of interest.
- Sequential Attention GAN for Interactive Image Editing114
A novel Sequential Attention Generative Adversarial Network (SeqAttnGAN) is proposed, which applies a neural state tracker to encode the previous image and the textual description in each turn of the sequence, and uses a GAN framework to generate a modified version of the image that is consistent with the preceding images and coherent with the description.
- Sequential Attention GAN for Interactive Image Editing via Dialogue.114
A novel Sequential Attention Generative Adversarial Network (SeqAttnGAN) is proposed, which applies a neural state tracker to encode the previous image and the textual description in each turn of the sequence, and uses a GAN framework to generate a modified version of the image that is consistent with the preceding images and coherent with the description.
- Improved YOLOv3 Network for Insulator Detection in Aerial Images with Diverse Background Interference112
Experimental results and analysis validate the claim that the proposed YOLOv3-dense network achieves good performance in the detection of different size insulators amid diverse background interference.
- Multi-step Reasoning via Recurrent Dual Attention for Visual Dialog111
This paper presents a new model for visual dialog, Recurrent Dual Attention Network (ReDAN), using multi-step reasoning to answer a series of questions about an image, and demonstrates that ReDAN can locate context-relevant visual and textual clues via iterative refinement, which can lead to the correct answer step-by-step.
- Target Detection of Forward-Looking Sonar Image Based on Improved YOLOv5109
The experimental results show that the improved model based on YOLOv5 network is superior to the original YOL olv5 network and other popular deep neural networks for target detection in the forward-looking sonar image, which has a reference significance for underwater target detection.
- MTI-YOLO: A Light-Weight and Real-Time Deep Neural Network for Insulator Detection in Complex Aerial Images105
This work proposes the modified YOLO-tiny for insulator (MTI-YOLO) network, which achieves good performance in both complex backgrounds and bright illumination conditions and improves the detection accuracy of different sizes of insulator.
- LightningDOT: Pre-training Visual-Semantic Embeddings for Real-Time Image-Text Retrieval95
A simple yet highly effective approach, LightningDOT is proposed that accelerates the inference time of ITR by thousands of times, without sacrificing accuracy, and achieves superior performance across mainstream ITR benchmarks such as Flickr30k and COCO datasets.
- A Hybrid Retrieval-Generation Neural Conversation Model94
This paper proposes a hybrid neural conversation model that combines the merits of both response retrieval and generation methods and shows that the proposed model outperforms both retrieval- based methods and generation-based methods under both automatic evaluation metrics and human evaluation.
- Contrastive Distillation on Intermediate Representations for Language Model Compression87
CoDIR is proposed, a principled knowledge distillation framework where the student is trained to distill knowledge through intermediate layers of the teacher via a contrastive objective, and achieves superb performance on the GLUE benchmark, outperforming state-of-the-art compression methods.
- Violin: A Large-Scale Dataset for Video-and-Language Inference81
A new large-scale dataset, named Violin (VIdeO-and-Language INference), is introduced for this task, which consists of 95,322 video-hypothesis pairs from 15,887 video clips, spanning over 582 hours of video.
- Video search re-ranking via multi-graph propagation81
Experimental results verify the effectiveness of the graph-based propagation approach combined with the object-sensitive query analysis approach, which brings significant improvement to the baseline of text-based video search.
- What Makes A Good Story? Designing Composite Rewards for Visual Storytelling76
This paper proposes three assessment criteria: relevance, coherence and expressiveness, which are observed through empirical analysis could constitute a “high-quality” story to the human eye and proposes a reinforcement learning framework, ReCo-RL, with reward functions designed to capture the essence of these quality criteria.
- Discriminative Deep Dyna-Q: Robust Planning for Dialogue Policy Learning69
Experiments show that D3Q significantly outperforms DDQ by controlling the quality of simulated experience used for planning and is further demonstrated in a domain extension setting, where the agent’s capability of adapting to a changing environment is tested.
- An improved CACO algorithm based on adaptive method and multi-variant strategies66
An improved CACO algorithm based on adaptive multi-variant strategies (CACOAMS) is proposed in this paper, which can effectively improve system control precision and guarantee feasibility and effectiveness.
- Playing Lottery Tickets with Vision and Language64
This work performs the first empirical study to assess whether trainable subnetworks also exist in pre-trained VL models, and finds that LTH remains relevant when using other training methods (e.g., adversarial training).
- An Improved Method Based on Deep Learning for Insulator Fault Detection in Diverse Aerial Images63
The improved YOLOv3 model achieves good performance for insulator fault detection in aerial images with diverse backgrounds and is the smallest between the four compared models.
- 50
- BachGAN: High-Resolution Image Synthesis From Salient Object Layout44
The proposed Background Hallucination Generative Adversarial Network (BachGAN), which leverages a background retrieval module to first select a set of segmentation maps from a large candidate pool, then encodes these candidate layouts via a background fusion module to hallucinate a suitable background for the given objects.
- 39
- Idempotence and Perceptual Image Compression37
This work proposes a new paradigm of perceptual image codec by inverting unconditional generative model with idempotence constraints, and shows that this approach outperforms state-of-the-art methods such as HiFiC and ILLM, in terms of Fr\'echet Inception Distance.
- Cluster-Former: Clustering-based Sparse Transformer for Question Answering35
Cluster-Former is proposed, a novel clustering-based sparse Transformer to perform attention across chunked sequences that allows information integration beyond local windows, which is especially beneficial for question answering (QA) tasks that rely on long-range dependencies.
- Calibration-Free BEV Representation for Infrastructure Perception34
A Calibration-free BEV Representation (CBR) network is proposed, which achieves 3D detection based on BEV representation without calibration parameters and additional depth supervision and can serve as a baseline for future research addressing practical challenges of infrastructure perception.
- Efficient image encryption and compression based on a VAE generative model34
An image encryption method based on variational auto-encoder generation model that is fast and easy to encrypt, the algorithm is simple and the distortion rate of the decrypt image is low, and it is strong practicality is shown.
- VIMI: Vehicle-Infrastructure Multi-view Intermediate Fusion for Camera-based 3D Object Detection33
This work proposes a novel 3D object detection framework, Vehicles-Infrastructure Multi-view Intermediate fusion (VIMI), and proposes a Multi-scale Cross Attention (MCA) module that fuses infrastructure and vehicle features on selective multi-scales to correct the calibration noise introduced by camera asynchrony.
- Self-Guided Adaptation: Progressive Representation Alignment for Domain Adaptive Object Detection33
This work proposes a Self-Guided Adaptation model, targeting at aligning feature representation and transferring object detection models across domains while considering the instantaneous alignment difficulty, and shows that SGA improves the state-of-the-art methods with significant margins especially on large domain shift cases.
- An image sharpening operator combined with framelet for image deblurring31
The proposed model is convex and hence it can be solved efficiently by the semi-proximal alternating direction method of multipliers (sPADMM) with guaranteed linear rate convergence, which covers the classical ADMM.
- 27
- Dialogue-Oriented Review Summary Generation for Spoken Dialogue Recommendation Systems27
This paper describes an approach using statistical models such as decision trees and SVMs to select the most representative phrases from the extracted phrase set and explains how to generate informative yet concise review summaries for dialogue purposes.
- A nonconvex l1(l1−l2) model for image restoration with impulse noise25
The experimental outcomes demonstrate that the proposed approach is better than those existing methods in terms of standard signal-to-noise ratio (PSNR), relative error (ReErr), and visual quality.
- 25
- 23
- Fatigue behaviors prediction method of welded joints based on soft computing methods23
The simulation results show the proposed method can predict effectively the fatigue behaviors of aluminum alloy welded joints and the RST–ACO–BPNN prediction method is provided with the merits of building model easily, simple structure, high precision and good generalization.
- 21
- 19
- 17
- Med-CMR: A Fine-Grained Benchmark Integrating Visual Evidence and Clinical Logic for Medical Complex Multimodal Reasoning15
Med-CMR is presented, a fine-grained Medical Complex Multimodal Reasoning benchmark that provides a stress test for visual-reasoning integration and rare-case robustness in medical MLLMs, and a rigorous yardstick for future clinical systems.
- SpiderMesh: Spatial-aware Demand-guided Recursive Meshing for RGB-T Semantic Segmentation14
This work systematically proposes a Spatial-aware Demand-guided Recursive Meshing (SpiderMesh) framework that proactively compensates inadequate contextual semantics in optically-impaired regions via a demand-guided target masking algorithm and refines multimodal semantic features with recursive meshing to improve pixel-level semantic analysis performance.
- 14
- 13
- Contrastive Private Data Synthesis via Weighted Multi-PLM Fusion12
A novel contrAstive private data Synthesis via Weighted multiple Pre-trained language models (PLM) framework, named as WASP, which utilizes limited private samples for more accurate private data distribution estimation via a Top-Q voting mechanism, and leverages low-quality synthetic samples for contrastive generation via collaboration among dynamically weighted multiple pre-trained models.
- 11
- 10
- Conditional Perceptual Quality Preserving Image Compression9
It is shown that conditional perceptual quality has similar theoretical properties as rate-distortion-perception trade-off \citep{blau2019rethinking}.
- When Data Geometry Meets Deep Function: Generalizing Offline Reinforcement Learning9
DOGE marries dataset geometry with deep function approximators in offline RL, and enables exploitation in generalizable OOD areas rather than strictly constraining policy within data distribution, demonstrating the superiority of the approach to existing methods that are solely based on data distribution or support constraints.
- 9
- K-means Clustering Based Feature Consistency Alignment for Label-free Model Evaluation8
A novel method called K-means Clustering Based Feature Consistency Alignment (KCFCA), which is tailored to handle the distribution shifts of various datasets, and a dynamic regression model to capture the relationship between the shifts in distribution and model accuracy.
- WSRC: Weakly Supervised Faster RCNN Toward Accurate Traffic Object Detection8
A Weakly Supervised Faster RCNN (WSRC) that aims to train accurate traffic object detectors without using any box annotations, addressing the phenomenon that the traffic object detector trained by traditional method cannot be applied in the real traffic scene due to the lack of data annotation.
- Multi-Level Head-Wise Match and Aggregation in Transformer for Textual Sequence Matching8
Experiments show that the proposed new approach to sequence pair matching with Transformer can achieve new state-of-the-art performance on multiple tasks that rely only on pre-computed sequence-vector-representation, such as SNLI, MNLI-match,MNLI-mismatch, QQP, and SQuAD-binary.
- 8
- 6
- A fractional-order regularization with sparsity constraint for blind restoration of images6
Numerical experiments show that the proposed model with fractional-order regularization and sparsity constraint for blind restoration can effectively remove different kinds of blur without requiring any prior information of the blur kernels and obtain higher signal-to-noise ratios and lower relative errors.
- Detecting moving objects from dynamic background combining subspace learning with mixed norm approach6
The proposed approach outperforms the existing state-of-the-art approaches, particularly for the cases with dynamic background, which indicates the worth to integrate the subspace learning, the ℓ2,1 mixed norm and total variation into a low-rank representation model.
- 6
- 5
- 3D Face Tracking and Multi-Scale, Spatio-temporal Analysis of Linguistically Significant Facial Expressions and Head Positions in ASL5
New methods for the recognition of nonmanual grammatical markers in American Sign Language (ASL) based on a 2-level learning framework to combine low- and high-level features of differing spatio-temporal scales are presented.
- Adaptive Model for Robust Pedestrian Counting5
A model-based approach based on a whole pedestrian model built up in advance, adaptive models are dynamically determined by the occlusion conditions of corresponding body parts to examine visibility of certain body part.
- Automatic detection of fastener safety wire twisting direction based on machine vision4
The experimental results show that the proposed machine vision-based safety wire twisting direction method is effective, has high accuracy and robustness, and can achieve fastener and safety wire location and safety Wire twisting direction detection in complex aircraft maintenance scenarios.
- Adaptive Learning Rates with Maximum Variation Averaging4
An adaptive learning rate rule is proposed in which the running mean squared gradient is replaced by a weighted mean, with weights chosen to maximize the estimated variance of each coordinate, which gives a worst-case estimate for the local gradient variance.
- Research on optical model for microfacet reflectance lightness of stereo paper product surface with BRDF4
The optical model simulation system could sufficiently accomplish reflectance lightness, which provides a novel way to predict and reproduce printing quality.
- Alternating method based on frameletl0-norm and TV regularization for image restoration4
Experimental results show that the proposed alternating minimization method is robust for a different type of blur and noise and does some comparisons with related existing methods to demonstrate that the method has a more significant effect.
- DCAN: Dynamic Channel Attention Network for Multi-Scale Distortion Correction3
A dynamic channel attention network (DCAN) that employs a multi-scale design and utilizes the optical flow network for distortion feature extraction, effectively balancing global structural consistency and local detail preservation under varying levels of distortion is proposed.
- IESSP: Information Extraction-Based Sparse Stripe Pruning Method for Deep Neural Networks3
This work proposes an information extraction-based sparse stripe pruning (IESSP) method, which introduces an information extraction module (IEM), which enhances stripe selection through a mask-based mechanism, promoting inter-layer interactions and directing the network’s focus toward key features.
- 3
- Moving object detection in gigapixel-level videos using manifold sparse representation3
The detection performance on PANDA is improved by an average of 5%, which suggests that the proposed MSR method for detecting moving objects in gigapixel-level videos is promising for gigapixel-level videos.
- 3
- 3
- 3
- 3
- Semi-implicit Image Denoising Algorithm for Different Boundary Conditions3
The Crank-Nicolson semi-implicit difference scheme in matrix form is applied to discrete the Rudin-Osher-Fatemi model and its recovery effects are better than the explicit discrete scheme and the antireflective boundary conditions and Neumann boundary conditions can better maintain the continuity of the boundary in image denoising.
- Global Context and Enhanced Feature Guided Residual Refinement Network for 3D Cardiovascular Image Segmentation2
A novel deep neural network to segment the blood pool and myocardium from three dimensional cardiovascular images is introduced by fully exploiting the global context and complementary information encoded in different feature extraction layers, referred to as GCEFG-R2Net briefly.
- Advanced wang–landau monte carlo‐based tracker for abrupt motions2
A novel tracking algorithm combining the background subtraction method with the Wang–Landau Monte Carlo (WLMC) sampling method for dealing with abrupt motions effectively is proposed, and efficiently samples and tracks the target not only in position but also in scale variance accurately and robustly.
- Exploiting multispectral and contextual information to improve human detection2
Exploiting Multispectral and Contextual Information to Improve Human Detection and to improve human detection skills is explored.
- 1
- A Remeshing Method via Adaptive Multiple Original-Facet-Clipping and Centroidal Voronoi Tessellation1
This work proposes a CVT-based surface remeshing method that balances optimization between quality and efficiency via curvature-adaptive multi-clipping of 3D Centroidal Voronoi cells using original surface facets using original surface facets.
- 1
- A novel Federated Learning method with Domain Adaptation and Model Selection for Intrusion Detection1
- 1
- 1
- Neural RAPT: deep learning-based pitch tracking with prior algorithmic knowledge instillation1
The proposed Neural RAPT is an interpretable neural network-based pitch tracking model that incorporates signal processing knowledge from RAPT and exhibits superiority over existing deep learning-based pitch tracking methods.
- 1
- 1
- 1
- 1
- Visual tracking in high-dimensional particle filter1
A framework using high-order Monte Carlo Markov Chain which considers more information and performs more discriminative and efficient on moving objects than the traditional first-order particle filtering is presented.
- 1
- Research on Feature Fusion Image Defogging Algorithms based on Transformers–
Qualitative results demonstrate the algorithm's superior performance in restoring image color fidelity, contrast, and detail integrity, effectively addressing issues such as incomplete defogging, color bias, or edge blurring present in existing methods.
- Bi-Level Unsupervised Feature Selection–
This work proposes a novel bi-level unsupervised feature selection (BLUFS) method, including a clustering level and a feature level, and is the first work to combine a bi-level framework with the $\ell_{2,0}$-norm.
- –
- IROAM: Improving Roadside Monocular 3D Object Detection Learning from Autonomous Vehicle Data Domain–
This work proposes IROAM, a semantic-geometry decoupled contrastive learning framework, which takes vehicle-side and roadside data as input simultaneously and demonstrates the effectiveness of IROAM in improving roadside detector's performance.
- –
- UnA-Mix: Rethinking Image Mixtures for Unsupervised Person Re-Identification–
This study is the first to explore the impact of the mixup technique on unsupervised person re-identification, which is a downstream task of contrastive learning, in detail and proposes an uncertainty-aware mixup (UnA-Mix) method, which reduces the over-learning of simple person samples and avoids the information damage that occurs when information-rich person samples are mixed.
- –
- –
- Metropolis Theorem and Its Applications in Single Image Detail Enhancement–
The process of searching and matching is treated as a thermodynamic process, where the Metropolis theorem can minimize the internal energy and get the global optimal solution of this task, that is, to find a more suitable feature for a better detail enhancement performance.
- –
- –
- –
- –
- –
- An improved fuzzy c-means algorithm for unbalanced sized clusters–
The improved fuzzy c-means algorithm based on cluster height information is considered and introduced to the distance function to adjust the conventional Euclidean distance, thus to control the effect on classification from cluster size difference.
- The Research on Image Watermark Spread Spectrum Method–
A kind of image watermark spread spectrum method carried out in DCT transform domain is discussed and discriptions in details are given on how to execute comparisons and analysis between different image water mark spread spectrum methods with perceptibility and robustness evaluation standards so that they can get a comparably reasonable conclusion.
Publication data from OpenAlex, with missing venues and authors filled in from Crossref; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-14. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.
Report an error
Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.