Is this you? Claim this profile to correct it, add a bio and choose the work people see first.
Claim this profileWorks249 from public data
- Generative Image Inpainting with Contextual Attention2,597
This work proposes a new deep generative model-based approach which can not only synthesize novel image structures but also explicitly utilize surrounding image features as references during network training to make better predictions.
- Free-Form Image Inpainting With Gated Convolution2,102
The proposed gated convolution solves the issue of vanilla convolution that treats all input pixels as valid ones, generalizes partial convolution by providing a learnable dynamic feature selection mechanism for each channel at each spatial location across all layers.
- 1,363
- Top-Down Neural Attention by Excitation Backprop1,087
A new backpropagation scheme, called Excitation Backprop, is proposed to pass along top-down signals downwards in the network hierarchy via a probabilistic Winner-Take-All process, and the concept of contrastive attention is introduced to make the top- down attention maps more discriminative.
- Interactive Facial Feature Localization1,015
An improvement to the Active Shape Model is proposed that allows for greater independence among the facial components and improves on the appearance fitting step by introducing a Viterbi optimization process that operates along the facial contours.
- MAttNet: Modular Attention Network for Referring Expression Comprehension1,003
This work proposes to decompose expressions into three modular components related to subject appearance, location, and relationship to other objects, which allows for flexibly adapt to expressions containing different types of information in an end-to-end framework.
- High-Resolution Image Inpainting Using Multi-scale Neural Patch Synthesis877
This work proposes a multi-scale neural patch synthesis approach based on joint optimization of image content and texture constraints, which not only preserves contextual structures but also produces high-frequency details by matching and adapting patches with the most similar mid-layer feature correlations of a deep classification network.
- 829
- 784
- MAT: Mask-Aware Transformer for Large Hole Image Inpainting539
A novel transformer-based model for large hole inpainting is presented, which unifies the merits of transformers and convolutions to efficiently process high-resolution images.
- 460
- 435
- RAPID434
The RAPID (RAting PIctorial aesthetics using Deep learning) system is presented, which adopts a novel deep neural network approach to enable automatic feature learning and style attributes of images to help improve the aesthetic quality categorization accuracy.
- InstantBooth: Personalized Text-to-Image Generation without Test-Time Finetuning428
InstantBooth is an innovative approach leveraging existing text-to-image models for instantaneous text-guided image personalization, eliminating the need for test-time finetuning and boasting a 100-fold increase in generation speed.
- Foreground-Aware Image Inpainting402
This work proposes a foreground-aware image inpainting system that explicitly disentangles structure inference and content completion, and shows that by such disentanglement, the contour completion model predicts reasonable contours of objects, and further substantially improves the performance of image inPainting.
- SmartBrush: Text and Shape Guided Object Inpainting with Diffusion Model395
This work proposes a new diffusion-based model named SmartBrush for completing a missing region with an object using both text and shape-guidance and shows that this model outperforms all baselines in terms of visual quality, mask controllability, and background preservation.
- Video-P2P: Video Editing with Cross-Attention Control391
This work proposes to first tune a Text-to-Set (T2S) model to complete an approximate inversion and then optimize a shared unconditional embedding to achieve accurate video inversion with a small memory cost, and proves that it is crucial for consistent video editing.
- Image Super-Resolution by Neural Texture Transfer379
An end-to-end deep model which enriches HR details by adaptively transferring the texture from Ref images according to their textural similarity is designed, which facilitates multi-scale neural transfer that allows the model to benefit more from those semantically related Ref patches, and gracefully degrade to SISR performance on the least relevant Ref inputs.
- 337
- Scene Graph Generation With External Knowledge and Image Reconstruction322
This paper proposes a novel scene graph generation algorithm with external knowledge and image reconstruction loss to overcome dataset issues, and extracts commonsense knowledge from the external knowledge base to refine object and phrase features for improving generalizability inscene graph generation.
- 321
- Recurrent Multimodal Interaction for Referring Image Segmentation315
It is argued that learning word-to-image interaction is more native in the sense of jointly modeling two modalities for the image segmentation task, and a convolutional multimodal LSTM to encode the sequential interactions between individual words, visual information, and spatial information is proposed.
- Deep Image Harmonization314
This work proposes an end-to-end deep convolutional neural network for image harmonization, which can capture both the context and semantic information of the composite images during harmonization and introduces an efficient way to collect large-scale and high-quality training data that can facilitate the training process.
- 289
- 273
- Towards High-Resolution Salient Object Detection242
Experiments illustrate that the proposed novel approach outperforms existing state-of-the-art methods on high-resolution saliency datasets by a large margin, and achieves comparable or even better performance than them on some widely used saliency benchmarks.
- 239
- Contextual-Based Image Inpainting: Infer, Match, and Translate234
This work proposes a learning-based approach to generate visually coherent completion given a high-resolution image with missing components and shows that it generates results of better visual quality than previous state-of-the-art methods.
- 230
- Multimodal Contrastive Training for Visual Representation Learning229
This work develops an approach to learning visual representations that embraces multimodal data, driven by a combination of intra- and inter-modal similarity preservation objectives, and exploits intrinsic data properties within each modality and semantic information from cross- modal correlation simultaneously, hence improving the quality of learned visual representations.
- 224
- Temporally Distributed Networks for Fast Video Semantic Segmentation224
TDNet, a temporally distributed network designed for fast and accurate video semantic segmentation, is presented, observing that features extracted from a certain high-level layer of a deep CNN can be approximated by composing features extracts from several shallower sub-networks.
- Inner and Inter Label Propagation: Salient Object Detection in the Wild197
A co-transduction algorithm is devised to fuse both boundary and objectness labels based on an inter propagation scheme, which achieves superior performance compared with the newest state-of-the-arts in terms of different evaluation metrics.
- High-Resolution Image Inpainting with Iterative Confidence Feedback and Guided Upsampling187
A deep generative model which not only outputs an inpainting result but also a corresponding confidence map is introduced, which progressively fills the hole by trusting only high-confidence pixels inside the hole at each iteration and focuses on the remaining pixels in the next iteration.
- Lite Vision Transformer with Enhanced Self-Attention178
Lite Vision Transformer (LVT), a novel light-weight transformer network with two enhanced self-attention mechanisms to improve the model performances for mobile deployment and Recursive Atrous Self-Attention (RASA), which utilizes the multi-scale context when calculating the similarity map and a recursive mechanism to increase the representation capability with marginal extra parameter cost.
- 178
- 168
- 165
- Photo-Sketching: Inferring Contour Drawings From Images163
Surprisingly, when the model fine-tunes on BSDS500, the model achieves the state-of-the-art performance in salient boundary detection, suggesting contour drawing might be a scalable alternative to boundary annotation, which at the same time is easier and more interesting for annotators to draw.
- Mask Guided Matting via Progressive Refinement Network160
A robust matting framework that takes a general coarse mask as guidance, which can generalize to unseen types of guidance masks such as trimap and low-quality alpha matte, making it suitable for various application pipelines.
- PhraseCut: Language-Based Image Segmentation in the Wild160
This work considers the problem of segmenting image regions given a natural language phrase, and studies it on a novel dataset of 77,262 images and 345,486 phrase-region pairs, collected on top of the Visual Genome dataset.
- Real-Time Semantic Segmentation With Fast Attention159
The proposed architecture relies on the fast spatial attention, which is a simple yet efficient modification of the popular self-attention mechanism and captures the same rich spatial context at a small fraction of the computational cost, by changing the order of operations.
- Learning to Predict Visual Attributes in the Wild154
A large-scale in-the-wild visual attribute prediction dataset consisting of over 927K attribute annotations for over 260K object instances is introduced, including a base model that utilizes both low- and high-level CNN features with multi-hop attention, reweighting and resampling techniques, and a novel supervised attribute-aware contrastive learning algorithm.
- 153
- 148
- Uncovering the Disentanglement Capability in Text-to-Image Diffusion Models146
This work finds that for stable diffusion models, by partially changing the input text embedding from a neutral description to one with style, the generated images can be modified towards the target style without changing the semantic content.
- 143
- 142
- Eigen-PEP for Video Face Recognition135
The Eigen-PEP model is presented, built upon the recent success of the probabilistic elastic part (PEP) model, which produces an intermediate high dimensional, part-based, and pose-invariant representation of a face subject.
- 129
- Joint Object and Part Segmentation Using Deep Learned Potentials128
This paper proposes a joint solution that tackles semantic object and part segmentation simultaneously, in which higher object-level context is provided to guide part segmentsation, and more detailed part-level localization is utilized to refine object segmentation.
- 121
- Open-Vocabulary Instance Segmentation via Robust Cross-Modal Pseudo-Labeling119
This work proposes a cross-modal pseudo-labeling framework, which generates training pseudo masks by aligning word semantics in captions with visual features of object masks in images and designs a robust student model that selectively distills mask knowledge by estimating the mask noise levels, hence mitigating the adverse impact of noisy pseudo masks.
- Video Scene Parsing with Predictive Feature Learning117
It is experimentally proved that the learned predictive features in the model are able to significantly enhance the video parsing performance by combining with the standard image parsing network.
- Automatic Image Cropping using Visual Composition, Boundary Simplicity and Content Preservation Models117
This paper proposes a new, aesthetic photo cropping system which combines three models: visual composition, boundary simplicity, and content preservation, and achieves significant improvement over recent cropping methods in both quantitative and qualitative evaluation.
- High Quality Entity Segmentation115
CropFormer is the first query-based Transformer architecture that can effectively fuse mask predictions from multiple image views, by learning queries that effectively associate the same entities across the full image and its crop.
- 113
- Skeleton Key: Image Captioning by Skeleton-Attribute Decomposition112
A coarse-to-fine method that decomposes the original image description into a skeleton sentence and its attributes, and generates the skeleton sentences and attribute phrases separately, which enables image description generation that better accommodates user preferences.
- SSH: A Self-Supervised Framework for Image Harmonization108
This work reformulate the image harmonization problem from a representation fusion perspective, which separately processes the foreground and background examples, to address the background occlusion issue and builds a real-world harmonization dataset as carefully created by expert users, for evaluation and benchmarking purposes.
- Open World Entity Segmentation103
This work proposes a CondInst-like fully-convolutional architecture with two novel modules specifically designed to exploit the class-agnostic and non-overlapping requirements of ES, and investigates the feasibility of convolutional center-based representation to segment things and stuffs in a unified manner.
- 102
- 98
- Multimodal Style Transfer via Graph Cuts97
This paper introduces a more flexible and general universal style transfer technique: multimodal style transfer (MST), which explicitly considers the matching of semantic patterns in content and style images.
- 94
- Expressing Visual Relationships via Language93
This work introduces a new language-guided image editing dataset that contains a large number of real image pairs with corresponding editing instructions and proposes a new relational speaker model based on an encoder-decoder architecture with static relational attention and sequential multi-head attention and extended with dynamic relational attention.
- 91
- 88
- 85
- 84
- Locker: Locally Constrained Self-Attentive Sequential Recommendation77
A simple framework, (Locker), that enhances short-term user dynamics modeling, while retaining the long-term semantics captured by standard self-attentive encoders, is proposed in a plug-and-play fashion.
- Multi-Scale Aligned Distillation for Low-Resolution Detection74
The idea of spatially aligning feature maps between models of varying input resolutions by shifting feature pyramid position is explored and crossed feature-level fusion is proposed to dynamically fuse teacher’s multi-resolution features to guide the student better.
- Sky is not the limit69
An automatic background replacement algorithm that can generate realistic, artifact-free images with a diverse styles of skies by utilizing visual semantics to guide the entire process including sky segmentation, search and replacement is proposed.
- 69
- Content-Aware GAN Compression67
A novel content-aware method is proposed to guide the processes of both pruning and distillation of unconditional GANs, which forms a smoother and better disentangled latent manifold, making it more effective for image editing.
- 66
- 65
- Automatic Content-Aware Color and Tone Stylization64
A new technique is introduced that automatically generates diverse, visually compelling stylizations for a photograph in an unsupervised manner by learning style ranking for a given input using a large photo collection and selecting a diverse subset of matching styles for final style transfer.
- 63
- High Quality Segmentation for Ultra High-resolution Images63
The Continuous Refinement Model (CRM) continuously aligns the feature map with the refinement target and aggregates features to reconstruct these image details and shows its significant generalization ability to fill the resolution gap between low-resolution training images and ultra high-resolution testing ones.
- SketchEdit: Mask-Free Local Image Manipulation with Partial Sketches62
This work investigates a new paradigm of sketch-based image manipulation: mask-free local image manipulation, which only requires sketch inputs from users and utilizes the entire original image.
- 62
- Scene Parsing with Global Context Embedding62
This work trains a context network based on scene similarities to generate feature representations for global contexts and designs modules to embed the feature representations and the priors into the segmentation network as additional global context cues.
- SceneComposer: Any-Level Semantic Image Synthesis61
Experimental results show that the proposed method can generate high-quality images following the layout at given precision, and compares favorably against existing methods.
- FoveaNet: Perspective-Aware Urban Scene Parsing61
This work proposes a new FoveaNet model to fully exploit the perspective geometry of scene images and address the common failures of generic parsing models, and introduces a new dense CRFs model that takes the Perspective geometry as a prior potential.
- Consensus of Regression for Occlusion-Robust Facial Feature Localization61
This work proposes an occlusion-robust regression method, which is trained to estimate facial feature locations under the precondition that a particular pre-defined region of the face is occluded, and achieves consistently better results over state-of-the-art methods for facial landmark localization and Occlusion detection.
- 59
- 59
- Harnessing the Spatial-Temporal Attention of Diffusion Models for High-Fidelity Text-to-Image Synthesis58
This paper proposes a new text-to-image algorithm that adds explicit control over spatial-temporal cross-attention in diffusion models, and generates images with higher fidelity compared to diffusion-model-based baselines without fine-tuning the diffusion model.
- Mobile Product Image Search by Automatic Query Object Extraction58
A weighted object mask voting algorithm is proposed based on a spatially-constrained model, which allows robust localization and segmentation of the query object, and achieves significantly better retrieval accuracy than previous methods.
- DeepLens55
A novel neural network model comprised of a depth prediction module, a lens blur module, and a guided upsampling module that can generate high-quality shallow DoF images at high resolution, and produces significantly fewer artifacts than the baselines and existing solutions for single image shallow doF synthesis.
- Joint Image-Text Representation by Gaussian Visual-Semantic Embedding55
A novel Gaussian Visual-Semantic Embedding model is presented, which leverages the visual information to model text concepts as Gaussian distributions in semantic space, with higher accuracy and better robustness.
- 53
- 52
- Context-Aware Group Captioning via Self-Attention and Contrastive Features51
This paper introduces a new task, context-aware group captioning, which aims to describe a group of target images in the context of another group of related reference images with a framework combining self-attention mechanism with contrastive feature construction to effectively summarize common information from each image group while capturing discriminative information between them.
- Active Object Perceiver: Recognition-Guided Policy Learning for Object Searching on Mobile Robots49
A novel framework that integrates a deep neural network based object recognition module and a deep reinforcement learning based action prediction mechanism is proposed, which outperforms competing methods in both average trajectory length and success rate.
- 49
- Latent Feature-Guided Diffusion Models for Shadow Removal47
This paper proposes the use of diffusion models as they offer a promising approach to gradually refine the details of shadow regions during the diffusion process, and improves this process by conditioning on a learned latent feature space that inherits the characteristics of shadow-free images, thus avoiding the limitation of conventional methods.
- 46
- ALADIN: All Layer Adaptive Instance Normalization for Fine-grained Style Similarity46
ALADIN sets a new state of the art accuracy for style-based visual search over both coarse labelled style data (BAM) and BAM-FG; a new 2.62 million image dataset of 310,000 fine-grained style groupings also contributed by this work.
- 45
- 45
- 44
- Open-Edit: Open-Domain Image Manipulation with Open-Vocabulary Instructions42
This work proposes a novel algorithm, named Open-Edit, which is the first attempt on open-domain image manipulation with open-vocabulary instructions, which takes advantage of the unified visual-semantic embedding space pretrained on a general image-caption dataset, and manipulates the embedded visual features by applying text-guided vector arithmetic on the image feature maps.
- Multiple Instance Visual-Semantic Embedding42
This work presents Multi-Instance visual-semantic Embedding model (MIE) for embedding images associated with either single or multiple labels, and demonstrates the superiority of the method over the state-of-the-art on two tasks, including multi-label image annotation and zero-shot learning.
- Deep Image Compositing41
A dense-connected multi-stream fusion network is proposed to effectively fuse the information from the foreground and back-ground images at different scales to automatically generate high-quality composites.
- Unsupervised Video Object Segmentation with Joint Hotspot Tracking40
This work reformulates the problem of detecting and tracking of salient object spots as a new task called object hotspot tracking, and proposes a Weighted Correlation Siamese Network (WCS-Net) which employs a Weighting Correlation Block for encoding the pixel-wise correspondence between a template frame and the search frame.
- 39
- 38
- 37
- 35
- 32
- 30
- 30
- Stochastic Downsampling for Cost-Adjustable Inference and Improved Regularization in Convolutional Networks29
This work proposes a novel approach for cost-adjustable inference in CNNs - Stochastic Downsampling Point (SDPoint), which is validated through extensive experiments on image classification.
- 28
- 27
- Neural Rejuvenation: Improving Deep Network Training by Enhancing Computational Resource Utilization26
- 25
- Human MotionFormer: Transferring Human Motions with Vision Transformers25
A hierarchical ViT framework that leverages global and local perceptions to capture large and subtle motion matching, respectively, and set the new state-of-the-art performance both qualitatively and quantitatively is proposed.
- 25
- 25
- 24
- 22
- 22
- 22
- 21
- 20
- 20
- 20
- 19
- Scaling Object Detection by Transferring Classification Weights19
This paper first introduces input and feature normalization schemes to curb the under-fitting during training of a vanilla WTN, and proposes autoencoder-WTN (AE- WTN) which uses reconstruction loss to preserve classification network's information over all classes in the target latent space to ensure generalization to novel classes.
- 19
- 18
- 18
- 18
- 18
- 18
- 17
- 17
- 17
- 17
- 15
- 15
- 15
- 14
- 14
- 14
- 13
- 13
- 13
- 13
- 12
- 12
- 12
- 12
- 11
- 11
- 11
- 11
- 10
- 10
- 10
- 9
- 9
- 9
- 9
- 9
- 9
- Scene Graph Modification Based on Natural Language Commands9
This paper explores the novel problem of graph modification, where the systems need to learn how to update an existing scene graph given a new user’s command.
- 9
- 9
- 9
- 8
- 8
- 8
- 8
- 7
- 7
- 7
- 7
- 7
- 7
- 6
- 6
- 6
- 6
- 6
- 6
- 6
- 6
- 5
- 5
- 5
- 5
- 5
- 5
- 5
- 5
- 5
- 5
- 5
- 5
- 4
- 4
- 4
- 4
- 3
- 3
- 3
- 3
- 3
- 3
- 3
- 3
- 3
- 2
- 2
- 2
- 2
- 2
- 2
- 2
- 2
- 2
- 2
- 2
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- Conditional Random Fields for Image Region Labeling with Global Observation1
A new approach based on Conditional Random Fields (CRF) to assign the semantic labels to the corresponding regions of images by incorporating the global observation into the region labeling framework with the harness of spatial context modeling of CRF model.
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
Publication data from OpenAlex, with missing venues and authors filled in from Crossref; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-13. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.
Report an error
Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.