Is this you? Claim this profile to correct it, add a bio and choose the work people see first.
Claim this profileAcademic lineage
Possible advisorsa guess from early papers, not confirmed
- Xiaoou TangSuggested from co-authorship
- Fei ShaSuggested from co-authorship
Is this you? Claim this profile to confirm or dismiss it.
Works131 from public data
- 2,458
- Large-Scale Long-Tailed Recognition in an Open World1,483
An integrated OLTR algorithm is developed that maps an image to a feature space such that visual concepts can easily relate to each other based on a learned metric that respects the closed-world classification while acknowledging the novelty of the open world.
- Synthesized Classifiers for Zero-Shot Learning756
This work introduces a set of "phantom" object classes whose coordinates live in both the semantic space and the model space and demonstrates superior accuracy of this approach over the state of the art on four benchmark datasets for zero-shot learning.
- Adversarial Examples Improve Image Recognition634
This work proposes AdvProp, an enhanced adversarial training scheme which treats adversarial examples as additional examples, to prevent overfitting, and shows that AdvProp improves a wide range of models on various image recognition tasks and performs better when the models are bigger.
- PolarNet: An Improved Grid Representation for Online LiDAR Point Clouds Semantic Segmentation634
This work proposes a new LiDAR-specific, KNN-free segmentation algorithm - PolarNet, which greatly increases the mIoU in three drastically different real urban LiDar single-scan segmentation datasets while retaining ultra low latency and near real-time throughput.
- An Empirical Study and Analysis of Generalized Zero-Shot Learning for Object Recognition in the Wild629
It is shown that there is a large gap between the performance of existing approaches and the performance limit of GZSL, suggesting that improving the quality of class semantic embeddings is vital to improving ZSL.
- Spatiotemporal Contrastive Video Representation Learning600
This work proposes a temporally consistent spatial augmentation method to impose strong spatial augmentations on each frame of the video while maintaining the temporal consistency across frames, and proposes a sampling-based temporal augmentation methods to avoid overly enforcing invariance on clips that are distant in time.
- Connecting the Dots with Landmarks: Discriminatively Learning Domain-Invariant Features for Unsupervised Domain Adaptation522
This paper automatically discovers the existence of landmarks and uses them to bridge the source to the target by constructing provably easier auxiliary domain adaptation tasks, and shows how this composition can be optimized discriminatively without requiring labels from the target domain.
- Curriculum Domain Adaptation for Semantic Segmentation of Urban Scenes494
This work proposes a curriculum-style learning approach to minimize the domain gap in semantic segmentation, which significantly outperforms the baselines as well as the only known existing approach to the same problem.
- Diverse Sequential Subset Selection for Supervised Video Summarization485
This work proposes the sequential determinantal point process (seqDPP), a probabilistic model for diverse sequential subset selection, which heeds the inherent sequential structures in video data, thus overcoming the deficiency of the standard DPP.
- Domain Randomization and Pyramid Consistency: Simulation-to-Real Generalization Without Accessing Target Domain Data483
A new approach of domain randomization and pyramid consistency to learn a model with high generalizability for semantic segmentation of real-world self-driving scenes in a domain generalization fashion is proposed.
- A Fast and Accurate One-Stage Approach to Visual Grounding462
A simple, fast, and accurate one-stage approach to visual grounding that enables end-to-end joint optimization and shows great potential in terms of both accuracy and speed for both phrase localization and referring expression comprehension.
- MoViNets: Mobile Video Networks for Efficient Video Recognition313
Three progressive techniques allow MoViNets to achieve state-of-the-art accuracy and efficiency on the Kinetics, Moments in Time, and Charades video action recognition datasets and a simple ensembling technique to improve accuracy further without sacrificing efficiency is proposed.
- Rethinking Class-Balanced Methods for Long-Tailed Visual Recognition From a Domain Adaptation Perspective312
This work connects existing class-balanced methods for long-tailed classification to target shift to reveal that these methods implicitly assume that the training data and test data share the same class-conditioned distribution, which does not hold in general and especially for the tail classes.
- Surrogate Gap Minimization Improves Sharpness-Aware Training240
GSAM seeks a region with both small loss and low sharpness, giving rise to a model with high generalization capabilities, and theoretically, shows the convergence of GSAM and provably better generalization than SAM.
- Constructing Self-Motivated Pyramid Curriculums for Cross-Domain Semantic Segmentation: A Non-Adversarial Approach239
This work proposes a new approach, called self-motivated pyramid curriculum domain adaptation (PyCDA), to facilitate the adaptation of semantic segmentation neural networks from synthetic source domains to real target domains and reports state-of-the-art results for the adaptation from both GTAV and SYNTHIA to Cityscapes.
- End-to-End Learning of Motion Representation for Video Understanding230
This work proposes TVNet, a novel end-to-end trainable neural network, to learn optical-flow-like features from data, which achieves better accuracies than all compared methods, while being competitive with the fastest counterpart in terms of features extraction time.
- Learning Attributes Equals Multi-Source Domain Generalization207
This work provides a novel perspective to attribute detection and proposes to gear the techniques in multi-source domain generalization for the purpose of learning cross-category generalizable attribute detectors.
- Contrastive Learning for Label Efficient Semantic Segmentation204
A simple and effective contrastive learning-based training strategy in which the network is pretrain the network using a pixel-wise, label-based contrastive loss, and then fine-tune it using the cross-entropy loss, which increases intra-class compactness and inter-class separability, thereby resulting in a better pixel classifier.
- Complete & Label: A Domain Adaptation Approach to Semantic Segmentation of LiDAR Point Clouds198
A Sparse Voxel Completion Network (SVCN) is designed to complete the 3D surfaces of a sparse point cloud to serve as a canonical domain from which semantic labels can transfer across different LiDAR sensors.
- Adversarially Adaptive Normalization for Single Domain Generalization176
A generic normalization approach, adaptive standardization and rescaling normalization (ASR-Norm), to complement the missing part in previous works and bring consistent improvement to the state-of-the-art ADA approaches.
- Look, Listen, and Act: Towards Audio-Visual Embodied Navigation174
This paper attempts to approach the problem of Audio-Visual Embodied Navigation, the task of planning the shortest path from a random starting location in a scene to the sound source in an indoor environment, given only raw egocentric visual and audio sensory data.
- Query-Focused Video Summarization: Dataset, Evaluation, and a Memory Network Based Approach155
This work explores the recently proposed query-focused video summarization which introduces user preferences in the form of text queries about the video into the summarization process and proposes a memory network parameterized sequential determinantal point process in order to attend the user query onto different video frames and shots.
- Open Compound Domain Adaptation146
This work proposes a new approach based on two technical insights into OCDA: a curriculum domain adaptation strategy to bootstrap generalization across domains in a data-driven self-organizing fashion and a memory module to increase the model's agility towards novel domains.
- A Curriculum Domain Adaptation Approach to the Semantic Segmentation of Urban Scenes145
This work proposes a curriculum-style learning approach to minimizing the domain gap in urban scene semantic segmentation, which outperforms the baselines on two datasets and three backbone networks.
- 142
- Reshaping Visual Datasets for Domain Adaptation142
This work devise a nonparametric formulation and efficient optimization procedure that can successfully discover domains among both training and test data and ensures that a strong discriminative model can be learned from the domain.
- VQS: Linking Segmentations to Questions and Answers for Supervised Attention in VQA and Question-Focused Semantic Segmentation140
This paper presents the preliminary work of linking the instance segmentations provided by COCO to the questions and answers (QAs) in the VQA dataset, and name the collected links visual questions and segmentation answers (VQS).
- Query-Focused Extractive Video Summarization138
A probabilistic model, Sequential and Hierarchical Determinantal Point Process (SH-DPP), is developed, for query-focused extractive video summarization, which returns a summary by selecting key shots from the video.
- 134
- 126
- VideoPrism: A Foundational Visual Encoder for Video Understanding124
The pretraining approach improves upon masked autoencoding by global-local distillation of semantic video embeddings and a token shuffling scheme, enabling VideoPrism to focus primarily on the video modality while leveraging the invaluable text associated with videos.
- Automatic facial expression recognition on a single 3D face by exploring shape deformation122
Unlike previous methods which recognize a facial expression with the help of manually labeled key points and/or a neutral face, this method works on a single 3D face without any manual assistance.
- A Semi-Supervised Two-Stage Approach to Learning from Noisy Labels107
This paper contends that it is actually better to ignore the labels of some of the data points than to keep them if the labels are incorrect, especially when the noisy rate is high, and suggests a two-stage framework for the learning from noisy labels.
- Webly-Supervised Video Recognition by Mutually Voting for Relevant Web Images and Web Video Frames104
A novel approach to mutually voting for relevant Web images and video frames, where two forces are balanced, i.e. aggressive matching and passive video frame selection is proposed and validated on three large-scale video recognition datasets.
- 101
- Learning Kernels for Unsupervised Domain Adaptation with Applications to Visual Object Recognition101
This paper addresses the challenging scenario of unsupervised domain adaptation, where the target domain does not provide any annotated data to assist in adapting the classifier.
- Instruct-Imagen: Image Generation with Multi-modal Instruction98
Human evaluation on various image generation datasets re-veals that Instruct-Imagen matches or surpasses prior task-specific models in-domain and demonstrates promising generalization to unseen and more complex tasks.
- Improving Facial Attribute Prediction Using Semantic Segmentation95
The proposed facial attribute prediction model harnesses the localization cues learned by the semantic segmentation to guide the attention of the attribute prediction to the regions where different attributes naturally show up, and is able to localize the attributes.
- 92
- 90
- Robust and Accurate Object Detection via Adversarial Learning83
This work augments the fine-tuning stage for object detectors by exploring adversarial examples, which can be viewed as a model-dependent data augmentation, and dynamically selects the stronger adversarial images sourced from a detector’s classification and localization branches and evolves with the detector to ensure the augmentation policy stays current and relevant.
- Federated Multi-Target Domain Adaptation82
This work considers a more practical scenario where the distributed client data is unlabeled, and a centralized labeled dataset is available on the server, and proposes an effective DualAdapt method to address the new challenges.
- Improving Object Detection with Selective Self-supervised Self-training75
A selective net is proposed to rectify the supervision signals in Web images and not only identifies positive bounding boxes but also creates a safe zone for mining hard negative boxes.
- 72
- Ranking Neural Checkpoints71
The results suggest that the linear separability of the features extracted by the checkpoints is a strong indicator of transferability, and a new ranking measure, ${\mathcal{N}}$LEEP, which gives rise to the best performance in the experiments.
- End-to-End Video Captioning With Multitask Reinforcement Learning67
This paper proposes a multitask reinforcement learning approach to training an E2E video captioning model that is trained end-to-end from the raw video input to the caption output, and shows that such a model outperforms existing ones to a large margin on two benchmarkVideo captioning datasets.
- 61
- 60
- 56
- Neural Networks Are More Productive Teachers Than Human Raters: Active Mixup for Data-Efficient Knowledge Distillation From a Blackbox Model54
This work proposes an approach that blends mixup and active learning that effectively augments the few unlabeled images by a big pool of synthetic images sampled from the convex hull of the original images, and actively chooses from the pool hard examples for the student neural network and query their labels from the teacher model.
- Classifier and Exemplar Synthesis for Zero-Shot Learning54
This paper proposes two ZSL frameworks that learn to synthesize parameters for novel unseen classes through a flexible approach that synthesizes “classifiers” for the unseen classes.
- Improving Sequential Determinantal Point Processes for Supervised Video Summarization52
A new probabilistic distribution is designed such that, when it is integrated into SeqDPP, the resulting model accepts user input about the expected length of the summary, and significantly extend a popular video summarization dataset.
- MosaicOS: A Simple and Effective Use of Object-Centric Images for Long-Tailed Object Detection51
MosaicOS is proposed, a simple and novel framework that is surprisingly effective at tackling the challenges of long-tailed object detection and can be compatibly used with other existing approaches to achieve even further gains.
- Large-margin determinantal point processes51
This work shows how to reparameterize a DPP's kernel matrix with multiple kernel functions, thus enhancing modeling flexibility, and proposes a novel parameter estimation technique based on the principle of large margin separation.
- On Calibrating Semantic Segmentation Models: Analyses and An Algorithm48
This work provides a system-atic study on the calibration of semantic segmentation models and proposes a simple yet effective approach, namely selective scaling, by separating correct/incorrect prediction for scaling and more focusing on misprediction logit smoothing.
- 48
- Open Long-Tailed Recognition in a Dynamic World47
The algorithm, OLTR++, maps images to a feature space such that visual concepts can relate to each other through a memory association mechanism and a learned metric that both respects the closed world classification of seen classes and acknowledges the novelty of open classes.
- Joint Modeling of Dense and Incomplete Trajectories for Citywide Traffic Volume Inference46
This paper proposes a novel framework for the citywide traffic volume inference using both dense GPS trajectories and incomplete trajectories captured by camera surveillance systems and employs a high-fidelity traffic simulator and deep reinforcement learning to recover full vehicle movements from the incomplete trajectoryories.
- Class-Balanced Distillation for Long-Tailed Visual Recognition45
A new training method, referred to as Class-Balanced Distillation (CBD), that leverages knowledge distillation to enhance feature representations and can naturally accommodate the usage of multiple teachers, unlocking the information from an ensemble of models to enhance recognition capabilities.
- How Local Is the Local Diversity? Reinforcing Sequential Determinantal Point Processes with Dynamic Ground Sets for Supervised Video Summarization45
This paper proposes a novel probabilistic model, built upon SeqDPP, to dynamically control the time span of a video segment upon which the local diversity is imposed, and enables SequDPP to learn to automatically infer how local the local Diversity is supposed to be from the input video.
- 42
- 41
- 40
- Controllable Image-to-Video Translation: A Case Study on Facial Expression Generation38
A novel neural network architecture is designed that can incorporate the user input into its skip connections and several improvements to the adversarial training method for the neural network are proposed.
- On Discrete Prompt Optimization for Diffusion Models36
Empirical evaluation on prompts collected from diverse sources suggests that the first gradient-based framework for prompt optimization in text-to-image diffusion models can discover prompts that substantially improve or destroy the faithfulness of images generated by the text-to-image diffusion model.
- 33
- CrossVQA: Scalably Generating Benchmarks for Systematically Testing VQA Generalization32
This paper proposes a semi-automatic framework for generating disentangled shifts by introducing a controllable visual question-answer generation (VQAG) module that is capable of generating highly-relevant and diverse question- answer pairs with the desired dataset style.
- Neptune: The Long Orbit to Benchmarking Long Video Understanding28
A scalable dataset creation pipeline is proposed which leverages large models (VLMs and LLMs), to automatically generate dense, time-aligned video captions, as well as tough question answer decoy sets for video segments (up to 15 minutes in length).
- Automatic Jailbreaking of the Text-to-Image Generative AI Systems27
This paper evaluates the safety of the commercial T2I generation systems, such as ChatGPT, Copilot, and Gemini, on copyright infringement with naive prompts, and proposes a stronger automated jailbreaking pipeline for T2I generation systems, which produces prompts that bypass their safety guards.
- Distilling Vision-Language Models on Millions of Videos27
This work fine-tunes a video-language model from a strong image-language baseline with syn-thesized instructional data, which is then used to auto-label millions of videos to generate high-quality captions and generates detailed descriptions for previously unseen videos.
- Contextualized Spatio-Temporal Contrastive Learning with Self-Supervision27
This paper designs a region-based pretext task which requires the model to transform instance representations from one view to another, guided by context features, and introduces a simple network design that successfully reconciles the simultaneous learning process of both holistic and local representations.
- 25
- Defending Against Adversarial Attacks Using Random Forest24
This paper proposes to use a simple yet very effective non-differentiable hybrid model that combines DNNs and random forests, rather than hide gradients from attackers, to defend against the attacks.
- 23
- Optimize Deep Convolutional Neural Network with Ternarized Weights and High Accuracy23
This work proposes statistical weight scaling and residual expansion methods to reduce the bit-width of the whole network weight parameters to ternary values to greatly reduce model size, computation cost and accuracy degradation caused by the model compression.
- 22
- Geodesic Flow Kernel and Landmarks: Kernel Methods for Unsupervised Domain Adaptation22
This chapter addresses the challenging scenario of unsupervised domain adaptation, where the target domain does not provide any annotated data to assist in adapting the classifier.
- The Crystal Ball Hypothesis in diffusion models: Anticipating object positions from initial noise21
This study identifies specific regions within the initial noise image that play a key role for object generation in the resulting images, termed trigger patches, and proposes a reject-sampling strategy to obtain optimal noise, aiming to improve prompt adherence and positional diversity in image generation.
- VideoGLUE: Video General Understanding Evaluation of Foundation Models20
The need and tremendous opportunities to conduct research on video-focused FMs are revealed, and both tasks and adaptation methods matter when it comes to the evaluation of FMs, which confirms that both tasks and adaptation methods matter.
- Facial Image-to-Video Translation by a Hidden Affine Transformation19
This work derives an inverse formulation to the affine transformation, enabling automatic inference of the facial expression intensities from videos, and could improve users' communication experience by enabling them to conveniently and creatively produce expression GIFs, which are popular art forms in online messaging and social networks.
- 18
- medXGAN: Visual Explanations for Medical Classifiers through a Generative Latent Space17
A novel medical imaging generative adversarial framework, medXGAN (medical eXplanation GAN), to visually explain what a medical classifier focuses on in its binary predictions, and optimize the latent space such that interpolation explains how the features contribute to the classifier’s output.
- OmnixR: Evaluating Omni-modality Language Models on Reasoning across Modalities16
OmnixR presents a unique evaluation towards assessing OLMs over a diverse mix of modalities, such as a question that involves video, audio, and text, providing a rigorous cross-modal reasoning testbed unlike any existing benchmarks.
- Domain Generalization with Adversarial Intensity Attack for Medical Image Segmentation15
This work introduces a novel DG method called Adversarial Intensity Attack (AdverIN), which leverages adversarial training to generate training data with an infinite number of styles and increase data diversity while preserving essential content information.
- Unified Visual Relationship Detection with Vision and Language Models15
UniVRD is proposed, a novel bottom-up method for Unified Visual Relationship Detection by leveraging vision and language models (VLMs) that provides well-aligned image and text embeddings, where similar relationships are optimized to be close to each other for semantic unification.
- 15
- 13
- Anti-Neuron Watermarking: Protecting Personal Data Against Unauthorized Neural Networks11
This work shows that when a user watermarks images using a specialized linear color transformation, a neural network classifier will be imprinted with the signature so that a third-party arbitrator can verify the potentially unauthorized usage of the user data by inferring the watermark signature from the neural network.
- 11
- Boosting 3D object retrieval by object flexibility11
This paper proposes a novel feature, called object flexibility, at a point of a 3D object to describe how the neighborhood of this point is massively connected to the object, and shows that this feature is stable to the deformation of objects' articulations, in addition to commonly concerned linear transforms.
- Identity Encoder for Personalized Diffusion10
This work learns an identity encoder which can extract an identity representation from a set of reference images of a subject, together with a diffusion generator that can generate new images of the subject conditioned on the identity representation.
- 2.5D visual relationship detection10
This work studies 2.5D visual relationship detection (2.5VRD), in which the goal is to jointly detect objects and predict their relative depth and occlusion relationships, and shows that existing models largely rely on semantic cues and simple heuristics to solve this task.
- 9
- 9
- Federated Learning of Shareable Bases for Personalization-Friendly Image Classification8
FedBasis is a novel PFL framework that learns a set of few shareable ``basis'' models, which can be linearly combined to form personalized models for clients, making it more parameter-efficient, robust, and accurate than competitive PFL baselines, especially in the low data regime.
- 8
- 8
- 7
- Compound Domain Adaptation in an Open World.7
A class-confusion loss is proposed to disentangle the domain-dominant factors out of the data and then use them to schedule a curriculum domain adaptation strategy, and a memory-augmented neural network architecture is used to increase the network's capacity for handling previously unseen domains.
- 3D object retrieval with semantic attributes7
This work builds a user-friendly 3D object retrieval system, where the user can browse the database and search for targeted objects using semantic attributes, and has obtained the best retrieval performance on three popular benchmarks.
- 6
- Video Timeline Modeling For News Story Understanding5
This paper curates a realistic benchmark dataset, YouTube-News-Timeline, consisting of over $12$k timelines and $300$k YouTube news videos, and proposes a set of quantitative metrics to comprehensively evaluate and compare methodologies.
- AdverIN: Monotonic adversarial intensity attack for domain generalization in medical image segmentation4
This work proposes a novel domain generalization technique termed Adversarial Intensity Attack (AdverIN), which leverages an adversarial training strategy to augment data diversity by synthesizing a spectrum of intensity variations while preserving essential contextual information within images.
- BabyVLM-V2: Toward Developmentally Grounded Pretraining and Benchmarking of Vision Foundation Models4
This work introduces BabyVLM-V2, a developmentally grounded framework for infant-inspired vision-language modeling that extensively improves upon BabyVLM-V1 through a longitudinal, multifaceted pretraining set, a versatile model, and, most importantly, DevCV Toolbox for cognitive evaluation.
- 4
- Robust Pre-Processing: A Robust Defense Method Against Adversary Attack4
The proposed defense method robust pre-processing achieves the best accuracy among the current state of the art defenses and eliminates the need for adversarial training, which makes it a more generalized defense method with almost half training overhead and much-improved accuracy.
- 4
- 4
- Image Diffusion Preview with Consistency Solver3
Diffusion Preview is introduced, a novel paradigm employing rapid, low-step sampling to generate preliminary outputs for user evaluation, deferring full-step refinement until the preview is deemed satisfactory, making it ideal for efficient preview-and-refine workflows.
- 3
- DAVE: Diverse Atomic Visual Elements Dataset with High Representation of Vulnerable Road Users in Complex and Unpredictable Environments2
DAVE is introduced, a manually annotated dataset emphasizing Vulnerable Road Users (VRUs) — including pedestrians, animals, motorbikes, and bicycles — in complex, real-world environments and proposes DAVE-DETR, a transformer-based detector achieving state-of-the-art performance on DAVE.
- 2
- Lifting Data-Tracing Machine Unlearning to Knowledge-Tracing for Foundation Models1
This position paper proposes to lift data-tracing machine unlearning to knowledge-tracing for foundation models (FMs) and discusses the nontrivial challenges in the knowledge-tracing machine unlearning paradigm.
- 1
- Towards A Unified Neural Architecture for Visual Recognition and Reasoning1
This framework enables the principled investigation of how different visual recognition tasks, datasets, and inductive biases can help enable spatiotemporal reasoning capabilities and finds that object detection, which requires spatial localization of individual objects, is the most beneficial recognition task for reasoning.
- 1
- 1
- A Multisource Domain Generalization Approach to Visual Attribute Detection1
This work provides a novel perspective to attribute detection and proposes to gear the techniques in multisource domain generalization for the purpose of learning cross-category generalizable attribute detectors.
- 1
- –
- –
- –
- –
- –
- –
- –
- –
- Multi-modal Domain Adaptation for REG via Relation Transfer–
This work proposes a novel approach to effectively transfer multi-modal knowledge through a specially relation-tailored approach for the REG problem, which is to localize an image region described by a natural language expression.
- –
- –
Publication data from OpenAlex, with missing venues and authors filled in from Crossref; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-14. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.
Report an error
Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.