Is this you? Claim this profile to correct it, add a bio and choose the work people see first.
Claim this profileWorks240 from public data
- VITON: An Image-Based Virtual Try-on Network787
An image-based VIirtual Try-On Network (VITON) without using 3D information in any form is presented, which seamlessly transfers a desired clothing item onto the corresponding region of a person using a coarse-to-fine strategy.
- BlockDrop: Dynamic Inference Paths in Residual Networks554
BlockDrop, an approach that learns to dynamically choose which layers of a deep network to execute during inference so as to best reduce total computation without degrading prediction accuracy, is introduced.
- Modeling Spatial-Temporal Clues in a Hybrid Deep Learning Framework for Video Classification479
This work proposes a hybrid deep learning framework for video classification, which is able to model static spatial information, short-term motion, as well as long-term temporal clues in the videos, and achieves very competitive performance on two popular and challenging benchmarks.
- M2TR: Multi-modal Multi-scale Transformers for Deepfake Detection436
A Multi-modal Multi-scale TRansformer (M2TR), which operates on patches of different sizes to detect local inconsistencies in images at different spatial levels, and learns to detect forgery artifacts in the frequency domain to complement RGB information through a carefully designed cross modality fusion block.
- Exploiting Feature and Class Relationships in Video Categorization with Regularized Deep Neural Networks385
Through arming the DNN with better capability of harnessing both the feature and the class relationships, the proposed regularized DNN (rDNN) is more suitable for modeling video semantics.
- AdaViT: Adaptive Vision Transformers for Efficient Image Recognition355
AdaViT is introduced, an adaptive computation framework that learns to derive usage policies on which patches, self-attention heads and transformer blocks to use throughout the backbone on a per-input basis, aiming to improve inference efficiency of vision transformers with a minimal drop of accuracy for image recognition.
- Automatic Spatially-Aware Fashion Concept Discovery341
This paper proposes an automatic spatially-aware concept discovery approach using weakly labeled image-text data from shopping websites, and decomposes the visual-semantic embedding space into multiple concept-specific subspaces to facilitate structured browsing and attribute-feedback product retrieval.
- Making an Invisibility Cloak: Real World Adversarial Attacks on Object Detectors334
A systematic study of adversarial attacks on state-of-the-art object detection frameworks, and a detailed study of physical world attacks using printed posters and wearable clothes, to quantify the performance of such attacks with different metrics.
- A Survey on Video Diffusion Models318
This article presents a comprehensive review of video diffusion models in the AIGC era, and presents an overview of research on diffusion models in the video domain, categorizing the work into three key areas: video generation, video editing, and other video understanding tasks.
- BEVT: BERT Pretraining of Video Transformers279
BEVT is introduced which decouples video representation learning into spatial representation learning and temporal dynamics learning and achieves state-of-the-art performance on three challenging video benchmarks where BEVT achieves very promising results.
- DCAN: Dual Channel-Wise Alignment Networks for Unsupervised Scene Adaptation276
Dual Channel-wise Alignment Networks (DCAN) are presented, a simple yet effective approach to reduce domain shift at both pixel-level and feature-level in deep neural networks for semantic segmentation.
- AdaFrame: Adaptive Frame Selection for Fast Video Recognition272
It is qualitatively demonstrate learned frame usage can indicate the difficulty of making classification decisions; easier samples need fewer frames while harder ones require more, both at instance-level within the same class and at class-level among different categories.
- ObjectFormer for Image Manipulation Detection and Localization240
This paper proposes ObjectFormer, a method to detect and localize image manipulations that outperforms state-of-the-art tampering detection and localization methods, and uses a set of learnable object prototypes as mid-level representations to model the object-level consistencies among different regions.
- Cross-Domain Contrastive Learning for Unsupervised Domain Adaptation238
This work builds upon contrastive self-supervised learning to align features so as to reduce the domain discrepancy between training and testing sets, and introduces a simple yet effective framework CDCL, for domain alignment.
- The Regretful Agent: Heuristic-Aided Navigation Through Progress Estimation208
This paper proposes to use a progress monitor developed in prior work as a learnable heuristic for search, and proposes two modules incorporated into an end-to-end architecture that significantly outperforms current state-of-the-art methods using greedy action selection.
- Multi-Stream Multi-Class Fusion of Deep Networks for Video Classification188
A multi-stream framework that is able to exploit multimodal features that are more comprehensive than those previously attempted and produces significantly better results than the state of the arts on two popular benchmarks, 92.2% on UCF-101 (without using audio) and 84.9% on Columbia Consumer Videos.
- Secrets of RLHF in Large Language Models Part II: Reward Modeling179
This report introduces contrastive learning to enhance the ability of reward models to distinguish between chosen and rejected responses, thereby improving model generalization and employs meta-learning to enable the reward model to maintain the ability to differentiate subtle differences in out-of-distribution samples.
- Towards Transferable Adversarial Attacks on Vision Transformers175
A dual attack framework is introduced, which contains a Pay No Attention (PNA) attack and a PatchOut attack, to improve the transferability of adversarial samples across different ViTs, and it is shown that skipping the gradients of attention during backpropagation can generate adversarial examples with high transferability.
- M3DETR: Multi-representation, Multi-scale, Mutual-relation 3D Object Detection with Transformers174
M3DETR is the first approach that unifies multiple point cloud representations, feature scales, as well as models mutual relationships between point clouds simultaneously using transformers, and ranks 1st on Waymo Open Dataset with single frame point cloud input.
- 165
- Llama Scope: Extracting Millions of Features from Llama-3.1-8B with Sparse Autoencoders160
A suite of 256 SAEs, trained on each layer and sublayer of the Llama-3.1-8B-Base model, with 32K and 128K features are introduced and modifications to a state-of-the-art SAE variant, Top-K SAEs, are evaluated across multiple dimensions.
- Exploring Inter-feature and Inter-class Relationships with Deep Neural Networks for Video Classification154
A novel unified framework that jointly learns feature relationships and exploits the class relationships for improved video classification performance is proposed and demonstrates that the proposed framework exhibits superior performance over several state-of-the-art approaches.
- DiffusionAD: Norm-Guided One-Step Denoising Diffusion for Anomaly Detection142
DiffusionAD, a novel anomaly detection pipeline comprising a reconstruction sub-network and a segmentation sub-network, is introduced and a norm-guided paradigm is proposed to integrate the benefits of multiple noise scales, enhancing the fidelity of reconstructions.
- SVFormer: Semi-supervised Video Transformer for Action Recognition142
SVFormer is introduced, which adopts a steady pseudo-labeling framework (i.e., EMA-Teacher) to cope with unlabeled video samples, and a novel augmentation strategy, Tube Token-Mix, tailored for video data where video clips are mixed via a mask with consistent masked tokens over the temporal axis.
- Masked Video Distillation: Rethinking Masked Feature Modeling for Self-supervised Video Representation Learning135
This paper proposes masked video distillation (MVD), a simple yet effective two-stage masked feature modeling framework for video representation learning that demonstrates that video transformers pre-trained with spatial-temporal co-teaching outperform models distilled with a single teacher on a multitude of video datasets.
- FLAG: Adversarial Data Augmentation for Graph Neural Networks133
This work proposes a simple but effective solution, FLAG (Free Large-scale Adversarial Augmentation on Graphs), which iteratively augments node features with gradient-based adversarial perturbations during training, and boosts performance at test time.
- Prototypical Residual Networks for Anomaly Detection and Localization129
A framework called Prototypical Residual Network (PRN), which learns feature residuals of varying scales and sizes between anomalous and normal patterns to accurately reconstruct the segmentation maps of anomalous regions, is proposed.
- SimDA: Simple Diffusion Adapter for Efficient Video Generation128
A Simple Dif-fusion Adapter (SimDA) is proposed that fine-tunes only 24M out of I.IB parameters of a strong T2I model, adapting it to video generation in a parameter-efficient way and could minimize the training effort with extremely few tunable parameters for model adaptation.
- Evaluating Two-Stream CNN for Video Classification126
An in-depth study is conducted to investigate important implementation options that may affect the performance of deep nets on video classification, and very competitive results are attained on two popular video classification benchmarks.
- Modeling Multimodal Clues in a Hybrid Deep Learning Framework for Video Classification123
A novel hybrid deep learning framework is introduced that integrates useful clues from multiple modalities, including static spatial appearance information, motion patterns within a short time window, audio information, as well as long-range temporal dynamics.
- Robust Optimization as Data Augmentation for Large-scale Graphs118
This work proposes FLAG (Free Large-scale Adversarial Augmentation on Graphs), which iteratively augments node features with gradient-based adversarial perturbations during training, which universally works in node classification, link prediction, and graph classification tasks.
- Rethinking Pseudo Labels for Semi-supervised Object Detection117
This paper introduces certainty-aware pseudo labels tailored for object detection, which can effectively estimate the classification and localization quality of derived pseudo labels by converting conventional localization as a classification task followed by refinement.
- StableAnimator: High-Quality Identity-Preserving Human Image Animation114
StableAnimator is presented, the first end-to-end ID-preserving video diffusion framework, which synthesizes high-quality videos without any post-processing, conditioned on a reference image and a sequence of poses.
- Learning From Noisy Anchors for One-Stage Object Detection114
This paper proposes to mitigate noise incurred by imperfect label assignment such that the contributions of anchors are dynamically determined by a carefully constructed cleanliness score associated with each anchor, and conducts extensive experiments on COCO.
- AgentGym: Evolving Large Language Model-based Agents across Diverse Environments113
This paper proposes AgentGym, a new framework featuring a variety of environments and tasks for broad, real-time, uni-format, and concurrent agent exploration, and proposes a novel method, AgentEvol, to investigate the potential of agent self-evolution beyond previously seen data across tasks and environments.
- ACE: Adapting to Changing Environments for Semantic Segmentation110
This work presents ACE, a framework for semantic segmentation that dynamically adapts to changing environments over time, and introduces a memory that stores feature statistics from previously seen domains, thus preventing catastrophic forgetting.
- Efficient Video Transformers with Spatial-Temporal Token Selection103
STTS is a token selection framework that dynamically selects a few informative tokens in both temporal and spatial dimensions conditioned on input video samples that achieves similar results while requiring 20% less computation.
- 93
- 89
- FiNet: Compatible and Diverse Fashion Image Inpainting87
Fashion Inpainting Networks (FiNet) is presented, a two-stage image-to-image generation framework that is able to perform compatible and diverse inpainting and is readily extended to clothing reconstruction and fashion transfer.
- MotionEditor: Editing Video Motion via Content-Aware Diffusion84
MotionEditor is the first to use diffusion models specifically for video motion editing, considering the origin dynamic background and camera movement, and incorporates a novel content-aware motion adapter into ControlNet to capture temporal motion correspondence.
- BEVNeXt: Reviving Dense BEV Frameworks for 3D Object Detection80
This paper aims to address the drawbacks of the existing dense BEV-based 3D object detectors by introducing the proposed enhanced components, including a CRF-modulated depth estimation module enforcing object-level consistencies, a long-term temporal aggregation module with extended receptive fields, and a two-stage object decoder combining perspective techniques with CRF-modulated depth embedding.
- Enhancing the Self-Universality for Transferable Targeted Attacks73
A novel transfer-based targeted attack method that optimizes the adversarial perturbations without any extra training efforts for auxiliary networks on training data is proposed, named as Self-Universality (SU) attack.
- Implicit Temporal Modeling with Learnable Alignment for Video Recognition72
This paper proposes a novel Implicit Learnable Alignment (ILA) method, which minimizes the temporal modeling effort while achieving incredibly high performance and allows eliminating the costly or insufficient temporal self-attention in video.
- Fudan-Huawei at MediaEval 2015: Detecting Violent Scenes and Affective Impact in Movies with Deep Learning72
This work trains a Convolutional Neural Network model with a subset of ImageNet classes selected particularly for violence detection, and adopts a specially designed two-stream CNN framework to extract features on both static frames and motion optical ows.
- Towards Scalable Neural Representation for Diverse Videos69
D-NeRV is proposed, a novel neural representation framework designed to encode diverse videos by decoupling clip-specific visual content from motion information, introducing temporal reasoning into the implicit neural network, and employing the task-oriented flow as intermediate output to reduce spatial redundancies.
- Cross-Modal Transferable Adversarial Attacks from Images to Videos64
This paper proposes a simple yet effective cross-modal attack method, named as Image To Video (I2V) attack, which generates adversarial frames by minimizing the cosine similarity between features of pretrained image models from adversarial and benign examples, then combines the generated adversarial Frames to perform black-box attacks on video recognition models.
- Look Before You Match: Instance Understanding Matters in Video Object Segmentation61
It is argued that instance understanding matters in VOS, and integrating it with memory-based matching can enjoy the synergy, which is intuitively sensible from the definition of VOS task, i.e., identifying and segmenting object instances within the video.
- ResFormer: Scaling ViTs with Multi-Resolution Training60
ResFormer, a framework that is built upon the seminal idea of multi-resolution training for improved performance on a wide spectrum of, mostly unseen, testing resolutions, is introduced and can be easily extended to semantic segmentation, object detection and video action recognition.
- AgentGym: Evaluating and Training Large Language Model-based Agents across Diverse Environments57
A GENT G YM is proposed, a framework featuring 7 real-world scenarios, 14 environments, and 89 tasks for unified, real-time, and concurrent agent interaction that takes the initial step to develop LLM-based agents that can handle diverse tasks via methods like self-improvement or reinforcement learning.
- 56
- Semi-supervised Vision Transformers55
A joint semi-supervised learning framework, Semiformer, which contains a transformer stream, a convolutional stream and a carefully designed fusion module for knowledge sharing between these streams, which is compatible with most modern transformer and Convolutional neural architectures.
- Enhancing LLM Reasoning via Critique Models with Test-Time and Training-Time Supervision54
This paper delves into a two-player paradigm that separates the roles of reasoning and critique models, where the critique model provides step-level feedback to supervise the reasoning (actor) model during both test-time and train-time, and proposes a critique-in-the-loop self-improvement method.
- VideoLT: Large-scale Long-tailed Video Recognition54
FrameStack is proposed, a simple yet effective method for long-tailed video recognition that performs sampling at the frame-level in order to balance class distributions, and the sampling ratio is dynamically determined using knowledge derived from the network during training.
- Boosting the Transferability of Video Adversarial Examples via Temporal Translation51
A temporal translation attack method, which optimizes the adversarial perturbations over a set of temporal translated video clips to boost the transferability of video adversarial examples for black-box attacks on video recognition models.
- Intentonomy: a Dataset and Study towards Human Intent Understanding51
This paper introduces an intent dataset, Intentonomy, comprising 14K images covering a wide range of everyday scenes, and systematically studies whether, and to what extent, commonly used visual information, i.e., object and context, contribute to human motive understanding.
- Emotion in Context51
The proposed framework can realize recognition in real-time and achieves state-of-the-art performance on two challenging emotion recognition benchmarks, 50.6 on VideoEmotion and 51.8 on Ekman.
- A Coarse-to-Fine Framework for Resource Efficient Video Recognition43
LiteEval is a coarse-to-fine framework that dynamically allocates computation on a per-video basis, and can be deployed in both online and offline settings, and adaptively determines on the fly when to read in more discriminative yet computationally expensive features.
- Toward Generalizable Evaluation in the LLM Era: A Survey Beyond Benchmarks39
This survey probes the core challenges that the rise of LLMs poses for evaluation, and identifies and analyzes two pivotal transitions: from task-specific to capability-based evaluation and from manual to automated evaluation.
- 38
- FT-TDR: Frequency-Guided Transformer and Top-Down Refinement Network for Blind Face Inpainting37
A transformer-based network is used to detect the corrupted regions to be inpainted as masks by modeling the relation among different patches and a top-down refinement network is proposed to hierarchically restore features at different levels and generate contents that are semantically consistent with the unmasked face regions.
- MotionFollower: Editing Video Motion via Lightweight Score-Guided Diffusion35
The proposed MotionFollower is a lightweight score-guided diffusion model for video motion editing that achieves an approximately 80% reduction in GPU memory while delivering superior motion editing performance and exclusively supporting large camera movements and actions.
- Attacking Video Recognition Models with Bullet-Screen Comments30
This paper introduces a novel adversarial attack, the bullet-screen comment (BSC) attack, which attacks video recognition models with BSCs, generated with a Reinforcement Learning (RL) framework.
- 30
- 29
- 29
- MouSi: Poly-Visual-Expert Vision-Language Models28
This paper proposes the use of ensemble experts technique to synergizes the capabilities of individual visual encoders, including those skilled in image-text matching, OCR, image segmentation, etc, while bridging the gap between image encoders and pre-trained LLMs.
- 26
- 24
- 24
- 23
- 23
- 21
- On the Importance of Spatial Relations for Few-shot Action Recognition21
It is found that the spatial misalignment between objects also occurs in videos, notably more common than the temporal inconsistency, and a novel Spatial Alignment Cross Transformer (SA-CT) which learns to re-adjust the spatial relations and incorporates the temporal information is contributed.
- 21
- GCMA: Generative Cross-Modal Transferable Adversarial Attacks from Images to Videos21
This paper investigates cross-modal transferable attacks from image domains to video domains with the generator-oriented approach, i.e., crafting adversarial perturbations for each frame of video clips with the perturbation generator trained in the ImageNet domain to attack target video models.
- Self-Supervised Learning for Semi-Supervised Temporal Language Grounding21
The results demonstrate that the proposed proposed Temporal Language Grounding can achieve competitive performance compared to fully-supervised state-of-the-art methods while only requiring a small portion of temporal annotations.
- 21
- 21
- 20
- 19
- 19
- A Glimpse to Compress: Dynamic Visual Token Pruning for Large Vision-Language Models18
A dynamic pruning framework inspired by human cognition that takes a data-driven “glimpse” and prunes irrelevant visual tokens in a single forward pass before answer generation and reduces the time and memory costs of fine-tuning with negligible performance loss.
- 18
- 18
- 17
- 17
- 16
- Fighting Malicious Media Data: A Survey on Tampering Detection and Deepfake Detection14
A comprehensive review of the current media TD approaches is provided and the challenges and trends in this field are discussed, to discuss the challenges and trends in this field for future research.
- 13
- 13
- Visual Content Recognition by Exploiting Semantic Feature Map with Attention and Multi-task Learning12
- 12
- 12
- 11
- 11
- 10
- 10
- 10
- 10
- 10
- 10
- HCMS: Hierarchical and Conditional Modality Selection for Efficient Video Recognition9
Hierarchical and Conditional Modality Selection (HCMS) is introduced, a simple yet efficient multimodal learning framework for efficient video recognition that dynamically decides on-the-fly whether to use computationally expensive modalities on a per-input basis.
- ModelLock: Locking Your Model With a Spell8
A diffusion-based framework dubbed ModelLock is proposed that explores text-guided image editing to transform the private finetuning data into unique styles or blend new objects into the background to open up a new direction for intellectual property protection of private models.
- 8
- 8
- 7
- 7
- INST-IT: Boosting Instance Understanding via Explicit Visual Prompt Instruction Tuning7
Experimental results show that the proposed Inst-IT models not only achieve outstanding performance on Inst-IT Bench and other instance understanding benchmarks, but also demonstrate significant improvements across various generic image and video understanding benchmarks, highlighting that the method not only boosts instance-level understanding but also strengthens the overall capabilities of generic image and video comprehension.
- 7
- 7
- 7
- 7
- 7
- 6
- Open-VCLIP: Transforming CLIP to an Open-vocabulary Video Model via Interpolated Weight Optimization6
- 6
- 6
- 6
- 5
- 5
- 5
- 5
- 5
- 4
- 4
- 4
- 4
- 4
- 3
- 3
- 3
- 3
- 3
- 2
- 2
- 2
- 2
- 2
- UniGen: Enhanced Training & Test-Time Strategies for Unified Multimodal Understanding and Generation2
- 2
- 2
- 2
- 2
- 2
- 2
- 1
- 1
- 1
- 1
- Downstream Transfer Attack: Adversarial Attacks on Downstream Models with Pre-trained Vision Transformers1
Downstream Transfer Attack (DTA) is proposed, a novel attack method that leverages a pre-trained ViT model to craft the adversarial example and then applies the adversarial example to attack a fine-tuned version of the model on a downstream dataset.
- 1
- 1
- 1
- 1
- 1
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- COSMO-RL: Towards Trustworthy LMRMs via Joint Safety and Stability–
COSMO-RL is presented, a mixed reinforcement learning framework that trains reasoning oriented LMRMs under multimodal, multitask, and multiobjective signals, and the resulting model, COSMO-R1, improves safety while maintaining-and often improving multimodal reasoning and instruction following.
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
Publication data from OpenAlex; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-13. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.
Report an error
Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.