Is this you? Claim this profile to correct it, add a bio and choose the work people see first.
Claim this profileAcademic lineage
Possible advisorsa guess from early papers, not confirmed
- Shuicheng YanSuggested from co-authorship
- Bingfeng ZhouSuggested from co-authorship
Is this you? Claim this profile to confirm or dismiss it.
Works137 from public data
- Deep High-Resolution Representation Learning for Visual Recognition5,145
The superiority of the proposed HRNet in a wide range of applications, including human pose estimation, semantic segmentation, and object detection, is shown, suggesting that the HRNet is a stronger backbone for computer vision problems.
- Fast Fourier Convolution848
FFC is a generic operator that can directly replace vanilla convolutions in a large body of existing networks, without any adjustments and with comparable complexity metrics ( e.g. , FLOPs).
- 614
- Pyramidal Flow Matching for Efficient Video Generative Modeling395
A unified pyramidal flow matching algorithm that reinterprets the original denoising trajectory as a series of pyramid stages, where only the final stage operates at the full resolution, thereby enabling more efficient video generative modeling.
- 332
- 196
- Weakly-Supervised Action Localization by Generative Attention Modeling184
This paper proposes to model the class-agnostic frame-wise probability conditioned on the frame attention using conditional Variational Auto-Encoder (VAE), and demonstrates advantage of the method and effectiveness in handling action-context confusion problem.
- High-Capacity Convolutional Video Steganography with Temporal Residual Modeling175
This work proposes a novel solution to the problem of hiding a video into another video by developing the model based on deep convolutional neural networks, which is the first of its kind in the literature of video steganography.
- Attention-Based Multi-Context Guiding for Few-Shot Semantic Segmentation165
An Attentionbased Multi-Context Guiding (A-MCG) network is proposed, which consists of three branches: the support branch, the query branch, the feature fusion branch, and adopts a spatial attention along the fusion branch to highlight context information from several scales, enhancing self-supervision in one-shot learning.
- 163
- InstructScene: Instruction-Driven 3D Indoor Scene Synthesis with Semantic Graph Prior143
Ind InstructScene, a novel generative framework that integrates a semantic graph prior and a layout decoder to improve controllability and fidelity for 3D scene synthesis, is introduced.
- Video-LaVIT: Unified Video-Language Pre-training with Decoupled Visual-Motional Tokenization109
This paper proposes an efficient video decomposition that represents each video as keyframes and temporal motions, then adapted to an LLM using well-designed tokenizers that discretize visual and temporal information as a few tokens, thus enabling unified generative pre-training of videos, images, and text.
- Unified Language-Vision Pretraining in LLM with Dynamic Discrete Visual Tokenization104
A well-designed visual tokenizer is introduced to translate the non-linguistic image into a sequence of discrete tokens like a foreign language that LLM can read, which empowers LaVIT to serve as an impressive generalist interface to understand and generate multi-modal content simultaneously.
- 95
- 85
- 82
- 80
- OmniPhysGS: 3D Constitutive Gaussians for General Physics-Based Dynamics Generation77
Comprehensive experiments demonstrate that OmniPhysGS achieves more general and realistic physical dynamics across a broader spectrum of materials, including elastic, viscoelastic, plastic, and fluid substances, as well as interactions between different materials.
- Non-Metric Locality-Sensitive Hashing77
This paper utilizes the norm-keeping property of p-stable functions to ensure that two data's collision probability reflects their non-metric distance in original feature space and investigates various concrete examples to validate the proposed algorithm.
- 72
- 70
- Learning End-to-End Autonomous Steering Model from Spatial and Temporal Visual Cues67
This work focuses on a vision-based model that directly maps raw input images to steering angles using deep networks, and demonstrates the effectiveness and robustness of the model, even under scenarios like drastic lighting changes and abrupt turning.
- 66
- DiffSplat: Repurposing Image Diffusion Models for Scalable Gaussian Splat Generation65
DiffSplat is introduced, a novel 3D generative framework that natively generates 3D Gaussian splats by taming large-scale text-to-image diffusion models by effectively utilizing web-scale 2D priors while maintaining 3D consistency in a unified model.
- Efficient large-scale image annotation by probabilistic collaborative multi-label propagation64
A novel multi-label propagation scheme that encodes the tag information of an image as a unit label confidence vector, which naturally imposes inter-label constraints and manipulates labels interactively and outperforms the state-of-the-art algorithms.
- Large-Scale Unsupervised Hashing with Shared Structure Learning62
To make hash functions not only preserve the local neighborhood structure but also capture the global cluster distribution of the whole data, an objective function incorporating spectral embedding loss, binary quantization loss, and shared subspace contribution is introduced to guide the hash function learning.
- Scale Matters: Temporal Scale Aggregation Network For Precise Action Localization In Untrimmed Videos57
This work proposes a novel integrated temporal scale aggregation network (TSA-Net), based on the formulation of boundary point detection, that demonstrates clear and consistent better performances and recalibrates new state-of-the-art on THUMOS14 benchmark.
- Submodular video hashing57
A novel framework for efficient large-scale video retrieval that integrates feature pooling and hashing in a single framework, and shows that the influence maximization problem is submodular, which allows a greedy optimization method to achieve a nearly optimal solution.
- 56
- 55
- Scene Aligned Pooling for Complex Video Recognition55
A new visual representation, namely scene aligned pooling, for the task of event recognition in complex videos is proposed and can consistently improve various kinds of visual features such as different low-level color and texture features, or middle-level histogram of local descriptors such as SIFT, or space-time interest points, and high level semantic model features.
- Fast Non-Local Neural Networks with Spectral Residual Learning52
Spectral residual learning is proposed, a novel network architectural design for achieving fully global receptive field and its equivalence to conducting residual learning in some spectral domain is shown and performance improvement by large margins is shown.
- Patch-based Knowledge Distillation for Lifelong Person Re-Identification51
Extensive experiments on twelve person re-identification datasets clearly validate the superiority of the patch-based new solution over state-of-the-art competitors by large performance margins.
- 46
- 46
- Classification by Retrieval44
A generic formulation that significantly expedites the training and deployment of image classification models, particularly under the scenarios of many image categories and high feature dimensions, and proposes a novel bit-flipping procedure which enjoys high efficacy and a local optimality guarantee.
- Regularizing Second-Order Influences for Continual Learning43
This work dissects the interaction of sequential selection steps within a framework built on influence functions to identify a new class of second-order influences that will gradually amplify incidental bias in the replay buffer and compromise the selection process.
- Dense Events Grounding in Video41
This work proposes Dense Events Propagation Network (DepNet), which adaptively aggregates temporal and semantic information of dense events into a compact set through a second-order attention pooling, then selectively propagates the aggregated information to each single event with soft attention.
- Computational facial attractiveness prediction by aesthetics-aware features41
This paper addresses a novel topic, i.e., facial attractiveness prediction, which has both theoretic interest and numerous potential applications such as online dating and face beautification and introduces a new dataset with high-quality landmark annotations and multiple facial attractiveness ratings.
- MoVieS: Motion-Aware 4D Dynamic View Synthesis in One Second40
MoVieS, a Motion-aware View Synthesis model that reconstructs 4D dynamic scenes from monocular videos in one second, allows for the unified modeling of appearance, geometry and motion from monocular videos, and enables reconstruction, view synthesis and 3D point tracking within a single learning-based framework.
- 38
- Stochastic Gradient Made Stable: A Manifold Propagation Approach for Large-Scale Optimization37
A novel stratified semi-stochastic gradient descent (S3GD) algorithm, which elegantly balances the geometric algorithmic convergence rate against the space and time complexities during the optimization of a large family of composite convex functions.
- 35
- Distributed Low-Rank Subspace Segmentation35
This work introduces novel applications of LRR-based subspace segmentation to large-scale semi-supervised learning for multimedia event detection, concept detection, and image tagging and proposes a novel divide-and-conquer algorithm that can cope with LRR's non-decomposable constraints and maintains L RR's strong recovery guarantees.
- Two-Stream Video Classification with Cross-Modality Attention33
A cross-modality attention operation, which can obtain information from other modality in a more effective way than two-stream, is proposed and a compatible block named CMA block is implemented, which is a wrapper of the proposed attention operation.
- 32
- IBM Research and Columbia University TRECVID-2011 Multimedia Event Detection (MED) System32
The IBM Research/Columbia team investigated a novel range of low-level and high-level features and their combination for the TRECVID Multimedia Event Detection (MED) task and submitted four runs exploring various methods of extraction, modeling and fusing of low level features and hundreds of high- level semantic concepts.
- Image Completion with Heterogeneously Filtered Spectral Hints30
A new StyleGAN-based image completion network, Spectral Hint GAN (SH-GAN), inside which a carefully designed spectral processing module, Spectra Hint Unit, is introduced and two novel 2D spectral processing strategies, Heterogeneous Filtering and Gaussian Split are proposed.
- Large-scale multi-task image labeling with adaptive relevance discovery and feature hashing30
A novel multi-label classification approach that seamlessly incorporates the idea of multi-task feature hashing learning that can capture the task relationships at task-level as well as feature-level.
- 30
- Supervised deep learning with auxiliary networks28
Comparing prior works on unsupervised deep networks and supervised learning, SUGAR better balances numerical tractability and the flexible utilization of supervision information, and can effectively improve the performance by using the auxiliary networks, on both shallow and deep architectures.
- Text-Controlled Motion Mamba: Text-Instructed Temporal Grounding of Human Motion27
This study proposes Text-controlled Motion Mamba (TM-Mamba), a unified model that integrates temporal global context, language query control, and spatial graph topology with only linear memory cost and introduces BABEL-Grounding, the first text-motion dataset that provides detailed textual descriptions of human actions along with their corresponding temporal segments.
- Learning Instance-Level Representation for Large-Scale Multi-Modal Pretraining in E-Commerce27
This paper proposes an instance-centric multi-modal pretraining paradigm called ECLIP, a decoder architecture that introduces a set of learnable instance queries to explicitly aggregate instance-level semantics and demonstrates the strong transferability to real-world E-commerce applications.
- Randomized Locality Sensitive Vocabularies for Bag-of-Features Model27
This paper proposes the random locality sensitive vocabulary (RLSV) scheme, a simple scheme that generates and aggregates multiple visual vocabularies based on random projections, without taking clustering or training efforts.
- Mixed image-keyword query adaptive hashing over multilabel images26
A Boosting style algorithm for simultaneously optimizing both the optimal label subsets and hashing functions in a unified formulation is developed, and a query-adaptive retrieval mechanism based on hash bit selection for mixed queries is proposed, no matter whether or not the query words exist in the training data.
- 25
- Applications of deep learning to relativistic hydrodynamics24
It is demonstrated that a successfully designed and trained deep neural network, called stacked U-net, can capture the main features of the non-linear evolution of hydrodynamics, which also rapidly predicts the final profiles for various testing initial conditions.
- Multimedia semantics-aware query-adaptive hashing with bits reconfigurability22
The basic idea of the proposed indexing framework is to maintain a large pool of over-complete hashing functions, which are randomly generated and shared when indexing diverse multimedia semantics, and proposes a sequential bit-selection algorithm based on local consistency and global regularization.
- Compact hashing for mixed image-keyword query over multi-label images21
This paper addresses the new topic of hashing with multi-label data, in which images in the database are assumed to be associated with missing or noisy multiple labels and each query consists of a query image and several textual search terms, similar to the new "Search with Image" function introduced by the Google Image Search.
- IBM Research and Columbia University TRECVID-2012 Multimedia Event Detection (MED), Multimedia Event Recounting (MER), and Semantic Indexing (SIN) Systems20
A range of new modeling strategies were studied, including those that take into account temporal dynamics of event semantics, optimize fusion of system components, provide linear approximations of non-linear kernels, and generate synthetic data for the limited exemplar condition.
- Efficient region-aware large graph construction towards scalable multi-label propagation20
Extensive experiments on NUS-WIDE (260k images) and COREL-5k datasets validate the effectiveness and efficiency of the proposed framework for region-aware and scalable multi-label propagation.
- Temporal Binary Coding for Large-Scale Video Search19
This work proposes a temporal binary coding solution in an unsupervised manner, which simultaneously considers the intrinsic relations among the visual content and the temporal consistency among the successive frames, and formulate the video hashing problem as a joint learning of the binary codes, the hash functions and the temporal variation.
- 18
- 18
- A Stochastic Attribute Grammar for Robust Cross-View Human Tracking18
A learning method to train this attribute grammar model from weakly supervised training data is developed and an alternative inference method to employ both topdown and bottom-up computations to search the optimal solution is developed.
- Learning Sample Importance for Cross-Scenario Video Temporal Grounding17
This paper proposes a novel method called Debiased Temporal Language Localizer (Debias-TLL) to prevent the model from naively memorizing the biases and enforce it to ground the query sentence based on true inter-modal relationship.
- Cap2Seg: Inferring Semantic and Spatial Context from Captions for Zero-Shot Image Segmentation16
Cap2Seg is described, a novel solution of zero-shot image segmentation that harnesses accompanying image captions for intelligently inferring spatial and semantic context for the zero- shot image segmentations task.
- Boosting Complementary Hash Tables for Fast Nearest Neighbor Search16
Extensive experiments carried out on two popular tasks including Euclidean and semantic nearest neighbor search demonstrate that the proposed boosted complementary hash-tables method enjoys the strong table complementarity and significantly outperforms the state-of-the-arts.
- Accelerated Large Scale Optimization by Concomitant Hashing16
A hashing scheme for accelerating min/max inner product is proposed, which exploits properties of order statistics of statistically correlated random vectors and proposes a general framework for accelerating a large variety of optimization procedures in computer vision.
- 15
- Co-segmentation of Image Pairs with Quadratic Global Constraint in MRFs15
Unlike conventional linear global terms in MRFs, this paper proposes a sum-of-squared-difference (SSD) based global constraint and deduce its equivalent quadratic form which takes into account the pairwise relations in feature space.
- 14
- 14
- 14
- 14
- Learning reconfigurable hashing for diverse semantics13
A sequential bit-selection algorithm based on local consistency and global regularization is proposed, thereby achieving reusability of the pre-computed hashing bits in a reusable scheme that benefits the indexing and retrieval of large-scale dataset.
- Coordinate discrete optimization for efficient cross-view image retrieval12
This paper attacks the crossview hashing problem by simultaneously capturing semantic neighboring relations and maximizing the generative probability of the learned hash codes in each view, and develops a novel formulation and optimization scheme for cross-view hashing.
- Non-uniform multiple kernel learning with cluster-based gating functions12
A ''non-uniform'' MKL method with a data-dependent gating mechanism, i.e., adaptively determine the kernel weights for the samples, utilize a soft clustering algorithm and then tune the weight for each cluster under the graph embedding (GE) framework.
- Weakly-Supervised Affordance Grounding Guided by Part-Level Semantic Priors11
This work develops a supervised training pipeline based on pseudo labels for weakly supervised affordance grounding, where a model is trained to identify affordance regions on objects using human-object interaction images and egocentric object images without dense labels.
- 11
- Diffused Fourier Network for Video Action Segmentation10
This work proposes a novel model, dubbed as diffused Fourier network (DFN), which harnesses the parameter-free and light-weighted Fast Fourier Transform (FFT) for efficient spectral-temporal feature learning and treats over-segments as noises corrupting the true temporal boundaries.
- 10
- Co-Salient Object Detection with Semantic-Level Consensus Extraction and Dispersion10
This work proposes a Transformer-based dispersion module that takes into account the variation of the co-salient object in different scenes and distributes the consensus to the image feature maps in an image-specific way while making full use of interactions within the group.
- 10
- 10
- Deep Hashing: A Joint Approach for Image Signature Learning10
This paper proposes a novel algorithm that concurrently performs feature engineering and non-linear supervised hashing function learning, and proposes a pre-training scheme inspired by recent advance in deep network based image classification.
- 9
- InstructLayout: Instruction-Driven 2D and 3D Layout Synthesis With Semantic Graph Prior9
Ind InstructLayout is introduced, a novel generative framework that integrates a semantic graph prior and a layout decoder to improve controllability and fidelity for 2D and 3D layout synthesis.
- NavQ: Learning a Q-Model for Foresighted Vision-and-Language Navigation9
Q-learning is drawn upon to train a Q-model using large-scale unlabeled trajectory data, in order to learn the general knowledge regarding the layout and object relations within indoor scenes within indoor scenes, in order to develop a foresighted agent.
- Weakly-Supervised Spatio-Temporal Video Grounding with Variational Cross-Modal Alignment9
This model re-formulates the original STVG task as two cross-modal alignment sub-problems: region-phrase and frame-sentence and treats them as latent variables and learn to model the joint conditional distribution by reconstructing the interactions of entities in the video.
- Learning Solution-Aware Transformers for Efficiently Solving Quadratic Assignment Problem9
This work proposes the first solution of its kind for QAP in the learn-to-improve category, which encodes facility and location nodes separately, instead of forming computationally intensive association graphs prevalent in current approaches, and enables scalability to larger problem sizes.
- Fixed-Rank Supervised Metric Learning on Riemannian Manifold9
This paper focuses on learning an optimal Mahalanobis distance matrix (parameterized by a positive semi-definite matrix W) in the setting of supervised learning by enforcing fixed-rank constraint on the matrix W and devise a novel second-order Riemannian retraction operator.
- IBM Research and Columbia University TRECVID-2013 Multimedia Event Detection (MED), Multimedia Event Recounting (MER), Surveillance Event Detection (SED), and Semantic Indexing (SIN) Systems.9
The current IBM-Columbia MER system exploits nine observations about human cognition, language, and visual perception in order to produce an effective video recounting of an event, and achieves better results on the retrospective and interactive tasks than last year.
- Large-scale multilabel propagation based on efficient sparse graph construction8
A novel sparse graph based multilabel propagation (SGMP) scheme for super large scale datasets, which encodes the label information of an image as a unit label confidence vector and naturally imposes inter-label constraints and manipulates labels interactively.
- Joint Hand-Object Pose Estimation with Differentiably-Learned Physical Contact Point Analysis7
Experimental results show that both the physical affinity loss and the context-aware graph network can effectively capture the relationship and improve the accuracy of 3D pose estimation.
- Localize, Assemble, and Predicate: Contextual Object Proposal Embedding for Visual Relation Detection7
This work proposes localize-assemble-predicate network (LAP-Net), which decomposes VRD into three sub-tasks: localizing individual objects, assembling and predicting the subject-object pairs, and alleviating the bias in popular relations induced by training data.
- Learning 3-D Human Pose Estimation from Catadioptric Videos6
This work explores a novel way of obtaining gigantic 3-D human pose data without manual annotations by jointly harnessing the epipolar geometry and human skeleton priors and can boil down to an optimization problem over two sets of 2-D estimations.
- Self-Supervised Video Action Localization with Adversarial Temporal Transforms6
This work designs a set of temporal transform operations, including naive temporal down-sampling to learnable attention-piloted time warping, and devise a self-refine module to enhance the completeness of action intervals harnessing temporal and semantic contexts.
- Learning single-shot vehicle orientation estimation from large-scale street panoramas5
This work proposes a solution to the following problem: can one precisely estimate an autonomous vehicle’s orientation relative to the road by taking a picture from within the vehicle, and defines for the first time and proposes a two-stream deep network for attacking this problem.
- 5
- Transferable Video Moment Localization by Moment-Guided Query Prompting4
This work proposes a Moment-Guided Query Prompting (MGQP) method, which is comprised of a prompt primitive constructor, a multimodal prompt refiner, and a holistic prompt incorporator to generate multiple distinct and complementary prompt primitives through stratification of the original queries.
- 4
- Multi-Target Invisibly Trojaned Networks for Visual Recognition and Detection4
This work forms trigger embedding as an image steganography-and-steganalysis problem that conceals a secret image in another image in a decipherable and almost invisible way, and proposes a generic split-and-merge scheme for training a trojaned model.
- 4
- Granularity-Adaptive Spatial Evidence Tokenization for Video Question Answering3
This work proposes a granularity-adaptive spatial evidence tokenization model for video question answering that highlights spatially activated patches at low resolutions through a granularity weighting module and adaptively encodes these activated patches at high resolution for detail supplementation.
- Curriculum Graph Poisoning3
This paper revisits the bi-level optimization problem in graph poisoning and proposes a novel graph poisoning method, termed Curriculum Graph Poisoning (CuGPo), inspired by curriculum learning, which achieves state-of-the-art performance inGraph poisoning attacks.
- 3
- 3
- Google Helps YouTube: Learning Few-Shot Video Classification from Historic Tasks and Cross-Domain Sample Transfer3
This work proposes meta-learning input-transformer (MLIT), a novel deep network that tames the noisy source data such that they are more amenable for being used in the target domain, which significantly elevates the accuracy of the unseen video category.
- High-Precision Camera Localization in Scenes with Repetitive Patterns3
A high-precision multi-modal approach for localizing moving cameras with monocular videos, which has wide potentials in many intelligent applications, including robotics, autonomous vehicles, and so on is presented.
- VSCC'20173
A brief summary of the first workshop on visual analysis for smart and connected communities (VSCC'2017), which is held in conjunction with the ACM Conference on Multimedia, 2017 is presented.
- Contextual motion field-based distance for video analysis3
Estimating distance between two frames is equivalent to calculating the distance between their corresponding Gaussian mixture models, which is solved via earth mover distance (EMD) in this paper.
- 2
- 2
- Tree-Structured Trajectory Encoding for Vision-and-Language Navigation2
This work proposes a novel tree-structured trajectory encoding strategy for VLN, and implements the holistic agent based on cross-modal transformer and train it with a newly-proposed Tree-nDTW reward.
- PointerNet: Spatiotemporal Modeling for Crowd Counting in Videos2
A novel end-to-end video crowd counting framework, named PointerNet (PseudO-3D (P3D) CNNs INtegrated with Temporal channEl-awaRe (TCA) block, which causes the framework to possess greater structural diversity and go deep, while having a limited computational cost and memory demand.
- 2
- Searching Motion Graphs for Human Motion Synthesis2
This work proposes a graph search based method for human motion sequence synthesis, complementing the modern generative model (e.g., variational auto-encoder or Gaussian process) based solutions that currently dominate this task and showing strong advantages at several aspects.
- Question-Guided Semantic Dual-Graph Visual Reasoning with Novel Answers2
A matching based VQA model is designed, in which a novel Single-Source Graph Convolutional Network (SSGCN) module is designed to jointly leverage question guidance and dual semantic answer-graphs to produce more discriminative and relevant answer embeddings.
- 2
- A stochastic image grammar for fine-grained 3D scene reconstruction2
Results with comparisons on public image dataset show that the stochastic grammar for fine-grained 3D scene reconstruction from a single image clearly outperforms the alternate state-of-the-art single-view reconstruction methods.
- Biomechanics-Guided Residual Approach to Generalizable Human Motion Generation and Estimation1
BioVAE achieves state-of-the-art performance on multiple benchmarks, bridging the gap between data-driven motion synthesis and biomechanical authenticity while setting new standards for physically accurate motion generation and pose estimation.
- 1
- Text vectorization based on character recognition and character stroke modeling1
A text vectorization method is proposed using OCR (Optical Character Recognition) and character stroke modeling based on the observation that for a particular character, its font glyphs may have different shapes, but often share same stroke structures.
- Divide-and-Conquer Subspace Segmentation1
This work introduces the DFC-LRR algorithm as a scalable solution to the subspace segmentation problem, and presents results that illustrate the scalability and accuracy of DFC relative to LRR.
- MC-JBIG2: an improved algorithm for Chinese textual image compression1
An improved algorithm called MC-JBIG2 is developed, which aims at improving compression ratio for Chinese textual images by first multiple features are extracted from the characters in the images, and a cascade of clusters are introduced to accomplish the pattern-matching task.
- –
- –
- –
- –
- –
- Video2Subtitle: Matching Weakly-Synchronized Sequences via Dynamic Temporal Alignment–
A novel algorithm is proposed with the key traits of heterogeneous multi-cue fusion and dynamic temporal alignment for Video2Subtitle, which demonstrates excellent performances in comparison with several state-of-the-art cross-modal matching methods.
- –
- Guest Editorial: Special issue on large scale multimedia semantic indexing–
Though recent years have witnessed plenty of research effort on driving for higher accuracy and improved efficiency of multimedia semantic indexing algorithms, there are still many open problems in this research field.
- –
- –
- –
Publication data from OpenAlex, with missing venues and authors filled in from Crossref; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-15. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.
Report an error
Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.