Is this you? Claim this profile to correct it, add a bio and choose the work people see first.

Claim this profile

Academic lineage

Possible advisorsa guess from early papers, not confirmed

  • Jiwen Lu

    Possible advisor · last author on 3 of their early first-author papers, 2022

    Suggested from co-authorship
  • Jie Zhou

    Possible advisor · last author on 10 of their early first-author papers, 2017–2021

    Suggested from co-authorship

Is this you? Claim this profile to confirm or dismiss it.

Works83 from public data

TitleCited by
  • Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modeling

    Xumin Yu, Lulu Tang, Yongming Rao, Tiejun Huang, Jie Zhou, Jiwen Lu

    IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · 2022

    Point-BERT, a new paradigm for learning Transformers to generalize the concept of BERT to 3D point cloud, is presented and it is shown that a pure Transformer architecture attains 93.8% accuracy on ModelNet40 and 83.1% accuracy in the hardest setting of ScanObjectNN, surpassing carefully designed point cloud models with much fewer hand-made designs.

    1,232
  • DenseCLIP: Language-Guided Dense Prediction with Context-Aware Prompting

    Yongming Rao, Wenliang Zhao, Guangyi Chen, Yansong Tang, Zheng Zhu, Guan Huang, Jie Zhou, Jiwen Lu

    IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · 2022

    This work presents a new framework for dense prediction by implicitly and explicitly leveraging the pre-trained knowledge from CLIP and converts the original image-text matching problem in CLIP to a pixel- text matching problem and uses the pixel-text score maps to guide the learning of dense prediction models.

    886
  • PoinTr: Diverse Point Cloud Completion with Geometry-Aware Transformers

    Xumin Yu, Yongming Rao, Ziyi Wang, Zuyan Liu, Jiwen Lu, Jie Zhou

    IEEE/CVF International Conference on Computer Vision (ICCV) · 2021

    A new method is presented that reformulates point cloud completion as a set-to-set translation problem and design a new model, called PoinTr, that adopts a transformer encoder-decoder architecture for point clouds completion that outperforms state-of-the-art methods by a large margin.

    679
  • Generative Multimodal Models are In-Context Learners

    Quan Sun, Yufeng Cui, Xiaosong Zhang, Fan Zhang, Qiying Yu, Yueze Wang, Yongming Rao, Jingjing Liu, +2 more

    IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · 2024

    This work demonstrates that the task-agnostic in-context learning capabilities of large multimodal models can be significantly enhanced by effective scaling-up, and introduces Emu2, a generative multimodal model with 37 billion parameters, trained on large-scale multimodal sequences with a unified autoregressive objective.

    545
  • Runtime Neural Pruning

    Ji Lin, Yongming Rao, Jiwen Lu, Jie Zhou

    neural information processing systems · 2017

    A Runtime Neural Pruning (RNP) framework which prunes the deep neural network dynamically at the runtime and preserves the full ability of the original network and conducts pruning according to the input image and current feature maps adaptively.

    536
  • COIN: A Large-Scale Dataset for Comprehensive Instructional Video Analysis

    Yansong Tang, Dajun Ding, Yongming Rao, Yu Zheng, Danyang Zhang, Lili Zhao, Jiwen Lu, Jie Zhou

    IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · 2019

    A simple yet effective method to capture the dependencies among different steps, which can be easily plugged into conventional proposal-based action detection methods for localizing important steps in instruction videos is proposed.

    477
  • Stochastic Trajectory Prediction via Motion Indeterminacy Diffusion

    Tianpei Gu, Guangyi Chen, Junlong Li, Chunze Lin, Yongming Rao, Jie Zhou, Jiwen Lu

    IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · 2022

    This paper presents a new framework to formulate the trajectory prediction task as a reverse process of motion indeterminacy diffusion (MID), in which the history behavior information and the social interactions are encoded as a state embedding and a Transformer-based diffusion model is devised to capture the temporal dependencies of trajectories.

    438
  • Structure-Preserving Super Resolution With Gradient Guidance

    Cheng Ma, Yongming Rao, Yean Cheng, Ce Chen, Jiwen Lu, Jie Zhou

    IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · 2020

    A structure-preserving super resolution method which exploits gradient maps of images to guide the recovery in two aspects and proposes a gradient loss which imposes a second-order restriction on the super-resolved images.

    415
  • Counterfactual Attention Learning for Fine-Grained Visual Categorization and Re-identification

    Yongming Rao, Guangyi Chen, Jiwen Lu, Jie Zhou

    IEEE/CVF International Conference on Computer Vision (ICCV) · 2021

    This paper analyzes the effect of the learned visual attention on network prediction through counterfactual intervention and maximize the effect to encourage the network to learn more useful attention for fine-grained image recognition.

    380
  • Unleashing Text-to-Image Diffusion Models for Visual Perception

    Wenliang Zhao, Yongming Rao, Zuyan Liu, Benlin Liu, Jie Zhou, Jiwen Lu

    IEEE/CVF International Conference on Computer Vision (ICCV) · 2023

    It is shown that vision-language pre-trained diffusion models can be faster adapted to downstream visual perception tasks using the proposed VPD, a new framework that exploits the semantic information of a pre-trained text-to-image diffusion model in visual perception tasks.

    366
  • DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsification

    Yongming Rao, Wenliang Zhao, Benlin Liu, Jiwen Lu, Jie Zhou, Cho‐Jui Hsieh

    arXiv · 2021

    306
  • NerfingMVS: Guided Optimization of Neural Radiance Fields for Indoor Multi-view Stereo

    Yi Wei, Shaohui Liu, Yongming Rao, Wang Zhao, Jiwen Lu, Jie Zhou

    IEEE/CVF International Conference on Computer Vision (ICCV) · 2021

    A new multi-view depth estimation method that utilizes both conventional SfM reconstruction and learning-based priors over the recently proposed neural radiance fields (NeRF), with surprising findings presented on the effectiveness of correspondence-based opti-mization and NeRF-based optimization over the adapted depth priors.

    293
  • HorNet: Efficient High-Order Spatial Interactions with Recursive Gated Convolutions

    Yongming Rao, Wenliang Zhao, Yansong Tang, Jie Zhou, Ser Nam Lim, Jiwen Lu

    neural information processing systems · 2022

    217
  • FineDiving: A Fine-grained Dataset for Procedure-aware Action Quality Assessment

    Jinglin Xu, Yongming Rao, Xumin Yu, Guangyi Chen, Jie Zhou, Jiwen Lu

    IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · 2022

    This work proposes a procedure-aware approach for action quality assessment, learned by a new Temporal Segmentation Attention module, and proposes to parse pairwise query and exemplar action instances into consecutive steps with diverse semantic and temporal correspondences.

    203
  • Deep Face Super-Resolution With Iterative Collaboration Between Attentive Recovery and Landmark Estimation

    Cheng Ma, Zhenyu Jiang, Yongming Rao, Jiwen Lu, Jie Zhou

    IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · 2020

    Results show the proposed FSR method with iterative collaboration between two recurrent networks which focus on facial image recovery and landmark estimation significantly outperforms state-of-the-art FSR methods in recovering high-quality face images.

    201
  • AdaPoinTr: Diverse Point Cloud Completion With Adaptive Geometry-Aware Transformers

    Xumin Yu, Yongming Rao, Ziyi Wang, Jiwen Lu, Jie Zhou

    IEEE Transactions on Pattern Analysis and Machine Intelligence · 2023

    A Transformer encoder-decoder architecture, called PoinTr, is proposed, which reformulates point cloud completion as a set-to-set translation problem and employs a geometry-aware block to model local geometric relationships explicitly and can achieve promising performance with higher throughputs and fewer FLOPs compared with the previous best methods in practice.

    190
  • Attention-Aware Deep Reinforcement Learning for Video Face Recognition

    Yongming Rao, Jiwen Lu, Jie Zhou

    IEEE International Conference on Computer Vision (ICCV) · 2017

    182
  • Group-aware Contrastive Regression for Action Quality Assessment

    Xumin Yu, Yongming Rao, Wenliang Zhao, Jiwen Lu, Jie Zhou

    IEEE/CVF International Conference on Computer Vision (ICCV) · 2021

    It is shown that the relations among videos can provide important clues for more accurate action quality assessment during both training and inference, and a new Contrastive Regression framework is proposed to learn the relative scores by pair-wise comparison, which highlights the differences between videos and guides the models to learning the key hints for assessment.

    176
  • Spherical Fractal Convolutional Neural Networks for Point Cloud Recognition

    Yongming Rao, Jiwen Lu, Jie Zhou

    IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · 2019

    173
  • DiffSwap: High-Fidelity and Controllable Face Swapping via 3D-Aware Masked Diffusion

    Wenliang Zhao, Yongming Rao, Weikang Shi, Zuyan Liu, Jie Zhou, Jiwen Lu

    IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · 2023

    157
  • Insight-V: Exploring Long-Chain Visual Reasoning with Multimodal Large Language Models

    Yuhao Dong, Zuyan Liu, Hailong Sun, Jingkang Yang, Winston Hu, Yongming Rao, Ziwei Liu

    IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · 2025

    InsightV, an early effort to scalably produce long and robust reasoning data for complex multi-modal tasks, and an effective training pipeline to enhance the reasoning capabilities of multi-modal large language models (MLLMs), is presented.

    153
  • Global-Local Bidirectional Reasoning for Unsupervised Representation Learning of 3D Point Clouds

    Yongming Rao, Jiwen Lu, Jie Zhou

    IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · 2020

    This work hypothesizes that a powerful representation of a 3D object should model the attributes that are shared between parts and the whole object, and distinguishable from other objects, and proposes to learn point cloud representation by bidirectional reasoning between the local structures at different abstraction hierarchies and the global shape without human supervision.

    152
  • PV-RAFT: Point-Voxel Correlation Fields for Scene Flow Estimation of Point Clouds

    Yi Wei, Ziyi Wang, Yongming Rao, Jiwen Lu, Jie Zhou

    IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · 2021

    Experimental results show that PV-RAFT outperforms state-of-the-art methods by remarkable margins and is evaluated on the FlyingThings3D and KITTI Scene Flow 2015 datasets.

    127
  • Structure-Preserving Image Super-Resolution

    Cheng Ma, Yongming Rao, Jiwen Lu, Jie Zhou

    IEEE Transactions on Pattern Analysis and Machine Intelligence · 2021

    A structure-preserving super-resolution (SPSR) method to alleviate the above issue while maintaining the merits of GAN-based methods to generate perceptual-pleasant details, and proposes two self-supervised structure learning methods, contrastive prediction and solving jigsaw puzzles, to train the NSEs.

    123
  • SurroundDepth: Entangling Surrounding Views for Self-Supervised Multi-Camera Depth Estimation

    Yi Wei, Linqing Zhao, Wenzhao Zheng, Zheng Dong Zhu, Yongming Rao, Guan Huang, Jiwen Lu, Jie Zhou

    arXiv · 2022

    This paper proposes a SurroundDepth method to incorporate the information from multiple surrounding views to predict depth maps across cameras, and adopts the two-frame structure-from-motion to extract scale-aware pseudo depths to pretrain the models.

    112

Show all 83 works

Publication data from OpenAlex; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-16. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.

Report an error

Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.

Sign in to report