Is this you? Claim this profile to correct it, add a bio and choose the work people see first.

Claim this profile

Academic lineage

Possible advisorsa guess from early papers, not confirmed

  • Chi–Keung Tang

    Possible advisor · last author on 4 of their early first-author papers, 2020–2023

    Suggested from co-authorship
  • Fisher Yu

    Possible advisor · last author on 6 of their early first-author papers, 2021–2023

    Suggested from co-authorship

Is this you? Claim this profile to confirm or dismiss it.

Works46 from public data

TitleCited by
  • Memory-Attended Recurrent Network for Video Captioning

    Wenjie Pei, Jiyuan Zhang, Xiangrong Wang, Lei Ke, Xiaoyong Shen, Yu‐Wing Tai

    IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · 2019

    The Memory-Attended Recurrent Network (MARN) for video captioning is proposed, in which a memory structure is designed to explore the full-spectrum correspondence between a word and its various similar visual contexts across videos in training data.

    240
  • Deep Occlusion-Aware Instance Segmentation with Overlapping BiLayers

    Lei Ke, Yu‐Wing Tai, Chi–Keung Tang

    IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · 2021

    The explicit modeling of occlusion relationship with bilayer structure naturally decouples the boundaries of both the occluding and occluded instances, and considers the interaction between them during mask regression, to validate the efficacy of bilayer decoupling on both one-stage and two-stage object detectors with different backbones and network layer choices.

    239
  • Gaussian Grouping: Segment and Edit Anything in 3D Scenes

    Mingqiao Ye, Martin Danelljan, Fisher Yu, Lei Ke

    Lecture notes in computer science · 2024

    200
  • Cascaded Deep Monocular 3D Human Pose Estimation With Evolutionary Training Data

    Shichao Li, Lei Ke, Kevin Pratama, Yu‐Wing Tai, Chi–Keung Tang, Kwang-Ting Tim Cheng

    IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · 2020

    A novel data augmentation method that is scalable for synthesizing massive amount of training data for training 2D-to-3D networks, can effectively reduce dataset bias and generalizes significantly better to unseen and rare poses.

    185
  • Mask Transfiner for High-Quality Instance Segmentation

    Lei Ke, Martin Danelljan, Xia Li, Yu‐Wing Tai, Chi–Keung Tang, Fisher Yu

    IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · 2022

    This paper presents Mask Transfiner, a transformer-based approach to instance segmentation that decomposes and represents the image regions as a quadtree, allowing it to predict highly accurate instance masks, at a low computational cost.

    173
  • Reflective Decoding Network for Image Captioning

    Lei Ke, Wenjie Pei, Ruiyu Li, Xiaoyong Shen, Yu‐Wing Tai

    IEEE/CVF International Conference on Computer Vision (ICCV) · 2019

    It is shown that vocabulary coherence between words and syntactic paradigm of sentences are also important to generate high-quality image captioning, and the proposed Reflective Decoding Network (RDN) enhances both the long-sequence dependency and position perception of words in a caption decoder.

    109
  • Segment Anything in High Quality

    Lei Ke, Mingqiao Ye, Martin Danelljan, Yifan Liu, Yu‐Wing Tai, Chi–Keung Tang, Fisher Yu

    neural information processing systems · 2023

    102
  • LLM-Seg: Bridging Image Segmentation and Large Language Model Reasoning

    Junchi Wang, Lei Ke

    IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · 2024

    This work delves into reasoning segmentation, a novel task that enables segmentation system to reason and interpret implicit user intention via large language model reasoning and then segment the corresponding target.

    87
  • OVTrack: Open-Vocabulary Multiple Object Tracking

    Siyuan Li, Tobias Fischer, Lei Ke, Henghui Ding, Martin Danelljan, Fisher Yu

    IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · 2023

    OVTrack is developed, an open-vocabulary tracker that is capable of tracking arbitrary object classes and its design is based on two key ingredients, leveraging vision-language models for both classification and association via knowledge distillation and a data hallucination strategy for robust appearance feature learning from denoising diffusion probabilistic models.

    86
  • Matching Anything by Segmenting Anything

    Siyuan Li, Lei Ke, Martin Danelljan, Luigi Piccinelli, Mattia Segù, Luc Van Gool, Fisher Yu

    IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · 2024

    This work proposes MASA, a novel method for robust instance association learning, capable of matching any objects within videos across diverse domains without tracking labels, Leveraging the rich object segmentation from the Segment Anything Model (SAM).

    83
  • Cascade-DETR: Delving into High-Quality Universal Object Detection

    Mingqiao Ye, Lei Ke, Siyuan Li, Yu‐Wing Tai, Chi–Keung Tang, Martin Danelljan, Fisher Yu

    IEEE/CVF International Conference on Computer Vision (ICCV) · 2023

    Cascade-DETR is introduced for high-quality universal object detection, and the Cascade Attention layer is proposed, which explicitly integrates object-centric information into the detection decoder by limiting the attention to the previous box prediction.

    69
  • GSNet: Joint Vehicle Pose and Shape Reconstruction with Geometrical and Scene-Aware Supervision

    Lei Ke, Shichao Li, Yanan Sun, Yu‐Wing Tai, Chi–Keung Tang

    Lecture notes in computer science · 2020

    A novel end-to-end framework named GSNet (Geometric and Scene-aware Network), which jointly estimates 6DoF poses and reconstructs detailed 3D car shapes from single urban street view, and inspires a new multi-objective loss function to regularize network training.

    56
  • Occlusion-Aware Video Object Inpainting

    Lei Ke, Yu‐Wing Tai, Chi–Keung Tang

    IEEE/CVF International Conference on Computer Vision (ICCV) · 2021

    Experimental results clearly demonstrate the efficacy of the occlusion-aware video object inpainting, which recovers both the complete shape and appearance for occluded objects in videos given their visible mask segmentation.

    47
  • Occlusion-Aware Instance Segmentation Via BiLayer Network Architectures

    Lei Ke, Yu‐Wing Tai, Chi–Keung Tang

    IEEE Transactions on Pattern Analysis and Machine Intelligence · 2023

    Large and consistent improvements using one/two-stage and query-based object detectors with various backbones and network layer choices validate the generalization ability of bilayer decoupling, as shown by extensive experiments on image instance segmentation benchmarks.

    37
  • The First Visual Object Tracking Segmentation VOTS2023 Challenge Results

    Matej Kristan, Jiřı́ Matas, Martin Danelljan, Michael Felsberg, Hyung Jin Chang, Luka Čehovin Zajc, Alan Lukežič, Ondřej Drbohlav, +22 more

    IEEE/CVF International Conference on Computer Vision Workshops (ICCVW) · 2023

    35
  • Prototypical Cross-Attention Networks for Multiple Object Tracking and Segmentation

    Lei Ke, Xia Li, Martin Danelljan, Yu‐Wing Tai, Chi–Keung Tang, Fisher Yu

    Repository for Publications and Research Data (ETH Zurich) · 2021

    35
  • Mask-Free Video Instance Segmentation

    Lei Ke, Martin Danelljan, Henghui Ding, Yu‐Wing Tai, Chi–Keung Tang, Fisher Yu

    IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · 2023

    This work proposes MaskFreeVIS, achieving highly competitive VIS performance, while only using bounding box annotations for the object state, by introducing the Temporal KNN-patch Loss (TK-Loss), providing strong mask supervision without any labels.

    32
  • Commonality-Parsing Network Across Shape and Appearance for Partially Supervised Instance Segmentation

    Qi Fan, Lei Ke, Wenjie Pei, Chi–Keung Tang, Yu‐Wing Tai

    Lecture notes in computer science · 2020

    This work proposes to learn the underlying class-agnostic commonalities that can be generalized from mask-annotated categories to novel categories, and significantly outperforms the state-of-the-art methods on both partially supervised setting and few-shot setting for instance segmentation on COCO dataset.

    32
  • BiMatting: Efficient Video Matting via Binarization

    Haotong Qin, Lei Ke, Xudong Ma, Martin Danelljan, Yu‐Wing Tai, Chi–Keung Tang, Xianglong Liu, Fisher Yu

    neural information processing systems · 2023

    BiMatting is proposed, an accurate and efficient video matting model using binarization that outperforms other binarized video matting models, including state-of-the-art (SOTA) binarization methods, by a significant margin and performs comparably to the full-precision counterpart in visual quality.

    28
  • Video Mask Transfiner for High-Quality Video Instance Segmentation

    Lei Ke, Henghui Ding, Martin Danelljan, Yu‐Wing Tai, Chi–Keung Tang, Fisher Yu

    Lecture notes in computer science · 2022

    21
  • BiVM: Accurate Binarized Neural Network for Efficient Video Matting

    Haotong Qin, Xianglong Liu, Xudong Ma, Lei Ke, Yulun Zhang, Jie Luo, Michele Magno

    IEEE Transactions on Pattern Analysis and Machine Intelligence · 2025

    The proposed BiVM surpasses alternative binarized video matting networks, including state-of-the-art (SOTA) binarization methods, by a substantial margin and achieves significant savings in computation and storage costs.

    19
  • Stable Segment Anything Model

    Qi Fan, Xin Tao, Lei Ke, Mingqiao Ye, Yuan Zhang, Pengfei Wan, Zhongyuan Wang, Yu‐Wing Tai, +1 more

    arXiv · 2023

    The first comprehensive analysis on SAM's segmentation stability across a diverse spectrum of prompt qualities is presented, revealing that given low-quality prompts, SAM's mask decoder tends to activate image features that are biased towards the background or confined to specific object parts.

    18
  • DreamScene4D: Dynamic Multi-Object Scene Generation from Monocular Videos

    Wen-Hsuan Chu, Lei Ke, Katerina Fragkiadaki

    neural information processing systems · 2024

    16
  • SLAck: Semantic, Location, and Appearance Aware Open-Vocabulary Tracking

    Siyuan Li, Lei Ke, Yung-Hsu Yang, Luigi Piccinelli, Mattia Segù, Martin Danelljan, Luc Van Gool

    Lecture notes in computer science · 2024

    7
  • TAPIP3D: Tracking Any Point in Persistent 3D Geometry

    Bowei Zhang, Lei Ke, Adam W. Harley, Katerina Fragkiadaki

    neural information processing systems · 2025

    6
  • ProReflow: Progressive Reflow with Decomposed Velocity

    Lei Ke, Haohang Xu, Xuefei Ning, Yu Li, Jiajun Li, Haoling Li, Yuxuan Lin, Dongsheng Jiang, +2 more

    IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · 2025

    This paper suggests that the original training pipeline of reflow is not optimal and introduces progressive reflow, which progressively reflows the diffusion models in local timesteps until the whole diffusion progresses, reducing the difficulty of flow matching.

    6
  • Robust Multi-Object 4D Generation for In-the-wild Videos

    Wen-Hsuan Chu, Lei Ke, Jianmeng Liu, Mingxiao Huo, Pavel Tokmakov, Katerina Fragkiadaki

    IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · 2025

    5
  • EntitySAM: Segment Everything in Video

    Mingqiao Ye, Seoung Wug Oh, Lei Ke, Joon‐Young Lee

    IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · 2025

    3
  • AnchorWorld: Embodied Egocentric World Simulation with View-based Evolution Customization

    Yu Li, Menghan Xia, Gongye Liu, Xintao Wang, C.J. Zhang, Lei Ke, Yuxuan Lin, Ruihang Chu, +3 more

    arXiv · 2026

    AnchorWorld is presented, a framework that advances egocentric simulation through enhanced interaction integrity and a flexible mechanism for world customization and proposes a simple yet effective mechanism for customizing self-evolving worlds.

    2
  • Generative 4D Scene Gaussian Splatting with Object View-Synthesis Priors

    Wen-Hsuan Chu, Lei Ke, Liu, Jianmeng, Mingxiao Huo, Pavel Tokmakov, Katerina Fragkiadaki

    arXiv · 2025

    GenMOJO, a novel approach that integrates rendering-based deformable 3D Gaussian optimization with generative priors for view synthesis, generates more realistic novel views of scenes and produces more accurate point tracks compared to existing approaches.

    1
  • Stream3D-VLM: Online 3D Spatial Understanding with Incremental Geometry Priors

    Hanxun Yu, Xuan Qu, Lei Ke, Boqiang Zhang, Yuxin Wang, Jianke Zhu, Dong Yu

    Lecture notes in computer science · 2026

    –
  • RePlan: Reasoning-Guided Region Planning for Complex Instruction-Based Image Editing

    Tianyuan Qu, Lei Ke, Xiaohang Zhan, Longxiang Tang, Yuqi Liu, Bohao Peng, Bei Yu, Dong Yu, +1 more

    Lecture notes in computer science · 2026

    –
  • Beyond VLM-Based Rewards: Diffusion-Native Latent Reward Modeling

    Gongye Liu, Bo Yang, Yida Zhi, Zhizhou Zhong, Lei Ke, Didan Deng, Han Gao, Yongxiang Huang, +3 more

    arXiv · 2026

    –
  • AudioX-Turbo: A Unified Framework for Efficient Anything-to-Audio Generation

    Zeyue Tian, Lei Ke, Zhaoyang Liu, Ruibin Yuan, Liumeng Xue, Yujiu Yang, Weijia Chen, Xu Tan, +3 more

    arXiv · 2026

    –
  • DVPSFormer: Efficient Online Depth-aware Video Panoptic Segmentation for Autonomous Driving

    Yung-Hsu Yang, Luigi Piccinelli, Siyuan Li, Mattia Segù, Lei Ke, Martin Danelljan, Yuqian Fu, Zuria Bauer, +3 more

    arXiv · 2026

    DVPSFormer is proposed, a unified online architecture designed for efficient 4D scene understanding that establishes a new state-of-the-art on the Cityscapes-DVPS and SemKITTI-DVPS benchmarks, offering a streamlined solution for online robotic perception.

    –
  • Recursive Synthesis for Long-Horizon Terminal Tasks

    Zhongzhi Li, Yucheng Shi, Zongxia Li, Ruhan Wang, Anhao Li, Zixun Huang, Junyao Yang, Lei Ke, +3 more

    arXiv · 2026

    –
  • Penguin-VL: Exploring the Efficiency Limits of VLM with LLM-based Vision Encoders

    Boqiang Zhang, Lei Ke, Ruihan Yang, Qi Gao, Tianyuan Qu, Rossell Chen, Dong Yu, Leoweiliang

    arXiv · 2026

    –
  • UI-Mate: Advancing Open-Weight Foundation GUI Agents with In-Context Demonstrations

    Zihan Ding, Longxu Dou, Qi Gao, Xiangwu Guo, Shengchao Hu, Zilong Huang, Zihang Jiang, Lei Ke, +21 more

    arXiv · 2026

    –
  • Unlocking Dense Metric Depth Estimation in VLMs

    Hanxun Yu, Xuan Qu, Wang, Yuxin, Jianke Zhu, Lei Ke

    arXiv · 2026

    –
  • HelixWorld: A Real-time Interactive Audio-Visual World Model

    Lei Ke, Jiahao Pan, Zeyue Tian, Jiaming Wang, Haoyuan Huang, Kam Man Wu, Pengjun Fang, Hongyu Liu, +8 more

    arXiv · 2026

    –
  • N3D-VLM: Native 3D Grounding Enables Accurate Spatial Reasoning in Vision-Language Models

    Wang, Yuxin, Lei Ke, Zhang, Boqiang, Qu, Tianyuan, Yu, Hanxun, Huang, Zhenpeng, Yu, Meng, Dan Xu, +1 more

    arXiv · 2025

    –
  • MotionEdit: Benchmarking and Learning Motion-Centric Image Editing

    Yixin Wan, Lei Ke, Wenhao Yu, Kai-Wei Chang, Yu, Dong

    arXiv · 2025

    –
  • Zero-P-to-3: Zero-Shot Partial-View Images to 3D Object

    Lin, Yuxuan, Ruihang Chu, Zhenyu Chen, Tang, Xiao, Lei Ke, Haoling Li, Yingji Zhong, Zhihao Li, +4 more

    arXiv · 2025

    This work proposes a novel training-free approach that integrates the local dense observations and multi-source priors for reconstruction, and introduces a fusion-based strategy to effectively align these priors in DDIM sampling, thereby generating multi-view consistent images to supervise invisible views.

    –
  • Stable and Efficient Single-Rollout RL for Multimodal Reasoning

    Rui Liu, Dian Yu, Lei Ke, Haolin Liu, Yujun Zhou, Zhenwen Liang, Haitao Mi, Pratap Tokekar, +1 more

    arXiv · 2025

    –
  • –
  • –

Publication data from OpenAlex; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-16. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.

Report an error

Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.

Sign in to report