Is this you? Claim this profile to correct it, add a bio and choose the work people see first.
Claim this profileAcademic lineage
Possible advisorsa guess from early papers, not confirmed
- Suggested from co-authorship
- Ping LuoSuggested from co-authorship
Is this you? Claim this profile to confirm or dismiss it.
Works47 from public data
- SEED-Bench: Benchmarking Multimodal LLMs with Generative Comprehension1,018
This work addresses the evaluation of generative comprehension in MLLMs as a preliminary step towards a comprehensive assessment ofGenerative models, by introducing a benchmark named SEED-Bench, which spans 12 evaluation dimensions including the comprehension of both the image and video modality.
- DeepFashion2: A Versatile Benchmark for Detection, Pose Estimation, Segmentation and Re-Identification of Clothing Images446
A strong baseline is proposed, called Match R- CNN, which builds upon Mask R-CNN to solve the above four tasks in an end-to-end manner to address issues of DeepFashion2.
- SEED-X: Multimodal Models with Unified Multi-granularity Comprehension and Generation365
This work presents a unified and versatile foundation model, namely, SEED-X, which is able to model multi-granularity visual semantics for comprehension and generation tasks and demonstrates its effectiveness in handling real-world applications across various domains after instruction tuning.
- 327
- Parser-Free Virtual Try-on via Distilling Appearance Flows267
This work proposes a novel approach, "teacher-tutor-student" knowledge distillation, which is able to produce highly photo-realistic images without human parsing, possessing several appealing advantages compared to prior arts.
- All in One: Exploring Unified Video-Language Pre-Training258
This work introduces an end-to-end VLP model, namely all-in-one Transformer, that embeds raw video and textual signals into joint representations using a unified backbone architecture and introduces a novel and effective token rolling operation to encode temporal representations from video clips in a non-parametric manner.
- Making LLaMA SEE and Draw with SEED Tokenizer227
SEED is introduced, an elaborate image tokenizer that empowers LLMs with the ability to SEE and Draw at the same time, and with SEED tokens, LLM is able to perform scalable multimodal autoregression under its original training recipe, i.e., next-word prediction.
- SEED-Bench-2-Plus: Benchmarking Multimodal Large Language Models with Text-Rich Visual Comprehension156
SEED-Bench-2-Plus is introduced, a benchmark specifically designed for evaluating text-rich visual comprehension of Multimodal Large Language Models and a thorough evaluation involving 34 prominent MLLMs is conducted, to emphasize the current limitations of MLLMs in text-rich visual comprehension.
- Planting a SEED of Vision in Large Language Model150
This study identifies two crucial principles for the architecture and training of SEED that effectively ease subsequent alignment with LLMs, and emphasizes the great potential of discrete visual tokens in versatile multimodal LLMs and the importance of proper image tokenizers in broader research.
- Bridging Video-text Retrieval with Multiple Choice Questions131
This work enables fine-grained video-text interactions while maintaining high efficiency for retrieval via a novel pretext task, dubbed as Multiple Choice Questions (MCQ), where a parametric module BridgeFormer is trained to answer the “questions” constructed by the text features via resorting to the video features.
- Disentangled Cycle Consistency for Highly-realistic Virtual Try-On130
The proposed Disentangled Cycle-consistency Try-On Network (DCTON) is able to produce highly-realistic try-on images by disentangling important components of virtual try- on including clothes warping, skin synthesis, and image composition.
- SEED-Data-Edit Technical Report: A Hybrid Dataset for Instructional Image Editing106
A pretrained Multimodal Large Language Model (MLLM) that unifies comprehension and generation with SEED-Data-Edit is fine-tuneed, indicating the potential and effectiveness of SEED-Data-Edit in advancing the field of instructional image editing.
- SEED-Bench-2: Benchmarking Multimodal Large Language Models103
This work categorizes the capabilities of MLLMs into hierarchical levels from $L_0 to $L-4 based on the modalities they can accept and generate, and proposes SEED-Bench-2, a comprehensive benchmark that evaluates the hierarchical capabilities ofMLLMs.
- SCAN: Self-and-Collaborative Attention Network for Video Person Re-Identification92
A novel and practical deep architecture for video person re-identification termed self-and-collaborative attention network (SCAN), which adopts the video pairs as the input and outputs their matching scores and can be predicted by the binary classifier.
- Moto: Latent Motion Token as the Bridging Language for Learning Robot Manipulation from Videos84
This work introduces Moto, which converts video content into latent Motion Token sequences by a Latent Motion Tokenizer, learning a bridging “language” of motion from videos in an unsupervised manner, and implements a co-fine-tuning strategy that seamlessly bridges latent motion token prediction and real robot control.
- EgoPlan-Bench: Benchmarking Multimodal Large Language Models for Human-Level Planning79
EgoPlan-Bench is introduced, a comprehensive benchmark to evaluate the planning abilities of MLLMs in real-world scenarios from an egocentric perspective, mirroring human perception, and EgoPlan-IT, a specialized instruction-tuning dataset that effectively enhances model performance on EgoPlan-Bench.
- TokLIP: Marry Visual Tokens to CLIP for Multimodal Comprehension and Generation64
TokLIP is introduced, a visual tokenizer that enhances comprehension by semanticizing vector-quantized (VQ) tokens and incorporating CLIP-level semantics while enabling end-to-end multimodal autoregressive training with standard VQ tokens.
- 50
- TimeLens: Rethinking Video Temporal Grounding with Multimodal LLMs49
TimeLens is presented, a systematic investigation into building MLLMs with strong VTG ability, along two primary dimensions: data quality and algorithmic design, which culminate in a family of MLLMs with state-of-the-art VTG performance among open-source models and even surpass proprietary models.
- EgoPlan-Bench2: A Benchmark for Multimodal Large Language Model Planning in Real-World Scenarios46
It is found that MLLMs’ planning performance on EgoPlan-Bench2 is critically dependent on temporally structured action sequences in historical task progress and interactions between objects and humans in current observation state, which underscores the necessity for strong reasoning abilities to integrate diverse multimodal cues and analysis before making final decision.
- VIT-LENS: Towards Omni-modal Representations46
Vit-lens provides a unified solution for representation learning of increasing modalities with two appealing advantages: unlocking the great potential of pretrained- ViTs to novel modalities effectively with efficient parameters and data regime and enabling emergent down- stream capabilities through modality alignment and shared ViT parameters.
- 33
- Exploring the Effect of Reinforcement Learning on Video Understanding: Insights from SEED-Bench-R127
SEED-Bench-R1 is introduced, a benchmark designed to systematically evaluate post-training methods for MLLMs in video understanding and identifies key limitations such as inconsistent reasoning and overlooked visual cues, and suggest future improvements in base model reasoning, reward modeling, and RL robustness against noisy signals.
- 22
- 21
Publication data from OpenAlex; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-13. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.
Report an error
Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.