Is this you? Claim this profile to correct it, add a bio and choose the work people see first.
Claim this profileAcademic lineage
Possible advisorsa guess from early papers, not confirmed
- Ping LuoSuggested from co-authorship
Is this you? Claim this profile to confirm or dismiss it.
Works100 from public data
- OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Models539
An Omnidirectionally calibrated Quantization technique for LLMs is introduced, which achieves good performance in diverse quantization settings while maintaining the computational efficiency of PTQ by efficiently optimizing various quantization parameters.
- SPHINX: The Joint Mixing of Weights, Tasks, and Visual Embeddings for Multi-modal Large Language Models313
SPHINX, a versatile multi-modal large language model (MLLM) with a joint mixing of model weights, tuning tasks, and visual embeddings is presented, and an efficient strategy aiming to better capture fine-grained appearances of high-resolution images is proposed.
- LVLM-EHub: A Comprehensive Evaluation Benchmark for Large Vision-Language Models296
A comprehensive evaluation of publicly available large multimodal models by building an LVLM evaluation Hub (LVLM-eHub), which consists of 13 representative LVLMs such as InstructBLIP and LLaVA, which are thoroughly evaluated by a quantitative capability evaluation and an online arena platform.
- OmniMedVQA: A New Large-Scale Comprehensive Evaluation Benchmark for Medical LVLM283
OmniMedVQA is introduced, a novel comprehensive medical Visual Question Answering (VQA) benchmark collected from 73 different medical datasets, including 12 different modalities and covering more than 20 distinct anatomical regions, to reveal the current limitations of LVLM in understanding real medical images and highlight the dataset's significance.
- MMT-Bench: A Comprehensive Multimodal Benchmark for Evaluating Large Vision-Language Models Towards Multitask AGI217
This study presents MMT-Bench, a comprehensive benchmark designed to assess LVLMs across massive multimodal tasks requiring expert knowledge and deliberate visual recognition, localization, reasoning, and planning, and hopes that MMT-Bench will inspire the community to develop next-generation multimodal foundation models aimed at achieving general-purpose multimodal intelligence.
- 202
- Towards World Simulator: Crafting Physical Commonsense-Based Benchmark for Video Generation190
The evaluation results and in-depth analysis demonstrate that current models struggle to generate videos that comply with physical commonsense, and it is hoped this study will inspire the community to prioritize the learning of physical commonsense in these models beyond entertainment applications.
- 186
- SPHINX-X: Scaling Data and Parameters for a Family of Multi-modal Large Language Models162
The SPHINX framework is modified by removing redundant visual encoders, bypassing fully-padded sub-images with skip tokens, and simplifying multi-stage training into a one-stage all-in-one paradigm to improve the architecture and training efficiency.
- Lumina-T2X: Transforming Text into Any Modality, Resolution, and Duration via Flow-based Large Diffusion Transformers147
This technical report introduces the Lumina-T2X family - a series of Flow-based Large Diffusion Transformers (Flag-DiT) equipped with zero-initialized attention, as a unified framework designed to transform noise into images, videos, multi-view 3D objects, and audio clips conditioned on text instructions.
- DiffRate : Differentiable Compression Rate for Efficient Vision Transformers110
Differentiable Compression Rate (DiffRate), a novel token compression method that has several appealing properties prior arts do not have, is proposed and achieves state-of-the-art performance.
- Beyond One-to-One: Rethinking the Referring Image Segmentation108
A Dual Multi-Modal Interaction (DMMI) Network is proposed, which contains two decoder branches and enables information flow in two directions and achieves state-of-the-art performance on different datasets, and the Ref-ZOM-trained model performs well on various types of text inputs.
- ChartAssisstant: A Universal Chart Multimodal Language Model via Chart-to-Table Pre-training and Multitask Instruction Tuning99
ChartAssistant, a chart-based vision-language model for universal chart comprehension and reasoning, is proposed, which leverages ChartSFT, a comprehensive dataset covering diverse chart-related tasks with basic and specialized chart types.
- MMIU: Multimodal Multi-image Understanding for Evaluating Large Vision-Language Models71
The Multimodal Multi-image Understanding (MMIU) benchmark is introduced, a comprehensive evaluation suite designed to assess LVLMs across a wide range of multi-image tasks, making it the most extensive benchmark of its kind.
- RoboCodeX: Multimodal Code Generation for Robotic Behavior Synthesis66
A tree-structured multimodal code generation framework for generalized robotic behavior synthesis, termed RoboCodeX, which decomposes high-level human instructions into multiple object-centric manipulation units consisting of physical preferences such as affordance and safety constraints, and applies code generation to introduce generalization ability across various robotics platforms.
- Tree-Planner: Efficient Close-loop Task Planning with Large Language Models66
Tree-Planner is proposed, which reframes task planning with Large Language Models into three distinct phases: plan sampling, action tree construction, and grounded deciding, which achieves state-of-the-art performance while maintaining high efficiency.
- BESA: Pruning Large Language Models with Blockwise Parameter-Efficient Sparsity Allocation62
A novel LLM pruning technique dubbed blockwise parameter-efficient sparsity allocation (BESA) by applying a blockwise reconstruction loss, characterized by two distinctive attributes: i) it targets the overall pruning error with respect to individual transformer blocks, and ii) it allocates layer-specific sparsity in a differentiable manner, both of which ensure reduced performance degradation after pruning.
- SSN: Learning Sparse Switchable Normalization via SparsestMax57
Sparse switchable normalization (SSN) is presented where the importance ratios are constrained to be sparse, and SparsestMax is a sparse version of softmax, which is guaranteed to select only one normalizer for each normalization layer, avoiding redundant computations and improving interpretability of normalizer selection.
- 55
- ZipVL: Efficient Large Vision-Language Models with Dynamic Token Sparsification53
ZipVL is presented, an efficient inference framework designed for LVLMs through a dynamic ratio allocation strategy of important tokens that improves efficiency for less complex tasks while maintaining high performance for more challenging ones.
- 51
- 50
- B-AVIBench: Toward Evaluating the Robustness of Large Vision-Language Model on Black-Box Adversarial Visual-Instructions40
B-AVIBench is introduced, a framework designed to analyze the robustness of LVLMs when facing various Black-box Adversarial Visual-Instructions (B-AVIs), including four types of image-based B-AVIs, ten types of text-based B-AVIs, and nine types of content bias B-AVIs (such as gender, violence, cultural, and racial biases, among others).
- Differentiable Learning-to-Group Channels via Groupable Convolutional Neural Networks40
This work presents Groupable ConvNet (GroupNet) built by using a novel dynamic grouping convolution (DGConv) operation, which is able to learn the number of groups in an end-to-end manner and outperforms its counterparts such as ResNet and ResNeXt in terms of accuracy and computational complexity.
- RoboScript: Code Generation for Free-Form Manipulation Tasks across Real and Simulation37
The RobotScript platform addresses the gap in deployability of generated code on real robots, by emphasizing the unified interface with both simulation and real robots, based on abstraction from the Robot Operating System (ROS), ensuring syntax compliance and simulation validation with Gazebo.
- OpenING: A Comprehensive Benchmark for Judging Open-ended Interleaved Image-Text Generation36
OpenING, a comprehensive benchmark comprising 5,400 high-quality human-annotated instances across 56 real-world tasks, is introduced and IntJudge, a judge model for evaluating open-ended multimodal generation methods, is presented.
- 34
- PhyBench: A Physical Commonsense Benchmark for Evaluating Text-to-Image Models34
PhyBench is introduced, a comprehensive T2I evaluation dataset comprising 700 prompts across 4 primary categories: mechanics, optics, thermodynamics, and material properties, encompassing 31 distinct physical scenarios, and it is demonstrated that incorporating physical principles into prompts enhances the models' ability to generate physically accurate images.
- MM-PRM: Enhancing Multimodal Mathematical Reasoning with Scalable Step-Level Supervision30
MM-PRM demonstrates that process supervision is a powerful tool for enhancing the logical robustness of multimodal reasoning systems and achieves significant improvements across both in-domain and out-of-domain benchmarks.
- Dynamic Multimodal Evaluation with Flexible Complexity by Vision-Language Bootstrapping30
VLB dynamically generates new visual question-answering samples through a multimodal bootstrapping module that modifies both images and language, while ensuring that newly generated samples remain consistent with the original ones by a judge module.
- Real-Time Controllable Denoising for Image and Video29
Real-time Controllable Denoising (RCD) is introduced, the first deep image and video denoising pipeline that provides a fully controllable user interface to edit arbitraryDenoising levels in real-time with only one-time network inference.
- SearchLVLMs: A Plug-and-Play Framework for Augmenting Large Vision-Language Models by Searching Up-to-Date Internet Knowledge26
This paper proposes a plug-and-play framework, for augmenting existing LVLMs in handling visual question answering (VQA) about up-to-date knowledge, dubbed SearchLVLMs, where a hierarchical filtering model is trained to effectively and efficiently find the most helpful content from the websites returned by a search engine to prompt LVLMs with up-to-date knowledge.
- SPHINX: A Mixer of Weights, Visual Embeddings and Image Scales for Multi-modal Large Language Models26
- Foundation Model is Efficient Multimodal Multitask Model Selector26
An efficient multi-task model selector (EMMS), which employs large-scale foundation models to transform diverse label formats such as categories, texts, and bounding boxes of different downstream tasks into a unified noisy label embedding, making it the first model selection method in the multi- task scenario.
- ConvBench: A Multi-Turn Conversation Evaluation Benchmark with Hierarchical Capability for Large Vision-Language Models25
ConvBench is presented, a novel multi-turn conversation evaluation benchmark tailored for Large Vision-Language Models (LVLMs) that adopts a three-level multimodal capability hierarchy, mimicking human cognitive processes by stacking up perception, reasoning, and creativity.
- CO^3: Cooperative Unsupervised 3D Representation Learning for Autonomous Driving25
This paper proposes CO^3, namely Cooperative Contrastive Learning and Contextual Shape Prediction, to learn 3D representation for outdoor-scene point clouds in an unsupervised manner and believes CO^3 will facilitate understanding LiDAR point clouds in outdoor scene.
- 25
- Open-Vocabulary Animal Keypoint Detection with Semantic-Feature Matching24
This work introduces the Open-Vocabulary Keypoint Detection (OVKD) task, which is innovatively designed to use text prompts for identifying arbitrary keypoints across any species, and develops a novel framework named Open-Vocabulary Keypoint Detection with Semantic-feature Matching (KDSM), which synergistically combines vision and language models.
- MLLMs-Augmented Visual-Language Representation Learning23
This work demonstrates that Multi-modal Large Language Models (MLLMs) can enhance visual-language representation learning by establishing richer image-text associations for image-text datasets.
- SAMRefiner: Taming Segment Anything Model for Universal Mask Refinement22
This paper introduces a multi-prompt excavation strategy to mine diverse input prompts for SAM from initial coarse masks, and introduces a split-then-merge (STM) pipeline that boosts the performance of the generic SAMRefiner on the target dataset.
- 22
- MDK12-Bench: A Multi-Discipline Benchmark for Evaluating Reasoning in Multimodal Large Language Models17
This work introduces MDK12-Bench, a multi-disciplinary benchmark assessing the reasoning capabilities of MLLMs via real-world K-12 examinations, and presents a novel dynamic evaluation framework to mitigate data contamination issues by bootstrapping question forms, question types, and image styles during evaluation.
- Two Trades is not Baffled: Condensing Graph via Crafting Rational Gradient Matching17
A novel graph condensation method named CTRL, which offers an optimized starting point closer to the original dataset's feature distribution and a more refined strategy for gradient matching and can effectively neutralize the impact of accumulated errors on the performance of condensed graphs.
- Forensics-Bench: A Comprehensive Forgery Detection Benchmark Suite for Large Vision Language Models16
Forensics-Bench is presented, a new forgery detection evaluation benchmark suite to assess Large Vision Language Models' discerning capabilities across massive forgery detection tasks, requiring comprehensive recognition, location and reasoning capabilities on diverse forgeries.
- 16
- RetroAgent: From Solving to Evolving via Retrospective Dual Intrinsic Feedback15
This work introduces RetroAgent, an online RL framework that trains agents to master interactive environments not merely by solving tasks, but by evolving across episodes, and proposes Similarity&Utility-Aware Upper Confidence Bound (SimUtil-UCB), a retrieval strategy that balances semantic relevance, historical utility, and exploration.
- PrefixQuant: Eliminating Outliers by Prefixed Tokens for Large Language Models Quantization14
This work proposes PrefixQuant, a novel quantization method that achieves state-of-the-art performance across various precision levels (W4A4KV4 and W4A8KV4) and granularities (dynamic and static quantization) by effectively isolating token-wise outliers.
- Step-wise Rubric Rewards for LLM Reasoning14
Step-wise Rubrics as Rewards is introduced, an RLVR framework that uses an LLM judge to attribute each rubric item to a specific reasoning step, normalizes per-step rubric scores across rollouts so only steps whose quality varies produce a learning signal, and combines the per-step reward with the outcome reward through a decoupled advantage estimator that keeps the outcome baseline stable.
- 14
- CPGD: Toward Stable Rule-based Reinforcement Learning for Language Models12
This work proposes Clipped Policy Gradient Optimization with Policy Drift (CPGD), a novel algorithm designed to stabilize policy learning in LMs and demonstrates through empirical analysis that it mitigates the instability observed in prior approaches.
- DiffAgent: Fast and Accurate Text-to-Image API Selection with Large Language Model12
This work introduces DiffAgent, an LLM agent designed to screen the accurate selection in seconds via API calls, and leverages a novel two-stage training framework, SFTA, enabling it to accurately align T2I API responses with user input in accordance with human preferences.
- Enhance-A-Video: Better Generated Video for Free11
This work introduces a training-free approach to enhance the coherence and quality of DiT-based generated videos, named Enhance-A-Video, which enhances the cross-frame correlations based on non-diagonal temporal attention distributions.
- D-GARA: A Dynamic Benchmarking Framework for GUI Agent Robustness in Real-World Anomalies10
D-GARA is modular and extensible, supporting the seamless integration of new tasks, anomaly types, and interaction scenarios to meet specific evaluation goals, and construct and annotate a benchmark featuring commonly used Android applications with embedded anomalies to support broader community research.
- 10
- Align, Adapt and Inject: Sound-guided Unified Image Generation10
This paper proposes a unified framework 'Align, Adapt, and Inject' (AAI), able to extract the dynamic information of varied sounds, while utilizing the formidable capability of existing T2I models to facilitate sound-guided image generation, editing, and stylization in a convenient and cost-effective manner.
- SPOT: Scalable 3D Pre-Training via Occupancy Prediction for Learning Transferable 3D Representations7
SPOT, namely Scalable Pre-training via Occupancy prediction for learning Transferable 3D representations under such a label-efficient fine-tuning paradigm, achieves effectiveness on various public datasets with different downstream tasks, showcasing its general representation power, cross-domain robustness and data scalability.
- Cached Transformers: Improving Transformers with Differentiable Memory Cache7
This work introduces a new Transformer model that uses Gated Recurrent Cached (GRC) attention to extend the self-attention mechanism with a differentiable memory cache of tokens, and surpasses previous memory-based techniques in tasks such as language modeling and displays the ability to be applied to a broader range of situations.
- 7
- Understanding Regularization in Batch Normalization7
This work analyzes BN by using a building block of neural networks, which consists of a weight layer, a BN layer, and a nonlinear activation function, and shows that BN enables training converged with large maximum and effective learning rates.
- TinyLVLM-eHub: Towards Comprehensive and Efficient Evaluation for Large Vision-Language Models6
A comprehensive and efficient evaluation framework, TinyLVLM-eHub, is introduced to assess LVLMs’ performance, including proprietary models, and reveals that closed-source API models like GPT4V and GeminiPro-V excel in most capabilities compared to previous open-source LVLMs, though they show some vulnerability in object hallucination.
- Truly Assessing Fluid Intelligence of Large Language Models through Dynamic Reasoning Evaluation6
DRE-Bench is proposed, a dynamic reasoning evaluation benchmark grounded in a hierarchical cognitive framework that highlights the gap between current LLMs and true human-like fluid intelligence and offers a new path for systematically tracking reasoning progress in LLMs.
- Adapting LLaMA Decoder to Vision Transformer6
The tailored model, dubbed as image LLaMA (iLLaMA), is akin to LLaMA in architecture and enables direct supervised learning, and boosts computational efficiency and learns complex representation by elevating attention map ranks.
- 5
- MDK12-Bench: A Comprehensive Evaluation of Multimodal Large Language Models on Multidisciplinary Exams4
MDK12-Bench is introduced, a large-scale multidisciplinary benchmark built from real-world K-12 exams spanning six disciplines with 141K instances and 6,225 knowledge points organized in a six-layer taxonomy and a novel dynamic evaluation framework that introduces unfamiliar visual, textual, and question form shifts to challenge model generalization while improving benchmark objectivity and longevity by mitigating data contamination.
- Leveraging Error Diversity in Group Rollouts for Reinforcement Learning3
This work proposes Error Diversity Advantage Shaping (EDAS), a lightweight, algorithm-agnostic technique that modulates the advantage signal for incorrect rollouts based on intra-group error diversity and amplifies penalties for dominant, repeated errors and attenuates penalties for rare, exploratory ones, thereby encouraging the model to maintain diverse reasoning paths and discouraging error perseveration.
- 3
- 3
- 2
- 2
- 2
- 2
- 2
- 2
- 2
- 1
- 1
- DCP: Learning Accelerator Dataflow for Neural Networks via Propagation1
This work proposes an efficient data-centric approach, named Dataflow Code Propagation (DCP), to automatically find the optimal dataflow for DNN layers in seconds without human effort, which has several attractive benefits that prior studies lack.
- 1
- 1
- Temporal Overlapping Prediction: A Self-Supervised Pre-Training Method for LiDAR Moving Object Segmentation1
This paper proposes Temporal Overlapping Prediction (TOP), a self-supervised pre-training method designed to alleviate this annotation burden and consistently outperforms existing supervised and self-supervised pre-training baselines across both pointlevel Intersection-over-Union (IoU) and object-level Recall metrics.
- 1
- 1
- 1
- CLAP: Unsupervised 3D Representation Learning for Fusion 3D Perception via Curvature Sampling and Prototype Learning1
This work proposes a joint unsupervised differentiable-rendering-based pre-training method for images and point clouds, termed CLAP, short for Curvature sampLing and leArnable Prototype, and overcomes the computational hurdle by Curvature Sampling to select the more informative points/pixels for pre-training.
- Position: Towards Implicit Prompt For Text-To-Image Models1
This position paper presents a benchmark named ImplicitBench, designs and collects more than 2,000 implicit prompts of three aspects: General Symbols, Celebrity Privacy, and Not-Safe-For-Work (NSFW) Issues, and evaluates six well-known T2I models' capabilities under these implicit prompts.
- 1
- –
- –
- –
- AnyMod-LLVE: Low-Light Video Enhancement with Modality-Agnostic Inference–
A Spatial-Spectral Dual-Gated Translator that learns the correspondence between auxiliary modalities and RGB inputs, producing implicit auxiliary representations to support the robust enhancement of low-light video enhancement under modality absence conditions is introduced.
- Right Makes Might: Aligning Verified Hidden States Empowers RL Reasoning–
Hidden-Align is proposed, an auxiliary loss function that aligns the last-layer hidden states of correct rollouts at the anchor token during RL training, with zero overhead in both training and inference.
- –
- –
- –
- UniPruning: Unifying Local Metric and Global Feedback for Scalable Sparse LLMs–
UniPruning is presented, a unified post-training pruning framework that combines the speed of local saliency metrics with the stability of global coordination, enabled by a mirror descent based optimization, all without updating model weights.
- COSMO-RL: Towards Trustworthy LMRMs via Joint Safety and Stability–
COSMO-RL is presented, a mixed reinforcement learning framework that trains reasoning oriented LMRMs under multimodal, multitask, and multiobjective signals, and the resulting model, COSMO-R1, improves safety while maintaining-and often improving multimodal reasoning and instruction following.
- –
- –
- DCP: Learning Accelerator Dataflow for Neural Network via Propagation–
This work proposes an efficient data-centric approach, named Dataflow Code Propagation (DCP), to automatically find the optimal dataflow for DNN layers in seconds without human effort, and surpasses the GAMMA method that performs a full search using thousands of samples.
- –
Publication data from OpenAlex, with missing venues and authors filled in from Crossref; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-12. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.
Report an error
Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.