Is this you? Claim this profile to correct it, add a bio and choose the work people see first.
Claim this profileWorks84 from public data
- Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis1,687
Video-MME is introduced, the first-ever full-spectrum, Multi-Modal Evaluation benchmark of MLLMs in Video analysis, and it is revealed that Gemini 1.5 Pro is the best-performing commercial model, significantly outperforming the open-source models with an average accuracy of 75%, compared to 71.9% for GPT-4o.
- Woodpecker: hallucination correction for multimodal large language models296
Implemented in a post-remedy manner, Woodpecker can easily serve different MLLMs, while being interpretable by accessing intermediate outputs of the five stages, and shows the huge potential of this new paradigm.
- SeqTR: A Simple Yet Universal Network for Visual Grounding247
Experiments on five benchmark datasets demonstrate that the proposed SeqTR outperforms (or is on par with) the existing state-of-the-arts, proving that a simple yet universal approach for visual grounding is indeed feasible.
- VITA: Towards Open-Source Interactive Omni Multimodal LLM204
VITA, the first-ever open-source Multimodal Large Language Model (MLLM) adept at simultaneous processing and analysis of Video, Image, Text, and Audio modalities, and meanwhile has an advanced multimodal interactive experience is introduced.
- Fusion-Mamba for Cross-Modality Object Detection190
This work investigates a novel cross-modality fusion approach by associating cross-modal features in a hidden state space based on an improved Mamba with a gating attention mechanism, and establishes a new baseline for cross-modality object detection.
- 143
- 130
- CF-ViT: A General Coarse-to-Fine Method for Vision Transformer121
A coarse-to-fine vision transformer (CF-ViT) to relieve computational burden while retaining performance in this paper, motivated by two important observations in modern ViT models: The coarse-grained patch splitting can locate informative regions of an input image and network inference in a two-stage manner.
- 116
- Aligning and Prompting Everything All at Once for Universal Visual Perception100
APE advances the convergence of detection and grounding by reformulating language-guided grounding as open-vocabulary detection, which efficiently scales up model prompting to thousands of category vocab-ularies and region descriptions while maintaining the ef-fectiveness of cross-modality fusion.
- Active Teacher for Semi-Supervised Object Detection98
The experimental results not only validate the superior performance gain of Active Teacher over the compared methods, but also show that it enables the baseline network, i.e., Faster-RCNN, to achieve 100% supervised performance with much less label expenditure,i.e. 40% labeled examples on MS-COCO.
- 80
- Cantor: Inspiring Multimodal Chain-of-Thought of MLLM74
It is argued that converging visual context acquisition and logical reasoning is pivotal for tackling visual reasoning tasks and an innovative multimodal CoT framework, termed Cantor, characterized by a perception-decision architecture is proposed, demonstrating the efficacy of the proposed framework.
- A Challenger to GPT-4V? Early Explorations of Gemini in Visual Expertise68
A preliminary exploration of Gemini Pro's visual understanding proficiency, which comprehensively covers four domains: fundamental perception, advanced cognition, challenging vision tasks, and various expert capacities, demonstrates the potential of Gemini to be a strong challenger to GPT-4V.
- Dynamic-LLaVA: Efficient Multimodal Large Language Models via Dynamic Vision-language Context Sparsification66
A dynamic vision-language context sparsification framework Dynamic-LLaVA is proposed, which dynamically reduces the redundancy of vision context in the prefill stage and decreases the memory and computation overhead of the generated language context during decoding.
- LCTR: On Awakening the Local Continuity of Transformer for Weakly Supervised Object Localization65
A novel framework built upon the transformer, termed LCTR (Local Continuity TRansformer), which targets at enhancing the local perception capability of global features among long-range feature dependencies and a relational patch-attention module (RPAM), which considers cross-patch information on a global basis.
- Weakly Supervised Object Detection via Object-Specific Pixel Gradient65
The OPG is trained by using image-level annotations alone, which performs in an iterative manner to localize potential objects in a given image robustly and efficiently, and significantly improves the detection speed, i.e., achieving three times faster compared with the state-of-the-art method.
- MISSU: 3D Medical Image Segmentation via Self-Distilling TransUNet64
This work proposes to self-distill a Transformer-based UNet for medical image segmentation, which simultaneously learns global semantic information and local spatial-detailed features and achieves the best performance over previous state-of-the-art methods.
- 50
- Solving the Catastrophic Forgetting Problem in Generalized Category Discovery43
This work designs a novel learning approach, LegoGCD, which is seamlessly integrated into previous methods to enhance the discrimination of novel classes while maintaining performance on previously encountered known classes and effectively addresses the known category forgetting issue across all datasets.
- 43
- VITA-Audio: Fast Interleaved Cross-Modal Token Generation for Efficient Large Speech-Language Model42
VITA-Audio is the first multi-modal large language model capable of generating audio output during the first forward pass, enabling real-time conversational capabilities with minimal latency and significantly outperforms open-source models of similar model size on multiple benchmarks.
- 37
- UWSOD: Toward Fully-Supervised-Level Capacity Weakly Supervised Object Detection37
The proposed UWSOD achieves competitive results with the state-of-the-art WSOD methods while not requiring external modules or additional supervision, and the upper-bound performance of UWSOD with class-agnostic ground-truth bounding boxes approaches Faster R-CNN, which demonstrates UWSOD has fully-supervised-level capacity.
- Open Vocabulary Object Detection with Proposal Mining and Prediction Equalization36
This work designs an online proposal mining to refine the inherited vision-semantic knowledge from coarse to fine, allowing for proposal-level detection-oriented feature alignment and introduces a class-wise backdoor adjustment to reinforce the predictions on novel categories to improve the overall OVD performance.
Publication data from OpenAlex; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-15. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.
Report an error
Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.