Is this you? Claim this profile to correct it, add a bio and choose the work people see first.
Claim this profileWorks84 from public data
- Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis1,687
Video-MME is introduced, the first-ever full-spectrum, Multi-Modal Evaluation benchmark of MLLMs in Video analysis, and it is revealed that Gemini 1.5 Pro is the best-performing commercial model, significantly outperforming the open-source models with an average accuracy of 75%, compared to 71.9% for GPT-4o.
- Woodpecker: hallucination correction for multimodal large language models296
Implemented in a post-remedy manner, Woodpecker can easily serve different MLLMs, while being interpretable by accessing intermediate outputs of the five stages, and shows the huge potential of this new paradigm.
- SeqTR: A Simple Yet Universal Network for Visual Grounding247
Experiments on five benchmark datasets demonstrate that the proposed SeqTR outperforms (or is on par with) the existing state-of-the-arts, proving that a simple yet universal approach for visual grounding is indeed feasible.
- VITA: Towards Open-Source Interactive Omni Multimodal LLM204
VITA, the first-ever open-source Multimodal Large Language Model (MLLM) adept at simultaneous processing and analysis of Video, Image, Text, and Audio modalities, and meanwhile has an advanced multimodal interactive experience is introduced.
- Fusion-Mamba for Cross-Modality Object Detection190
This work investigates a novel cross-modality fusion approach by associating cross-modal features in a hidden state space based on an improved Mamba with a gating attention mechanism, and establishes a new baseline for cross-modality object detection.
- 143
- 130
- CF-ViT: A General Coarse-to-Fine Method for Vision Transformer121
A coarse-to-fine vision transformer (CF-ViT) to relieve computational burden while retaining performance in this paper, motivated by two important observations in modern ViT models: The coarse-grained patch splitting can locate informative regions of an input image and network inference in a two-stage manner.
- 116
- Aligning and Prompting Everything All at Once for Universal Visual Perception100
APE advances the convergence of detection and grounding by reformulating language-guided grounding as open-vocabulary detection, which efficiently scales up model prompting to thousands of category vocab-ularies and region descriptions while maintaining the ef-fectiveness of cross-modality fusion.
- Active Teacher for Semi-Supervised Object Detection98
The experimental results not only validate the superior performance gain of Active Teacher over the compared methods, but also show that it enables the baseline network, i.e., Faster-RCNN, to achieve 100% supervised performance with much less label expenditure,i.e. 40% labeled examples on MS-COCO.
- 80
- Cantor: Inspiring Multimodal Chain-of-Thought of MLLM74
It is argued that converging visual context acquisition and logical reasoning is pivotal for tackling visual reasoning tasks and an innovative multimodal CoT framework, termed Cantor, characterized by a perception-decision architecture is proposed, demonstrating the efficacy of the proposed framework.
- A Challenger to GPT-4V? Early Explorations of Gemini in Visual Expertise68
A preliminary exploration of Gemini Pro's visual understanding proficiency, which comprehensively covers four domains: fundamental perception, advanced cognition, challenging vision tasks, and various expert capacities, demonstrates the potential of Gemini to be a strong challenger to GPT-4V.
- Dynamic-LLaVA: Efficient Multimodal Large Language Models via Dynamic Vision-language Context Sparsification66
A dynamic vision-language context sparsification framework Dynamic-LLaVA is proposed, which dynamically reduces the redundancy of vision context in the prefill stage and decreases the memory and computation overhead of the generated language context during decoding.
- LCTR: On Awakening the Local Continuity of Transformer for Weakly Supervised Object Localization65
A novel framework built upon the transformer, termed LCTR (Local Continuity TRansformer), which targets at enhancing the local perception capability of global features among long-range feature dependencies and a relational patch-attention module (RPAM), which considers cross-patch information on a global basis.
- Weakly Supervised Object Detection via Object-Specific Pixel Gradient65
The OPG is trained by using image-level annotations alone, which performs in an iterative manner to localize potential objects in a given image robustly and efficiently, and significantly improves the detection speed, i.e., achieving three times faster compared with the state-of-the-art method.
- MISSU: 3D Medical Image Segmentation via Self-Distilling TransUNet64
This work proposes to self-distill a Transformer-based UNet for medical image segmentation, which simultaneously learns global semantic information and local spatial-detailed features and achieves the best performance over previous state-of-the-art methods.
- 50
- Solving the Catastrophic Forgetting Problem in Generalized Category Discovery43
This work designs a novel learning approach, LegoGCD, which is seamlessly integrated into previous methods to enhance the discrimination of novel classes while maintaining performance on previously encountered known classes and effectively addresses the known category forgetting issue across all datasets.
- 43
- VITA-Audio: Fast Interleaved Cross-Modal Token Generation for Efficient Large Speech-Language Model42
VITA-Audio is the first multi-modal large language model capable of generating audio output during the first forward pass, enabling real-time conversational capabilities with minimal latency and significantly outperforms open-source models of similar model size on multiple benchmarks.
- 37
- UWSOD: Toward Fully-Supervised-Level Capacity Weakly Supervised Object Detection37
The proposed UWSOD achieves competitive results with the state-of-the-art WSOD methods while not requiring external modules or additional supervision, and the upper-bound performance of UWSOD with class-agnostic ground-truth bounding boxes approaches Faster R-CNN, which demonstrates UWSOD has fully-supervised-level capacity.
- Open Vocabulary Object Detection with Proposal Mining and Prediction Equalization36
This work designs an online proposal mining to refine the inherited vision-semantic knowledge from coarse to fine, allowing for proposal-level detection-oriented feature alignment and introduces a class-wise backdoor adjustment to reinforce the predictions on novel categories to improve the overall OVD performance.
- 36
- Robotic Free Writing of Chinese Characters via Human–Robot Interactions25
This paper aims to demonstrate the efforts towards in-situ applicability of EMMARM, as to provide real-time information about concrete mechanical properties such as E-modulus and compressive strength.
- 24
- LUCY: Linguistic Understanding and Control Yielding Early Stage of Her23
LUCY is a E2E speech model that senses and responds to user's emotion, senses and responds to user's emotion, delivers responses in a succinct and natural style, and uses external tool to answer real-time inquiries.
- Adaptive Hierarchy-Branch Fusion for Online Knowledge Distillation22
A novel Adaptive Hierarchy-Branch Fusion framework for Online Knowledge Distillation, termed AHBF-OKD, is proposed, which designs hierarchical branches and adaptive hierarchy-branch fusion module to boost the model diversity and aggregate complementary knowledge.
- 22
- 21
- 21
- Fine-grained Data Distribution Alignment for Post-Training Quantization21
A fine-grained data distribution alignment (FDDA) method to boost the performance of post-training quantization on ImageNet, based on two important properties of batch normalization statistics (BNS) observed in deep layers of the trained network.
- 21
- 20
- 19
- Semi-Supervised Blind Image Quality Assessment through Knowledge Distillation and Incremental Learning18
A unified framework that combines semi-supervised and incremental learning to address the issue of catastrophic forgetting in Blind Image Quality Assessment and demonstrates its potential to tackle the challenges in real-world production processes.
- 17
- Hacking Chinese Touclick CAPTCHA by Multi-Scale Corner Structure Model with Fast Pattern Matching15
This paper proposes a multi-scale Corner based Structure Model, termed CSM, with a very efficient pattern matching scheme that can accurately capture the intrinsic statistics of touclick Chinese CAPTCHA against background clutters.
- Incidence, mortality, and global burden of retinoblastoma in 204 countries worldwide from 1990 to 2021: Data and systematic analysis from the Global Burden of Disease Study 202113
Retinoblastoma burden remains higher in low SDI regions, reflecting socioeconomic disparities, and females under two bear a higher retinoblastoma burden than males.
- E2Net: Excitative-Expansile Learning for Weakly Supervised Object Localization13
A novel end-to-end Excitation-Expansion network to localize entire objects with only image-level labels, coined as E$^2$Net, which outperforms the previous state-of-the-art WSOL methods and also significantly improves classification performance.
- VITA-VLA: Efficiently Teaching Vision-Language Models to Act via Action Expert Distillation11
This work proposes a simple yet effective distillation-based framework that equips VLMs with action-execution capability by transferring knowledge from pretrained small action models, and yields substantial efficiency gains over training large VLA models from scratch.
- 11
- 11
- The distributed system for inverted multi-index visual retrieval11
A distributed visual retrieval system based on inverted multi-index is proposed to generate a huge codebook with very low memory consumption and time cost and a vector space decomposition strategy is performed by affinity propagation clustering to improve the performance of product quantization.
- 10
- A General and Efficient Training for Transformer via Token Expansion10
This paper proposes a novel token growth scheme Token Expansion (termed ToE) to achieve consistent training acceleration for ViTs, and introduces an “initialization-expansion-merging” pipeline to maintain the integrity of the intermediate feature distribution of original transformers, preventing the loss of crucial learnable information in the training process.
- 10
- LAB-Net: LAB Color-Space Oriented Lightweight Network for Shadow Removal10
A novel lightweight deep neural network that processes shadow images in the LAB color space that well outperforms state-of-the-art methods and its parameters and computational costs are reduced by several orders of magnitude.
- A Part Power Set Model for Scale-Free Person Retrieval10
An end-to-end part power set model with multi-scale features is proposed, which captures the discriminative parts of pedestrians from global to local, and from coarse to fine, enabling part-based scale-free person re-ID.
- THANet: Transferring Human Pose Estimation to Animal Pose Estimation9
The proposed vision transformer and cross-domain method significantly improve the model’s accuracy and generalization ability for APE, and is inspired by the success of ViTPose in HPE.
- Learning Dynamic Prior Knowledge for Text-to-Face Pixel Synthesis9
A novel dynamic parameter generation module, PixelFace, is proposed, which transforms text features into dynamic knowledge embeddings for end-to-end pixel regression and validates its merits over existing T2F methods in both text-image matching and inference speed.
- Few-Shot Image Quality Assessment via Adaptation of Vision-Language Models8
The Gradient-Regulated MetaPrompt IQA Framework (GRMP-IQA), designed to efficiently adapt the visual-language pre-trained model, CLIP, to IQA tasks, achieving high accuracy even with limited data, is proposed.
- VEGA: Learning Interleaved Image-Text Comprehension in Vision-Language Large Models8
By employing a multi-task, multi-scale post-training strategy, this work sets a robust baseline for MLLMs on the IITC task, attaining an $85.8% accuracy rate in image association and a $0.508$ Rouge score, which validate the effectiveness of the dataset in improving MLLMs capabilities for nuanced image-text comprehension.
- FlexiReID: Adaptive Mixture of Expert for Multi-Modal Person Re-Identification7
FlexiReID is proposed, a flexible framework that supports seven retrieval modes across four modalities: rgb, infrared, sketches, and text, and introduces an adaptive mixture-of-experts (MoE) mechanism to dynamically integrate diverse modality features and a cross-modal query fusion module to enhance multimodal feature extraction.
- 7
- 7
- The burden of refraction disorders in 204 countries and territories from 1990 to 2021: A systematic analysis from the global burden of disease 20217
Refraction disorders remain a significant burden in regions with lower Socio-demographic Index (SDI), particularly in parts of South Asia and Western Sub-Saharan Africa, particularly in parts of South Asia and Western Sub-Saharan Africa.
- BUFF: Bayesian Uncertainty Guided Diffusion Probabilistic Model for Single Image Super-Resolution6
The Bayesian Uncertainty Guided Diffusion Probabilistic Model (BUFF) is introduced, incorporating a Bayesian network to generate high-resolution uncertainty masks that guide the diffusion process and significantly mitigates artifacts and blurring in areas characterized by complex textures and fine details.
- Multimodal Inplace Prompt Tuning for Open-set Object Detection6
This study introduces a redundancy assessment metric to identify uniform attention patterns and incorporates multimodal inplace prompt tuning (MIPT) to enrich the text prompt with visual clues in areas with high redundancy.
- Adaptive Feature Selection for No-Reference Image Quality Assessment by Mitigating Semantic Noise Sensitivity6
A Quality-Aware Feature Matching IQA metric (QFM-IQM) is proposed that employs contrastive learning to remove harmful features from the upstream task and achieves superior performance to the state-of-the-art NR-IQA.
- From Objects to Events: Unlocking Complex Visual Understanding in Object Detectors Via LLM-guided Symbolic Reasoning5
A novel framework that expands the capability of standard object detectors beyond mere object recognition to complex event understanding through LLM-guided symbolic reasoning, bridging the semantic gap between object detection and event understanding without requiring expensive taskspecific training.
- Multi-Modal Prompt Learning on Blind Image Quality Assessment5
This paper introduces an innovative multi-modal prompt-based methodology for IQA that employs carefully crafted prompts that synergistically mine incremental semantic information from both visual and linguistic data.
- Filter Pruning for Efficient CNNs via Knowledge-driven Differential Filter Sampler4
A novel Knowledge-driven Differential Filter Sampler~(KDFS) with Masked Filter Modeling~(MFM) framework for filter pruning, which globally prunes the redundant filters based on the prior knowledge of a pre-trained model in a differential and non-alternative optimization.
- Multi-scale Features for Weakly Supervised Lesion Detection of Cerebral Hemorrhage with Collaborative Learning4
This work quantitatively analyze the cerebral hemorrhage dataset and proposes a Multi-scale Feature with Collaborative Learning (MFCL) strategy in terms of Weakly Supervised Lesion Detection (WSLD), which not only adapts to the characteristics of detecting small lesions but also introduces the global constraint classification objective in training.
- NoTVLA: Semantics-Preserving Robot Adaptation via Narrative Action Interfaces3
The Narrowing of Trajectory VLA (NoTVLA) framework is proposed, a novel approach that narrows its focus to sparse trajectories, thereby avoiding the catastrophic forgetting associated with dense trajectory fine-tuning and enabling zero-shot generalization in specific scenarios.
- Probability-Density-aware Semi-supervised Learning3
It is proved that traditional first-order similarity pseudo-labeling could be viewed as a particular case of PMLP, which provides a comprehensive theoretical understanding of PMLP's superior performance.
- LocLoc: Low-level Cues and Local-area Guides for Weakly Supervised Object Localization3
This paper introduces a GrabCut-Enhanced Generator to learn global semantic representations for localization based on graph cuts to enhance low-level information based on long-range dependencies captured by the transformer and designs a Local Feature Digging Module (LFDM) that utilizes low- level cues to guide the learning route of local feature representations for accurate classification.
- 3
- Pseudo-Label Quality Decoupling and Correction for Semi-Supervised Instance Segmentation2
This paper introduces a novel Pseudo-Label Quality Decoupling and Correction (PL-DC) framework for SSIS, which achieves significant performance improvements and introduces a pixel-level mask uncertainty-aware mechanism at the pixel level to re-weight the mask loss for different pixels, thereby reducing the impact of noise introduced by pixel-level mask pseudo-labels.
- What You Perceive Is What You Conceive: A Cognition-Inspired Framework for Open Vocabulary Image Segmentation2
A novel Cognition-Inspired Framework for open vocabulary image segmentation that emulates the human visual recognition process: first forming a conceptual understanding of an object, then perceiving its spatial extent, offering enhanced flexibility in recognizing unseen categories.
- MASA: Rethinking the Representational Bottleneck in LoRA with Multi-A Shared Adaptation2
MASA (Multi-$A$ Shared Adaptation), an architecture that implements a multi-$A$, single-$B$ structure where the multi-$A$ expert ensemble is asymmetrically shared across layers to ensure parameter efficiency, is proposed.
- 2
- Self-Supervised Learning for Multi-Modal Medical Image Segmentation2
To address the shortcomings of underutilized multi-modal features in self-supervised learning methods for medical images, a self-supervised learning method considering multi-modal complementary information is proposed, named SLeM.
- 2
- 2
- 1
- Towards Universal Perception through Language-Guided Open-World Object Detection1
UniPerception is proposed, a novel universal perception framework based on open-vocabulary object detection that achieves strong open-world generalization and multi-modal understanding, outperforming the existing state-of-the-art methods and establishing itself as a unified, instruction-driven perception system.
- HUWSOD: Holistic Self-training for Unified Weakly Supervised Object Detection1
This paper introduces a unified, high-capacity weakly supervised object detection network called HUWSOD, which utilizes a comprehensive self-training framework without needing external modules or additional supervision, and indicates that randomly initialized boxes, although significantly different from well-designed offline object proposals, are effective for WSOD training.
- 1
- Global, regional, and national burden and temporal trends of depressive disorders in women of child-bearing age, 1990 to 2021: A worldwide analysis–
The rising global incidence of DD, coupled with regional variations in prevalence and DALYs, underscores the urgent need for innovative prevention and healthcare strategies to mitigate the burden among WCBA worldwide.
- –
- Sensitive Information Detection on Cyber-Space–
Two algorithms are proposed, i.e., iterative based semi-supervised deep learning model and humming melody based search model, to detect abnormal visual and audio objects respectively and show the effectiveness of these algorithms.
Publication data from OpenAlex; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-15. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.
Report an error
Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.