Is this you? Claim this profile to correct it, add a bio and choose the work people see first.
Claim this profileWorks153 from public data
- GLaMM: Pixel Grounding Large Multimodal Model594
This work presents Grounding LMM (GLaMM), the first model that can generate natural language responses seamlessly in-tertwined with corresponding object segmentation masks and is flexible enough to accept both textual and optional visual prompts (region of interest) as input.
- 443
- 417
- 362
- 324
- Binary patterns encoded convolutional neural networks for texture recognition and remote sensing scene classification287
This paper shows that Local Binary Patterns (LBP) encoded CNN models, codenamed TEX-Nets, trained using mapped coded images with explicit LBP based texture information provide complementary information to the standard RGB deep models.
- 281
- Mask-Guided Attention Network for Occluded Pedestrian Detection223
A novel mask-guided attention network is introduced that emphasizes on visible pedestrian regions while suppressing the occluded ones by modulating full body features and it is empirically demonstrated that coarse-level segmentation annotations provide reasonable approximation to their dense pixel-wise counterparts.
- SipMask: Spatial Information Preservation for Fast Image and Video Instance Segmentation200
A fast single-stage instance segmentation method that preserves instance-specific spatial information by separating mask prediction of an instance to different sub-regions of a detected bounding-box, leading to improved mask predictions and a mask alignment weighting loss and a feature alignment scheme to better correlate mask prediction with object detection.
- Person Image Synthesis via Denoising Diffusion Model186
This work shows how denoising diffusion models can be applied for high-fidelity person image synthesis with strong sample diversity and enhanced mode coverage of the learnt data distribution and introduces a ‘texture diffusion module’ based on cross-attention to accurately model the correspondences between appearance and pose information available in source and target images.
- Foundational Models Defining a New Era in Vision: A Survey and Outlook172
A comprehensive review of emerging foundational models in computer vision, including typical architecture designs to combine different modalities, training objectives, pre-training datasets, fine-tuning mechanisms, and the common prompting patterns; textual, visual, and heterogeneous.
- Spatio-temporal Relation Modeling for Few-shot Action Recognition159
A novel few-shot action recognition framework, STRM, which enhances class-specific feature discrim-inability while simultaneously learning higher-order temporal representations and introduces a query-class similarity classifier on the patch-level enriched features to enhance class- specific feature dis-criminability.
- XrayGPT: Chest Radiographs Summarization using Large Medical Vision-Language Models145
This paper introduces XrayGPT, a conversational medical vision-language (VLMs) model that can analyze and answer open-ended questions about chest radiographs, and aligns both medical visual encoder with a fine-tuned LLM to possess visual conversation abilities, grounded in an understanding of radiographs and medical knowledge.
- Class-Agnostic Object Detection with Multi-modal Transformer138
This paper advocates that existing methods lack a top-down supervision signal governed by human-understandable semantics and demonstrates that Multi-modal Vision Transformers (MViT) trained with aligned image-text pairs can effectively bridge this gap.
- Deep Contextual Attention for Human-Object Interaction Detection138
This work proposes a contextual attention framework for human-object interaction detection that leverages context by learning contextually-aware appearance features for human and object instances and adaptively selects relevant instance-centric context information to highlight image regions likely to contain human- object interactions.
- Rethinking Transformers Pre-training for Multi-Spectral Satellite Imagery135
The proposed SatMAE++ with multi-scale pre-training is equally effective for both optical as well as multi-spectral imagery and achieves mean average precision (mAP) gain of 2.5% for multi-label classification task on BigEarthNet dataset.
- LLM Post-Training: A Deep Dive Into Reasoning Large Language Models132
This survey provides a systematic exploration of post-training methodologies, analyzing their role in refining LLMs beyond pretraining, addressing key challenges such as catastrophic forgetting, reward hacking, and inference-time trade-offs.
- Remote Sensing Change Detection With Transformers Trained From Scratch129
This work develops an end-to-end CD approach with transformers that is trained from scratch and yet achieves state-of-the-art performance on five benchmarks, and introduces a change-enhanced feature fusion (CEFF) module to fuse the features from input image pairs by performing a per-channel re-weighting.
- 115
- 114
- Coloring Action Recognition in Still Images113
The experiments demonstrate that incorporating color information considerably improves recognition performance, and that a descriptor based on color names outperforms pure color descriptors and that late fusion of color and shape information outperforms other approaches on action recognition.
- 97
- Handwriting Transformers90
A novel transformer-based styled handwritten text image generation approach, HWT, that strives to learn both style-content entanglement as well as global and local style patterns, and is the first to introduce a transformer- based network for styling handwritten text generation.
- 89
- PSTR: End-to-End One-Step Person Search With Transformers88
The proposed PSTR is the first to propose an end-to-end one-step transformer-based person search framework that jointly performs person detection and re-identification (re-id) in a single architecture and sets a new state-of-the-art on both benchmarks.
- 76
- PSC-Net: learning part spatial co-occurrence for occluded pedestrian detection61
The proposed PSC-Net contains a dedicated module that is designed to explicitly capture both inter and intra-part co-occurrence information of different pedestrian body parts through a graph convolutional network (GCN).
- MobiLlama: Towards Accurate and Lightweight Fully Transparent GPT60
MobiLlama is a SLM design that initiates from a larger model and applies a careful parameter sharing scheme to reduce both the pre-training and the deployment cost, and strives to not only bridge the gap in open-source SLMs but also ensures full transparency.
- Energy-based Latent Aligner for Incremental Learning59
This work proposes ELI: Energy-based Latent Aligner for Incremental Learning, which first learns an energy manifold for the latent representations such that previous task latents will have low energy and the current task latent values have high energy.
- DriveLMM-o1: A Step-by-Step Reasoning Dataset and Large Multimodal Model for Driving Scenario Understanding57
A large multimodal model is introduced that is fine-tuned on the authors' reasoning dataset, demonstrating robust performance in complex driving scenarios, and various open-source and closed-source methods are benchmarked on this proposed dataset, systematically comparing their reasoning capabilities for autonomous driving tasks.
- 57
- AgroGPT : Efficient Agricultural Vision-Language Model with Expert Tuning55
This work utilizes diverse agricultural datasets spanning multiple domains, curate class-specific information, and employ large language models to construct an expert-tuning set, resulting in a 70k AgroInstruct, an efficient LMM that can hold complex agriculture-related conversations and provide useful insights.
- 55
- Compact color–texture description for texture classification55
This paper shows that combining multiple texture description methods significantly improves the recognition performance compared to using a single best texture method alone, and proposes to use information theoretic compression approach to compress high-dimensional multi-texture features into a compact heterogeneous texture representation.
- 54
- 51
- 3D Vision with Transformers: A Survey46
A systematic and thorough review of more than 100 transformers methods for different 3D vision tasks, including classification, segmentation, detection, completion, pose estimation, and others, and compares their performance to common non-transformer methods on 12 3D benchmarks.
- BiMediX: Bilingual Medical Mixture of Experts LLM40
BiMediX, the first bilingual medical mixture of experts LLM designed for seamless interaction in both English and Arabic, is introduced and outperforms the generic Arabic-English bilingual LLM, Jais-30B, by average absolute gains of 10% on the authors' Arabic medical benchmark and 15% on bilingual evaluations across multiple datasets.
- Discriminative Co-Saliency and Background Mining Transformer for Co-Salient Object Detection40
This paper proposes a Discriminative co-salient and background Mining Transformer framework based on several economical multi-grained correlation modules to explicitly mine both co- saliency and background information and effectively model their discrimination.
- Open-YOLO 3D: Towards Fast and Accurate Open-Vocabulary 3D Instance Segmentation39
This work proposes a fast yet accurate open-vocabulary 3D instance segmentation approach that effectively leverages only 2D object detection from multi-view RGB images for open-vocabulary 3D instance segmentation, and empirically finds that a better performance of matching text prompts to 3D masks can be achieved in a faster fashion with a 2D object detector.
- Composed Video Retrieval via Enriched Context and Discriminative Embeddings39
This work introduces a novel CoVR framework that leverages detailed language descriptions to explicitly encode query-specific contextual information and learns discriminative embeddings of vision only, text only and vision-text for better alignment to accurately retrieve matched target videos.
- Palo: A Polyglot Large Multimodal Model for 5B People37
A semiautomated translation approach to adapt the multimodal instruction dataset from English to the target languages using a fine-tuned Large Language Model, thereby ensuring high linguistic fidelity while allowing scalability due to minimal manual effort.
- DoodleFormer: Creative Sketch Drawing with Transformers37
A novel coarse-to-fine two-stage framework, DoodleFormer, that decomposes the creative sketch generation problem into the creation of coarse sketch composition followed by the incorporation of fine-details in the sketch, yielding realistic and diverse creative sketches.
- 35
- Count- and Similarity-Aware R-CNN for Pedestrian Detection35
A count-weighted detection loss function that assigns higher weights to the detection errors occurring at highly overlapping pedestrians, and a count-and-similarity branch within the two-stage detection framework, which predicts pedestrian count as well as proposal similarity to identify distinct proposals.
- Scale coding bag of deep features for human attribute and action recognition30
It is argued that entirely scale-invariant image representations are sub-optimal and approaches to scale coding within a bag of deep features framework are investigated and two strategies to encode multi-scale information explicitly in the final image representation are proposed.
- Combining Holistic and Part-based Deep Representations for Computational Painting Categorization29
This work presents an approach that encodes both the global scene layout and discriminative latent common structures for computational painting categorization, and achieves state-of-the-art results outperforming a recent multi-scale deep features based approach.
- DFormer: Diffusion-guided Transformer for Universal Image Segmentation28
DFormer takes deep pixel-level features along with the noisy masks as inputs to generate mask features and attention masks, employing diffusion-based decoder to perform mask prediction gradually and directly predicts the masks and corresponding categories from a set of randomly-generated masks.
- Learning a Dynamic Cross-Modal Network for Multispectral Pedestrian Detection28
This work introduces a multispectral pedestrian detection framework that comprises a novel dynamic cross-modal network (DCMNet), which strives to adaptively utilize the local and non-local complementary information between multi- modal features.
- EvoLMM: Self-Evolving Large Multimodal Models with Continuous Rewards26
This work proposes a self-evolving framework, named EvoLMM, that instantiates two cooperative agents from a single backbone model: a Proposer and a Solver, which solves problems through internal consistency and encourages both the generation of informative queries and the refinement of structured reasoning without relying on ground-truth or human judgments.
- Semi-supervised Open-World Object Detection23
A novel SS-OWOD detector that utilizes a feature-alignment scheme to better align the object query representations between the original and augmented images to leverage the large unlabeled and few labeled data and a pseudo-labeling scheme for unknown detection that exploits the inherent capability of decoder object queries to capture object-specific information.
- RAGNet: Large-Scale Reasoning-Based Affordance Segmentation Benchmark Towards General Grasping22
A large-scale grasping-oriented affordance segmentation benchmark with human-like instructions, named RAGNet, and a comprehensive affordance-based grasping framework, named AffordanceNet, which consists of a VLM pretrained on the authors' massive affordance data and a grasping network that conditions an affordance map to grasp the target.
- Compact Deep Color Features for Remote Sensing Scene Classification22
This work shows that combining several deep color models significantly improves the recognition performance compared to using the RGB network alone, and proposes to use an information theoretic compression approach to counter this issue, leading to a compact deep color feature set without any significant loss in accuracy.
- Arabic Mini-ClimateGPT : A Climate Change and Sustainability Tailored Arabic LLM20
This work proposes a light-weight Arabic Mini-ClimateGPT that is built on an open-source LLM and is specifically fine-tuned on a conversational-style instruction tuning curated Arabic dataset Clima500-Instruct with over 500k instructions about climate change and sustainability.
- 19
- 18
- 18
- CDCHAT: A Large Multimodal Model for Remote Sensing Change Description17
A change description instruction dataset is introduced that can be utilized to finetune an LMM and provide better change descriptions for RS images and it is shown that the LLaVA-1.5 model, with slight modifications, can be finetuned on the change description instruction dataset and achieve favorably better performance.
- 17
- 17
- 17
- Agent-X: Evaluating Deep Multimodal Reasoning in Vision-Centric Agentic Tasks16
Agent-X is introduced, a large-scale benchmark for evaluating vision-centric agents multi-step and deep reasoning capabilities in real-world, multimodal settings and proposes a fine-grained, step-level evaluation framework that assesses the correctness and logical coherence of each reasoning step and the effectiveness of tool usage throughout the task.
- DB-SAM: Delving into High Quality Universal Medical Image Segmentation16
This work proposes a dual-branch adapted SAM framework, named DB-SAM, that strives to effectively bridge the domain gap between natural and 2D/3D medical data and achieves an absolute gain of 8.8%, compared to a recent medical SAM adapter in the literature.
- Color Contribution to Part-Based Person Detection in Different Types of Scenarios16
This paper challenges RGB space with the opponent color space (OPP), which is inspired in the human vision system, and trains and test the part-based human classifier proposed by Felzenszwalb et al. using PASCAL VOC challenge protocols and person database.
- Multi-grained Temporal Prototype Learning for Few-shot Video Object Segmentation15
This work proposes to leverage multi-grained temporal guidance information for handling the temporal correlation nature of video data, and decomposes the query video information into a clip prototype and a memory prototype for capturing local and long-term internal temporal guidance.
- SA2-Net: Scale-aware Attention Network for Microscopic Image Segmentation15
SA2-Net is introduced, an attention-guided method that leverages multi-scale feature learning to effectively handle diverse structures within microscopic images and addresses the issue of blurred region boundaries by introducing a novel upsampling strategy called the Adaptive Up-Attention (AuA) module.
- Opponent Colors for Human Detection15
By feeding OPP space in the baseline framework of Dalal et al. for human detection (based on RGB, HOG and linear SVM), the authors will obtain better detection performance than by using RGB space, which suggests that in the future it could be worth to compute co-occurrence matrices, self-similarity features, etc., also on top of OPPspace, i.e., as they have done with HOG in this paper.
- AIN: The Arabic INclusive Large Multimodal Model13
AIN is an English-Arabic bilingual LMM designed to excel in English and Arabic, leveraging carefully constructed 3.6 million high-quality Arabic-English multimodal data samples to demonstrate state-of-the-art Arabic performance, while also possessing strong English-language visual capabilities.
- Modulate Your Spectrum in Self-Supervised Learning13
Spectral Transformation (ST) is introduced, a framework to modulate the spectrum of embedding and to seek for functions beyond whitening that can avoid dimensional collapse, and its empirical investigations unveil other ST instances capable of preventing collapse.
- TEX-Nets12
This work shows that CNN models, which it is called TEX-Nets, trained using mapped coded images with explicit texture information provide complementary information to the standard deep models trained on RGB patches, and investigates two deep architectures, namely early and late fusion, to combine the texture and color information.
- 11
- Deep Semantic Pyramids for Human Attributes and Action Recognition11
This work states that convolutional neural networks (CNNs) or deep features have shown to improve the performance over the conventional shallow features in the context of object recognition and object detection.
- 10
- 10
- On the Robustness of 3D Object Detectors10
This work is the first to investigate the robustness of point-based 3D object detectors against several data corruptions and shows that methods that integrate Transformers at a patch or object level lead to increased robustness, compared to using Transformers at the point level.
- MediX-R1: Open Ended Medical Reinforcement Learning9
The results demonstrate that open-ended RL with comprehensive reward signals and LLM-based evaluation is a practical path toward reliable medical reasoning in multimodal models.
- Beyond Simple Edits: Composed Video Retrieval with Dense Modifications9
This work introduces a novel dataset that captures both fine-grained and composed actions across diverse video segments, enabling more detailed compositional changes in retrieved video content and develops a new model that integrates visual and textual information through Cross-Attention (CA) fusion using grounded text encoder.
- 9
- Adapting In-Domain Few-Shot Segmentation to New Domains Without Source Domain Retraining8
The resulting Informative Structure Adaptation (ISA) method effectively addresses domain shifts and equips existing well-trained in-domain FSS models with flexible adaptation capabilities for new domains, eliminating the need to redesign or retrain in-domain FSS models on base data.
- Ask, Solve, Generate: Self-Evolving Unified Multimodal Understanding and Generation via Self-Consistency Rewards7
This work proposes a self-evolving training framework with three internal roles: a Proposer that generates visual questions, a Solver that answers and evaluates them, and a Generator that synthesizes images, and introduces Solver Token Entropy (STE), a continuous difficulty signal based on token-level prediction uncertainty that remains useful even when sample-level consistency becomes unreliable.
- 7
- 7
- 7
- 7
- Video Instance Segmentation in an Open-World6
This work proposes the first open-world VIS approach, named OW-VISFormer, that introduces a novel feature enrichment mechanism and a spatio-temporal objectness (STO) module, and shows the generalizability of the contributions for the open-world detection (OWOD) setting, outperforming the best existing OWOD method in the literature.
- Structured Latent Embeddings for Recognizing Unseen Classes in Unseen Domains6
This work proposes a novel approach that learns domain-agnostic structured latent embeddings by projecting images from different domains as well as class-specific semantic text-based representations to a common latent space.
- 5
- 5
- Open3DTrack: Towards Open-Vocabulary 3D Multi-Object Tracking5
This work proposes a novel approach that integrates open-vocabulary capabilities into a 3D tracking framework, allowing for generalization to unseen object classes, and is the first to address open-vocabulary 3D tracking.
- 5
- 5
- 5
- Generative Multiplane Neural Radiance for 3D-Aware Image Generation5
The proposed multiplane neural radiance model, named GMNR, consists of a novel α-guided view-dependent representation (α-VdR) module for learning view-dependent information and a view-consistency loss to enforce photometric similarity across multiple views.
- 4
- TAViS: Text-bridged Audio-Visual Segmentation with Foundation Models4
TAViS is presented, a novel framework that couples the knowledge of multimodal foundation models (ImageBind) for cross-modal alignment and a segmentation foundation model (SAM2) for precise segmentation and achieves superior performance on single-source, multi-source, semantic datasets, and excels in zero-shot settings.
- MAviS: A Multimodal Conversational Assistant For Avian Species4
The MAviS-Dataset is introduced, a large-scale multimodal avian species dataset that integrates image, audio, and text modalities for over 1,000 bird species, and MAviS-Chat, a multimodal LLM that supports audio, vision, and text and is designed for fine-grained species understanding, multimodal question answering, and scene-specific description generation.
- 4
- Audit After Segmentation: Reference-Free Mask Quality Assessment for Language-Referred Audio-Visual Segmentation3
MQ-Auditor is proposed, a multimodal large language model (MLLM)-based auditor that explicitly reasons over multimodal cues and mask information to produce quantitative and qualitative mask quality assessments and can be integrated with existing Ref-AVS systems to detect segmentation failures and support downstream segmentation improvement.
- MedROV: Towards Real-Time Open-Vocabulary Detection Across Diverse Medical Imaging Modalities3
This work introduces MedROV, the first Real-time Open Vocabulary detection model for medical imaging, and curates a large-scale dataset, Omnis, with 600K detection samples across nine imaging modalities and introduces a pseudo-labeling strategy to handle missing annotations from multi-source datasets.
- 3
- 3
- 3
- 2
- 2
- 2
- 2
- Multi-modal Generation via Cross-Modal In-Context Learning2
A Multi-modal Generation via Cross-Modal In-Context Learning (MGCC) method that generates novel images from complex multimodal prompt sequences by leveraging the combined capabilities of large language models (LLMs) and diffusion models.
- 2
- 2
- Multi-stream Convolutional Networks for Indoor Scene Recognition2
This work investigates CNN architectures by augmenting RGB images with estimated depth and texture information, as multiple streams, for monocular indoor scene recognition and trains a CNN model to exploit the successful Local Binary Patterns (LBP) to exploit texture information available in indoor scenes.
- Top-Down Deep Appearance Attention for Action Recognition2
This paper proposes a video representation which combines deep appearance and motion based local convolutional features within the bag-of-deep- features framework and outperforms both standard approaches of early and late feature fusion.
- 2
- 1
- CLIMB-3D: Continual Learning for Imbalanced 3D Instance Segmentation1
A class-balanced re-weighting (CBR) scheme that estimates object frequencies from pseudo-labels and dynamically adjusts training bias, without requiring access to past data is proposed.
- MATRIX: Multimodal Agent Tuning for Robust Tool-Use Reasoning1
A vision-centric agent tuning framework that automatically synthesizes multimodal trajectories, generates step-wise preference pairs, and trains a VLM controller for robust tool-use reasoning, demonstrating scalable and effective multimodal tool use.
- 1
- How Good are Foundation Models in Step-by-Step Embodied Reasoning?1
The Foundation Model Embodied Reasoning (FoMER) benchmark is proposed, designed to evaluate the reasoning capabilities of LMMs in complex embodied decision-making scenarios, and highlights both the potential and current limitations of LMMs in embodied reasoning.
- 1
- 1
- LEAPS: End-to-End One-Step Person Search With Learnable Proposals1
The proposed LEAPS employs a dynamic person search head to directly perform person detection and corresponding re-id feature generation without non-maximum suppression post-processing, and introduces a diverse re-ID sample matching strategy, instead of bipartite matching in detection head.
- Multi-scale Feature Aggregation for Crowd Counting1
Extensive experiments on four challenging datasets demonstrate the proposed easy-to-implement MSFANet can achieve promising results when compared with the previous state-of-the-art approaches.
- 1
- 1
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- Continual Learning and Unknown Object Discovery in 3D Scenes via Self-distillation–
This paper proposes a new exemplar-free approach for 3D continual learning and unknown object discovery through continual self-distillation that leverages the pseudo-labels generated by the model from the preceding task to improve the unknown predictions during training while simultaneously mitigating catastrophic forgetting.
- –
- –
- –
- –
- CMR3D–
This work proposes Contextualized Multi-Stage Refinement for 3D Object Detection (CMR3D), which takes a 3D scene as an input and strives to explicitly integrate useful contextual information of the scene at multiple levels to predict a set of object bounding-boxes along with their corresponding semantic labels.
- –
- –
- –
- –
- –
Publication data from OpenAlex, with missing venues and authors filled in from Crossref; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-16. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.
Report an error
Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.