Deepu Rajan lists you as their PhD student. Claim this profile to confirm it and keep the rest of your record right.
Claim this profileAcademic lineage
View as a treeAdvisors
Works134 from public data
- GLaMM: Pixel Grounding Large Multimodal Model594
This work presents Grounding LMM (GLaMM), the first model that can generate natural language responses seamlessly in-tertwined with corresponding object segmentation masks and is flexible enough to accept both textual and optional visual prompts (region of interest) as input.
- 443
- 417
- 361
- 281
- SipMask: Spatial Information Preservation for Fast Image and Video Instance Segmentation200
A fast single-stage instance segmentation method that preserves instance-specific spatial information by separating mask prediction of an instance to different sub-regions of a detected bounding-box, leading to improved mask predictions and a mask alignment weighting loss and a feature alignment scheme to better correlate mask prediction with object detection.
- Person Image Synthesis via Denoising Diffusion Model186
This work shows how denoising diffusion models can be applied for high-fidelity person image synthesis with strong sample diversity and enhanced mode coverage of the learnt data distribution and introduces a ‘texture diffusion module’ based on cross-attention to accurately model the correspondences between appearance and pose information available in source and target images.
- 3C-Net: Category Count and Center Loss for Weakly-Supervised Action Localization183
This work proposes a framework, called 3C-Net, which only requires video-level supervision (weak supervision) in the form of action category labels and the corresponding count to learn discriminative action features with enhanced localization capabilities.
- Foundational Models Defining a New Era in Vision: A Survey and Outlook172
A comprehensive review of emerging foundational models in computer vision, including typical architecture designs to combine different modalities, training objectives, pre-training datasets, fine-tuning mechanisms, and the common prompting patterns; textual, visual, and heterogeneous.
- Fine-Grained Recognition: Accounting for Subtle Differences between Similar Classes147
The main requisite for fine-grained recognition task is to focus on subtle discriminative details that make the subordinate classes different from each other, and this work proposes to explicitly force the network to find the subtle differences among closely related classes.
- XrayGPT: Chest Radiographs Summarization using Large Medical Vision-Language Models145
This paper introduces XrayGPT, a conversational medical vision-language (VLMs) model that can analyze and answer open-ended questions about chest radiographs, and aligns both medical visual encoder with a fine-tuned LLM to possess visual conversation abilities, grounded in an understanding of radiographs and medical knowledge.
- Object Counting and Instance Segmentation With Image-Level Supervision136
This work is the first to propose image-level supervised density map estimation for common object counting and demonstrate its effectiveness in image- level supervised instance segmentation and outperforms existing methods, including those using instance-level supervision, on both datasets for common objects counting.
- Rethinking Transformers Pre-training for Multi-Spectral Satellite Imagery135
The proposed SatMAE++ with multi-scale pre-training is equally effective for both optical as well as multi-spectral imagery and achieves mean average precision (mAP) gain of 2.5% for multi-label classification task on BigEarthNet dataset.
- LLM Post-Training: A Deep Dive Into Reasoning Large Language Models132
This survey provides a systematic exploration of post-training methodologies, analyzing their role in refining LLMs beyond pretraining, addressing key challenges such as catastrophic forgetting, reward hacking, and inference-time trade-offs.
- Remote Sensing Change Detection With Transformers Trained From Scratch129
This work develops an end-to-end CD approach with transformers that is trained from scratch and yet achieves state-of-the-art performance on five benchmarks, and introduces a change-enhanced feature fusion (CEFF) module to fuse the features from input image pairs by performing a per-channel re-weighting.
- ELGC-Net: Efficient Local–Global Context Aggregation for Remote Sensing Change Detection119
This work proposes an efficient change detection (CD) framework, efficient global and local context aggregation network (ELGC-Net), which leverages rich contextual information to precisely estimate change regions while reducing the model size, and introduces ELGC-Net-LW, a lighter variant with significantly reduced computational complexity, suitable for resource-constrained settings.
- 115
- 97
- Handwriting Transformers90
A novel transformer-based styled handwritten text image generation approach, HWT, that strives to learn both style-content entanglement as well as global and local style patterns, and is the first to introduce a transformer- based network for styling handwritten text generation.
- 89
- PSTR: End-to-End One-Step Person Search With Transformers88
The proposed PSTR is the first to propose an end-to-end one-step transformer-based person search framework that jointly performs person detection and re-identification (re-id) in a single architecture and sets a new state-of-the-art on both benchmarks.
- D2-Net: Weakly-Supervised Action Localization via Discriminative Embeddings and Denoised Activations69
A novel loss formulation is introduced, which jointly enhances the discriminability of latent embeddings and robustness of the output temporal class activations with respect to foreground-background noise caused by weak supervision, in a weakly-supervised temporal action localization framework, called D2-Net.
- PSC-Net: learning part spatial co-occurrence for occluded pedestrian detection61
The proposed PSC-Net contains a dedicated module that is designed to explicitly capture both inter and intra-part co-occurrence information of different pedestrian body parts through a graph convolutional network (GCN).
- MobiLlama: Towards Accurate and Lightweight Fully Transparent GPT60
MobiLlama is a SLM design that initiates from a larger model and applies a careful parameter sharing scheme to reduce both the pre-training and the deployment cost, and strives to not only bridge the gap in open-source SLMs but also ensures full transparency.
- DriveLMM-o1: A Step-by-Step Reasoning Dataset and Large Multimodal Model for Driving Scenario Understanding57
A large multimodal model is introduced that is fine-tuned on the authors' reasoning dataset, demonstrating robust performance in complex driving scenarios, and various open-source and closed-source methods are benchmarked on this proposed dataset, systematically comparing their reasoning capabilities for autonomous driving tasks.
- 57
- AgroGPT : Efficient Agricultural Vision-Language Model with Expert Tuning55
This work utilizes diverse agricultural datasets spanning multiple domains, curate class-specific information, and employ large language models to construct an expert-tuning set, resulting in a 70k AgroInstruct, an efficient LMM that can hold complex agriculture-related conversations and provide useful insights.
- 55
- Self-Evolving Multi-Agent Simulations for Realistic Clinical Interactions51
This work introduces MedAgentSim, an open-source simulated clinical environment with doctor, patient, and measurement agents designed to evaluate and enhance LLM performance in dynamic diagnostic settings, and introduces an evaluation benchmark for assessing the LLM's ability to engage in dynamic, context-aware diagnostic interactions.
- Sparse Coding for Alpha Matting49
A multi-frame graph model, as opposed to a single image as for image matting, is proposed that can be solved efficiently in closed form and outperforms the current stateoftheart in image and video matting.
- 3D Vision with Transformers: A Survey46
A systematic and thorough review of more than 100 transformers methods for different 3D vision tasks, including classification, segmentation, detection, completion, pose estimation, and others, and compares their performance to common non-transformer methods on 12 3D benchmarks.
- BiMediX: Bilingual Medical Mixture of Experts LLM40
BiMediX, the first bilingual medical mixture of experts LLM designed for seamless interaction in both English and Arabic, is introduced and outperforms the generic Arabic-English bilingual LLM, Jais-30B, by average absolute gains of 10% on the authors' Arabic medical benchmark and 15% on bilingual evaluations across multiple datasets.
- Discriminative Co-Saliency and Background Mining Transformer for Co-Salient Object Detection40
This paper proposes a Discriminative co-salient and background Mining Transformer framework based on several economical multi-grained correlation modules to explicitly mine both co- saliency and background information and effectively model their discrimination.
- Open-YOLO 3D: Towards Fast and Accurate Open-Vocabulary 3D Instance Segmentation39
This work proposes a fast yet accurate open-vocabulary 3D instance segmentation approach that effectively leverages only 2D object detection from multi-view RGB images for open-vocabulary 3D instance segmentation, and empirically finds that a better performance of matching text prompts to 3D masks can be achieved in a faster fashion with a 2D object detector.
- Palo: A Polyglot Large Multimodal Model for 5B People37
A semiautomated translation approach to adapt the multimodal instruction dataset from English to the target languages using a fine-tuned Large Language Model, thereby ensuring high linguistic fidelity while allowing scalability due to minimal manual effort.
- DoodleFormer: Creative Sketch Drawing with Transformers37
A novel coarse-to-fine two-stage framework, DoodleFormer, that decomposes the creative sketch generation problem into the creation of coarse sketch composition followed by the incorporation of fine-details in the sketch, yielding realistic and diverse creative sketches.
- AVisT: A Benchmark for Visual Object Tracking in Adverse Visibility36
AVisT is introduced, a dedicated benchmark for visual tracking in diverse scenarios with adverse visibility and can greatly benefit the tracking community by complementing the existing benchmarks, in developing new creative tracking solutions in order to continue pushing the boundaries of the state-of-the-art.
- 35
- Count- and Similarity-Aware R-CNN for Pedestrian Detection35
A count-weighted detection loss function that assigns higher weights to the detection errors occurring at highly overlapping pedestrians, and a count-and-similarity branch within the two-stage detection framework, which predicts pedestrian count as well as proposal similarity to identify distinct proposals.
- TransRadar: Adaptive-Directional Transformer for Real-Time Multi-View Radar Semantic Segmentation34
This work proposes a novel approach to the semantic segmentation of radar scenes using a multi-input fusion of radar data through a novel architecture and loss functions that are tailored to tackle the drawbacks of radar perception.
- 31
- Learning a Dynamic Cross-Modal Network for Multispectral Pedestrian Detection28
This work introduces a multispectral pedestrian detection framework that comprises a novel dynamic cross-modal network (DCMNet), which strives to adaptively utilize the local and non-local complementary information between multi- modal features.
- Sparse codes as Alpha Matte28
In this paper, image matting is cast as a sparse coding problem wherein the sparse codes directly give the estimate of the alpha matte, and there is no need to use the matting equation that restricts the estimates of α from a single pair of foreground (F) and background (B) samples.
- EvoLMM: Self-Evolving Large Multimodal Models with Continuous Rewards26
This work proposes a self-evolving framework, named EvoLMM, that instantiates two cooperative agents from a single backbone model: a Proposer and a Solver, which solves problems through internal consistency and encourages both the generation of informative queries and the refinement of structured reasoning without relying on ground-truth or human judgments.
- 25
- Semi-supervised Open-World Object Detection23
A novel SS-OWOD detector that utilizes a feature-alignment scheme to better align the object query representations between the original and augmented images to leverage the large unlabeled and few labeled data and a pseudo-labeling scheme for unknown detection that exploits the inherent capability of decoder object queries to capture object-specific information.
- Salient Mask-Guided Vision Transformer for Fine-Grained Classification22
This work introduces a simple yet effective Salient Mask-Guided Vision Transformer (SM-ViT), where the discriminability of the standard ViT`s attention maps is boosted through salient masking of potentially discriminative foreground regions.
- 21
- Arabic Mini-ClimateGPT : A Climate Change and Sustainability Tailored Arabic LLM20
This work proposes a light-weight Arabic Mini-ClimateGPT that is built on an open-source LLM and is specifically fine-tuned on a conversational-style instruction tuning curated Arabic dataset Clima500-Instruct with over 500k instructions about climate change and sustainability.
- 20
- 19
- 18
- 18
- Top-down saliency with Locality-constrained Contextual Sparse Coding18
The proposed top-down saliency achieves state-of-the-art results at patch- level on two challenging datasets-Graz-02 and PASCAL VOC-07 and a novel Gaussian-weighted interpolation further improves pixel-level saliency map derived from the patch-level map.
- CDCHAT: A Large Multimodal Model for Remote Sensing Change Description17
A change description instruction dataset is introduced that can be utilized to finetune an LMM and provide better change descriptions for RS images and it is shown that the LLaVA-1.5 model, with slight modifications, can be finetuned on the change description instruction dataset and achieve favorably better performance.
- 17
- 17
- Towards Partial Supervision for Generic Object Counting in Natural Scenes17
This work introduces a partially supervised setting that significantly reduces the supervision level required for generic object counting, and proposes two novel frameworks, named lower-count (LC) and reduced lower- count (RLC), to enable object counting under this setting.
- Agent-X: Evaluating Deep Multimodal Reasoning in Vision-Centric Agentic Tasks16
Agent-X is introduced, a large-scale benchmark for evaluating vision-centric agents multi-step and deep reasoning capabilities in real-world, multimodal settings and proposes a fine-grained, step-level evaluation framework that assesses the correctness and logical coherence of each reasoning step and the effectiveness of tool usage throughout the task.
- Transgene-free generation of mouse post-gastrulation whole embryo models solely from naive ESCs and iPSCs16
It is demonstrated that mouse nESCs and naive induced pluripotent stem cells (niPSCs) can be simultaneously co-induced, via signaling pathway modulation, to generate PrE and TE extra-embryonic cells that self-organize into embryonic day (E) 8.5-E8.75 transgene-free (TF) SEMs.
- CNN-based gender classification in near-infrared periocular images16
Two convolutional neural network (CNN)-based approaches for gender classification using near-infrared images of the periocular region are explored, showing higher classification accuracy than the baseline methods and accuracy is consistently higher than the existing eyebrow feature-based method.
- Multi-grained Temporal Prototype Learning for Few-shot Video Object Segmentation15
This work proposes to leverage multi-grained temporal guidance information for handling the temporal correlation nature of video data, and decomposes the query video information into a clip prototype and a memory prototype for capturing local and long-term internal temporal guidance.
- SA2-Net: Scale-aware Attention Network for Microscopic Image Segmentation15
SA2-Net is introduced, an attention-guided method that leverages multi-scale feature learning to effectively handle diverse structures within microscopic images and addresses the issue of blurred region boundaries by introducing a novel upsampling strategy called the Adaptive Up-Attention (AuA) module.
- A classifier-guided approach for top-down salient object detection15
A framework for top-down salient object detection that incorporates a tightly coupled image classification module that is trained on novel category-aware sparse codes computed on object dictionaries used for saliency modeling is proposed.
- Fixing Localization Errors to Improve Image Classification14
This work proposes a loss function that seeks to minimize peaks within the negative CAMs, called ‘ Homogeneous Negative CAM ’ loss, which provides an extra supervisory signal that helps the model to better discriminate between similar classes.
- AIN: The Arabic INclusive Large Multimodal Model13
AIN is an English-Arabic bilingual LMM designed to excel in English and Arabic, leveraging carefully constructed 3.6 million high-quality Arabic-English multimodal data samples to demonstrate state-of-the-art Arabic performance, while also possessing strong English-language visual capabilities.
- ChangeBind: A Hybrid Change Encoder for Remote Sensing Change Detection13
This work proposes an effective Siamese-based framework to encode the semantic changes occurring in the bi-temporal RS images by introducing a change encoder that leverages local and global feature representations to capture both subtle and large change feature information from multi-scale features to precisely estimate the change regions.
- On the Robustness of 3D Object Detectors10
This work is the first to investigate the robustness of point-based 3D object detectors against several data corruptions and shows that methods that integrate Transformers at a patch or object level lead to increased robustness, compared to using Transformers at the point level.
- MediX-R1: Open Ended Medical Reinforcement Learning9
The results demonstrate that open-ended RL with comprehensive reward signals and LLM-based evaluation is a practical path toward reliable medical reasoning in multimodal models.
- 9
- CLIP meets DINO for Tuning Zero-Shot Classifier using Unlabeled Image Collections9
A label-free prompt-tuning method that leverages the rich visual features of self-supervised learning models (DINO) and the broad textual knowledge of large language models (LLMs) to largely enhance CLIP-based image classification performance using unlabeled images, resulting in a powerful and efficient approach that surpasses state-of-the-art label-free classification methods.
- Adapting In-Domain Few-Shot Segmentation to New Domains Without Source Domain Retraining8
The resulting Informative Structure Adaptation (ISA) method effectively addresses domain shifts and equips existing well-trained in-domain FSS models with flexible adaptation capabilities for new domains, eliminating the need to redesign or retrain in-domain FSS models on base data.
- Ask, Solve, Generate: Self-Evolving Unified Multimodal Understanding and Generation via Self-Consistency Rewards7
This work proposes a self-evolving training framework with three internal roles: a Proposer that generates visual questions, a Solver that answers and evaluates them, and a Generator that synthesizes images, and introduces Solver Token Entropy (STE), a continuous difficulty signal based on token-level prediction uncertainty that remains useful even when sample-level consistency becomes unreliable.
- Medical Image Segmentation Using Directional Window Attention7
This work introduces DwinFormer, a hierarchical encoder-decoder architecture for medical image segmentation comprising a directional window (Dwin) attention and global self-attention (GSA) for feature encoding and introduces Dwin block within DwinFormer that effectively captures local and global information along the horizontal, vertical, and depthwise directions of the input feature map.
- DDAM-PS: Diligent Domain Adaptive Mixer for Person Search7
A diligent domain adaptive mixer (DDAM) for person search (DDAP-PS) framework that aims to bridge a gap to improve knowledge transfer from the labeled source domain to the unlabeled target domain is proposed.
- 7
- 7
- 7
- 6
- Video Instance Segmentation in an Open-World6
This work proposes the first open-world VIS approach, named OW-VISFormer, that introduces a novel feature enrichment mechanism and a spatio-temporal objectness (STO) module, and shows the generalizability of the contributions for the open-world detection (OWOD) setting, outperforming the best existing OWOD method in the literature.
- Improving 3-D Medical Image Segmentation at Boundary Regions Using Local Self-Attention and Global Volume Mixing6
A novel hierarchical encoder–decoder-based framework that strives to explicitly capture the local and global dependencies for volumetric 3-D medical image segmentation and introduces a novel volumetric multi-layer perceptron (MLP)-mixer to capture the global dependencies at low-resolution feature representations, respectively.
- Structured Latent Embeddings for Recognizing Unseen Classes in Unseen Domains6
This work proposes a novel approach that learns domain-agnostic structured latent embeddings by projecting images from different domains as well as class-specific semantic text-based representations to a common latent space.
- 5
- Open3DTrack: Towards Open-Vocabulary 3D Multi-Object Tracking5
This work proposes a novel approach that integrates open-vocabulary capabilities into a 3D tracking framework, allowing for generalization to unseen object classes, and is the first to address open-vocabulary 3D tracking.
- 5
- 5
- Generative Multiplane Neural Radiance for 3D-Aware Image Generation5
The proposed multiplane neural radiance model, named GMNR, consists of a novel α-guided view-dependent representation (α-VdR) module for learning view-dependent information and a view-consistency loss to enforce photometric similarity across multiple views.
- 4
- TAViS: Text-bridged Audio-Visual Segmentation with Foundation Models4
TAViS is presented, a novel framework that couples the knowledge of multimodal foundation models (ImageBind) for cross-modal alignment and a segmentation foundation model (SAM2) for precise segmentation and achieves superior performance on single-source, multi-source, semantic datasets, and excels in zero-shot settings.
- MAviS: A Multimodal Conversational Assistant For Avian Species4
The MAviS-Dataset is introduced, a large-scale multimodal avian species dataset that integrates image, audio, and text modalities for over 1,000 bird species, and MAviS-Chat, a multimodal LLM that supports audio, vision, and text and is designed for fine-grained species understanding, multimodal question answering, and scene-specific description generation.
- Audit After Segmentation: Reference-Free Mask Quality Assessment for Language-Referred Audio-Visual Segmentation3
MQ-Auditor is proposed, a multimodal large language model (MLLM)-based auditor that explicitly reasons over multimodal cues and mask information to produce quantitative and qualitative mask quality assessments and can be integrated with existing Ref-AVS systems to detect segmentation failures and support downstream segmentation improvement.
- MedROV: Towards Real-Time Open-Vocabulary Detection Across Diverse Medical Imaging Modalities3
This work introduces MedROV, the first Real-time Open Vocabulary detection model for medical imaging, and curates a large-scale dataset, Omnis, with 600K detection samples across nine imaging modalities and introduces a pseudo-labeling strategy to handle missing annotations from multi-source datasets.
- 3
- 3
- 3
- 3
- Hyret-Change: A Hybrid Retentive Network for Remote Sensing Change Detection3
This work introduces a novel feature difference module to exploit both convolutions and multi-head retention mechanisms in a parallel manner to capture complementary information and proposes an adaptive local-global interactive context awareness mechanism that enables mutual learning and enhances discrimination capability through information exchange.
- 3
- 3
- 3
- 2
- 2
- Multi-modal Generation via Cross-Modal In-Context Learning2
A Multi-modal Generation via Cross-Modal In-Context Learning (MGCC) method that generates novel images from complex multimodal prompt sequences by leveraging the combined capabilities of large language models (LLMs) and diffusion models.
- 2
- L1-Regularized Reconstruction Error as Alpha Matte2
Qualitative and quantitative evaluations on a dataset exclusively for video matting demonstrate the effectiveness of the proposed matting algorithm that uses L1-regularized reconstruction error of F and B samples as a measure of the alpha matte.
- Mobile-O: Unified Multimodal Understanding and Generation on Mobile Device1
Running in only ~3s per 512x512 image on an iPhone, Mobile-O establishes the first practical framework for real-time unified multimodal understanding and generation on edge devices, which it hopes will ease future research in real-time unified multimodal intelligence running entirely on-device with no cloud dependency.
- 1
- CoME-VL: Scaling Complementary Multi-Encoder Vision-Language Learning1
This work proposes CoME-VL: Complementary Multi-Encoder Vision-Language, a modular fusion framework that integrates a contrastively trained vision encoder with a self-supervised DINO encoder to scale the fusion of complementary visual representations for vision-language modeling.
- 1
- CLIMB-3D: Continual Learning for Imbalanced 3D Instance Segmentation1
A class-balanced re-weighting (CBR) scheme that estimates object frequencies from pseudo-labels and dynamically adjusts training bias, without requiring access to past data is proposed.
- 1
- How Good are Foundation Models in Step-by-Step Embodied Reasoning?1
The Foundation Model Embodied Reasoning (FoMER) benchmark is proposed, designed to evaluate the reasoning capabilities of LMMs in complex embodied decision-making scenarios, and highlights both the potential and current limitations of LMMs in embodied reasoning.
- 1
- Multi-scale Feature Aggregation for Crowd Counting1
Extensive experiments on four challenging datasets demonstrate the proposed easy-to-implement MSFANet can achieve promising results when compared with the previous state-of-the-art approaches.
- 1
- –
- –
- –
- –
- –
- –
- SafeDiffusion-R1: Online Reward Steering for Safe Diffusion Post-Training–
A novel online reinforcement learning framework that addresses both data scarcity and model degradation through post-training with Group Relative Policy Optimization (GRPO) on both negative and positive text prompts, and introduces a steering reward mechanism that exploits an inherent property of CLIP embeddings: steering text representations toward positive safety directions and away from negative ones in the embedding space.
- –
- –
- Open-Set Semi-Supervised Learning for Long-Tailed Medical Datasets–
An open-set learning method for highly imbal-anced medical datasets using a semi-supervised approach that addresses the impact of long-tail data in classification significantly improves the overall performance of the network in terms of closed-set and open-set accuracies on all datasets.
- –
- –
- InceptionMamba: Efficient Multi-Stage Feature Enhancement with Selective State Space Model for Microscopic Medical Image Segmentation–
This work proposes an efficient framework for the segmentation task, named InceptionMamba, which encodes multi-stage rich features and offers both performance and computational efficiency, and achieves state-of-the-art performance on two challenging microscopic segmentation datasets and two skin lesion segmentation datasets.
- –
- Continual Learning and Unknown Object Discovery in 3D Scenes via Self-distillation–
This paper proposes a new exemplar-free approach for 3D continual learning and unknown object discovery through continual self-distillation that leverages the pseudo-labels generated by the model from the preceding task to improve the unknown predictions during training while simultaneously mitigating catastrophic forgetting.
- –
- –
- –
- CMR3D–
This work proposes Contextualized Multi-Stage Refinement for 3D Object Detection (CMR3D), which takes a 3D scene as an input and strives to explicitly integrate useful contextual information of the scene at multiple levels to predict a set of object bounding-boxes along with their corresponding semantic labels.
Publication data from OpenAlex, with missing venues and authors filled in from Crossref; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-15. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.
Report an error
Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.