Is this you? Claim this profile to correct it, add a bio and choose the work people see first.
Claim this profileAcademic lineage
Possible advisorsa guess from early papers, not confirmed
- Mohan KankanhalliSuggested from co-authorship
Is this you? Claim this profile to confirm or dismiss it.
Works41 from public data
- BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models9,471
BLIP-2 achieves state-of-the-art performance on various vision-language tasks, despite having significantly fewer trainable parameters than existing methods, and is demonstrated's emerging capabilities of zero-shot image-to-text generation that can follow natural language instructions.
- InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning4,052
This paper conducts a systematic and comprehensive study on vision-language instruction tuning based on the pretrained BLIP-2 models, and introduces an instruction-aware Query Transformer, which extracts informative features tailored to the given instruction.
- 846
- LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understanding811
The LongVideoBench is introduced, a question-answering benchmark that features video-language interleaved inputs up to an hour long, designed to comprehensively evaluate LMMs on long-term multimodal understanding.
- BLIP-Diffusion: Pre-trained Subject Representation for Controllable Text-to-Image Generation and Editing593
This work introduces BLIP-Diffusion, a new subject-driven image generation model that supports multimodal control which consumes inputs of subject images and text prompts and designs a subject representation learning task which enables a diffusion model to leverage such visual representation and generates new subject renditions.
- Align and Prompt: Video-and-Language Pre-training with Entity Prompts238
This paper proposes Align and Prompt: a new video-and-language pre-training framework (AlPro), which operates on sparsely-sampled video frames and achieves more effective cross-modal alignment without explicit object detectors.
- From Images to Textual Prompts: Zero-shot Visual Question Answering with Frozen Large Language Models202
Img2LLM is a plug-and-play module that provides LLM prompts to enable LLMs to perform zeroshot VQA tasks without end-to-end training and eliminates the need to specialize LLMs using end-to-end finetuning and serve highly specialized LLMs to end users, thereby reducing cost.
- 164
- Interact as You Intend: Intention-Driven Human-Object Interaction Detection127
The proposed human intention-driven HOI detection (iHOI) framework models human pose with the relative distances from body joints to the object instances and utilizes human gaze to guide the attended contextual regions in a weakly-supervised setting.
- Open Vocabulary Object Detection with Pseudo Bounding-Box Labels122
This work proposes a method to automatically generate pseudo bounding-box annotations of diverse objects from large-scale image-caption pairs that leverages the localization ability of pre-trained vision-language models to generate pseudo Bounding- box labels and then directly uses them for training object detectors.
- Dual-Glance Model for Deciphering Social Relationships95
This work has proposed a dualglance model for social relationship recognition, where the first glance fixates at the individual pair of interest and the second glance deploys attention mechanism to explore contextual cues.
- X-InstructBLIP: A Framework for aligning X-Modal instruction-aware representations to LLMs and Emergent Cross-modal Reasoning92
An efficient and effective framework that integrates multiple modalities (images, 3D, audio and video) to a frozen LLM and demonstrates an emergent ability for cross-modal reasoning (2+ modality inputs).
- 78
- LAVIS: A Library for Language-Vision Intelligence71
LAVIS aims to serve as a one-stop comprehensive library that brings recent advancements in the language-vision field accessible for researchers and practitioners, as well as fertilizing future research and development.
- Video Storytelling: Textual Summaries for Events56
This paper introduces the problem of video storytelling, which aims at generating coherent and succinct stories for long videos, and proposes a context-aware framework for multimodal embedding learning, where a residual bidirectional recurrent neural network is designed to leverage contextual information from past and future.
- BLIP3o-NEXT: Next Frontier of Native Image Generation52
BLIP3o-NEXT unifies text-to-image generation and image editing within a single architecture, demonstrating strong image generation and image editing capabilities.
- LSTM-based multi-label video event detection38
A novel architecture for jointly recognizing multiple events in a given surveillance video, motivated by the sequence to sequence network is proposed, which can predict what happens in a video directly without the preprocessing of object detection and tracking.
- Hierarchical & multimodal video captioning: Discovering and transferring multimodal knowledge for vision to language36
A Hierarchical & Multimodal Video Caption (HMVC) model is proposed to jointly learn the dynamics within both visual and textual modalities for video caption task, which infers an arbitrary length sentence according to the input video with arbitrary number of frames.
- 19
- 17
- WALT: Web Agents that Learn Tools14
WALT (Web Agents that Learn Tools), a framework that reverse-engineers latent website functionality into reusable invocable tools, achieves higher success with fewer steps and less LLM-dependent reasoning, establishing a robust and generalizable paradigm for browser automation.
- A Multi-sensor Framework for Personal Presentation Analytics13
This work proposes a novel multi-sensor self-quantification system for presentations, which is designed based on a new proposed assessment rubric, and conducts a series of user studies with the speakers and an interview with an English communication expert, which reveals positive and promising feedback.
- 12
- 12
- UniSec9
UniSec is presented, a unified programming framework for fast security functions development while improving performance with SmartNIC acceleration and shows that the code has been reduced by 65% and processing performance improved up to 76%, compared with the software-only implementation.
- 7
- 7
- 7
- Self-supervised Representation Learning Using 360° Data7
This paper proposes middle-out, a new self-supervised learning task, which leverages the spatial configuration of normal field-of-view images sampled from a 360-degree image as supervisory signal and can enforce the ConvNet to extract high-level semantic concepts, an ability which previous self- supervised learning methods have not acquired.
- 4
- 4
- Design and Realization of Intelligent Voice-Control Car Based on Raspberry Pi3
An intelligent voice-control car based on Raspberry Pi that can work efficiently with high correction of speech recognition and Turing robot technology enables the car to dialogue with the users.
- 2
- 1
- 1
- BotSIM: An End-to-End Bot Simulation Toolkit for Commercial Task-Oriented Dialog Systems1
This report introduces BotSIM, a modular, open-source Bot SIM ulation environment with dialog generation, user simulation and conversation analytics capabilities, and focuses on the technical designs of various system components.
- 1
- –
- –
- –
- –
Publication data from OpenAlex, with missing venues and authors filled in from Crossref; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-12. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.
Report an error
Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.