Is this you? Claim this profile to correct it, add a bio and choose the work people see first.
Claim this profileWorks32 from public data
- LLaVA-OneVision: Easy Visual Task Transfer3,160
LLaVA-OneVision is the first single model that can simultaneously push the performance boundaries of open LMMs in three important computer vision scenarios: single-image, multi-image, and video scenarios.
- VBench: Comprehensive Benchmark Suite for Video Generative Models2,062
VBench is a comprehensive benchmark suite that dissects “video generation quality” into specific, hierarchical, and disentangled dimensions, each with tailored prompts and evaluation methods, and investi-gate the gaps between video and image generation models.
- Otter: A Multi-Modal Model With In-Context Instruction Tuning716
Otter builds upon Flamingo with Perceiver architecture, and has been instruction tuned for general purpose multi-modal assistant, and substantially enhances model convergence and generalization capabilities.
- Long Context Transfer from Language to Vision590
To effectively measure LMMs' ability to generalize to long contexts in the vision modality, V-NIAH (Visual Needle-In-A-Haystack), a purely synthetic long vision benchmark inspired by the language model's NIAH test is developed.
- 483
- 454
- LMMs-Eval: Reality Check on the Evaluation of Large Multimodal Models373
This work introduces LMMS-EVAL, a unified and standardized multimodal benchmark framework with over 50 tasks and more than 10 models to promote transparent and reproducible evaluations, and introduces LMMS-EVAL LITE, a pruned evaluation toolkit that emphasizes both coverage and efficiency.
- MIMIC-IT: Multi-Modal In-Context Instruction Tuning316
MultI-Modal In-Context Instruction Tuning (MIMIC-IT), a dataset comprising 2.8 million multimodal instruction-response pairs, with 2.2 million unique instructions derived from images and videos, is presented and a large VLM named Otter is trained.
- VBench-2.0: Advancing Video Generation Benchmark Suite for Intrinsic Faithfulness299
VBench-2.0 aims to set a new standard for the next generation of video generative models in pursuit of intrinsic faithfulness, a next-generation benchmark designed to automatically evaluate video generative models for their intrinsic faithfulness.
- Neural Prompt Search196
Neural prOmpt seArcH (NOAH) is a novel approach that learns, for large vision models, the optimal design of prompt modules through a neural architecture search algorithm, specifically for each downstream dataset.
- Learning Without Forgetting for Vision-Language Models125
ProjectiOn Fusion (Proof) is proposed that enables VLMs to learn without forgetting and the fusion module is proposed to better utilize the cross-modality information.
- Octopus: Embodied Vision-Language Programmer from Environmental Feedback104
Octopus, an embodied vision-language programmer that uses executable code generation as a medium to connect planning and manipulation, is introduced and compelling results are presented, showing that the proposed RLEF refines the agent's decision-making.
- OtterHD: A High-Resolution Multi-modality Model87
This study highlights the critical role of flexibility and high-resolution input capabilities in large multimodal models and also exemplifies the potential inherent in the Fuyu architecture's simplicity for handling complex visual data.
- LLaVA-OneVision-2: Towards Next-Generation Perceptual Intelligence36
This work introduces LLaVA-OneVision-2 (LLaVA-OV-2), the most capable vision-language model in the LLaVA-OneVision series to date, achieving superior performance across a broad range of multimodal benchmarks, and introduces JumpScore, a temporal-localization benchmark targeting fine-grained grounding in high-frequency, densely repeated motion.
- Bamboo: Building Mega-Scale Vision Dataset Continually with Human–Machine Synergy28
This work proposes a novel active learning framework for realistic dataset annotation and builds a high-quality vision dataset—Bamboo, which consists of 69M image classification annotations with 119K categories and 28M object bounding box annotations with 809 categories that achieves superior performance among various downstream tasks.
- Multimodal Foundation Models for Zero-shot Animal Species Recognition in Camera Trap Images22
This work proposes WildMatch, a novel zero-shot species classification framework that leverages multimodal foundation models and investigates techniques to build instruction tuning datasets for detailed animal description generation and proposes a novel knowledge augmentation technique to enhance caption quality.
- 18
- 17
- New frontiers in artificial intelligence for biodiversity research and conservation with multimodal language models16
The transformative potential of multimodal language models for biodiversity research is explored and several possible applications in biodiversity research are discussed.
- OneVision-Encoder: Codec-Aligned Sparsity as a Foundational Principle for Multimodal Intelligence12
The results validate the core hypothesis: efficiency and accuracy are not a trade-off; they are positively correlated, enabling OV-Encoder as a scalable engine for next-generation visual generalists.
- 3D Point Cloud Pre-training with Knowledge Distillation from 2D Images12
This work introduces a cross-attention mechanism to extract concept features from 3D point cloud and compare them with the semantic information from 2D images to acquire knowledge directly from the 2D representation learning model, particularly the image encoder of CLIP.
- 10
- 9
- On-Device Domain Generalization8
A systematic study of domain generalization (DG) for tiny neural networks finds that knowledge distillation (KD), a well-known technique for model compression, is much better for tackling the on-device DG problem than conventional DG methods.
- From Pixels to Words -- Towards Native One-Vision Models at Scale6
NEO-ov is introduced, a native foundation model that learns cross-frame and pixel-word correspondence end-to-end without any external encoders, auxiliary adapters, or post-hoc fusion, validating that native"one-vision"architectures are not only feasible but competitive at scale.
Publication data from OpenAlex; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-15. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.
Report an error
Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.