Is this you? Claim this profile to correct it, add a bio and choose the work people see first.
Claim this profileWorks70 from public data
- Intern VL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks3,476
A large-scale vision-language foun-dation model (Intern VL), which scales up the vision foun-dation model to 6 billion parameters and progressively aligns it with the LLM, using web-scale image-text data from various sources is designed.
- InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models1,936
InternVL3 incorporates variable visual position encoding (V2PE) to support extended multimodal contexts, employs advanced post-training techniques such as supervised fine-tuning (SFT) and mixed preference optimization (MPO), and adopts test-time scaling strategies alongside an optimized training infrastructure.
- VL-BERT: Pre-training of Generic Visual-Linguistic Representations1,860
A new pre-trainable generic representation for visual-linguistic tasks, called Visual-Linguistic BERT (VL-BERT), which adopts the simple yet powerful Transformer model as the backbone, and extends it to take both visual and linguistic embedded features as input.
- 1,837
- Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling1,813
This work explores the relationship between model scaling and performance, systematically exploring the performance trends in vision encoders, language models, dataset sizes, and test-time configurations and hopes this model contributes to the open-source community by setting new standards for developing and applying multimodal AI systems.
- Planning-oriented Autonomous Driving1,690
This work introduces Unified Autonomous Driving (UniAD), a comprehensive framework up-to-date that incorporates full-stack driving tasks in one network and is exquisitely devised to leverage advantages of each module, and provide complementary feature abstractions for agent interaction from a global perspective.
- How far are we to GPT-4V? Closing the gap to commercial multimodal models with open-source suites1,306
InternVL 1.5, an open-source multimodal large language model (MLLM) to bridge the capability gap between open-source and proprietary commercial models in multimodal understanding, shows competitive performance, achieving state-of-the-art results in 8 of 18 multimodal benchmarks.
- InternImage: Exploring Large-Scale Vision Foundation Models with Deformable Convolutions1,217
This work presents a new large-scale CNN-based foundation model, termed InternImage, which can obtain the gain from increasing parameters and training data like ViTs, and reduces the strict inductive bias of traditional CNNs and makes it possible to learn stronger and more robust patterns with large- scale parameters from massive datalike ViTs.
- BEVFormer v2: Adapting Modern Image Backbones to Bird's-Eye-View Recognition via Perspective Supervision490
A novel bird's-eye-view (BEV) detector with perspective supervision, which converges faster and bet-suits modern image backbones and achieves new SoTA results on the large-scale nuScenes dataset.
- Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memory368
Ghost in the Minecraft (GITM), a novel framework that integrates Large Language Models (LLMs) with text-based knowledge and memory, aiming to create Generally Capable Agents (GCAs) in Minecraft, shows the potential of LLMs in developing capable agents for handling long-horizon, complex tasks and adapting to uncertainties in open-world environments.
- DriveMLM: aligning multi-modal large language models with behavioral planning states for autonomous driving319
DriveMLM is introduced, an LLM-based AD framework that can perform close-loop autonomous driving in realistic simulators and can plug-and-play in existing AD systems such as Autopilot and Apollo for close-loop driving.
- Scene as Occupancy297
OccNet is proposed, a multi-view vision-centric pipeline with a cascade and temporal voxel decoder to reconstruct 3D occupancy embedding to represent 3D physical world and there are evident performance gain across multiple tasks, demonstrating the superiority of this method.
- Efficient Deformable ConvNets: Rethinking Dynamic and Sparse Operator for Vision Applications259
The advancements in speed and efficiency of DCNv4, combined with its robust performance across diverse vision tasks, show its potential as a foundational building block for future vision models.
- Delving Into the Devils of Bird’s-Eye-View Perception: A Review, Evaluation and Recipe252
A full suite of practical guidebook to improve the performance of BEV perception tasks, including camera, LiDAR and fusion inputs is introduced, and the future research directions in this area are pointed out.
- FuseFormer: Fusing Fine-Grained Information in Transformers for Video Inpainting180
This work proposes FuseFormer, a Transformer model designed for video inpainting via fine-grained feature fusion based on novel Soft Split and Soft Composition operations, which surpasses state-of-the-art methods in both quantitative and qualitative evaluations.
- Vision-RWKV: Efficient and Scalable Visual Perception with RWKV-Like Architectures166
Vision-RWKV (VRWKV), a model adapted from the RWKV model used in the NLP field with necessary modifications for vision tasks, surpasses ViT's performance in image classification and has significantly faster speeds and lower memory usage processing high-resolution inputs.
- 140
- Scaling Spatial Intelligence with Multimodal Foundation Models85
This work explores scaling up multimodal foundation models to cultivate spatial intelligence within the SenseNova-SI family, built upon established multimodal foundations including visual understanding models and unified understanding and generation models (i.e., Bagel).
- MM-Interleaved: Interleaved Image-Text Generative Modeling via Multi-modal Feature Synchronizer83
MM-Interleaved is an end-to-end generative model for interleaved image-text data that introduces a multi-scale and multi-image feature synchronizer module, allowing direct access to fine-grained image features in the previous context during the generation process.
- Towards All-in-One Pre-Training via Maximizing Multi-Modal Mutual Information58
This paper proposes a general multimodal mutual information formula as a unified optimization target and demonstrates that all mainstream approaches are special cases of the authors' framework, named Maximizing Multi-modal Mutual Information Pre-Training (M3I Pre-training).
- 57
- MMInstruct: a high-quality multi-modal instruction tuning dataset with extensive diversity51
The proposed instruction generation data engine enables semi-automatic, low-cost, and multi-domain instruction generation at 1/6 the cost of manual construction and could significantly improve the performance of VLLMs, e.g., the model fine-tuning on MMInstruct achieves new state-of-the-art performance on 10 out of 12 benchmarks.
- SynerGen-VL: Towards Synergistic Image Understanding and Generation with Vision Experts and Token Folding49
To address challenges identified in existing encoder-free unified MLLMs, the token folding mechanism and the vision-expert-based progressive alignment pretraining strategy are introduced, which effectively support high-resolution image understanding while reducing training complexity.
- 46
- Visual Embodied Brain: Let Multimodal Large Language Models See, Think, and Control in Spaces39
The Visual Embodied Brain (VeBrain), a unified framework for perception, reasoning, and control in real world, is presented and a novel robotic adapter is proposed to convert textual control signals from MLLMs to motion policies of real robots.
Publication data from OpenAlex; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-12. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.
Report an error
Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.