Is this you? Claim this profile to correct it, add a bio and choose the work people see first.
Claim this profileWorks70 from public data
- Intern VL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks3,476
A large-scale vision-language foun-dation model (Intern VL), which scales up the vision foun-dation model to 6 billion parameters and progressively aligns it with the LLM, using web-scale image-text data from various sources is designed.
- InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models1,936
InternVL3 incorporates variable visual position encoding (V2PE) to support extended multimodal contexts, employs advanced post-training techniques such as supervised fine-tuning (SFT) and mixed preference optimization (MPO), and adopts test-time scaling strategies alongside an optimized training infrastructure.
- VL-BERT: Pre-training of Generic Visual-Linguistic Representations1,860
A new pre-trainable generic representation for visual-linguistic tasks, called Visual-Linguistic BERT (VL-BERT), which adopts the simple yet powerful Transformer model as the backbone, and extends it to take both visual and linguistic embedded features as input.
- 1,837
- Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling1,813
This work explores the relationship between model scaling and performance, systematically exploring the performance trends in vision encoders, language models, dataset sizes, and test-time configurations and hopes this model contributes to the open-source community by setting new standards for developing and applying multimodal AI systems.
- Planning-oriented Autonomous Driving1,690
This work introduces Unified Autonomous Driving (UniAD), a comprehensive framework up-to-date that incorporates full-stack driving tasks in one network and is exquisitely devised to leverage advantages of each module, and provide complementary feature abstractions for agent interaction from a global perspective.
- How far are we to GPT-4V? Closing the gap to commercial multimodal models with open-source suites1,306
InternVL 1.5, an open-source multimodal large language model (MLLM) to bridge the capability gap between open-source and proprietary commercial models in multimodal understanding, shows competitive performance, achieving state-of-the-art results in 8 of 18 multimodal benchmarks.
- InternImage: Exploring Large-Scale Vision Foundation Models with Deformable Convolutions1,217
This work presents a new large-scale CNN-based foundation model, termed InternImage, which can obtain the gain from increasing parameters and training data like ViTs, and reduces the strict inductive bias of traditional CNNs and makes it possible to learn stronger and more robust patterns with large- scale parameters from massive datalike ViTs.
- BEVFormer v2: Adapting Modern Image Backbones to Bird's-Eye-View Recognition via Perspective Supervision490
A novel bird's-eye-view (BEV) detector with perspective supervision, which converges faster and bet-suits modern image backbones and achieves new SoTA results on the large-scale nuScenes dataset.
- Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memory368
Ghost in the Minecraft (GITM), a novel framework that integrates Large Language Models (LLMs) with text-based knowledge and memory, aiming to create Generally Capable Agents (GCAs) in Minecraft, shows the potential of LLMs in developing capable agents for handling long-horizon, complex tasks and adapting to uncertainties in open-world environments.
- DriveMLM: aligning multi-modal large language models with behavioral planning states for autonomous driving319
DriveMLM is introduced, an LLM-based AD framework that can perform close-loop autonomous driving in realistic simulators and can plug-and-play in existing AD systems such as Autopilot and Apollo for close-loop driving.
- Scene as Occupancy297
OccNet is proposed, a multi-view vision-centric pipeline with a cascade and temporal voxel decoder to reconstruct 3D occupancy embedding to represent 3D physical world and there are evident performance gain across multiple tasks, demonstrating the superiority of this method.
- Efficient Deformable ConvNets: Rethinking Dynamic and Sparse Operator for Vision Applications259
The advancements in speed and efficiency of DCNv4, combined with its robust performance across diverse vision tasks, show its potential as a foundational building block for future vision models.
- Delving Into the Devils of Bird’s-Eye-View Perception: A Review, Evaluation and Recipe252
A full suite of practical guidebook to improve the performance of BEV perception tasks, including camera, LiDAR and fusion inputs is introduced, and the future research directions in this area are pointed out.
- FuseFormer: Fusing Fine-Grained Information in Transformers for Video Inpainting180
This work proposes FuseFormer, a Transformer model designed for video inpainting via fine-grained feature fusion based on novel Soft Split and Soft Composition operations, which surpasses state-of-the-art methods in both quantitative and qualitative evaluations.
- Vision-RWKV: Efficient and Scalable Visual Perception with RWKV-Like Architectures166
Vision-RWKV (VRWKV), a model adapted from the RWKV model used in the NLP field with necessary modifications for vision tasks, surpasses ViT's performance in image classification and has significantly faster speeds and lower memory usage processing high-resolution inputs.
- 140
- Scaling Spatial Intelligence with Multimodal Foundation Models85
This work explores scaling up multimodal foundation models to cultivate spatial intelligence within the SenseNova-SI family, built upon established multimodal foundations including visual understanding models and unified understanding and generation models (i.e., Bagel).
- MM-Interleaved: Interleaved Image-Text Generative Modeling via Multi-modal Feature Synchronizer83
MM-Interleaved is an end-to-end generative model for interleaved image-text data that introduces a multi-scale and multi-image feature synchronizer module, allowing direct access to fine-grained image features in the previous context during the generation process.
- Towards All-in-One Pre-Training via Maximizing Multi-Modal Mutual Information58
This paper proposes a general multimodal mutual information formula as a unified optimization target and demonstrates that all mainstream approaches are special cases of the authors' framework, named Maximizing Multi-modal Mutual Information Pre-Training (M3I Pre-training).
- 57
- MMInstruct: a high-quality multi-modal instruction tuning dataset with extensive diversity51
The proposed instruction generation data engine enables semi-automatic, low-cost, and multi-domain instruction generation at 1/6 the cost of manual construction and could significantly improve the performance of VLLMs, e.g., the model fine-tuning on MMInstruct achieves new state-of-the-art performance on 10 out of 12 benchmarks.
- SynerGen-VL: Towards Synergistic Image Understanding and Generation with Vision Experts and Token Folding49
To address challenges identified in existing encoder-free unified MLLMs, the token folding mechanism and the vision-expert-based progressive alignment pretraining strategy are introduced, which effectively support high-resolution image understanding while reducing training complexity.
- 46
- Visual Embodied Brain: Let Multimodal Large Language Models See, Think, and Control in Spaces39
The Visual Embodied Brain (VeBrain), a unified framework for perception, reasoning, and control in real world, is presented and a novel robotic adapter is proposed to convert textual control signals from MLLMs to motion policies of real robots.
- Docopilot: Improving Multimodal Models for Document-Level Understanding38
This work presents a high-quality document-level dataset, Doc-750K, designed to support in-depth understanding of multimodal documents, and develops a native multimodal model, Docopilot, which can accurately handle document-level dependencies without relying on RAG.
- MaskGWM: A Generalizable Driving World Model with Video Mask Reconstruction37
This paper proposes MaskGWM: a Generalizable driving World Model embodied with Video Mask reconstruction, which contains two variants: MaskGWM-long, focusing on long-horizon prediction, and MaskGWM-mview, dedicated to multi-view generation.
- SenseNova-MARS: Empowering Multimodal Agentic Reasoning and Search via Reinforcement Learning32
This work introduces SenseNova-MARS, a novel Multimodal Agentic Reasoning and Search framework that empowers VLMs with interleaved visual reasoning and tool-use capabilities via reinforcement learning (RL).
- 32
- PVC: Progressive Visual Token Compression for Unified Image and Video Processing in Large Vision-Language Models31
This work extends each image into a "static" video and introduces a unified token compression strategy called Progressive Visual Token Compression (PVC), where the tokens of each frame are progressively encoded and adaptively compressed to supplement the information not extracted from previous frames.
- SenseNova-U1: Unifying Multimodal Understanding and Generation with NEO-unify Architecture28
This work introduces SenseNova-U1, a native unified multimodal paradigm built upon NEO-unify, in which understanding and generation evolve as synergistic views of a single underlying process that points toward a broader roadmap where models do not translate between modalities, but think and act across them in a native manner.
- HoloDrive: Holistic 2D-3D Multi-Modal Street Scene Generation for Autonomous Driving26
This paper proposes a framework to jointly generate the camera images and LiDAR point clouds and employs BEV-to-Camera and Camera-to-BEV transform modules between heterogeneous generative models, and introduces a depth prediction branch in the 2D generative model to disambiguate the un-projecting from image space to BEV space.
- Learning to Prompt Segment Anything Models25
SSPrompt introduces spatial-semantic prompt learning (SSPrompt) that learns effective semantic and spatial prompts for better SAMs and optimize spatial prompts and semantic prompts directly over the embedding space and selectively leverage the knowledge encoded in pre-trained prompt encoders.
- 23
- ADDP: Learning General Representations for Image Recognition and Generation with Alternating Denoising Diffusion Process22
This method represents the first successful development of general representations applicable to both generation and dense recognition tasks and can be used to generate diverse high-fidelity images and also demonstrate excellent transfer performance on recognition tasks.
- 17
- Weakly Supervised Monocular 3D Detection with a Single-View Image17
SKD-WM3D is proposed, a weakly supervised monocular 3D detection framework that exploits depth information to achieve M3D with a single-view image exclusively without any 3D annotations or other training data and surpasses the state-of-the-art clearly and is even on par with many fully supervised methods.
- 16
- 14
- Demystify Transformers & Convolutions in Modern Image Deep Networks13
This paper aims to identify the real gains of popular convolution and attention operators through a detailed study, and finds that the key difference among these feature transformation modules, such as attention or convolution, lies in their spatial feature aggregation approach, known as the “spatial token mixer” (STM).
- 12
- HoVLE: Unleashing the Power of Monolithic Vision-Language Models with Holistic Vision-Language Embedding11
This paper presents a novel high-performance monolithic VLM named HoVLE, which introduces a holistic embedding module that converts visual and textual inputs into a shared space, allowing LLMs to process images in the same way as texts.
- OpenMobile: Building Open Mobile Agents with Task and Trajectory Synthesis10
OpenMobile is presented, an open-source framework that synthesizes high-quality task instructions and agent trajectories, with two key components: a scalable task synthesis pipeline that constructs a global environment memory from exploration, then leverages it to generate diverse and grounded instructions.
- Multimodal 3D Reasoning Segmentation with Complex Scenes10
A large-scale and high-quality benchmark that integrates 3D segmentation masks and 3D spatial relations with generated question-answer pairs and a novel 3D reasoning network that works with queries of multiple objects and is tailored for 3D scene understanding.
- 9
- 8
- EVA: Efficient Reinforcement Learning for End-to-End Video Agent7
Evo, an Efficient Reinforcement Learning framework for End-to-End Video Agent, which enables planning-before-perception through iterative summary-plan-action-reflection reasoning, achieving query-driven and efficient video understanding.
- From Pixels to Words -- Towards Native One-Vision Models at Scale6
NEO-ov is introduced, a native foundation model that learns cross-frame and pixel-word correspondence end-to-end without any external encoders, auxiliary adapters, or post-hoc fusion, validating that native"one-vision"architectures are not only feasible but competitive at scale.
- Masked AutoDecoder is Effective Multi-Task Vision Generalist6
This work designs Masked AutoDecoder, an effective multitask vision generalist that achieves superior performance and inference efficiency compared to autoregressive counterparts while obtaining competitive accuracy with task-specific models.
- ICA: Information-Aware Credit Assignment for Visually Grounded Long-Horizon Information-Seeking Agents3
Information-Aware Credit Assignment is introduced, a post hoc reward propagation method that estimates turn-level utility scores from rollout success rates and assigns dense rewards to intermediate steps that introduced high-utility information.
- SEAL: Speech Embedding Alignment Learning for Speech Large Language Model with Retrieval-Augmented Generation3
A unified embedding framework is proposed that eliminates the need for intermediate text representations in speech larage language models and reduces pipeline latency by 50\% while achieving higher retrieval accuracy compared to traditional two-stage methods.
- Exploring the Potential of Flexible 8-bit Format: Design and Algorithm3
This work proposes a flexible mixed-precision quantization framework that supports various number systems, enabling optimal selection of the most appropriate quantization format for different neural network architectures.
- 3
- ACPO: Counteracting Likelihood Displacement in Vision-Language Alignment with Asymmetric Constraints2
Asymmetric Constrained Preference Optimization (ACPO), a modality-agnostic alignment mechanism that applies dynamic, target-oriented scaling to preference optimization, is proposed, which effectively reverses the chosen-reward degradation of standard DPO.
- EgoPro-Bench: Benchmarking Personalized Proactive Interaction in Egocentric Video Streams1
The experiments demonstrate that EgoPro-Bench substantially enhances the intention understanding capabilities of MLLMs and enables accurate identification of appropriate timings for HMI, thereby laying a solid foundation for next-generation user-centric proactive interactive agents.
- GA-Drive: Geometry-Appearance Decoupled Modeling for Free-viewpoint Driving Scene Generation1
GA-Drive, a novel simulation framework capable of generating camera views along user-specified novel trajectories through Geometry-Appearance Decoupling and Diffusion-Based Generation, substantially outperforms existing methods in terms of NTA-IoU, NTL-IoU, and FID scores.
- 1
- Multi-scale 2D Temporal Map Diffusion Models for Natural Language Video Localization1
A novel approach to NLVL that involves the direct generation of a global 2D temporal map via a conditional denoising diffusion process, based on the input video and language query, and develops an innovative diffusion decoder.
- 1
- –
- –
- –
- InterSketch: An Interleaved Reasoning Model with Self-correcting Visual Sketch and Stepwise Reward–
InterSketch is introduced, an interleaved reasoning model to enhance the VT-CoT capability via self-correcting and stepwise reward mechanisms to mitigate the sparsity of reward signals inherent in end-only supervision over long-horizon reasoning.
- –
- –
- –
- –
- –
- –
- –
Publication data from OpenAlex; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-12. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.
Report an error
Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.