Is this you? Claim this profile to correct it, add a bio and choose the work people see first.
Claim this profileWorks103 from public data
- MagicDrive: Street View Generation with Diverse 3D Geometry Control310
This paper introduces MagicDrive, a novel street view generation framework, offering diverse 3D geometry controls including camera poses, road maps, and 3D bounding boxes, together with textual descriptions, achieved through tailored encoding strategies.
- DDP: Diffusion Model for Dense Visual Prediction254
The method, called DDP, efficiently extends the denoising diffusion process into the modern perception pipeline and shows attractive properties such as dynamic inference and uncertainty awareness, in contrast to previous single-step discriminative methods.
- G-LLaVA: Solving Geometric Problem with Multi-Modal Large Language Model233
G-LLaVA is developed, which demonstrates exceptional performance in solving geometric problems, significantly outperforming GPT-4-V on the MathVista benchmark with only 7B parameters.
- Self-Supervised Aggregation of Diverse Experts for Test-Agnostic Long-Tailed Recognition215
This paper proposes a novel approach, called Self-supervised Aggregation of Diverse Experts, which consists of a new skill-diverse expert learning strategy that trains multiple experts from a single and stationary long-tailed dataset to separately handle different class distributions.
- OoD-Bench: Quantifying and Understanding Two Dimensions of Out-of-Distribution Generalization145
This work first identifies and measure two distinct kinds of distribution shifts that are ubiquitous in various datasets, and compares OoD generalization algorithms across two groups of benchmarks, revealing their strengths on one shift as well as limitations on the other shift.
- 132
- Mixture of Cluster-Conditional LoRA Experts for Vision-Language Instruction Tuning128
A novel Mixture of Experts (MoE) architecture designed to activate task-customized model parameters based on instruction clusters is proposed, and a separate universal expert is further incorporated to improve generalization abilities of MoCLE for novel instructions.
- ORDisCo: Effective and Efficient Usage of Incremental Unlabeled Data for Semi-supervised Continual Learning115
This work considers semi-supervised continual learning (SSCL) that incrementally learns from partially labeled data, and proposes deep Online Replay with Discriminator Consistency (ORDisCo) to interdependently learn a classifier with a conditional generative adversarial network (GAN), which continually passes the learned data distribution to the classifier.
- MagicDrive-V2: High-Resolution Long Video Generation for Autonomous Driving with Adaptive Control111
This work proposes MagicDriveV2, a novel approach that integrates the MVDiT block and spatial-temporal conditional encoding to enable multiview video generation and precise geometric control, and introduces an efficient method for obtaining contextual descriptions for videos to support diverse textual control.
- Automated Evaluation of Large Vision-Language Models on Self-Driving Corner Cases105
This work proposes CODA-LM, the very first benchmark for the automatic evaluation of LVLMs for self-driving corner cases, and builds CODA-VLM, a new driving LVLM surpassing all open-sourced counterparts on CODA-LM.
- 93
- 90
- DecAug: Out-of-Distribution Generalization via Decomposed Feature Representation and Semantic Augmentation89
DecAug is proposed, a novel decomposed feature representation and semantic augmentation approach for OoD generalization that outperforms other state-of-the-art methods on various OoD datasets, which is among the very few methods that can deal with different types of OoDgeneralization challenges.
- 88
- MagicDrive3D: Controllable 3D Generation for Any-View Rendering in Street Scenes86
Experiments show that MagicDrive3D generates diverse, high-quality 3D driving scenes, supports any-view rendering, and enhances downstream tasks like BEV segmentation, demonstrating its potential for autonomous driving simulation and beyond.
- A random-effects Wiener degradation model based on accelerated failure time86
An inverse Gaussian (IG) distribution can be used to characterize the unit-specific heterogeneity in degradation paths, which overcomes the disadvantages of the traditional models and provides more flexibility in the degradation modeling using Wiener processes.
- EMOVA: Empowering Language Models to See, Hear and Speak with Vivid Emotions64
This work proposes the EMOVA (EMotionally Omni-present Voice Assistant), to enable Large Language Models with end-to-end speech abilities while maintaining the leading vision-language performance and surprisingly notices that omni-modal alignment can further enhance vision-language and speech abilities compared with the bi-modal aligned counterparts.
- MultiSiam: Self-supervised Multi-instance Siamese Representation Learning for Autonomous Driving64
The Multi-instance Siamese Network (MultiSiam) remarkably improves generalization ability and achieves state-of-the-art transfer performance on autonomous driving benchmarks, while existing SSL counterparts like MoCo, MoCo-v2, and BYOL show significant performance drop.
- ILLUME+: Illuminating Unified MLLM with Dual Visual Tokenization and Diffusion Refinement62
ILLUME+ introduces a unified dual visual tokenizer, DualViTok, which preserves both fine-grained textures and text-aligned semantics while enabling a coarse-to-fine image representation strategy for multimodal understanding and generation.
- Mixed Autoencoder for Self-Supervised Visual Representation Learning62
The proposed Mixed Autoencoder (MixedAE) achieves the state-of-the-art transfer results among masked image modeling (MIM) augmentations on different downstream tasks with significant efficiency and is the very first work to consider mixing for MIM from the perspective of pretext task design.
- When is Acceleration Unnecessary in a Degradation Test?55
This study identifies situations where acceleration is unnecessary when some common stochastic process models are used, including the Wiener, gamma, and inverse Gaussian processes.
- DetDiffusion: Synergizing Generative and Perceptive Models for Enhanced Data Generation and Perception49
DetDiffusion, for the first time, harmonizes generative and perceptive models, tackling the challenges in generating effective data for perceptive models, and introduces perception-aware loss through segmentation, improving both quality and controllability.
- 49
- Adversarial Robustness for Unsupervised Domain Adaptation47
This work proposes to leverage intermediate representations learned by robust ImageNet models to improve the robustness of UDA models by aligning the features of the UDA model with the robust features learned by ImageNet pre-trained models along with domain adaptation training.
- NAS-OoD: Neural Architecture Search for Out-of-Distribution Generalization47
Robust Neural Architecture Search for OoD generalization (NAS-OoD), which optimizes the architecture with respect to its performance on generated OoD data by gradient descent, and reduces the error rate by more than 70% compared with the state-of-the-art method.
- Highly Controllable and Silicon-Compatible Ferroelectric Photovoltaic Synapses for Neuromorphic Computing47
This study paves a new way toward highly controllable and silicon-compatible synapses for neuromorphic computing that use polarization-controlled photocurrent as the readout and thus have no limitations on the forms and thicknesses of the constituent ferroelectric and electrode materials.
- Make-A-Protagonist: Generic Video Editing with An Ensemble of Experts45
This work proposes a generic video editing framework called Make-A-Protagonist, which utilizes textual and visual clues to edit videos with the goal of empowering individuals to become the protagonists, and proposes a visual-textual-based video generation model that employs mask-guided denoising sampling to generate the desired output.
- ConQueR: Query Contrast Voxel-DETR for 3D Object Detection42
This paper proposes a simple yet effective sparse 3D detector, named Query Contrast Voxel-DETR (Con-QueR), to eliminate the challenging false positives, and achieve more accurate and sparser predictions, and closes the gap of sparse and dense 3D detectors.
- 39
- DiffGuard: Semantic Mismatch-Guided Out-of-Distribution Detection using Pre-trained Diffusion Models39
This work proposes to directly use pre-trained diffusion models for semantic mismatch-guided OOD detection, named DiffGuard, and tries to enlarge the semantic difference between the reconstructed OOD image under these conditions and the original input image.
- 39
- MetaAugment: Sample-Aware Data Augmentation Policy Learning36
This paper theoretically prove the convergence of the training procedure and further derive the exact convergence rate and learns a sample-aware data augmentation policy efficiently by formulating it as a sample reweighting problem.
- 36
- 36
- CoCA: Regaining Safety-awareness of Multimodal Large Language Models with Constitutional Calibration35
This paper raises the question: ``Do the MLLMs possess safety-awareness against malicious image inputs?", and introduces a simple yet effective technique termed CoCA, which amplifies the safety-awareness of the MLLM by calibrating its output distribution.
- Generalizing Few-Shot NAS with Gradient Matching33
A gradient matching score (GM) that leverages gradient information at the shared weight for making informed splitting decisions is proposed that significantly outperforms its Few-Shot counterparts while surpassing previous comparable methods in terms of the accuracy of derived architectures.
- Test-Agnostic Long-Tailed Recognition by Test-Time Aggregating Diverse Experts with Self-Supervision32
- 31
- MetaBEV: Solving Sensor Failures for BEV Detection and Map Segmentation27
A robust framework to address extreme real-world environments involving overall six sensor corruptions and two extreme sensor-missing situations, called MetaBEV, is proposed, which outperforms prior arts by a large margin on both full and corrupted modalities.
- 27
- Implicit Concept Removal of Diffusion Models26
The Geom-Erasing effectively mitigates the generation of implicit concepts, achieving the state-of-the-art results on the Inappropriate Image Prompts (I2P) and the authors' challenging Implicit Concept Dataset (ICD) benchmarks.
- 26
- 25
- ContraNeRF: Generalizable Neural Radiance Fields for Synthetic-to-real Novel View Synthesis via Contrastive Learning24
This work investigates the effects of synthetic data in synthetic-to-real novel view synthesis and surprisingly observes that models trained with synthetic data tend to produce sharper but less accurate volume densities, and proposes geometry-aware contrastive learning to learn multi-view consistent features with geometric constraints.
- ConsistentNeRF: Enhancing Neural Radiance Fields with 3D Consistency for Sparse View Synthesis24
This paper proposes ConsistentNeRF, a method that leverages depth information to regularize both multi-view and single-view 3D consistency among pixels, and employs depth-derived geometry information and a depth-invariant loss to concentrate on pixels that exhibit 3D correspondence and maintain consistent depth relationships.
- OoD-Bench: Benchmarking and Understanding Out-of-Distribution Generalization Datasets and Algorithms21
- Continual Learning by Modeling Intra-Class Variation19
This work examines memory-based continual learning and identifies that large variation in the representation space is crucial for avoiding catastrophic forgetting, and proposes to diversify representations by using two types of perturbations: model-agnostic variation and model-based variation.
- 18
- 17
- CVT-xRF: Contrastive In-Voxel Transformer for 3D Consistent Radiance Fields from Sparse Inputs16
A novel approach that models 3D spatial field consistency to improve NeRF's performance with sparse inputs by adopting a voxel-based ray sampling strategy and using a contrastive loss on the encoder output of the Transformer to further improve consistency within each voxel is proposed.
- PaMi-VDPO: Mitigating Video Hallucinations by Prompt-Aware Multi-Instance Video Preference Learning15
PaMi-VDPO is introduced, an online preference learning framework that eliminates the need for preference annotation by leveraging video augmentations to generate rejected samples while keeping responses fixed, and seamlessly integrates into existing VLLMs without additional parameters, GPT-4/human supervision.
- 14
- 14
- Mixture of insighTful Experts (MoTE): The Synergy of Reasoning Chains and Expert Mixtures in Self-Alignment13
Experimental results demonstrate that MoTE significantly improves model safety, jailbreak resistance, and over-refusal capabilities, achieving performance comparable to OpenAI's state-of-the-art o1 model.
- LLMs Can Evolve Continually on Modality for X-Modal Reasoning13
PathWeave is proposed, a flexible and scalable framework with modal-Path sWitching and ExpAnsion abilities that enables MLLMs to continually EVolve on modalities for $\mathbb{X}$-modal reasoning.
- Degradation-Based Reliability Modeling of Complex Systems in Dynamic Environments13
A summary of the state of the arts on the research of reliability modeling of complex systems by taking account of two aspects: how to deal with complex systems with more than one degradation indicators, and how to incorporate the effects of dynamic environments.
- Machine Learning Based Distinguishing between Ferroelectric and Non‐Ferroelectric Polarization–Electric Field Hysteresis Loops12
The high accuracy and efficiency demonstrate that the ML approaches significantly outperform the manual way for distinguishing ferroelectric from non‐ferroelectric P–E loops, which may greatly facilitate the research on ferroelectrics.
- Can Atomic Step Decomposition Enhance the Self-structured Reasoning of Multimodal Large Models?11
This paper proposes a paradigm of Self-structured Chain of Thought (SCoT), which is composed of minimal semantic atomic steps and designs a novel AtomThink framework, which can not only generate cognitive CoT structures for various complex tasks but also mitigates the phenomenon of overthinking.
- Contextualizing Meta-Learning via Learning to Decompose11
LeadNet is proposed to contextualize the meta-learned “support-to-target” strategy, leveraging the context of instances with one or mixed latent attributes in a support set, and demonstrates the superiority of LeadNet in various applications, including exploring multiple views of confusing data, out-of-distribution recognition, and few-shot image classification.
- 11
- InSight-o3: Empowering Multimodal Foundation Models with Generalized Visual Search10
InSight-o3 is proposed, a multi-agent framework consisting of a visual reasoning agent (vReasoner) and a visual search agent (vSearcher) for which the task of generalized visual search is introduced -- locating relational, fuzzy, or conceptual regions described in free-form language, beyond just simple objects or figures in natural images.
- 10
- 10
- Nonparametric link functions with shape constraints in stochastic degradation processes: Application to emerging contaminants9
This work uses stationary stochastic processes for parametric modeling of log-transformed EC degradation under each combination of the treatment conditions and finds that the use of shape information significantly improves the accuracy of estimates.
- SplatMesh: Interactive 3D Segmentation and Editing Using Mesh-Based Gaussian Splatting8
SplatMesh is introduced, a novel fine-grained interactive 3D segmentation and editing algorithm that integrates 3D Gaussian Splat with a precomputed mesh and could adjust the memory request based on the requirement.
- ECCV 2024 W-CODA: 1st Workshop on Multimodal Perception and Comprehension of Corner Cases in Autonomous Driving8
Details of the 1st W-CODA workshop are presented, held in conjunction with the ECCV 2024, to explore next-generation solutions for autonomous driving corner cases, empowered by state-of-the-art multimodal perception and comprehension techniques.
- 8
- 7
- 7
- VGGT-Det: Mining VGGT Internal Priors for Sensor-Geometry-Free Multi-View Indoor 3D Object Detection6
VGGT-Det is presented, the first framework tailored for SG-Free multi-view indoor 3D object detection, which integrates VGGT encoder into a transformer-based pipeline and introduces two novel key components: Attention-Guided Query Generation and Query-Driven Feature Aggregation.
- 6
- AtomThink: Multimodal Slow Thinking With Atomic Step Reasoning5
The core idea is that models can learn to adaptively use different levels of reasoning to tackle questions of varying complexity, and proposes a novel paradigm of Self-structured Chain of Thought (SCoT), which consists of minimal semantic atomic steps.
- 5
- Getting More Juice Out of Your Data: Hard Pair Refinement Enhances Visual-Language Models Without Extra Data4
HELIP is introduced, a cost-effective strategy that improves CLIP models by exploiting challenging text-image pairs within existing datasets in continuous training, which eliminates the need for additional data or extensive retraining.
- Certifying Language Model Robustness with Fuzzed Randomized Smoothing: An Efficient Defense Against Backdoor Attacks4
FRS integrates software robustness certification techniques with biphased model parameter smoothing, employing Monte Carlo tree search for proactive fuzzing to identify vulnerable textual segments within the Damerau-Levenshtein space, allowing for targeted and efficient text randomization.
- 4
- 4
- 4
- Mixture of insighTful Experts (MoTE): The Synergy of Thought Chains and Expert Mixtures in Self-Alignment3
The MoTE approach not only outperforms existing methods in aligning LLMs with human values but also highlights the benefits of using self-generated data, revealing the dual benefits of improved alignment and training efficiency.
- Robustness May be More Brittle than We Think under Different Degrees of Distribution Shifts3
It is shown that the robustness of models can be quite brittle and inconsistent under different degrees of distribution shifts, and therefore one should be more cautious when drawing conclusions from evaluations under a limited range of degrees.
- Dual-Curriculum Teacher for Domain-Inconsistent Object Detection in Autonomous Driving3
DucTeacher is proposed, a novel method that consists of two curriculums that can calibrate biased pseudo-labels and handle the domain-inconsistent SSOD problem effectively, and shows its advantages on SODA10M, the largest public semi-supervised autonomous driving dataset, and COCO, a widely used SSOD benchmark.
- StainFlow: Entity-Stain Tracking and Evidence Linking for Process Rewards in GUI Agents2
Inspired by stain-tracing mechanisms in network flow analysis, StainFlow is proposed, an entity-stain-flow process reward model for GUI Agents that relatively improves online RL success, and reduces the subjectivity of global partitioning.
- 2
- 2
- 2
- 2
- MIRAGE: Mobile Agents with Implicit Reasoning and Generative World Models1
MIRAGE is a framework that learns continuous latent reasoning representations from visible textual reasoning traces, enabling the agent to reason internally without decoding long rationales and turns hidden computation into both a compressed thought representation and a forward-looking model of environment dynamics.
- 1
- 1
- –
- AsySplat: Efficient Asymmetric 3D Gaussian Splatting for Long-Sequence Scene Modeling–
This model matches optimization-based methods while delivering nearly 800x speedup, and surpasses the zero-shot performance of state-of-the-art generalizable models with markedly fewer parameters and reduced training/inference overhead, achieving an overall efficiency improvement.
- –
- –
- –
- –
- –
- –
- Corrupted but Not Broken: Understanding and Mitigating the Negative Impacts of Corrupted Data in Visual Instruction Tuning–
It is found that disabling a small subset of parameters can almost fully restore performance and a corruption-robust training paradigm is introduced that significantly surpasses existing strategies for mitigating the effects of corrupted data.
- –
- –
- Out-of-distribution Few-shot Learning For Edge Devices without Model Fine-tuning–
A lightweight, plug-and-play FSL module called Task-aware Normalization (TANO) is proposed that enables efficient and task-aware adaptation of a deep neural network without backpropagation and outperforms recent methods in terms of accuracy, inference speed, and model size.
- –
- –
Publication data from OpenAlex, with missing venues and authors filled in from Crossref; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-15. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.
Report an error
Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.