Is this you? Claim this profile to correct it, add a bio and choose the work people see first.
Claim this profileWorks56 from public data
- Learning to Prompt for Vision-Language Models4,550
Context Optimization (CoOp) is proposed, a simple approach specifically for adapting CLIP-like vision-language models for downstream image recognition that achieves superb domain generalization performance compared with the zero-shot model using hand-crafted prompts.
- Conditional Prompt Learning for Vision-Language Models2,499
Conditional Context Optimization (CoCoOp), which extends CoOp by further learning a lightweight neural network to generate for each image an input-conditional token (vector), and yields stronger domain generalization performance as well.
- Generalized Out-of-Distribution Detection: A Survey1,568
This paper addresses the gap in recent technical developments in recent technical developments in the field of OOD detection by presenting a unified framework called generalized OOD detection, which encompasses the five aforementioned problems, i.e.,AD, ND, OSR, OOD detection, and OD.
- Long Context Transfer from Language to Vision590
To effectively measure LMMs' ability to generalize to long contexts in the vision modality, V-NIAH (Visual Needle-In-A-Haystack), a purely synthetic long vision benchmark inspired by the language model's NIAH test is developed.
- LMMs-Eval: Reality Check on the Evaluation of Large Multimodal Models373
This work introduces LMMS-EVAL, a unified and standardized multimodal benchmark framework with over 50 tasks and more than 10 models to promote transparent and reproducible evaluations, and introduces LMMS-EVAL LITE, a pruned evaluation toolkit that emphasizes both coverage and efficiency.
- MIMIC-IT: Multi-Modal In-Context Instruction Tuning316
MultI-Modal In-Context Instruction Tuning (MIMIC-IT), a dataset comprising 2.8 million multimodal instruction-response pairs, with 2.2 million unique instructions derived from images and videos, is presented and a large VLM named Otter is trained.
- 310
- 239
- Semantically Coherent Out-of-Distribution Detection161
This work re-designs the benchmarks and proposes the semantically coherent out-of-distribution detection (SC-OOD), and designs a concise framework featured by unsupervised dual grouping (UDG) for the joint modeling of ID and OOD data.
- 115
- EgoLife: Towards Egocentric Life Assistant113
This work introduces EgoLife, a project to develop an egocentric life assistant that accompanies and enhances personal efficiency through AI-powered wearable glasses, and introduces EgoBulter, an integrated system comprising EgoGPT and EgoRAG, an integrated system trained on egocentric datasets.
- Large Language Models are Visual Reasoning Coordinators108
This work proposes Cola, a novel paradigm that coordinates multiple VLMs for visual reasoning by facilitating natural language communication that leverages their distinct and complementary capabilities, and validate that a coordinator LLM indeed comprehends the instruction prompts as well as the separate functionalities of VL Ms and coordinates them to enable impressive visual reasoning capabilities.
- Full-Spectrum Out-of-Distribution Detection89
This paper introduces full-spectrum OOD (F-OOD) detection, a more realistic problem setting that considers both detecting semantic shift and being tolerant to covariate shift; and proposes SEM, a simple feature-based semantics score function.
- OtterHD: A High-Resolution Multi-modality Model87
This study highlights the critical role of flexibility and high-resolution input capabilities in large multimodal models and also exemplifies the potential inherent in the Fuyu architecture's simplicity for handling complex visual data.
- Panoptic Video Scene Graph Generation73
A high-quality PVSG dataset is contributed, which consists of 400 videos (289 third-person + 111 egocentric videos) with totally 150K frames labeled with panoptic segmentation masks as well as fine, temporal scene graphs.
- Ego-R1: Chain-of-Tool-Thought for Ultra-Long Egocentric Video Reasoning67
Extensive results demonstrate that the dynamic, tool-augmented chain-of-thought reasoning by the Ego-R1 Agent can effectively tackle the unique challenges of understanding ultra-long egocentric videos, significantly extending the time coverage from few hours to a week.
- Generalized Out-of-Distribution Detection and Beyond in Vision Language Model Era: A Survey48
This survey presents a generalized OOD detection v2, encapsulating the evolution of these fields in the VLM era, and highlights the significant shift in the definition, problem settings, and benchmarks.
- 4D Panoptic Scene Graph Generation33
To solve PSG-4D, the introduction of 4D Panoptic Scene Graph (PSG-4D), a new representation that bridges the raw visual data perceived in a dynamic 4D world and high-level visual understanding, and a real-world application example to demonstrate how the model can achieve dynamic scene understanding by integrating a large language model into the system.
- 29
- 28
- 26
- SAD: Segment Any RGBD25
The Segment Any RGBD (SAD) model is proposed, which is specifically designed to extract geometry information directly from images, inspired by the natural ability of humans to identify objects through the visualization of depth maps, thus providing cues with enhanced geometry information and mitigating the issue of over-segmentation.
- Search for a small egg by spermatozoa in restricted geometries25
A mathematical model of the search by swimmers for a small target in a bounded domain is presented and the exploration of the parameter space for spermatozoa motion in different uterus geometries leads to scaling laws for the search process.
- Deep mixed model for marginal epistasis detection and population stratification correction in genome-wide association studies23
A neural network method is proposed that can potentially model arbitrary interactions between SNPs in genetic association studies as an extension to the mixed models in correcting confounding factors and helps gain some explorative understandings of the genetic architecture of Alzheimer’s disease.
- WorldQA: Multimodal World Knowledge in Videos through Long-Chain Reasoning20
This paper presents WorldQA, a video understanding dataset designed to push the boundaries of multimodal world models with three appealing properties, and introduces WorldRetriever, an agent designed to synthesize expert knowledge into a coherent reasoning chain, thereby facilitating accurate responses to WorldQA queries.
- 19
- Multiple Confounders Correction with Regularized Linear Mixed Effect Models, with Application in Biological Processes19
The results suggest that sequence multiple confounding factors corrections behave the best when different confounder contribute equally to response variables, and when various confounders affect the response variable unevenly.
- Webly Supervised Image Classification with Self-contained Confidence16
Inspired by the ability of Deep Neural Networks (DNNs) in confidence prediction, Self-Contained Confidence (SCC) is introduced by adapting model uncertainty for WSL setting, and used to sample-wisely balance the balance between web label and supervised loss on sample level.
- Webly Supervised Image Classification with Metadata: Automatic Noisy Label Correction via Visual-Semantic Graph15
VSGraph-LC starts from anchor selection referring to the semantic similarity between metadata and correct label concepts, and then propagates correct labels from anchors on a visual graph using graph neural network (GNN).
- ENHANCING GEOMETRIC DEEP LEARNING VIA GRAPH FILTER DECONVOLUTION13
This paper incorporates a graph filter deconvolution step into the classical geometric convolutional neural network pipeline, under the assumption that the graph domain plays a role in the generation of the observed graph signals.
- Unsolvable Problem Detection: Robust Understanding Evaluation for Large Multimodal Models12
A novel task to evaluate the robust understanding capability of Large Multimodal Models, termed Unsolvable Problem Detection (UPD), which assesses the LMM's ability to withhold answers when encountering unsolvable problems of MCQA, verifying whether the model truly understands the answer.
- 12
- GIST: distributed training for large-scale graph convolutional networks12
GIST improves model performance, scales to training on arbitrarily large graphs, decreases wall-clock training time, and enables the training of markedly overparameterized GCN models.
- 11
- Sparse Mixture-of-Experts are Domain Generalizable Learners11
This work reveals the mixture-of-experts (MoE) model’s generalizability on DG by leveraging to distributively handle multiple aspects of the predictive features across domains by incorporating sparsity and fusion mechanisms into the MoE framework to keep the model both sparse and predictive.
- Learning 4D Panoptic Scene Graph Generation from Rich 2D Visual Scene9
A 4D Large Language Model (4D-LLM) integrated with a 3D mask decoder for end-to-end generation of 4D-PSG and a 2D-to-4D visual scene transfer learning framework, where a spatial-temporal scene transcending strategy effectively transfers dimension-invariant features from abundant 2D SG annotations to 4D scenes, effectively compensating for data scarcity in 4D-PSG.
- On-Device Domain Generalization8
A systematic study of domain generalization (DG) for tiny neural networks finds that knowledge distillation (KD), a well-known technique for model compression, is much better for tackling the on-device DG problem than conventional DG methods.
- Dummy trajectory generation scheme based on generative adversarial networks8
A dummy trajectory generation scheme with conditional generative adversary network (GAN), where the map features are extracted using convolutional neural network, which is regarded as a prior restriction of conditional GAN.
- 6
- Dummy Trajectory Generation Scheme Based on Deep Learning6
LSTM (Long Short-Term Memory) is used to design the real and dummy trajectory discriminator, and it is shown that data mining based on deep learning can eliminate more than 95% of the algorithm generated trajectories, and the error rate of real trajectory is less than 10%.
- 6
- 3
- 3
- 2
- 2
- 2
- 1
- 1
- A Map-Aware Destination Prediction Model for Location-Based Consumer Electronics–
A Map-Aware Full Attention Model tailored for LCE devices, featuring an encoder-only full attention network optimized for real-time trajectory prediction, and introduces a sufficient grid-size criterion for AOI-based map representation to reduce the risk of losing small but informative geographic regions.
- –
- –
- –
- –
- –
- –
- –
Publication data from OpenAlex, with missing venues and authors filled in from Crossref; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-12. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.
Report an error
Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.