Is this you? Claim this profile to correct it, add a bio and choose the work people see first.
Claim this profileWorks146 from public data
- Multimodal Transformer for Unaligned Multimodal Language Sequences2,529
Comprehensive experiments on both aligned and non-aligned multimodal time-series show that the MulT model outperforms state-of-the-art methods by a large margin, and empirical analysis suggests that correlated crossmodal signals are able to be captured by the proposed cross modal attention mechanism in MulT.
- Multimodal Language Analysis in the Wild: CMU-MOSEI Dataset and Interpretable Dynamic Fusion Graph1,810
This paper introduces CMU Multimodal Opinion Sentiment and Emotion Intensity (CMU-MOSEI), the largest dataset of sentiment analysis and emotion recognition to date and uses a novel multimodal fusion technique called the Dynamic Fusion Graph (DFG), which is highly interpretable and achieves competative performance when compared to the previous state of the art.
- Efficient Low-rank Multimodal Fusion With Modality-Specific Factors1,109
The Low-rank Multimodal Fusion method is proposed, which performs multimodal fusion using low-rank tensors to improve efficiency and is indeed much more efficient in both training and inference compared to other methods that utilize tensor representations.
- Memory Fusion Network for Multi-view Sequential Learning992
A new neural architecture for multi-view sequential learning called the Memory Fusion Network (MFN) is presented that explicitly accounts for both interactions in a neural architecture and continuously models them through time.
- Found in Translation: Learning Robust Joint Representations by Cyclic Translations between Modalities597
This paper proposes a method to learn robust joint representations by translating between modalities based on the key insight that translation from a source to a target modality provides a method of learning joint representations using only the source modality as input.
- Multi-attention Recurrent Network for Human Communication Comprehension591
The main strength of the model comes from discovering interactions between modalities through time using a neural component called the Multi-attention Block (MAB) and storing them in the hybrid memory of a recurrent part called the Long-short Term Hybrid Memory (LSTHM).
- 553
- Words Can Shift: Dynamically Adjusting Word Representations Using Nonverbal Behaviors494
The Recurrent Attended Variation Embedding Network (RAVEN) is proposed that models the fine-grained structure of nonverbal subword sequences and dynamically shifts word representations based on nonverbal cues to capture the dynamic nature of non verbal intents.
- OS-ATLAS: A Foundation Action Model for Generalist GUI Agents393
OS-Atlas is developed - a foundational GUI action model that excels at GUI grounding and OOD agentic tasks through innovations in both data and modeling and is releasing the largest open-source cross-platform GUI grounding corpus to date, which contains over 13 million GUI elements.
- Multimodal sentiment analysis with word-level fusion and reinforcement learning327
The Gated Multimodal Embedding LSTM with Temporal Attention model is proposed that is composed of 2 modules and able to perform modality fusion at the word level and is able to better model the multimodal structure of speech through time and perform better sentiment comprehension.
- Foundations & Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questions321
An overview of the computational and theoretical foundations of multimodal machine learning is provided, defining three key principles of modality heterogeneity, connections, and interactions that have driven subsequent innovations, and proposing a taxonomy of six core technical challenges covering historical and recent trends.
- Towards Debiasing Sentence Representations298
It is shown that Sent-Debias is effective in removing biases, and at the same time, preserves performance on sentence-level downstream tasks such as sentiment analysis, linguistic acceptability, and natural language understanding.
- Rethinking Architecture Design for Tackling Data Heterogeneity in Federated Learning244
This paper demonstrates that self-attention-based architectures (e.g., Transformers) are more robust to distribution shifts and hence improve federated learning over heterogeneous data and releases code and pretrained models to encourage future exploration in robust architectures as an alternative to current research efforts on the optimization front.
- 232
- MEM1: Learning to Synergize Memory and Reasoning for Efficient Long-Horizon Agents231
MEM1, an end-to-end reinforcement learning framework that enables agents to operate with constant memory across long multi-turn tasks, is introduced, an end-to-end reinforcement learning framework that enables agents to operate with constant memory across long multi-turn tasks.
- Multimodal Language Analysis with Recurrent Multistage Fusion228
The Recurrent Multistage Fusion Network (RMFN) is proposed which decomposes the fusion problem into multiple stages, each of them focused on a subset of multimodal signals for specialized, effective fusion.
- 194
- 158
- Deep Gamblers: Learning to Abstain with Portfolio Theory143
Inspired by portfolio theory, a loss function for the selective classification problem based on the doubling rate of gambling is proposed, which allows us to train neural networks and characterize the disconfidence of prediction in an end-to-end fashion.
- Factorized Contrastive Learning: Going Beyond Multi-view Redundancy139
FactorCL is a new multimodal representation learning method to go beyond multi-view redundancy and captures both shared and unique information and achieves state-of-the-art results on six benchmarks.
- Diverse and Admissible Trajectory Forecasting Through Multimodal Context Understanding137
A model that fully synthesizes multiple input signals from the multimodal world, the environment's scene context and interactions between multiple surrounding agents, to best model all diverse and admissible trajectories is proposed.
- Medical Hallucination in Foundation Models and Their Impact on Healthcare136
The results reveal that inference techniques such as Chain-of-Thought (CoT) and Search Augmented Generation can effectively reduce hallucination rates, however, despite these improvements, non-trivial levels of hallucination persist.
- 135
- Foundations and Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questions121
This paper proposes a taxonomy of 6 core technical challenges: representation, alignment, reasoning, generation, transference, and quantification covering historical and recent trends, and defines two key principles of modality heterogeneity and interconnections that have driven subsequent innovations.
- 121
- Learning Representations from Imperfect Time Series Data via Tensor Rank Regularization116
A regularization method based on tensor rank minimization is presented based on the observation that high-dimensional multimodal time series data often exhibit correlations across time and modalities which leads to low-rank tensor representations.
- CMU-MOSEAS: A Multimodal Language Dataset for Spanish, Portuguese, German and French88
The first large-scale multimodal language dataset for Spanish, Portuguese, German and French, called CMU-MOSEAS (CMU Multimodal Opinion Sentiment, Emotions and Attributes), is introduced, which is the largest of its kind with 40, 000 total labelled sentences.
- Towards Vision-Language Mechanistic Interpretability: A Causal Tracing Tool for BLIP78
This work converts a unimodal causal tracing tool to BLIP to enable the study of the neural mechanisms underlying image-conditioned text generation and releases the BLIP causal tracing tool as open source to enable further experimentation in vision-language mechanistic interpretability by the community.
- Seq2Seq2Sentiment: Multimodal Sequence to Sequence Models for Sentiment Analysis72
Two methods for unsupervised learning of joint multimodal representations using sequence to sequence (Seq2Seq) methods are proposed: a Seq2 Seq Modality Translation Model and a Hierarchical Seq1Seq Modalities Translation Model.
- Read and Reap the Rewards: Learning to Play Atari with the Help of Instruction Manuals65
It is hypothesized that the ability to utilize human-written instruction manuals to assist learning policies for specific tasks should lead to a more efficient and better-performing agent in the Read and Reward framework.
- DIME: Fine-grained Interpretations of Multimodal Models via Disentangled Local Explanations57
DIME enables accurate and fine-grained analysis of multimodal models while maintaining generality across arbitrary modalities, model architectures, and tasks, and presents a step towards debugging and improving these models for real-world deployment.
- MMoE: Enhancing Multimodal Models with Mixtures of Multimodal Interaction Experts55
The key idea in MMoE is to train separate expert models for each type of multimodal interaction, such as redundancy present in both modalities, uniqueness in one modality, or synergy that emerges when both modalities are fused.
- Foundations of Multimodal Co-learning55
The differences between unimodal learning and Multimodal Co-learning are studied both from empirical and theoretical standpoints, and strong theoretical justifications in favor of MCl are demonstrated.
- 54
- World-in-World: World Models in a Closed-Loop World49
This study curates four closed-loop environments that rigorously evaluate diverse WMs, prioritize task success as the primary metric, and move beyond the common focus on visual quality; it presents the first data scaling law for world models in embodied settings.
- StylePTB: A Compositional Benchmark for Fine-grained Controllable Text Style Transfer49
A large-scale benchmark, StylePTB, is introduced with paired sentences undergoing 21 fine-grained stylistic changes spanning atomic lexical, syntactic, semantic, and thematic transfers of text, as well as compositions of multiple transfers which allow modeling of fine- grained styling changes as building blocks for more complex, high-level transfers.
- Tutorial on Multimodal Machine Learning: Principles, Challenges, and Open Questions48
This tutorial is designed to provide an overview of the computational and theoretical foundations of multimodal machine learning by synthesizing a broad range of application domains and theoretical frameworks from both historical and recent perspectives.
- Multimodal Local-Global Ranking Fusion for Emotion Recognition47
This work approaches emotion recognition from both direct person-independent and relative person-dependent perspectives and displays excellent performance on an audio-visual emotion recognition benchmark and improves over other algorithms for multimodal fusion.
- VideoWebArena: Evaluating Long Context Multimodal Agents with Video Understanding Web Tasks42
This work introduces VideoWebArena (VideoWA), a benchmark for evaluating the capabilities of long-context multimodal agents for video understanding and provides a testbed for future development with long-context video agents.
- Cross-Modal Generalization: Learning in Low Resource Modalities via Meta-Alignment39
An effective solution based on meta-alignment, a novel method to align representation spaces using strongly and weakly paired cross-modal data while ensuring quick generalization to new tasks across different modalities is proposed.
- 37
- IoT-LM: Large Multisensory Language Models for the Internet of Things33
IoT-LM, an open-source large multisensory language model tailored for the IoT ecosystem, is introduced and demonstrates new interactive question-answering, reasoning, and dialog capabilities conditioned on IoT sensors.
- Multimodal Learning Without Labeled Multimodal Data: Guarantees and Applications32
This work proposes two lower bounds based on the shared information between modalities and the other based on disagreement between separately trained unimodal classifiers, and derive an upper bound through connections to approximate algorithms for min-entropy couplings and validate these estimated bounds and show how they accurately track true interactions.
- 31
- Strong and Simple Baselines for Multimodal Utterance Embeddings31
Two simple but strong baselines to learn embeddings of multimodal utterances are proposed that show strong performance on both supervised and semi-supervised multimodals prediction, as well as significant speedups over neural models during inference.
- Benchmarking Vision, Language, & Action Models on Robotic Learning Tasks29
This work presents a comprehensive evaluation framework and benchmark suite for assessing VLA models, and reveals that current VLA models show significant variation in performance across different tasks and robot platforms, with GPT-4o demonstrating the most consistent performance through sophisticated prompt engineering.
- 28
- 28
- CLIMB: Data Foundations for Large Scale Multimodal Clinical Foundation Models27
Clinical Large-Scale Integrative Multimodal Benchmark (CLIMB) is introduced, a comprehensive clinical benchmark unifying diverse clinical data across imaging, language, temporal, and graph modalities and provides a foundation for new architecture designs and pretraining strategies to advance clinical AI research.
- Learn Globally, Speak Locally: Bridging the Gaps in Multilingual Reasoning26
The proposed M2A, a novel method that combines multi-scale multilingual alignment with language-consistency rewards on machine-translated questions, training models to reason directly and accurately in the target language, significantly enhances multilingual reasoning fidelity.
- Can A Society of Generative Agents Simulate Human Behavior and Inform Public Health Policy? A Case Study on Vaccine Hesitancy25
This work investigates the feasibility of simulating health-related decision-making, using vaccine hesitancy, using LLM-driven simulations as a case study, and introduces the VacSim framework with 100 generative agents powered by Large Language Models.
- 24
- OpenFace 3.0: A Lightweight Multitask System for Comprehensive Facial Behavior Analysis23
OpenFace 3.0 is introduced, an open-source toolkit capable of facial landmark detection, facial action unit detection, eye-gaze estimation, and facial emotion recognition and exhibits improvements in prediction performance, inference speed, and memory efficiency over similar toolkits and rivals state-of-theart models.
- Quantifying & Modeling Multimodal Interactions: An Information Decomposition Framework23
This work proposes an information-theoretic approach to quantify the degree of redundancy, uniqueness, and synergy across input features, which is term the PID statistics of a multimodal distribution.
- 22
- Learning Language and Multimodal Privacy-Preserving Markers of Mood from Mobile Data21
This paper studies behavioral markers of daily mood using a recent dataset of mobile behaviors from adolescent populations at high risk of suicidal behaviors and finds that language and multimodal representations of mobile typed text are predictive of daily Mood, but models trained to predict mood often also capture private user identities in their intermediate representations.
- 21
- Tutorial on Multimodal Machine Learning20
This tutorial will describe an updated taxonomy on multimodal machine learning synthesizing its core technical challenges and major directions for future research.
- 19
- Multimodal Fusion Interactions: A Study of Human and Automatic Quantification18
A comparative study of how humans annotate two categorizations of multimodal interactions is performed and a method to automatically convert annotations of partial and counterfactual labels to information decomposition is proposed, yielding an accurate and efficient method for quantifying multimodals interactions.
- 17
- Progressive Compositionality in Text-to-Image Generative Models17
This work leverage large-language models (LLMs) to compose realistic, complex scenarios and harness Visual-Question Answering (VQA) systems alongside diffusion models to automatically curate a contrastive dataset, ConPair, consisting of 15k pairs of high-quality contrastive images.
- 17
- Variational Auto-Decoder17
It is shown that encoders cannot efficiently marginalize the input volatility caused by imputed missing values in the VAD latent space, which is a particular area of impact for VAD models.
- Variational Auto-Decoder: Neural Generative Modeling from Partial Data17
It is shown that encoders cannot efficiently marginalize the input volatility caused by imputed missing values in the VAD latent space, which is a particular area of impact for VAD models.
- Interactive Sketchpad: A Multimodal Tutoring System for Collaborative, Visual Problem-Solving16
Interactive Sketchpad is a tutoring system that combines language-based explanations with interactive visualizations to enhance learning, enabling natural multimodal interaction with the student.
- Foundation Models for CPS-IoT: Opportunities and Challenges16
This paper identifies essential desiderata that CPS-IoT domain-specific FMs and LLMs must satisfy to bridge the gap between the current capabilities of FMs and LLMs in the CPS-IoT domain and the requirements they must meet to be viable for CPS-IoT applications.
- 16
- High-Modality Multimodal Transformer: Quantifying Modality & Interaction Heterogeneity for High-Modality Representation Learning16
A general multimodal model that enables multitask and transfer learning: multitask learning with shared parameters enables stable parameter counts, and cross-modal transfer learning enables information sharing across modalities and tasks (addressing partial observability).
- On Emergent Communication in Competitive Multi-Agent Teams16
The results show that an external competitive influence leads to improved accuracy and generalization, as well as faster emergence of communicative languages that are more informative and compositional.
- Localized Symbolic Knowledge Distillation for Visual Commonsense Models15
This work builds Localized Visual Commonsense models, which allow users to specify (multiple) regions as input and results in more precise VL models of reasoning compared to a baseline of passing a generated referring expression to an LLM.
- 14
- 14
- 13
- MultiZoo & MultiBench: A Standardized Toolkit for Multimodal Deep Learning13
MultiZoo is released, a public toolkit consisting of standardized implementations of>20 core multimodal algorithms and MultiBench, a large-scale benchmark spanning 15 datasets, 10 modalities, 20 prediction tasks, and 6 research areas that provide an automated end-to-end machine learning pipeline that simplifies and standardizes data loading, experimental setup, and model evaluation.
- MultiViz: Towards Visualizing and Understanding Multimodal Models13
The complementary stages in MULTIVIZ together enable users to simulate model predictions, assign interpretable concepts to features, perform error analysis on model misclassifications, and use insights from error analysis to debug models.
- Multimodal Lecture Presentations Dataset: Understanding Multimodality in Educational Slides12
The Multimodal Lecture Presentations dataset is introduced as a large-scale benchmark testing the capabilities of machine learning models in multimodal understanding of educational content and PolyViLT, a multi-instance learning loss that is more effective than current approaches is introduced.
- 11
- Face-to-Face Contrastive Learning for Social Intelligence Question-Answering11
Face-to-Face Contrastive Learning (F2F-CL), a graph neural network designed to model social interactions using factorization nodes to contextualize the multimodal face- to-face interaction along the boundaries of the speaking turn is proposed.
- 11
- MultiMed: Massively Multimodal and Multitask Medical Understanding10
The analysis highlights the advantages of training large-scale medical models across many related modalities and tasks and enables studies of generalization across related medical concepts, robustness to real-world noisy data and distribution shifts, and novel modality combinations to improve prediction performance.
- Dialogues with AI Reduce Beliefs in Misinformation but Build No Lasting Discernment Skills9
While AI assistance produced immediate improvements during AI-assisted sessions, participants’ unassisted performance on new items declined significantly by 15.3% in week 4 compared to week 0, indicating that while AI may help immediately, it may ultimately degrade long-term misinformation detection abilities.
- Human Behavior Atlas: Benchmarking Unified Psychological and Social Behavior Understanding9
It is shown that training on Human Behavior Atlas enables models to consistently outperform existing multimodal LLMs across diverse behavioral tasks; with the targeted use of behavioral descriptors yielding meaningful performance gains.
- 9
- Nano: Nested Human-in-the-Loop Reward Learning for Few-shot Language Model Control9
Nano, a few-shot human-in-the-loop training algorithm that continuously learns from human feedback, is proposed, which achieves state-of- the-art results on single topic/attribute as well as quantified distribution control compared to previous works.
- 9
- 8
- Benchmarking Vision, Language, & Action Models in Procedurally Generated, Open Ended Action Environments7
MultiNet v0.2 is introduced, a comprehensive benchmark designed to evaluate and analyze the generalization performance of state-of-the-art VLMs and VLAs on diverse procedural tasks from the Procgen benchmark, and identifies critical areas for improvement in their application to out-of-distribution digital tasks.
- Abstract 3D Perception for Spatial Intelligence in Vision-Language Models7
SandboxVLM is introduced, a simple yet effective framework that leverages abstract bounding boxes to encode geometric structure and physical kinematics for VLM and demonstrates that equipping VLMs with a 3D abstraction substantially enhances their 3D reasoning ability without additional training, suggesting new possibilities for general-purpose embodied intelligence.
- Comparative Knowledge Distillation7
A simple change - performing high-dimensional comparisons instead of low-dimensional relations, which is termed Comparative Knowledge Distillation - vaults performance well over existing KD approaches, and hypothesize that they hold promise for reduced dependency on teacher models because they can augment the effective dataset size without additional teacher calls.
- Brainish: Formalizing A Multimodal Language for Intelligence and Consciousness7
It is argued that such an inner language of multimodal artificial intelligence will be important for advances in machine models of intelligence and consciousness.
- 6
- Demystify the Gravity Well in the Optimization Landscape (Student Abstract)6
The ASR escape time lower bound is provided to understand the ASR gravity well theoretically and an explanation of why normal distribution with high variance can lead to suboptimal plateaus from an energy function point of view is provided.
- 6
- Foundations of Multisensory Artificial Intelligence5
A theoretical framework formalizing how modalities interact with each other to give rise to new information for a task is presented, and how future work can leverage these ideas toward more general, interactive, and safe multisensory AI is discussed.
- MultiViz: Towards User-Centric Visualizations and Interpretations of Multimodal Models5
It is shown that the complementary stages in MultiViz together enable users to simulate model predictions, assign interpretable concepts to features, perform error analysis on model misclassifications, and use insights from error analysis to debug models.
- HIINT: Historical, Intra- and Inter- personal Dynamics Modeling with Cross-person Memory Transformer5
A Cross-person Memory Transformer (CPM-T) framework which explicitly models intra- and inter- personal dependencies in multi-modal non-verbal cues, which outperforms baseline models in average F1-scores on three publicly available datasets for joint engagement, rapport, and human belief prediction tasks.
- 5
- 5
- OmniSapiens: A Foundation Model for Social Behavior Processing via Heterogeneity-Aware Relative Policy Optimization4
Omnisapiens-7B 2.0 is developed, a foundation model for social behavior processing that explicitly addresses learning from heterogeneous behavioral data that achieves the best and most consistent performance across 10 behavioral tasks, while also attaining the best performance on all five held-out benchmarks.
- 4
- Multimodal AI for Human Sensing and Interaction4
This course will cover the fundamental principles and challenges of multimodal AI, identify the right AI approaches for their problems, prototype basic hardware systems for efficient and robust sensing, be aware of real-world concerns around ethics, interpretability, and privacy, and appreciate the range of human-centered applications enabled by multimodal AI and sensing.
- MINT: Multimodal Instruction Tuning with Multimodal Interaction Grouping4
This work introduces MINT, a simple yet surprisingly effective task-grouping strategy based on the type of multimodal interaction that greatly outperforms existing task grouping baselines for multimodal instruction tuning, striking an effective balance between generalization and specialization.
- The Role of Computing Resources in Publishing Foundation Model Research4
It is found that increased computing is correlated with national funding allocations and citations, but the findings don't observe the strong correlations with research environment (academic or industrial), domain, or study methodology.
- 4
- 4
- Alice: Proactive Learning with Teacher's Demonstrations for Weak-to-Strong Generalization3
Alice is introduced, a framework that leverages complementary knowledge between teacher and student to enhance the learning process and significantly enhances the W2SG performance, yielding substantial improvements in three key tasks compared to the original W2SG.
- When Reasoning Meets Its Laws3
The Laws of Reasoning (LoRe), a unified framework that characterizes intrinsic reasoning patterns in LRMs, is presented, and an effective finetuning approach that enforces compute-law compositionality is developed.
- 3
- 3
- DENALI: A Dataset Enabling Non-Line-of-Sight Spatial Reasoning with Low-Cost LiDARs2
DENALI is presented, the first large-scale real-world dataset of space-time histograms from low-cost LiDARs capturing hidden objects, and it is shown that consumer LiDARs can enable accurate, data-driven NLOS perception.
- EasyVFX: Frequency-Driven Decoupling for Resource-Efficient VFX Generation2
EasyVFX is introduced, a resource-efficient framework that achieves realistic VFX synthesis under stringent constraints and proves that frequency-aware learning is a key catalyst for democratizing professional-grade VFX.
- 2
- 2
- 2
- Balancing Multimodal Training Through Game-Theoretic Regularization2
The Multimodal Competition Regu-larizer (MCR) is proposed, a new loss component inspired by mutual information (MI) decomposition designed to prevent the adverse effects of competition in multimodal training.
- 2
- From Competition to Collaborative Smelling: Navigating the Olfactory Gap in Human-AI Interaction1
Smell with Genji is introduced, an Artificial Intelligence (AI)-mediated olfactory experience that adapts the traditional Japanese olfactory game Genji-kō into a research probe for structured comparison and articulation.
- SynIB: Informational Bottleneck for Maximizing Synergy in Multimodal Learning1
To prioritize learning synergy, SynIB motivates the model to predict accurately from all modalities while penalizing confidence when information from any modality is withheld, and improves accuracy on synergy-dependent examples by up to 7.8% and overall accuracy by up to 3.8%.
- Smell with Genji: Rediscovering Human Perception through an Olfactory Game with AI1
Smell with Genji is presented, an artificial intelligence (AI)-mediated olfactory interaction system that reinterprets Genji-kō as a collaborative human-AI sensory experience, illustrating how sensing-enabled AI can participate in olfactory experience alongside users.
- SHALA-LLM: Smartly Handling Ambiguous Labels in Aligning LLMs1
Experiments on ambiguity-sensitive NLI and ER benchmarks, including ChaosNLI, GoEmotions, and MSP-Podcast, demonstrate that SHALA-LLM improves agreement with annotator label distributions, showing that modeling annotator disagreement can also strengthen classification performance.
- 1
- 1
- 1
- ClassMind: Scaling Classroom Observation and Instructional Feedback with Multimodal AI1
ClassMind, an AI-driven classroom observation system that integrates generative AI and multimodal learning to analyze classroom artifacts and deliver timely, personalized feedback aligned with pedagogical practices, illustrates how multimodal AI can scale expert coaching and advance teacher development.
- ORION: Teaching Language Models to Reason Efficiently in the Language of Thought1
Results show that Mentalese-style compressed reasoning offers a step toward human-like cognitive efficiency, enabling real-time, cost-effective reasoning without sacrificing accuracy.
- 1
- Anchor & Transform: Learning Sparse Representations of Discrete Objects1
A Bayesian nonparametric prior for embeddings that encourages sparsity and leverages natural groupings among objects is design that demonstrates stronger performance with fewer parameters as compared to existing compression baselines.
- Modeling Spatiotemporal Multimodal Language with Recurrent Multistage Fusion1
The Recurrent Multistage Fusion Network (RMFN) is proposed which decomposes the spatiotemporal fusion problem into multiple stages, each of them focused on a subset of multimodal signals for specialized, effective fusion.
- Continuous First, Discrete Later: VQ-VAEs Without Dimensional Collapse–
Theoretical analysis predicts downstream performance as a function of warm-up length, enabling an adaptive criterion for switching from AE Warm-up to VQ-VAE training and empirically characterize how warm-up duration governs the achievable final loss.
- Learning Ophthalmologist Clinical Reasoning for Glaucoma Diagnosis from Fundus Images–
A reasoning-driven vision-language framework that explicitly models the ophthalmologist's diagnostic workflow by generating structured clinical reasoning prior to diagnosis is developed, demonstrating that explicitly modeling expert clinical reasoning simultaneously improves interpretability and diagnostic performance.
- –
- –
- –
- CultureScore: Evaluating Cultural Faithfulness in Video Generation Models–
CultureScore is proposed, a compositional evaluation framework that decomposes cultural faithfulness into three granular dimensions: Identity (who is represented), Context (culturally localized background), and Behavior (normative gestures and interactions).
- Sentence to Scent: A Computational Pipeline Transforming NLP into Scent Sequence–
This work proposes narrative resonance as an alternative to optimizing chemical precision by treating olfaction as a chronological sequence rather than a static chemical representation, and leverages an LLM’s one-shot domain knowledge to compose everyday olfactory descriptions into a sequence of 12 scents feasible for hardware release.
- SCATR: Simple Calibrated Test-Time Ranking–
SCATR is a simple and efficient BoN ranking method that learns a lightweight scorer from a small calibration set using hidden representations from the base model, and offers a strong accuracy-efficiency trade-off for scalable test-time selection.
- –
- –
- –
- –
- –
- –
- –
- –
- –
Publication data from OpenAlex, with missing venues and authors filled in from Crossref; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-12. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.
Report an error
Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.