Is this you? Claim this profile to correct it, add a bio and choose the work people see first.
Claim this profileWorks146 from public data
- Multimodal Transformer for Unaligned Multimodal Language Sequences2,529
Comprehensive experiments on both aligned and non-aligned multimodal time-series show that the MulT model outperforms state-of-the-art methods by a large margin, and empirical analysis suggests that correlated crossmodal signals are able to be captured by the proposed cross modal attention mechanism in MulT.
- Multimodal Language Analysis in the Wild: CMU-MOSEI Dataset and Interpretable Dynamic Fusion Graph1,810
This paper introduces CMU Multimodal Opinion Sentiment and Emotion Intensity (CMU-MOSEI), the largest dataset of sentiment analysis and emotion recognition to date and uses a novel multimodal fusion technique called the Dynamic Fusion Graph (DFG), which is highly interpretable and achieves competative performance when compared to the previous state of the art.
- Efficient Low-rank Multimodal Fusion With Modality-Specific Factors1,109
The Low-rank Multimodal Fusion method is proposed, which performs multimodal fusion using low-rank tensors to improve efficiency and is indeed much more efficient in both training and inference compared to other methods that utilize tensor representations.
- Memory Fusion Network for Multi-view Sequential Learning992
A new neural architecture for multi-view sequential learning called the Memory Fusion Network (MFN) is presented that explicitly accounts for both interactions in a neural architecture and continuously models them through time.
- Found in Translation: Learning Robust Joint Representations by Cyclic Translations between Modalities597
This paper proposes a method to learn robust joint representations by translating between modalities based on the key insight that translation from a source to a target modality provides a method of learning joint representations using only the source modality as input.
- Multi-attention Recurrent Network for Human Communication Comprehension591
The main strength of the model comes from discovering interactions between modalities through time using a neural component called the Multi-attention Block (MAB) and storing them in the hybrid memory of a recurrent part called the Long-short Term Hybrid Memory (LSTHM).
- 553
- Words Can Shift: Dynamically Adjusting Word Representations Using Nonverbal Behaviors494
The Recurrent Attended Variation Embedding Network (RAVEN) is proposed that models the fine-grained structure of nonverbal subword sequences and dynamically shifts word representations based on nonverbal cues to capture the dynamic nature of non verbal intents.
- OS-ATLAS: A Foundation Action Model for Generalist GUI Agents393
OS-Atlas is developed - a foundational GUI action model that excels at GUI grounding and OOD agentic tasks through innovations in both data and modeling and is releasing the largest open-source cross-platform GUI grounding corpus to date, which contains over 13 million GUI elements.
- Multimodal sentiment analysis with word-level fusion and reinforcement learning327
The Gated Multimodal Embedding LSTM with Temporal Attention model is proposed that is composed of 2 modules and able to perform modality fusion at the word level and is able to better model the multimodal structure of speech through time and perform better sentiment comprehension.
- Foundations & Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questions321
An overview of the computational and theoretical foundations of multimodal machine learning is provided, defining three key principles of modality heterogeneity, connections, and interactions that have driven subsequent innovations, and proposing a taxonomy of six core technical challenges covering historical and recent trends.
- Towards Debiasing Sentence Representations298
It is shown that Sent-Debias is effective in removing biases, and at the same time, preserves performance on sentence-level downstream tasks such as sentiment analysis, linguistic acceptability, and natural language understanding.
- Rethinking Architecture Design for Tackling Data Heterogeneity in Federated Learning244
This paper demonstrates that self-attention-based architectures (e.g., Transformers) are more robust to distribution shifts and hence improve federated learning over heterogeneous data and releases code and pretrained models to encourage future exploration in robust architectures as an alternative to current research efforts on the optimization front.
- 232
- MEM1: Learning to Synergize Memory and Reasoning for Efficient Long-Horizon Agents231
MEM1, an end-to-end reinforcement learning framework that enables agents to operate with constant memory across long multi-turn tasks, is introduced, an end-to-end reinforcement learning framework that enables agents to operate with constant memory across long multi-turn tasks.
- Multimodal Language Analysis with Recurrent Multistage Fusion228
The Recurrent Multistage Fusion Network (RMFN) is proposed which decomposes the fusion problem into multiple stages, each of them focused on a subset of multimodal signals for specialized, effective fusion.
- 194
- 158
- Deep Gamblers: Learning to Abstain with Portfolio Theory143
Inspired by portfolio theory, a loss function for the selective classification problem based on the doubling rate of gambling is proposed, which allows us to train neural networks and characterize the disconfidence of prediction in an end-to-end fashion.
- Factorized Contrastive Learning: Going Beyond Multi-view Redundancy139
FactorCL is a new multimodal representation learning method to go beyond multi-view redundancy and captures both shared and unique information and achieves state-of-the-art results on six benchmarks.
- Diverse and Admissible Trajectory Forecasting Through Multimodal Context Understanding137
A model that fully synthesizes multiple input signals from the multimodal world, the environment's scene context and interactions between multiple surrounding agents, to best model all diverse and admissible trajectories is proposed.
- Medical Hallucination in Foundation Models and Their Impact on Healthcare136
The results reveal that inference techniques such as Chain-of-Thought (CoT) and Search Augmented Generation can effectively reduce hallucination rates, however, despite these improvements, non-trivial levels of hallucination persist.
- 135
- Foundations and Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questions121
This paper proposes a taxonomy of 6 core technical challenges: representation, alignment, reasoning, generation, transference, and quantification covering historical and recent trends, and defines two key principles of modality heterogeneity and interconnections that have driven subsequent innovations.
- 121
Publication data from OpenAlex, with missing venues and authors filled in from Crossref; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-12. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.
Report an error
Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.