Is this you? Claim this profile to correct it, add a bio and choose the work people see first.
Claim this profileWorks155 from public data
- MMMU: A Massive Multi-Discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI2,743
The introduction of MMMU, a new benchmark designed to evaluate multimodal models on massive multi-discipline tasks demanding college-level subject knowledge and deliberate reasoning, will stimulate the community to build next-generation multimodal foundation models towards expert artificial general intelligence.
- 1,053
- FinQA: A Dataset of Numerical Reasoning over Financial Data890
A new large-scale dataset, FinQA, with Question-Answering pairs over Financial reports, written by financial experts, aiming to automate the analysis of a large corpus of financial documents, and demonstrates that popular, large, pre-trained models fall far short of expert humans in acquiring finance knowledge and in complex multi-step numerical reasoning on that knowledge.
- MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editing770
This work introduces MagicBrush, the first large-scale, manually annotated dataset for instruction-guided real image editing that covers diverse scenarios: single-turn, multi- turn, mask-provided, and mask-free editing, and fine-tune InstructPix2Pix on it and shows that the new model can produce much better images according to human evaluation.
- HybridQA: A Dataset of Multi-Hop Question Answering over Tabular and Textual Data452
HybridQA is presented, a new large-scale question-answering dataset that requires reasoning on heterogeneous information and can serve as a challenging benchmark to study question answering withheterogeneous information.
- Long-context LLMs Struggle with Long In-context Learning402
A benchmark for long in-context learning in extreme-label classification using six datasets with 28 to 174 classes and input lengths from 2K to 50K tokens is introduced and reveals that long context understanding and reasoning is still a challenging task for the existing LLMs.
- MERT: Acoustic Music Understanding Model with Large-Scale Self-supervised Training394
An acoustic Music undERstanding model with large-scale self-supervised Training (MERT), which incorporates teacher models to provide pseudo labels in the masked language modelling (MLM) style acoustic pre-training, which outperforms conventional speech and audio approaches in terms of performance.
- VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Tasks279
It is shown that VLMs are secretly strong embedding models that achieves an absolute average improvement of 10% to 20% over existing multimodal embedding models on both in-distribution and out-of-distribution datasets in MMEB.
- Re-Imagen: Retrieval-Augmented Text-to-Image Generator272
The Retrieval-Augmented Text-to-Image Generator (Re-Imagen), a generative model that uses retrieved information to produce high-fidelity and faithful images, even for rare or unseen entities, is presented.
- Video Captioning via Hierarchical Reinforcement Learning262
A novel hierarchical reinforcement learning framework for video captioning, where a high-level Manager module learns to design sub-goals and a low-level Worker module recognizes the primitive actions to fulfill the sub-goal.
- MANTIS: Interleaved Multi-Image Instruction Tuning257
This paper meticulously construct Mantis-Instruct containing 721K multi-image instruction data to train a family of Mantis models, which empowers Mantis with different multi-image skills like co-reference, comparison, reasoning, and temporal understanding, and shows its generalization ability.
- 216
- 215
- Logical Natural Language Generation from Open-Domain Tables187
A new NLG task where a model is tasked with generating natural language statements that can be logically entailed by the facts in an open-domain semi-structured table is suggested, and new automatic metrics to evaluate the fidelity of generation models w.r.t. logical inference are proposed.
- Explanations from Large Language Models Make Small Reasoners Better181
This paper considers the problem of leveraging the explanations generated by LLM to improve the training of small reasoners, which are more favorable in real-production deployment due to their low cost.
- No Metrics Are Perfect: Adversarial Reward Learning for Visual Storytelling165
Though automatic evaluation indicates slight performance boost over state-of-the-art (SOTA) methods in cloning expert behaviors, human evaluation shows that this approach achieves significant improvement in generating more human-like stories than SOTA systems.
- OmniEdit: Building Image Editing Generalist Models Through Specialist Supervision161
This paper presents \omniedit, which is an omnipotent editor to handle seven different image editing tasks with any aspect ratio seamlessly, and proposes a new editing architecture called EditNet to greatly boost the editing success rate.
- ConsistI2V: Enhancing Visual Consistency for Image-to-Video Generation159
This work proposes ConsistI2V, a diffusion-based method to enhance visual consistency for I2V generation that introduces spatiotemporal attention over the first frame to maintain spatial and motion consistency and noise initialization from the low-frequency band of the first frame to enhance layout consistency.
- LongRAG: Enhancing Retrieval-Augmented Generation with Long-context LLMs157
A new framework LongRAG, consisting of a `long retriever' and a `long reader' and a `long reader', is proposed, which processes the entire Wikipedia corpus into 4K-token units by grouping related documents by grouping related documents.
- Few-shot In-context Learning for Knowledge Base Question Answering156
This work proposes KB-BINDER, which for the first time enables few-shot in- context learning over KBQA tasks and can achieve a strong performance with only a few in-context demonstrations.
- Few-Shot NLG with Pre-Trained Language Model150
This work proposes the new task of few-shot natural language generation and proposes a simple yet effective approach that achieves very reasonable performances and outperforms the strongest baseline by an average of over 8.0 BLEU points improvement.
- MagicLens: Self-Supervised Image Retrieval with Open-Ended Instructions144
The core thesis of this paper is that text instructions can enable retrieving images with richer relations beyond visual similarity, and introduces MagicLens, a series of self-supervised image retrieval models that support open-ended instructions.
- Semantically Conditioned Dialog Response Generation via Hierarchical Disentangled Self-Attention134
A multi-layer hierarchical graph is exploited to build a hierarchical disentangled self-attention network, where each act is represented as a root-to-leaf route on the graph, and combinatorially many dialog act semantics can be modeled to control the neural response generation.
- 125
- 122
- Variational Knowledge Graph Reasoning121
This paper tackles a practical query answering task involving predicting the relation of a given entity pair and frames this prediction problem as an inference problem in a probabilistic graphical model and aims at resolving it from a variational inference perspective.
- Kosmos-G: Generating Images in Context with Multimodal Large Language Models114
Kosmos-G is presented, a model that leverages the advanced multimodal perception capabilities of Multimodal Large Language Models (MLLMs) to tackle the aforementioned challenge and demonstrates an impressive capability of zero-shot subject-driven generation with interleaved multi-image and text input.
- KGPT: Knowledge-Grounded Pre-Training for Data-to-Text Generation113
A knowledge-grounded pre-training (KGPT), which consists of two parts, 1) a general knowledge-grounded generation model to generate knowledge-enriched text and 2) a pre-training paradigm on a massive knowledge-grounded text corpus crawled from the web.
- 107
- Instruct-Imagen: Image Generation with Multi-modal Instruction98
Human evaluation on various image generation datasets re-veals that Instruct-Imagen matches or surpasses prior task-specific models in-domain and demonstrates promising generalization to unseen and more complex tasks.
- 97
- Generative Adversarial Zero-Shot Relational Learning for Knowledge Graphs95
A novel formulation of zero-shot learning is considered, which is model-agnostic that could be potentially applied to any version of KG embeddings, and consistently yields performance improvements on NELL and Wiki dataset.
- TIGERScore: Towards Building Explainable Metric for All Text Generation Tasks93
TIGERScore demonstrates the possibility of building universal explainable metrics to evaluate any text generation task and can even surpass the best existing reference-based metrics.
- Synthesizing Coherent Story with Auto-Regressive Latent Diffusion Models91
This work proposes AR-LDM, a latent diffusion model auto-regressively conditioned on history captions and generated images that can generalize to new characters through adaptation and extends the text-conditioned method to multimodal conditioning.
- MAP-Neo: Highly Capable and Transparent Bilingual Large Language Model Series89
The MAP-Neo is the first fully open-sourced bilingual LLM with comparable performance compared to existing state-of-the-art LLMs, and is hoped to enhance and strengthen the open research community and inspire more innovations and creativities to facilitate the further improvements of LLMs.
- ImagenHub: Standardizing the evaluation of conditional image generation models89
ImagenHub is proposed, which is a one-stop library to standardize the inference and evaluation of all the conditional image generation models and design two human evaluation scores, i.e. Semantic Consistency and Perceptual Quality, along with comprehensive guidelines to evaluate generated images.
- 88
- Meta Module Network for Compositional Visual Reasoning85
This work proposes Meta Module Network (MMN), a more powerful NMN architecture centered on a novel meta module, which can take in function recipes and morph into diverse instance modules dynamically, inheriting the strong explainability and compositionality of NMN.
- 81
- Violin: A Large-Scale Dataset for Video-and-Language Inference81
A new large-scale dataset, named Violin (VIdeO-and-Language INference), is introduced for this task, which consists of 95,322 video-hypothesis pairs from 15,887 video clips, spanning over 582 hours of video.
- Logic2Text: High-Fidelity Natural Language Generation from Logical Forms78
This work forms high-fidelity NLG as generation from logical forms in order to obtain controllable and faithful generations, and presents a new large-scale dataset, Logic2Text, with 10,753 descriptions involving common logic types paired with the underlying logical forms.
- 75
- Zero-shot Fact Verification by Claim Generation70
QACG, a framework for training a robust fact verification model by using automatically generated claims that can be supported, refuted, or unverifiable from evidence from Wikipedia, is developed.
- 69
- 67
- Critique Fine-Tuning: Learning to Critique is More Effective than Learning to Imitate64
Critique Fine-Tuning is proposed, a method more effective than SFT for reasoning tasks that trains models to critique noisy responses, inspired by human learning processes that emphasize critical thinking, deeper analysis, and nuanced understanding - traits often overlooked by standard SFT.
- T2V-Turbo-v2: Enhancing Video Generation Model Post-Training through Data, Reward, and Conditional Guidance Design59
This paper introduces a significant advancement by integrating various supervision signals, including high-quality training data, reward model feedback, and conditional guidance, into the consistency distillation process, and establishes a new state-of-the-art result on VBench, surpassing proprietary systems such as Gen-3 and Kling.
- Attacking Open-domain Question Answering by Injecting Misinformation53
The necessity of building a misinformation-aware QA system that integrates question-answering and misinformation detection in a joint fashion is discussed, with experiments showing that QA models are vulnerable to even small amounts of evidence contamination brought by misinformation.
- Interactive Natural Language Processing51
This paper provides a systematic classification of iNLP, dissecting its various components, including interactive objects, interaction interfaces, and interaction methods, and delves into the evaluation methodologies used in the field, exploring its diverse applications, and scrutinize its ethical and safety issues.
- TC-Bench: Benchmarking Temporal Compositionality in Text-to-Video and Image-to-Video Generation50
This study argues that generated videos should incorporate the emergence of new concepts and their relation transitions like in real-world videos as time progresses, and proposes TC-Bench, a benchmark of meticulously crafted text prompts, corresponding ground truth videos, and robust evaluation metrics.
- DreamEdit: Subject-driven Image Editing47
This work proposes two novel subject-driven sub-tasks, i.e., Subject Replacement and Subject Addition, and devise an innovative method DreamEditor to resolve these tasks by performing iterative generation, which enables a smooth adaptation to the customized subject.
- StructLM: Towards Building Generalist Models for Structured Knowledge Grounding46
This investigation reveals a notable deficiency in LLMs' ability to process structured data, e.g., ChatGPT lags behind state-of-the-art (SoTA) model by an average of 35% and suggests that structured knowledge grounding is still a challenging task and requires more innovative design to push to a new level.
- 43
- 43
- VAMBA: Understanding Hour-Long Videos with Hybrid Mamba-Transformers42
An orthogonal direction is explored to build a hybrid Mamba-Transformer model (VAMBA) that employs Mamba-2 blocks to encode video tokens with linear complexity that achieves at least 50 % reduction in GPU memory usage during training and inference, and nearly doubles the speed per training step compared to transformer-based LMMs.
- Augmenting Black-box LLMs with Medical Textbooks for Biomedical Question Answering42
This study presents a system called LLMs Augmented with Medical Textbooks (LLM-AMT), designed to enhance the proficiency of LLMs in specialized domains by integrating authoritative medical textbooks into the LLMs' framework using plug-and-play modules.
- Beyond Semantic Similarity: Rethinking Retrieval for Agentic Search via Direct Corpus Interaction41
Direct corpus interaction (DCI) is studied, where an agent searches the raw corpus directly with general-purpose terminal tools, without any embedding model, vector index, or retrieval API, and indicates that as language agents become stronger, retrieval quality depends not only on reasoning ability but also on the resolution of the interface through which the model interacts with the corpus.
- 40
- 40
- Reward Guided Latent Consistency Distillation39
Reward Guided LCD (RG-LCD), which integrates feedback from a reward model (RM) into the LCD process by augmenting the original LCD loss with the objective of maximizing the reward associated with LCM's single-step generation, is introduced.
- 37
- TUNA: Taming Unified Visual Representations for Native Unified Multimodal Models35
TUNA is presented, a native UMM that builds a unified continuous visual representation by cascading a VAE encoder with a representation encoder that avoids representation format mismatches introduced by separate encoders, outperforming decoupled alternatives in both understanding and generation.
- Triangular Architecture for Rare Language Translation35
Empirical results demonstrate that the proposed novel triangular training architecture (TA-NMT) significantly improves the translation quality of rare languages on MultiUN and IWSLT2012 datasets, and achieves even better performance combining back-translation methods.
- XL-NBT: A Cross-lingual Neural Belief Tracking Framework35
A cross-lingual state tracking framework is built that assumes that there exists a source language with dialog belief tracking annotations while the target languages have no annotated dialog data of any form, and discusses two types of common parallel resources: bilingual corpus and bilingual dictionary.
- Understanding Reasoning Ability of Language Models From the Perspective of Reasoning Paths Aggregation33
This work proposes that an LM can view an LM as deriving new conclusions by aggregating indirect reasoning paths seen at pre-training time, and formalizes the reasoning paths as random walk paths on the knowledge/reasoning graphs.
- 33
- How Large a Vocabulary Does Text Classification Need? A Variational Approach to Vocabulary Selection32
A more sophisticated variational vocabulary dropout (VVD) based on variational dropout to perform vocabulary selection, which can intelligently select the subset of the vocabulary to achieve the required performance.
- CodeEditorBench: Evaluating Code Editing Capability of Large Language Models31
Evaluation of 19 LLMs reveals that closed-source models, particularly Gemini-Ultra and GPT-4, outperform open-source models in CodeEditorBench, highlighting differences in model performance based on problem types and prompt sensitivities.
- 29
- 27
- 26
- 26
- Mining Algorithm Roadmap in Scientific Publications25
This work defines a new problem called mining algorithm roadmap in scientific publications, and proposes a new weakly supervised method to build the roadmap, and presents a proposed algorithm that shows its superiority over the baseline methods on the proposed task.
- 24
- 22
- Optical design and simulation of a compact multi-aperture camera based on a freeform microlens array22
- VideoEval-Pro: Robust and Realistic Long Video Understanding Evaluation21
VideoEval-Pro is proposed, a realistic LVU benchmark containing questions with open-ended short-answer, which truly require understanding the entire video, and shows that VideoEval-Pro offers a more realistic and reliable measure of long video understanding.
- 17
- 15
- 15
- 14
- 14
- 13
- 13
- 12
- UniRAG: Universal Retrieval Augmentation for Large Vision Language Models12
Evaluation results on the MSCOCO dataset with common entities show that both proprietary models like GPT-4o and Gemini-Pro and smaller open-source models like LLaVA, LaVIT, and Emu2 significantly enhance their generation quality when their input prompts are augmented with relevant information retrieved by Vision-Language retrievers like UniIR models.
- 12
- Task-adaptive Pre-training and Self-training are Complementary for Natural Language Understanding12
Experimental results show that TFS protocol can effectively utilize unlabeled data to achieve strong combined gains consistently across six datasets covering sentiment classification, paraphrase identification, natural language inference, named entity recognition and dialogue slot classification.
- 11
- 11
- QuickVideo: Real-Time Long Video Understanding with System Algorithm Co-Design10
QuickVideo is proposed, a system-algorithm co-design that substantially accelerates long-video understanding to support real-time downstream applications, enabling scalable, high-quality video understanding even on limited hardware.
- Improved dissipativity‐based analysis and synthesis of T‐S fuzzy systems with aperiodic memory sampled‐data controllers via an augmented looped‐functional9
The dissipativity‐based analysis and synthesis issues of Takagi‐Sugeno (T‐S) fuzzy systems by applying aperiodic memory sampling control by applying free‐matrix‐based integral inequalities consequently are investigated.
- Enhancing the Robustness of Prior Network in Out-of-Distribution Detection.9
A perturbed prior network architecture is proposed, which can efficiently separate model- level uncertainty from data-level uncertainty via prior entropy viaPrior entropy, and a concentration perturbation algorithm, which adaptively adds noise to concentration parameters so that the in- and out-of-distribution images are better separable.
- Bootstrap, Review, Decode: Using Out-of-Domain Textual Data to Improve Image Captioning.9
A novel way of using out-of-domain textual data to enhance the performance of existing image captioning systems is proposed and the bootstrap learning method can largely improve performance and help the model to generate more accurate and diverse captions.
- PixelWorld: How Far Are We from Perceiving Everything as Pixels?8
Experiments show that PEAP attains competitive accuracy on semantic-understanding tasks, indicating that a vision transformer can capture global textual semantics without explicit tokens, and P IXEL W ORLD serves as a practical benchmark for evaluating unified vision–language models and supports broader exploration of PEAP across diverse tasks.
- 8
- 8
- Critique-Coder: Enhancing Coder Models by Critique Reinforcement Learning7
Critique Reinforcement Learning (CRL) is proposed, where the model is tasked with generating a critique for a given (question, solution) pair and the reward is determined solely by whether the final judgment label of the generated critique aligns with the ground-truth judgment.
- 7
- 7
- Generative Bridging Network for Neural Sequence Prediction7
Three different GBNs, namely uniform GBN, language-model GBN and coaching GBN are proposed to penalize confidence, enhance language smoothness and relieve learning burden, show that they can yield significant improvements over strong baselines.
- PIN: A Knowledge-Intensive Dataset for Paired and Interleaved Multimodal Documents6
The PIN format is introduced, a novel data format designed to foster a deeper integration of visual and textual knowledge that provides the community with a versatile data format and substantial resources, offering a foundation for new research in pre-training strategies and the development of more powerful knowledge-intensive LMMs.
- 6
- 6
- 5
- Peer-to-Peer Transactive Computation–Electricity Trading for Interconnected Virtual Power Plant Buildings5
A peer-to-peer transactive computation–electricity trading framework for multiple-building virtual power plants (BVPPs) is proposed, in which the interconnected BVPPs can proactively trade their available computation–electricity with each other.
- 5
- 4
- 4
- Global Textual Relation Embedding for Relational Understanding4
This work investigates how to learn a general-purpose embedding of textual relations, defined as the shortest dependency path between entities, and creates the largest distant supervision dataset by linking the entire English ClueWeb09 corpus to Freebase.
- 4
- 3
- 3
- A Systematic Investigation of KB-Text Embedding Alignment at Scale3
This work conducts a large-scale, systematic investigation of aligning KB and text embeddings for joint reasoning, and demonstrates how such alignment can infuse textual information into KB embedDings for more accurate link prediction on emerging entities and events.
- Modeling Token-level Uncertainty to Learn Unknown Concepts in SLU via Calibrated Dirichlet Prior RNN3
- 2
- Interpreting and Improving Deep Neural SLU Models via Vocabulary Importance2
This paper studies the vocabulary importance using a novel Embedding Sparse Structure Learning (SparseEmb) approach, and utilizes SparseEmb to sanitize the training data based on the selected useless words as well as the model re-validation during training.
- Neural Sequence Prediction by Coaching.2
This paper introduces a hybrid-coach framework to combine these two existing algorithms so that the supervision from MLE can stabilize RL training while RL can incorporate task-level loss into MLE training.
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- Comparison and analysis of teaching quality assessment between students and teaching supervisors1
The scores from the students and teaching supervisors are showing a relationship of identical character that can reflect the real teaching quality of the teachers and provide reliable fact for supervising and assuring the teaching quality.
- –
- –
- –
- –
- DisProtEdit: Exploring Disentangled Representations for Multi-Attribute Protein Editing–
DisProtEdit is introduced, a controllable protein editing framework that leverages dual-channel natural language supervision to learn disentangled representations of structural and functional properties, and performs competitively with existing methods while providing improved interpretability and controllability.
- –
- –
- –
- –
- –
- –
- –
- –
- –
- Dynamic Equilibrium Management Model and Applied Study under the Restriction Condition on Wild Animals–
The theoretic model including the simultaneous equations of wild animal resources under economics system and biology system which be based of the common logistic differential equation and selected the musk deer resources to establish the applied model shows that the factual variable's coefficient and change direction is not as same as the theoretic assumptions.
- –
- –
- Application of Foreign Teaching Methods in Vocational Writing Courses–
Based on the analysis of Freire's Autonomous Learning, Wagenschein's Example Teaching, and Skinner's Programmed Instruction theories, this paper proposes that these three theories should be integrated in vocational writing courses.
- –
- On the Models of Catastrophic Disease Insurance in China–
In- depth analysis and research have been conducted on the coverage, financing, operation as well as supervision of catastrophic disease insurance for urban and rural residents in China.
- Comparison and Optimizing among Comprehensive Evaluation Methods for the Medical Quality–
RSR, prehensive index, AHP and TOPSIS methods are used to evaluate the medical quality for the object, and the results of every evaluation method are summarized through comparing pre- with post-evaluation results.
- –
- –
- –
- –
- –
- –
Publication data from OpenAlex, with missing venues and authors filled in from Crossref; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-15. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.
Report an error
Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.