Is this you? Claim this profile to correct it, add a bio and choose the work people see first.
Claim this profileAcademic lineage
Possible advisorsa guess from early papers, not confirmed
- Kai-Fu LeeSuggested from co-authorship
- Changning HuangSuggested from co-authorship
Is this you? Claim this profile to confirm or dismiss it.
Works414 from public data
- Embedding Entities and Relations for Learning and Inference in Knowledge Bases3,965
It is found that embeddings learned from the bilinear objective are particularly good at capturing relational semantics and that the composition of relations is characterized by matrix multiplication.
- MS MARCO: A Human Generated MAchine Reading COmprehension Dataset.3,691
This new dataset is aimed to overcome a number of well-known weaknesses of previous publicly available datasets for the same task of reading comprehension and question answering, and is the most comprehensive real-world dataset of its kind in both quantity and quality.
- PIQA: Reasoning about Physical Commonsense in Natural Language3,673
The task of physical commonsense reasoning and a corresponding benchmark dataset Physical Interaction: Question Answering or PIQA are introduced and analysis about the dimensions of knowledge that existing models lack are provided, which offers significant opportunities for future research.
- A Diversity-Promoting Objective Function for Neural Conversation Models2,879
This work proposes using Maximum Mutual Information (MMI) as the objective function in neural models, and demonstrates that the proposed MMI models produce more diverse, interesting, and appropriate responses, yielding substantive gains in BLEU scores on two conversational datasets and in human evaluations.
- Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing2,830
It is shown that for domains with abundant unlabeled text, such as biomedicine, pretraining language models from scratch results in substantial gains over continual pretraining of general-domain language models.
- DIALOGPT : Large-Scale Generative Pre-training for Conversational Response Generation1,771
It is shown that conversational systems that leverage DialoGPT generate more relevant, contentful and context-consistent responses than strong baseline systems.
- Deep Learning--based Text Classification1,537
A comprehensive review of more than 150 deep learning--based models for text classification developed in recent years is provided, and their technical contributions, similarities, and strengths are discussed.
- Deep Reinforcement Learning for Dialogue Generation1,434
This work simulates dialogues between two virtual agents, using policy gradient methods to reward sequences that display three useful conversational properties: informativity, non-repetitive turns, coherence, and ease of answering.
- Multi-Task Deep Neural Networks for Natural Language Understanding1,376
A Multi-Task Deep Neural Network (MT-DNN) for learning representations across multiple natural language understanding (NLU) tasks that allows domain adaptation with substantially fewer in-domain labels than the pre-trained BERT representations.
- 1,188
- Large Language Models: A Survey1,136
This paper reviews some of the most prominent LLMs, including three popular LLM families (GPT, LLaMA, PaLM), and discusses their characteristics, contributions and limitations, and gives an overview of techniques developed to build, and augment LLMs.
- A Neural Network Approach to Context-Sensitive Generation of Conversational Responses915
A neural network architecture is used to address sparsity issues that arise when integrating contextual information into classic statistical models, allowing the system to take into account previous dialog utterances.
- 906
- 889
- Semantic Parsing via Staged Query Graph Generation: Question Answering with Knowledge Base847
This work proposes a novel semantic parsing framework for question answering using a knowledge base that leverages the knowledge base in an early stage to prune the search space and thus simplifies the semantic matching problem.
- The Design and Implementation of XiaoIce, an Empathetic Social Chatbot777
This article describes the development of Microsoft XiaoIce, the most popular social chatbot in the world, and shows how XiaoIce dynamically recognizes human feelings and states, understands user intent, and responds to user needs throughout long conversations.
- Neural Approaches to Conversational AI767
This tutorial surveys neural approaches to conversational AI that were developed in the last few years, and presents a review of state-of-the-art neural approaches, drawing the connection between neural approaches and traditional symbolic approaches.
- Learning semantic representations using convolutional neural networks for web search730
This paper presents a series of new latent semantic models based on a convolutional neural network to learn low-dimensional semantic vectors for search queries and Web documents that significantly outperforms other se-mantic models in retrieval performance.
- A Latent Semantic Model with Convolutional-Pooling Structure for Information Retrieval728
A new latent semantic model that incorporates a convolutional-pooling structure over word sequences to learn low-dimensional, semantic vector representations for search queries and Web documents is proposed.
- MIND: A Large-scale Dataset for News Recommendation718
This paper presents a large-scale dataset named MIND, constructed from the user click logs of Microsoft News, which contains 1 million users and more than 160k English news articles, each of which has rich textual content such as title, abstract and body.
- Scalable training of L 1 -regularized log-linear models671
This work presents an algorithm Orthant-Wise Limited-memory Quasi-Newton (OWL-QN), based on L-BFGS, that can efficiently optimize the L1-regularized log-likelihood of log-linear models with millions of parameters.
- A Knowledge-Grounded Neural Conversation Model595
This paper presents a novel, fully data-driven, and knowledge-grounded neural conversation model aimed at producing more contentful responses, generalizing the widely-used Sequence-to-Sequence (Seq2Seq) approach by conditioning responses on both conversation history and external “facts”, allowing the model to be versatile and applicable in an open-domain setting.
- Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedback552
A LLM-Augmenter system, which augments a black-box LLM with a set of plug-and-play modules to significantly reduces ChatGPT's hallucinations without sacrificing the fluency and informativeness of its responses.
- Multi-Domain Joint Semantic Frame Parsing Using Bi-Directional RNN-LSTM490
Experimental results show the power of a holistic multi-domain, multi-task modeling approach to estimate complete semantic frames for all user utterances addressed to a conversational system over alternative methods based on single domain/task deep learning.
- Cyclical Annealing Schedule: A Simple Approach to Mitigating KL Vanishing473
A cyclical annealing schedule is proposed, which simply repeats the process of increasing \beta multiple times, and allows to learn more meaningful latent codes progressively by leveraging the results of previous learning cycles as warm re-restart.
- 470
- Representation Learning Using Multi-Task Deep Neural Networks for Semantic Classification and Information Retrieval443
This work develops a multi-task DNN for learning representations across multiple tasks, not only leveraging large amounts of cross-task data, but also benefiting from a regularization effect that leads to more general representations to help tasks in new domains.
- 386
- 381
- Challenges in Building Intelligent Open-domain Dialog Systems359
This article reviews the recent work on neural approaches that are devoted to addressing three challenges in developing intelligent open-domain dialog systems: semantics, consistency, and interactiveness.
- Generation-Augmented Retrieval for Open-Domain Question Answering334
It is shown that generating diverse contexts for a query is beneficial as fusing their results consistently yields better retrieval accuracy, and as sparse and dense representations are often complementary, GAR can be easily combined with DPR to achieve even better performance.
- 332
- Understanding the Difficulty of Training Transformers328
It is revealed that for each layer in a multi-layer Transformer model, heavy dependency on its residual branch makes training unstable since it amplifies small parameter perturbations and result in significant disturbances in the model output, yet a light dependency limits the potential of model training and can lead to an inferior trained model.
- Towards End-to-End Reinforcement Learning of Dialogue Agents for Information Access312
This paper proposes KB-InfoBot - a multi-turn dialogue agent which helps users search Knowledge Bases without composing complicated queries by replacing symbolic queries with an induced “soft” posterior distribution over the KB that indicates which entities the user is interested in.
- ReasoNet309
A novel neural network architecture called the Reasoning Network (ReasoNet) for machine comprehension tasks, which makes use of multiple turns to effectively exploit and then reason over the relation among queries, documents, and answers.
- Dependence language model for information retrieval288
The linkage of a query is integrated as a hidden variable, which expresses the term dependencies within the query as an acyclic, planar, undirected graph, which extends the basic language modeling approach based on unigram by relaxing the independence assumption.
- Deep Reinforcement Learning with a Natural Language Action Space271
This paper introduces a novel architecture for reinforcement learning with deep neural networks designed to handle state and action spaces characterized by natural language, as found in text-based games, called a deep reinforcement relevance network (DRRN).
- 254
- 232
- Chinese Word Segmentation and Named Entity Recognition: A Pragmatic Approach231
A pragmatic mathematical framework in which segmenting known words and detecting unknown words of different types can be performed simultaneously in a unified way is proposed and implemented in an adaptive Chinese word segmenter, called MSRSeg, which is described in detail.
- S oloist : BuildingTask Bots at Scale with Transfer Learning and Machine Teaching230
A new method that uses transfer learning and machine teaching to build task bots at scale, Soloist, is presented, which parameterize classical modular task-oriented dialog systems using a Transformer-based auto-regressive language model, which subsumes different dialog modules into a single neural model.
- Few-shot Natural Language Generation for Task-Oriented Dialog213
FewshotWOZ is presented, the first NLG benchmark to simulate the few-shot learning setting in task-oriented dialog systems, and the proposed SC-GPT model significantly outperforms existing methods, measured by various automatic metrics and human evaluations.
- Optimus: Organizing Sentences via Pre-trained Modeling of a Latent Space205
This paper proposes the first large-scale language VAE model, Optimus, a universal latent embedding space for sentences that is first pre-trained on large text corpus, and then fine-tuned for various language generation and understanding tasks.
- Fine-tuning large neural language models for biomedical natural language processing203
Overall, domain-specific vocabulary and pretraining facilitate robust models for fine-tuning and establish a new state of the art on a wide range of biomedical NLP applications.
- Stochastic Answer Networks for Machine Reading Comprehension203
This work proposes a simple yet robust stochastic answer network (SAN) that simulates multi-step reasoning in machine reading comprehension that achieves results competitive to the state-of-the-art on the Stanford Question Answering Dataset, the Adversarial SQuAD, and the Microsoft MAchine Reading COmprehensionDataset.
- 200
- Composite Task-Completion Dialogue Policy Learning via Hierarchical Deep Reinforcement Learning197
This paper addresses the travel planning task by formulating the task in the mathematical framework of options over Markov Decision Processes (MDPs), and proposing a hierarchical deep reinforcement learning approach to learning a dialogue manager that operates at different temporal scales.
- Few-Shot Generative Conversational Query Rewriting193
This paper develops two methods, based on rules and self-supervised learning, to generate weak supervision data using large amounts of ad hoc search sessions, and to fine-tune GPT-2 to rewrite conversational queries.
- 191
- PlotMachines: Outline-Conditioned Generation with Dynamic Plot State Tracking190
PlotMachines is presented, a neural narrative model that learns to transform an outline into a coherent story by tracking the dynamic plot states, and is enriched with high-level discourse structure so that the model can learn different styles of writing corresponding to different parts of the narrative.
- 188
- How Much Do Language Models Copy From Their Training Data? Evaluating Linguistic Novelty in Text Generation Using RAVEN183
RAVEN, a suite of analyses for assessing the novelty of generated text, focusing on sequential structure (n-grams) and syntactic structure, is introduced, finding evidence that GPT-2 uses both compositional and analogical generalization mechanisms and showing that its novel text is usually well-formed morphologically and syntactically but has reasonably frequent semantic issues.
- Deep Dyna-Q: Integrating Planning for Task-Completion Dialogue Policy Learning183
Deep Dyna-Q is presented, which to the authors' knowledge is the first deep RL framework that integrates planning for task-completion dialogue policy learning and incorporates into the dialogue agent a model of the environment, referred to as the world model, to mimic real user response and generate simulated experience.
- Modeling Interestingness with Deep Neural Networks181
The results on large-scale, real-world datasets show that the semantics of documents are important for modeling interest-ingness and that the DSSM leads to significant quality improvement on both tasks, outperforming not only the classic document models that do not use semantics but also state-of-the-art topic models.
- 175
- 174
- Using Contextual Speller Techniques and Language Modeling for ESL Error Correction173
A decisiontree approach inspired by contextual spelling systems for detection and correction suggestions, and a large language model trained on the Gigaword corpus to provide additional information to filter out spurious suggestions are used.
- deltaBLEU: A Discriminative Metric for Generation Tasks with Intrinsically Diverse Targets159
In tasks involving generation of conversational responses, ∆BLEU correlates reasonably with human judgments and outperforms sentence-level and IBM BLEU in terms of both Spearman's ρ and Kendall’s τ.
- 158
- Rethinking Interpretability in the Era of Large Language Models155
It is argued that, despite their limitations, LLMs hold the opportunity to redefine interpretability with a more ambitious scope across many applications, including in auditing LLMs themselves.
- HittER: Hierarchical Transformers for Knowledge Graph Embeddings155
This paper proposes HittER, a Hierarchical Transformer model to jointly learn Entity-relation composition and Relational contextualization based on a source entity’s neighborhood and designs a masked entity prediction task to balance information from the relational context and the source entity itself.
- Improving query translation for cross-language information retrieval using statistical models155
Several methods to improve the previous dictionary-based query translation of English-Chinese CLIR collection result in significant improvements over the simple dictionary approaches, and achieve even better performance than a high-quality machine translation system.
- Generating Informative and Diverse Conversational Responses via Adversarial Information Maximization154
- End-to-End Memory Networks with Knowledge Carryover for Multi-Turn Spoken Language Understanding148
The experiments on Microsoft Cortana conversational data show that the proposed memory network architecture can effectively extract salient semantics for modeling knowledge carryover in the multi-turn conversations and outperform the results using the state-of-the-art recurrent neural network framework (RNN) designed for single-turn SLU.
- 148
- Toward a unified approach to statistical language modeling for Chinese147
This article presents a unified approach to Chinese statistical language modeling, which automatically and consistently gathers a high-quality training data set from the Web, creates ahigh-quality lexicon, segments the training data using this Lexicon, and compresses the language model by using the maximum likelihood principle, which is consistent with trigram model training.
- 146
- Robust Navigation with Language Pretraining and Stochastic Sampling133
This paper adapts large-scale pretrained language models to learn text representations that generalize better to previously unseen instructions and proposes a stochastic sampling scheme to reduce the considerable gap between the expert actions in training and sampled actions in test.
- A Large Scale Ranker-Based System for Search Query Spelling Correction133
The noisy channel model is subsumed by a more general ranker, which allows a variety of features to be easily incorporated and a distributed infrastructure is proposed for training and applying Web scale n-gram language models.
- 127
- SOLOIST: Few-shot Task-Oriented Dialog with A Single Pre-trained Auto-regressive Model127
A new method SOLOIST is presented, which uses transfer learning to efficiently build task-oriented dialog systems at scale using a Transformer-based auto-regressive language model, which subsumes different dialog modules into a single neural model.
- 122
- ConvLab-2: An Open-Source Toolkit for Building, Evaluating, and Diagnosing Dialogue Systems121
ConvLab-2, an open-source toolkit that enables researchers to build task-oriented dialogue systems with state-of-the-art models, perform an end-to-end evaluation, and diagnose the weakness of systems is presented.
- ConvLab: Multi-Domain End-to-End Dialog System Platform117
ConvLab is presented, an open-source multi-domain end-to-end dialog system platform that enables researchers to quickly set up experiments with reusable components and compare a large set of different approaches, ranging from conventional pipeline systems to end- to-end neural models, in common environments.
- A Nested Attention Neural Hybrid Model for Grammatical Error Correction117
Experiments show that the new model can effectively correct errors of both types by incorporating word and character-level information, and that the model significantly outperforms previous neural models for GEC as measured on the standard CoNLL-14 benchmark dataset.
- 116
- GO FIGURE: A Meta Evaluation of Factuality in Summarization115
GO FIGURE, a meta-evaluation framework for evaluating factuality evaluation metrics, is introduced and it is revealed that while QA metrics generally improve over standard metrics that measure factuality across domains, performance is highly dependent on the way in which questions are generated.
- Conversing by Reading: Contentful Neural Conversation with On-demand Machine Reading111
A new end-to-end approach to contentful neural conversation that jointly models response generation and on-demand machine reading is presented, allowing for more focused integration of external knowledge than has been possible in prior approaches.
- Multi-step Reasoning via Recurrent Dual Attention for Visual Dialog111
This paper presents a new model for visual dialog, Recurrent Dual Attention Network (ReDAN), using multi-step reasoning to answer a series of questions about an image, and demonstrates that ReDAN can locate context-relevant visual and textual clues via iterative refinement, which can lead to the correct answer step-by-step.
- Few-Shot Named Entity Recognition: An Empirical Baseline Study110
Three orthogonal schemes to improve model generalization ability in few-shot settings are investigated: meta-learning to construct prototypes for different entity types, task-specific supervised pre-training on noisy web data to extract entity-related representations and self-training to leverage unlabeled in-domain data.
- A comparison of Bayesian estimators for unsupervised Hidden Markov Model POS taggers108
This paper compares a variety of different Bayesian estimators for Hidden Markov Model POS taggers with various numbers of hidden states on data sets of different sizes and finds that Variational Bayes was the fastest of all the estimators, especially on large data sets, and that explicit Gibbs sampler were generally faster than their collapsed counterparts on largeData sets.
- An information-theoretic approach to automatic evaluation of summaries106
This paper introduces an information-theoretic approach to automatic evaluation of summaries based on the Jensen-Shannon divergence of distributions between an automatic summary and a set of reference summaries and results indicate that JS divergence-based evaluation method achieves comparable performance with the common automatic evaluation method ROUGE in single documents summarization task.
- Linear discriminant model for information retrieval105
Results show that in most test sets, LDM significantly outperforms the state-of-the-art language modeling approaches and the classical probabilistic retrieval model and it is more appropriate to train LDM using a measure of AP rather than likelihood if the IR system is graded on AP.
- Chinese named entity identification using class-based language model105
In this research, word segmentation and NE identification have been integrated into a unified framework that consists of several class-based language models and a hierarchical structure is adopted for one of the LMs so that the nested entities in organization names can be identified.
- Smoothing clickthrough data for web search ranking104
Two smoothing methods to expand clickthrough data are presented: query clustering via Random Walk on click graphs and a discounting method inspired by the Good-Turing estimator, which consistently outperform those trained on unsmoothed features.
- Clickthrough-based translation models for web search103
This paper provides a quantitative analysis of the language discrepancy issue, and explores the use of clickthrough data to bridge documents and queries, and demonstrates that standard statistical machine translation techniques can be adapted for building a better Web document retrieval system.
- Neural Approaches to Conversational Information Retrieval99
This book surveys recent advances in CIR, focusing on neural approaches that have been developed in the last few years, based on the authors' tutorial at SIGIR'2020 (Gao et al., 2020b).
- Clickthrough-based latent semantic models for web search98
Two new document ranking models for Web search based upon the methods of semantic representation and the statistical translation-based approach to information retrieval (IR) are presented.
- A Hybrid Retrieval-Generation Neural Conversation Model94
This paper proposes a hybrid neural conversation model that combines the merits of both response retrieval and generation methods and shows that the proposed model outperforms both retrieval- based methods and generation-based methods under both automatic evaluation metrics and human evaluation.
- Resolving query translation ambiguity using a decaying co-occurrence model and syntactic dependence relations94
The basic co-occurrence model is extended by adding a decaying factor that decreases the mutual information when the distance between the terms increases, and a triple translation model is incorporated, in which syntactic dependence relations (represented as triples) are integrated.
- A Controllable Model of Grounded Response Generation90
Quantitative and qualitative results show that, using this framework, a transformer based model with a novel inductive attention mechanism, trained on a conversation-like Reddit dataset, outperforms strong generation baselines.
- Indirect-HMM-based hypothesis alignment for combining outputs from machine translation systems86
The proposed indirect hidden Markov model (IHMM) is proposed to address the synonym matching and word ordering issues in hypothesis alignment and achieved the best Chinese-to-English translation result in the constrained training track of the 2008 NIST Open MT Evaluation.
- Self-Verification Improves Few-Shot Clinical Information Extraction85
This work explores a general mitigation framework using self-verification, which leverages the LLM to provide provenance for its own extraction and check its own outputs and consistently improves accuracy for various LLMs in standard clinical information extraction tasks.
- End-to-end joint learning of natural language understanding and dialogue manager85
Experiments show that the proposed model significantly outperforms the state-of-the-art pipeline models for both NLU and SAP, which indicates that the joint model is capable of mitigating the affects of noisy NLU outputs, and NLU model can be refined by error flows backpropagating from the extra supervised signals of system actions.
- Automatic Acquisition of Chinese–English Parallel Corpus from the Web85
A system that overcomes this limitation by automatically collecting high quality parallel bilingual corpora from the web by use of a k-nearest-neighbors classifier with multiple features, which is a significant improvement over earlier work.
- Augmenting interpretable models with large language models during training84
Two instantiations of Aug-imodels in natural-language processing are explored: Aug-Linear, which augments a linear model with decoupled embeddings from an LLM and Aug-Tree, which augmented a decision tree with LLM feature expansions, which outperform their non-augmented, interpretable counterparts.
- Discourse-Aware Neural Rewards for Coherent Text Generation84
Empirical results demonstrate that a generator trained with the learned reward produces more coherent and less repetitive text than models trained with cross-entropy or with reinforcement learning with commonly used scores as rewards.
- Jointly Optimizing Diversity and Relevance in Neural Response Generation83
A SpaceFusion model to jointly optimize diversity and relevance that essentially fuses the latent space of a sequence-to-sequence model and that of an autoencoder model by leveraging novel regularization terms is proposed.
- 82
- Segment and Recognize Anything at Any Granularity81
This work introduces Semantic-SAM, an augmented image segmentation foundation for segmenting and recognizing anything at desired granularities, and is the first attempt to jointly train a model on SA-1B, instance-level, and part-level segmentation datasets.
- 81
- 81
- A DOM tree alignment model for mining parallel data from the web81
Compared with previous mining schemes, the benchmarks show that this new mining scheme improves the mining coverage, reduces mining bandwidth, and enhances the quality of mined parallel sentences.
- GODEL: Large-Scale Pre-Training for Goal-Directed Dialog80
GODEL (Grounded Open Dialogue Language Model) is introduced, a large pre-trained language model for dialog that leverages a new phase of grounded pre-training designed to better support adapting GODEL to a wide range of downstream dialog tasks that require information external to the current conversation to produce good responses.
- 76
- Adversarial Advantage Actor-Critic Model for Task-Completion Dialogue Policy Learning76
This paper presents a new method - adversarial advantage actor-critic (Adversarial A2C), which significantly improves the efficiency of dialogue policy learning in task-completion dialogue systems.
- Exploring web scale language models for search query processing75
An extensive study on the language model properties of search queries and the three text streams associated with each web document shows that queries seem to be composed in a way most similar to how authors summarize documents in anchor texts or titles, offering a quantitative explanation to the observations in past work.
- Deep Reinforcement Learning with a Combinatorial Action Space for Predicting Popular Reddit Threads74
The proposed model, which represents dependence between sub-actions through a bi-directional LSTM, gives the best performance across different experimental configurations and domains, and it also generalizes well with varying numbers of recommendation requests.
- Overview of the Ninth Dialog System Technology Challenge: DSTC973
The task definition is described, provided datasets, baselines, and evaluation setup for each track, and the results of the submitted systems are summarized to highlight the general trends of the state-of-the-art technologies for the tasks.
- 73
- 73
- 72
- Bayesian semi-supervised Chinese word segmentation for statistical machine translation72
A Bayesian semi-supervised Chinese word segmentation model which uses both monolingual and bilingual information to derive a segmentation suitable for MT is proposed and improves a state-of-the-art MT system in a small and a large data environment.
- Implicit Deep Latent Variable Models for Text Generation69
An LVM to directly match the aggregated posterior to the prior is developed, which can be viewed as a natural extension of VAEs with a regularization of maximizing mutual information, mitigating the “posterior collapse” issue.
- Discriminative Deep Dyna-Q: Robust Planning for Dialogue Policy Learning69
Experiments show that D3Q significantly outperforms DDQ by controlling the quality of simulated experience used for planning and is further demonstrated in a domain extension setting, where the agent’s capability of adapting to a changing environment is tested.
- Improved source-channel models for Chinese word segmentation69
A Chinese word segmentation system that uses improved source-channel models of Chinese sentence generation and provides a unified approach to the four fundamental features of word-level Chinese language processing.
- 68
- ExACT: Teaching AI Agents to Explore with Reflective-MCTS and Exploratory Learning67
ExACT, an approach to combine test-time search and self-learning to build o1-like models for agentic applications, is presented, suggesting a promising research direction to enhance VLMs' capabilities for agentic applications via test-time search and self-learning.
- UnitedQA: A Hybrid Approach for Open Domain Question Answering67
It is demonstrated that a simple hybrid approach by combining answers from both readers can efficiently take advantages of extractive and generative answer inference strategies and outperforms single models as well as homogeneous ensembles.
- Adversarial Domain Adaptation for Machine Reading Comprehension67
An Adversarial Domain Adaptation framework for Machine Reading Comprehension (MRC), where pseudo questions are first generated for unlabeled passages in the target domain, and then a domain classifier is incorporated into an MRC model to predict which domain a given passage-question pair comes from.
- Human Parity on CommonsenseQA: Augmenting Self-Attention with External Attention66
It is found that the proposed external attention mechanism can significantly improve the performance of existing AI systems, allowing practitioners to easily customize foundation AI models to many diverse downstream applications.
- 66
- 64
- A Web-based English Proofing System for English as a Second Language Users.64
An algorithm that relies on web frequency counts to identify and correct writing errors made by non-native writers of English, suggesting that a web-based approach should be combined with local linguistic resources to achieve both effectiveness and efficiency.
- Knowledge-Grounded Dialogue Generation with a Unified Knowledge Representation63
PLUG is a language model that homogenizes different knowledge sources to a unified knowledge representation for knowledge-grounded dialogue generation tasks and achieves comparable performance with state-of-the-art methods in the fully-supervised setting and significantly outperforms other approaches in zero-shot and few-shot settings.
- The Microsoft Toolkit of Multi-Task Deep Neural Networks for Natural Language Understanding63
MT-DNN is presented, an open-source natural language understanding (NLU) toolkit that makes it easy for researchers and developers to train customized deep learning models, designed to facilitate rapid customization for a broad spectrum of NLU tasks.
- The Use of SVM for Chinese New Word Identification62
A study of new word identification (NWI) to improve the performance of a Chinese word segmenter and explores different features for NWI, including in-word probability of a character (IWP), the analogy between new words and lexicon words, anti-word list, and frequency in documents.
- 60
- Using Statistical Techniques and Web Search to Correct ESL Errors60
A system for automatic correction of errors made by learners of English where machine-learned classifiers trained on large amounts of native data and a very large language model are combined to optimize the precision of suggested corrections.
- 59
- 59
- Towards Generating Long and Coherent Text with Multi-Level Latent Variable Models58
Extensive experimental results demonstrate that the proposed multi-level VAE model produces more coherent and less repetitive long text compared to baselines as well as can mitigate the posterior-collapse issue.
- Structuring Latent Spaces for Stylized Response Generation58
StyleFusion is proposed, which bridges conversation modeling and non-parallel style transfer by sharing a structured latent space that allows the system to generate stylized relevant responses by sampling in the neighborhood of the conversation model prediction, and continuously control the style level.
- Efficient Exploration for Dialog Policy Learning with Deep BBQ Networks \& Replay Buffer Spiking.58
This work introduces an exploration technique based on Thompson sampling, drawing Monte Carlo samples from a Bayes-by-backprop neural network, demonstrating marked improvement over common approaches such as $\epsilon$-greedy and Boltzmann exploration.
- 56
- 55
- 52
- Extraction of Chinese compound words: an experimental study on a very large corpus52
Experimental results show that this method is efficient and robust compared with other approaches, and results on information retrieval to show the usefulness of extracted compounds.
- EmailSum: Abstractive Email Thread Summarization51
The results reveal the key challenges of current abstractive summarization models in this task, such as understanding the sender’s intent and identifying the roles of sender and receiver and emphasize the importance of human evaluation and the development of better metrics by the community.
- Multi-task Learning with Sample Re-weighting for Machine Reading Comprehension51
A novel sample re-weighting scheme to assign sample-specific weights to the loss of a joint Machine Reading Comprehension (MRC) model that can be applied to a wide range of MRC tasks in different domains.
- Switch-Based Active Deep Dyna-Q: Efficient Adaptive Planning for Task-Completion Dialogue Policy Learning51
By combining switcher and active learning, the new framework named as Switch-based Active Deep Dyna-Q (Switch-DDQ), leads to significant improvement over DDQ and Q-learning baselines in both simulation and human evaluations.
- 50
- Subgoal Discovery for Hierarchical Dialogue Policy Learning50
This paper proposes a divide-and-conquer approach that discovers and exploits the hidden structure of the task to enable efficient policy learning and demonstrates the method by building a dialogue agent for the composite task of travel planning.
- 49
- 49
- Language model size reduction by pruning and clustering48
Novel clustering techniques can be combined with Stolcke pruning to produce the smallest models at a given perplexity, which creates clustered models that are often larger than the unclustered models, but which can be pruned to model that are smaller than unclUSTered models with the same perplexity.
- 47
- 47
- Bi-directional Attention with Agreement for Dependency Parsing47
A novel bi-directional attention model for dependency parsing, which learns to agree on headword predictions from the forward and backward parsing directions, which achieves state-of-the-art unlabeled attachment scores on 6 languages.
- 47
- Data Augmentation for Abstractive Query-Focused Multi-Document Summarization46
This work builds abstractive end-to-end neural network models on the combined datasets that yield new state-of-the-art transfer results on DUC datasets, and introduces new hierarchical encoders that enable a more efficient encoding of the query together with multiple documents.
- 45
- Model adaptation via model interpolation and boosting for web search ranking45
This paper explores two classes of model adaptation methods for Web search ranking: Model Interpolation and error-driven learning approaches based on a boosting algorithm, and shows that model interpolation achieves the best results on all the open test sets where the test data is very different from the training data.
- 44
- The Use of Clustering Techniques for Language Modeling--Application to Asian Language44
Experimental results show that some novel clustering techniques presented work much better than previous methods, and achieve more than 40% size reduction at the same level of perplexity.
- Introduction to the special issue on statistical language modeling42
With the ever-increasing popularity of SLM, it is the right time to assemble a special issue reflecting recent advances in both its theory and applications.
- A system to mine large-scale bilingual dictionaries from monolingual web pages41
A system that automatically mines EnglishChinese translation pairs from large amount of monolingual Chinese web pages, showing that the mined bilingual dictionary greatly enlarges the coverage of an existing English-Chinese dictionary and improves query translation in cross-language information retrieval.
- 40
- Budgeted Policy Learning for Task-Oriented Dialogue Systems38
This paper presents a new approach that extends Deep Dyna-Q by incorporating a Budget-Conscious Scheduling (BCS) to best utilize a fixed, small amount of user interactions (budget) for learning task-oriented dialogue agents.
- Minimum sample risk methods for language modeling37
Evaluations on the task of Japanese text input show that MSR can handle a large number of features and training samples and outperforms the two widely applied discriminative methods, the boosting and the perceptron algorithms, by a small but statistically significant margin.
- 36
- Improving the Effectiveness of Information Retrieval with Clustering and Fusion36
This paper presents a novel fusion technique that can be combined with clustering to achieve consistent improvements and shows that this approach is more efficient than conventional approaches.
- 35
- 34
- Posterior Differential Regularization with f-divergence for Improving Model Robustness33
It is shown that regularizing the posterior difference with f-divergence can result in well-improved model robustness, indicating the great potential of the proposed framework for enhancing NLPmodel robustness.
- List Items One by One: A New Data Source and Learning Paradigm for Multimodal LLMs32
This work proposes a new learning paradigm:"list items one by one," which asks the model to enumerate and describe all visual tags placed on the image following the alphanumerics orders of tags, and finds that this new dataset, even in a relatively small size, significantly enhances visual reasoning capabilities and reduces hallucinations for MLLMs.
- Extending query translation to cross-language query expansion with markov chain models32
This paper model query translation by Markov Chains (MC), where query translation is viewed as a process of expanding query terms to their semantically similar terms in a different language, and performs query translation as a random walk in the graph, which propagates probabilities to related terms.
- 31
- 30
- 30
- 30
- 30
- 30
- 29
- 29
- 29
- 28
- 28
- 28
- 28
- 28
- 27
- 27
- 27
- 27
- 27
- 26
- 26
- 26
- Results of the Multi-Domain Task-Completion Dialog Challenge26
An overview of the “Multi-domain Task Completion” track (Track 1) at the 8th Dialog System Technology Challenge (DSTC-8) is provided, seeking to develop models that predict user responses when only limited in-domain data is available.
- 26
- 26
- 25
- 24
- 24
- 24
- 23
- Learning for Efficient Supervised Query Expansion via Two-stage Feature Selection23
It is pointed out that the cost of SQE mainly comes from term feature extraction, and a Two-stage Feature Selection framework (TFS) is proposed to address this problem, which can substantially reduce the time cost for SQE, while maintaining its effectiveness.
- 23
- 23
- 23
- 23
- 22
- 22
- 22
- 21
- 21
- 21
- 21
- 20
- 20
- 20
- 19
- 19
- 19
- 19
- 19
- 19
- 18
- 18
- 17
- Domain-Specific Pretraining for Vertical Search: Case Study on Biomedical Literature17
This work proposes a general approach for vertical search based on domain-specific pretraining and presents a case study for the biomedical domain, which performs comparably or better than the best systems in the official TREC-COVID evaluation, a COVID-related biomedical search competition.
- 17
- 17
- 16
- 16
- 16
- 16
- 16
- 16
- 16
- 15
- 15
- 15
- 15
- 15
- Unsupervised Deep Structured Semantic Models for Commonsense Reasoning15
Two neural network models based on the Deep Structured Semantic Models (DSSM) framework are proposed to tackle two classic commonsense reasoning tasks, Winograd Schema challenges (WSC) and Pronoun Disambiguation (PDP).
- 15
- 14
- 14
- 14
- 14
- 14
- 14
- Dyna-Think: Synergizing Reasoning, Acting, and World Model Simulation in AI Agents13
Dyna-Think, a thinking framework that integrates planning with an internal world model with reasoning and acting to enhance AI agent performance, is proposed and a promising research direction to integrate world model simulation into AI agents to enhance their reasoning, planning, and acting capabilities is suggested.
- 13
- 13
- Targeted Adversarial Training for Natural Language Understanding12
Experiments show that TAT can significantly improve accuracy over standard adversarial training on GLUE and attain new state-of-the-art zero-shot results on XNLI.
- 12
- 12
- 12
- 12
- 12
- 11
- 11
- 11
- 11
- 10
- 10
- 10
- 10
- 10
- 10
- 10
- 9
- 9
- 9
- 9
- 9
- 9
- 9
- 9
- Dyna-Mind: Learning to Simulate from Experience for Better AI Agents8
Dyna-Mind, a two-stage training framework that explicitly teaches (V)LM agents to integrate simulation ability into AI agents, and Dyna-GRPO, an online reinforcement learning method that leverages outcome and interaction-level signals to learn better policies for long-horizon, planning-intensive tasks are introduced.
- 8
- 8
- 8
- 8
- 8
- 8
- 7
- 7
- 7
- 7
- 7
- 7
- 7
- 7
- 7
- 7
- 7
- EfficientLLM: Efficiency in Large Language Models6
This study systematically explores three key axes: architecture pretraining, fine-tuning, and inference, and generalize across modalities, which provides essential guidance for researchers and engineers navigating the efficiency-performance landscape of next-generation foundation models.
- 6
- 6
- 6
- 6
- 6
- 6
- 6
- 6
- 6
- 6
- 6
- 5
- 5
- 5
- 5
- 5
- Multi-domain Task-oriented Dialog Challenge II5
This proposal extends the tasks by incorporating new datasets, creating new sub-tasks, and providing a new development platform, and specifically focuses on two aspects of dialog systems: language portability and end-to-end system complexity.
- 5
- 5
- 5
- 5
- 5
- 5
- 5
- 5
- 4
- 4
- 4
- 4
- 4
- 4
- 4
- 4
- 4
- 4
- 4
- 4
- 4
- 4
- 4
- 4
- 4
- 4
- 4
- 4
- 3
- 3
- 3
- 3
- 3
- 3
- 3
- 3
- 2
- 2
- 2
- 2
- 2
- 2
- 2
- 2
- 2
- 2
- 2
- 2
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
Publication data from OpenAlex, with missing venues and authors filled in from Crossref; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-13. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.
Report an error
Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.