Is this you? Claim this profile to correct it, add a bio and choose the work people see first.
Claim this profileAcademic lineage
Possible advisorsa guess from early papers, not confirmed
- Yunxin ZhaoSuggested from co-authorship
Is this you? Claim this profile to confirm or dismiss it.
Works306 from public data
- Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering4,946
A combined bottom-up and top-down attention mechanism that enables attention to be calculated at the level of objects and other salient image regions is proposed, demonstrating the broad applicability of this approach to VQA.
- Hierarchical Attention Networks for Document Classification4,944
Experiments conducted on six large scale text classification tasks demonstrate that the proposed architecture outperform previous methods by a substantial margin.
- Embedding Entities and Relations for Learning and Inference in Knowledge Bases3,965
It is found that embeddings learned from the bilinear objective are particularly good at capturing relational semantics and that the composition of relations is characterized by matrix multiplication.
- Learning deep structured semantic models for web search using clickthrough data2,323
A series of new latent semantic models with a deep structure that project queries and documents into a common low-dimensional space where the relevance of a document given a query is readily computed as the distance between them are developed.
- MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognition2,241
A benchmark task to recognize one million celebrities from their face images, by using all the possibly collected face images of this individual on the web as training data, which could lead to one of the largest classification problems in computer vision.
- Stacked Attention Networks for Image Question Answering2,028
A multiple-layer SAN is developed in which an image is queried multiple times to infer the answer progressively, and the progress that the SAN locates the relevant visual clues that lead to the answer of the question layer-by-layer.
- AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networks2,008
An Attentional Generative Adversarial Network that allows attention-driven, multi-stage refinement for fine-grained text-to-image generation and for the first time shows that the layered attentional GAN is able to automatically select the condition at the word level for generating different parts of the image.
- Stacked Cross Attention for Image-Text Matching1,425
Stacked Cross Attention to discover the full latent alignments using both image regions and words in sentence as context and infer the image-text similarity achieves the state-of-the-art results on the MS-COCO and Flickr30K datasets.
- From captions to visual concepts and back1,363
This paper uses multiple instance learning to train visual detectors for words that commonly occur in captions, including many different parts of speech such as nouns, verbs, and adjectives, and develops a maximum-entropy language model.
- A Corpus and Cloze Evaluation for Deeper Understanding of Commonsense Stories863
A new framework for evaluating story understanding and script learning: the 'Story Cloze Test', which requires a system to choose the correct ending to a four-sentence story, and a new corpus of ~50k five-sentence commonsense stories, ROCStories, to enable this evaluation.
- Semantic Parsing via Staged Query Graph Generation: Question Answering with Knowledge Base847
This work proposes a novel semantic parsing framework for question answering using a knowledge base that leverages the knowledge base in an early stage to prune the search space and thus simplifies the semantic matching problem.
- 823
- A Multi-View Deep Learning Approach for Cross Domain User Modeling in Recommendation Systems743
This work proposes a content-based recommendation system to address both the recommendation quality and the system scalability, and proposes to use a rich feature set to represent users, according to their web browsing history and search queries, using a Deep Learning approach.
- From Eliza to XiaoIce: challenges and opportunities with social chatbots742
The success metric for social chatbots is defined as conversation-turns per session (CPS), and it is shown how XiaoIce can dynamically recognize emotion and engage the user throughout long conversations with appropriate interpersonal responses.
- Learning semantic representations using convolutional neural networks for web search730
This paper presents a series of new latent semantic models based on a convolutional neural network to learn low-dimensional semantic vectors for search queries and Web documents that significantly outperforms other se-mantic models in retrieval performance.
- A Latent Semantic Model with Convolutional-Pooling Structure for Information Retrieval728
A new latent semantic model that incorporates a convolutional-pooling structure over word sequences to learn low-dimensional, semantic vector representations for search queries and Web documents is proposed.
- 591
- Domain Adaptation via Pseudo In-Domain Data Selection584
The results show that more training data is not always better, and that best results are attained via proper domain-relevant data selection, as well as combining in- and general-domain systems during decoding.
- Visual Storytelling551
Modelling concrete description as well as figurative and social language, as provided in this dataset and the storytelling task, has the potential to move artificial intelligence from basic understandings of typical visual scenes towards more and more human-like understanding of grounded event structure and subjective expression.
- Deep Learning with Low Precision by Half-Wave Gaussian Quantization537
An half-wave Gaussian quantizer (HWGQ) is proposed for forward approximation and shown to have efficient implementation, by exploiting the statistics of of network activations and batch normalization operations, and to achieve much closer performance to full precision networks than previously available low-precision networks.
- CleanNet: Transfer Learning for Scalable Image Classifier Training with Label Noise523
CleanNet, a joint neural embedding network, which only requires a fraction of the classes being manually verified to provide the knowledge of label noise that can be transferred to other classes is introduced, which can reduce label noise detection error rate on held-out classes where no human supervision available.
- Multimodal Intelligence: Representation Learning, Information Fusion, and Applications504
A technical review of available models and learning methods for multimodal intelligence, focusing on the combination of vision and natural language modalities, which has become an important topic in both the computer vision andnatural language processing research communities.
- Semantic Compositional Networks for Visual Captioning453
Experimental results show that the proposed method significantly outperforms prior state-of-the-art approaches, across multiple evaluation metrics.
- Investigation of recurrent-neural-network architectures and learning methods for spoken language understanding445
The results show that on this task, both types of recurrent networks outperform the CRF baseline substantially, and a bi-directional Jordantype network that takes into account both past and future dependencies among slots works best, outperforming a CRFbased baseline by 14% in relative error reduction.
- Representation Learning Using Multi-Task Deep Neural Networks for Semantic Classification and Information Retrieval443
This work develops a multi-task DNN for learning representations across multiple tasks, not only leveraging large amounts of cross-task data, but also benefiting from a regularization effect that leads to more general representations to help tasks in new domains.
- Semantic Parsing for Single-Relation Question Answering420
A semantic parsing framework based on semantic similarity for open domain question answering (QA) that achieves higher precision across different recall points compared to the previous approach, and can improve F1 by 7 points.
- Tips and Tricks for Visual Question Answering: Learnings from the 2017 Challenge403
This work presents a massive exploration of the effects of the myriad architectural and hyperparameter choices that must be made in generating a state-of-the-art model and provides a detailed analysis of the impact of each choice on model performance.
- Bottom-Up and Top-Down Attention for Image Captioning and VQA.357
A combined bottom-up and top-down attention mechanism that enables attention to be calculated at the level of objects and other salient image regions to enable deeper image understanding through multiple steps of reasoning is proposed.
- Object-Driven Text-To-Image Synthesis via Adversarial Training326
A thorough comparison between the classic grid attention and the new object-driven attention is provided through analyzing their mechanisms and visualizing their attention layers, showing insights of how the proposed model generates complex scenes in high quality.
- Deep Communicating Agents for Abstractive Summarization323
Empirical results demonstrate that multiple communicating encoders lead to a higher quality summary compared to several strong baselines, including those based on a single encoder or multiple non-communicating encoder.
- Generating Natural Questions About an Image318
This paper introduces the novel task of Visual Question Generation, where the system is tasked with asking a natural and engaging question when shown an image, and provides three datasets which cover a variety of images from object-centric to event-centric.
- 302
- Language Models for Image Captioning: The Quirks and What Works293
By combining key aspects of the ME and RNN methods, this paper achieves a new record performance over previously published results on the benchmark COCO dataset, however, the gains the authors see in BLEU do not translate to human judgments.
- Deep Reinforcement Learning with a Natural Language Action Space271
This paper introduces a novel architecture for reinforcement learning with deep neural networks designed to handle state and action spaces characterized by natural language, as found in text-based games, called a deep reinforcement relevance network (DRRN).
- 202
- Character-Level Question Answering with Attention191
It is shown that a character-level encoder-decoder framework can be successfully applied to question answering with a structured knowledge base and can be learned with significantly less data compared to previous work, which relies on data augmentation.
- Modeling Interestingness with Deep Neural Networks181
The results on large-scale, real-world datasets show that the semantics of documents are important for modeling interest-ingness and that the DSSM leads to significant quality improvement on both tasks, outperforming not only the classic document models that do not use semantics but also state-of-the-art topic models.
- Multi-Rate Deep Learning for Temporal Recommendation174
A novel deep neural network based architecture that models the combination of long-term static and short-term temporal user preferences to improve the recommendation performance and a novel pre-train method to reduce the number of free parameters significantly is proposed.
- Multi-hop Reading Comprehension across Multiple Documents by Reasoning over Heterogeneous Graphs171
This paper introduces a heterogeneous graph with different types of nodes and edges, which is named as Heterogeneous Document-Entity (HDE) graph, which contains different granularity levels of information including candidates, documents and entities in specific document contexts.
- Select, Answer and Explain: Interpretable Multi-Hop Reading Comprehension over Multiple Documents156
This paper proposes an effective and interpretable Select, Answer and Explain (SAE) system to solve the multi-document RC problem and achieves top competitive performance in distractor setting compared to other existing systems on the leaderboard.
- 141
- Enhancing personalized search by mining and modeling task behavior140
A method whereby other users performing similar tasks to the current user and leverage their on-task behavior to identify Web pages to promote in the current ranking yields promising gains in retrieval performance, and has direct implications for improving personalization in search systems.
- Table Cell Search for Question Answering135
This work proposes a novel table cell search framework that is comparable to state-of-the-art QA systems based on knowledge bases (KBs), while on Bing queries, it outperforms other systems with a 56.7% relative gain.
- 134
- Enhancing Automated Essay Scoring Performance via Fine-tuning Pre-trained Language Models with Combination of Regression and Ranking131
A new way to fine-tune pre-trained language models with multiple losses of the same task is found to improve AES’s performance, and the model outperforms not only state-of-the-art neural models near 3 percent but also the latest statistic model.
- Natural Language to Structured Query Generation via Meta-Learning130
This work explores a different learning protocol that treats each example as a unique pseudo-task, by reducing the original learning problem to a few-shot meta-learning scenario with the help of a domain-dependent relevance function.
- Rich Image Captioning in the Wild130
An image caption system that addresses new challenges of automatically describing images in the wild by developing a deep vision model that detects a broad range of visual concepts, an entity recognition model that identifies celebrities and landmarks, and a confidence model for the caption output.
- 124
- Orthogonal Relation Transforms with Graph Context Modeling for Knowledge Graph Embedding123
A novel distance-based approach for knowledge graph link prediction that extends the RotatE from 2D complex domain to high dimensional space with orthogonal transforms to model relations and improves prediction accuracy on the difficult N-to-1, 1- to-N and N-To-N cases.
- Learning Continuous Phrase Representations for Translation Modeling123
This paper improves the performance of a state-of-the-art phrase-based statistical machine translation system trained on WMT 2012 French-English data by up to 1.3 BLEU points.
- 121
- RevCore: Review-Augmented Conversational Recommendation120
A novel end-to-end framework, namely, Review-augmented Conversational Recommender (RevCore), where reviews are seamlessly incorporated to enrich item information and assist in generating both coherent and informative responses.
- Learning to extract cross-session search tasks105
A semi-supervised clustering model is proposed based on the latent structural SVM framework, and a set of effective automatic annotation rules are proposed as weak supervision to release the burden of manual annotation in identifying cross-session search tasks.
- Clickthrough-based translation models for web search103
This paper provides a quantitative analysis of the language discrepancy issue, and explores the use of clickthrough data to bridge documents and queries, and demonstrates that standard statistical machine translation techniques can be adapted for building a better Web document retrieval system.
- Learning Generic Sentence Representations Using Convolutional Neural Networks101
Experimental results on several benchmark datasets, and across a broad range of applications, demonstrate the superiority of the proposed model over competing methods.
- TopPPR100
PPR is proposed, an algorithm for top-k PPR queries that ensure at least ρ precision with at least 1 - 1/n probability, where ρ ∈;n (0, 1] is a user-specified parameter and n is the number of nodes in G.
- Hierarchically Structured Reinforcement Learning for Topically Coherent Visual Story Generation99
Empirical results from both automatic and human evaluations demonstrate that the proposed hierarchically structured reinforced training achieves significantly better performance compared to a strong flat deep reinforcement learning baseline.
- 94
- Self-Attention Guided Copy Mechanism for Abstractive Summarization90
A Transformer-based model is proposed to enhance the copy mechanism by identifying the importance of each source word based on the degree centrality with a directed graph built by the self-attention layer in the Transformer.
- Multimodal Joint Attribute Prediction and Value Extraction for E-commerce Product90
This paper proposes a multimodal method to jointly predict product attributes and extract values from textual product descriptions with the help of the product images, and demonstrates that explicitly modeling the relationship between attributes and values facilitates the method to establish the correspondence between them.
- 89
- Personalized ranking model adaptation for web search88
This paper proposes a general ranking model adaptation framework for personalized search that quickly learns to apply a series of linear transformations over the parameters of the given global ranking model such that the adapted model can better fit each individual user's search preferences.
- Keywords-Guided Abstractive Sentence Summarization86
This paper proposes an abstractive sentence summarization method by applying guidance signals of keywords to both the encoder and the decoder in the sequence-to-sequence model, and demonstrates that multi-task learning and keywords-oriented guidance facilitate sentence summarizing task.
- Deep Speaker Embedding Learning with Multi-level Pooling for Text-independent Speaker Verification86
Improvements to the widely used deep speaker embedding x-vector model are proposed, including a hybrid neural network structure using both time delay neural network (TDNN) and long short-term memory neural networks (LSTM) to generate complementary speaker information at different levels.
- Indirect-HMM-based hypothesis alignment for combining outputs from machine translation systems86
The proposed indirect hidden Markov model (IHMM) is proposed to address the synonym matching and word ordering issues in hypothesis alignment and achieved the best Chinese-to-English translation result in the constrained training track of the 2008 NIST Open MT Evaluation.
- 85
- Discourse-Aware Neural Rewards for Coherent Text Generation84
Empirical results demonstrate that a generator trained with the learned reward produces more coherent and less repetitive text than models trained with cross-entropy or with reinforcement learning with commonly used scores as rewards.
- Aspect-Aware Multimodal Summarization for Chinese E-Commerce Products82
This work proposes an aspect-aware multimodal summarization model that can effectively incorporate the visual information and also determine the most salient aspects of a product.
- 81
- Deep Reinforcement Learning with a Combinatorial Action Space for Predicting Popular Reddit Threads74
The proposed model, which represents dependence between sub-actions through a bi-directional LSTM, gives the best performance across different experimental configurations and domains, and it also generalizes well with varying numbers of recommendation requests.
- 74
- Maximum Expected BLEU Training of Phrase and Lexicon Translation Models71
The proposed method, evaluated on the Europarl German-to-English dataset, leads to a 1.1 BLEU point improvement over a state-of-the-art baseline translation system.
- Adapting deep RankNet for personalized search70
This paper continue-trained a variety of RankNets with different number of hidden layers and network structures over a previously trained global RankNet model, and observed that a deep neural network with five hidden layers gives the best performance.
- Knowledgeable Storyteller: A Commonsense-Driven Generative Model for Visual Storytelling64
This work presents a commonsensedriven generative model, which aims to introduce crucial commonsense from the external knowledge base for visual storytelling by adopting an elaborately designed vision-aware directional encoding schema to effectively integrate the most informative commonsense.
- 63
- 61
- Deep Reinforcement Learning for NLP59
A gentle introduction to the foundation of deep reinforcement learning, as well as some practical DRL solutions in NLP problems, with a special focus on generation, dialogue, and information extraction.
- Using word dependent transition models in HMM based word alignment for statistical machine translation58
A Bayesian Learning based method to train word dependent transition models for HMM based word alignment gives consistent and significant alignment error rate (AER) reduction and machine translation results show that word alignment can be used in a phrase-based machine translation system.
- Improving Prosody Modelling with Cross-Utterance Bert Embeddings for End-to-End Speech Synthesis56
Cross-utterance (CU) context vectors, which are produced by an additional CU encoder based on the sentence embeddings extracted by a pretrained BERT model, are used to augment the input of the Tacotron2 decoder.
- Two-Stage Synthesis Networks for Transfer Learning in Machine Comprehension55
A technique for transfer learning in machine comprehension (MC) using a novel two-stage synthesis network that aims to answer questions about documents in another domain, where there are no labeled data of question-answer pairs.
- 55
- Question-Answering with Grammatically-Interpretable Representations54
Support is found for the initial hypothesis that symbols can be interpreted as lexical-semantic word meanings, while roles can be interpretations as approximations of grammatical roles (or categories) such as subject, wh-word, determiner, etc.
- Query Prior Matters53
A machine reading comprehension based framework for MNER, namely MRC-MNER, which outperforms the current state-of-the-art models on Twitter2017, and yields competitive results on Twitter2015.
- 53
- 49
- Learning to Decouple Relations: Few-Shot Relation Classification with Entity-Guided Attention and Confusion-Aware Training48
CTEG is proposed, a model equipped with two novel mechanisms to learn to decouple these easily-confused relations, and the results show that the proposed model achieves comparable and even much better results to strong baselines in terms of accuracy.
- 48
- Bi-directional Attention with Agreement for Dependency Parsing47
A novel bi-directional attention model for dependency parsing, which learns to agree on headword predictions from the forward and backward parsing directions, which achieves state-of-the-art unlabeled attachment scores on 6 languages.
- Modeling action-level satisfaction for search task satisfaction prediction45
This work hypothesizes that searchers' latent action-level satisfaction influences their observed search behaviors and contributes to overall search satisfaction, and develops a latent structural learning method, whereby rich structured features and dependency relations unique to search satisfaction prediction are explored.
- 44
- 42
- Multimodal Sentence Summarization via Multimodal Selective Encoding40
A multimodal selective gate network that considers reciprocal relationships between textual and multi-level visual features to select highlights of the event when encoding the source sentence is proposed.
- 39
- 36
- 35
- 34
- Large-margin minimum classification error training: A theoretical risk minimization perspective34
This paper shows that the sigmoid bias in the conventional minimum classification error (MCE) training can be interpreted as a soft margin, and proposes a practical optimization strategy that adjusts the margin incrementally in the MCE training process so that a desirable balance between the empirical error rates on the training set and the margin can be achieved.
- 32
- 32
- 28
- 28
- 28
- 28
- 26
- 26
- LMAgent: A Large-Scale Multimodal Agents Society for Multi-User Simulation25
This paper presents LMAgent, a very large-scale and multimodal agents society based on multimodal LLMs, and introduces a self-consistency prompting mechanism to augment agents’ multimodal capabilities, resulting in significantly improved decision-making performance over the existing multi-agent system.
- 25
- Don’t Take It Literally: An Edit-Invariant Sequence Loss for Text Generation24
Experimental results show the novel Edit-Invariant Sequence Loss (EISL) method significantly outperforms the common CE loss and other strong baselines on all the tasks, and is easy to implement and efficient to compute with existing libraries.
- 24
- Discrete Trust-aware Matrix Factorization for Fast Recommendation24
This paper proposes a discrete trust-aware matrix factorization (DTMF) model to take dual advantages of both social relations and discrete technique for fast recommendation, and maps the latent representation of users and items into a joint hamming space by recovering the rating and trust interactions between users and Items.
- 23
- 23
- 23
- 23
- 22
- 22
- 22
- 21
- 21
- 21
- 20
- 20
- 19
- 19
- 19
- 19
- 19
- 18
- 18
- 18
- 17
- 17
- 17
- 17
- 16
- 16
- 16
- 16
- ChartMaster: Advancing Chart-to-Code Generation with Real-World Charts and Chart Similarity Reinforcement Learning15
The ChartMaster model, a GRPO-based reinforcement learning algorithm guided by a novel chart similarity reward, is developed, achieving SOTA results among 7B-parameter models and rivaling GPT-4o on various chart-to-code benchmarks.
- 15
- 14
- 13
- 12
- 12
- 12
- 12
- 12
- Gated Multimodal Fusion with Contrastive Learning for Turn-Taking Prediction in Human-Robot Dialogue11
- 11
- 11
- 11
- 11
- 11
- 10
- 10
- 10
- 10
- 10
- 10
- 10
- 10
- 9
- 9
- 9
- 9
- 9
- 9
- 8
- 8
- 8
- 8
- 8
- 8
- 8
- Cross-modal Contrastive Distillation for Instructional Activity Anticipation7
A novel cross-modal contrastive distillation (CCD) scheme is devised, which facilitates knowledge distillation between teacher and student in heterogeneous modalities with the proposed cross- modal distillation loss.
- 7
- 7
- 7
- 7
- 7
- 7
- 7
- 7
- 7
- 6
- 6
- 6
- 6
- 6
- 6
- 6
- 6
- 6
- 6
- 5
- 5
- 5
- 5
- 5
- 5
- 5
- 5
- Improved Monolingual Hypothesis Alignment for Machine Translation System Combination5
The proposed indirect hidden Markov model (IHMM) is proposed to address the synonym matching and word ordering issues in hypothesis alignment and achieved the best Chinese-to-English translation result in the constrained training track of the 2008 NIST Open MT Evaluation.
- 5
- 4
- 4
- 4
- 4
- 4
- 4
- 4
- 4
- 3
- 3
- 3
- 3
- 3
- 3
- 3
- 3
- 3
- 3
- 3
- 3
- 3
- 2
- 2
- 2
- 2
- 2
- 2
- 2
- 2
- 2
- 2
- 2
- 2
- 2
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
Publication data from OpenAlex, with missing venues and authors filled in from Crossref; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-13. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.
Report an error
Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.