Is this you? Claim this profile to correct it, add a bio and choose the work people see first.
Claim this profileWorks207 from public data
- CodeBERT: A Pre-Trained Model for Programming and Natural Languages4,421
This work develops CodeBERT with Transformer-based neural architecture, and trains it with a hybrid objective function that incorporates the pre-training task of replaced token detection, which is to detect plausible alternatives sampled from generators.
- 1,351
- Unicoder-VL: A Universal Encoder for Vision and Language by Cross-Modal Pre-Training983
After pretraining on large-scale image-caption pairs, Unicoder-VL is transferred to caption-based image-text retrieval and visual commonsense reasoning, with just one additional output layer, and shows the powerful ability of the cross-modal pre-training.
- K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapters648
K-Adapter is proposed, which remains the original parameters of the pre-trained model fixed and supports continual knowledge infusion and captures richer factual and commonsense knowledge than RoBERTa.
- Context-aware query suggestion by mining click-through and session data624
This paper proposes a novel context-aware query suggestion approach which is in two steps, and outperforms two baseline methods in both coverage and quality of suggestions.
- XGLUE: A New Benchmark Dataset for Cross-lingual Pre-training, Understanding and Generation404
A recent cross-lingual pre-trained model Unicoder is extended to cover both understanding and generation tasks, which is evaluated on XGLUE as a strong baseline and the base versions of Multilingual BERT, XLM and XLM-R are evaluated for comparison.
- 400
- NÜWA: Visual Synthesis Pre-training for Neural visUal World creAtion373
Compared to several strong baselines, N\"UWA achieves state-of-the-art results on text-to-image generation, text- to-video generation, video prediction, etc, and it also shows surprisingly good zero-shot capabilities onText-guided image and video manipulation tasks.
- On mining cross-graph quasi-cliques274
An efficient algorithm is developed, Crochet, which exploits several interesting and effective techniques and heuristics to efficaciously mine cross-graph quasi-cliques, which is generalized from several interesting applications such as cross-market customer segmentation and joint mining of gene expression data and protein interaction data.
- Unicoder: A Universal Language Encoder by Pre-training with Multiple Cross-lingual Tasks244
It is found that doing fine-tuning on multiple languages together can bring further improvement in Unicoder, a universal language encoder that is insensitive to different languages.
- Graph-Based Reasoning over Heterogeneous External Knowledge for Commonsense Question Answering232
This work proposes to automatically extract evidence from heterogeneous knowledge sources, and answer questions based on the extracted evidence, and achieves the state-of-the-art accuracy on the CommonsenseQA dataset.
- Context-aware query classification195
This paper incorporates context information into the problem of query classification by using conditional random field (CRF) models and shows that it can improve the F1 score by 52% as compared to other state-of-the-art baselines.
- 193
- Context-aware ranking in web search182
The experimental results clearly show that the context-aware ranking approach improves the ranking of a commercial search engine which ignores context information and outperforms a baseline method which considers context information in ranking.
- THE-X: Privacy-Preserving Transformer Inference with Homomorphic Encryption180
This work introduces THE-X, an approximation approach for transformers, which enables privacy-preserving inference of pre-trained models developed by popular frameworks, and proposes a workflow to deal with complex computation in transformer networks, including all the non-polynomial functions like GELU, softmax, and LayerNorm.
- Reinforced Multi-Teacher Selection for Knowledge Distillation172
This paper systematically develops a reinforced method to dynamically assign weights to teacher models for different training instances and optimize the performance of student models distilled.
- 163
- CoSQA: 20,000+ Web Queries for Code Search and Question Answering162
A contrastive learning method dubbed CoCLR is introduced to enhance text-code matching, which works as a data augmenter to bring more artificially generated training instances to bring better semantic matching between query and code.
- WhiteningBERT: An Easy Unsupervised Sentence Embedding Approach153
This work conducts a thorough examination of pretrained model based unsupervised sentence embeddings and concludes that an easy whitening-based vector normalization strategy with less than 10 lines of code consistently boosts the performance.
- Web Query Recommendation via Sequential Query Prediction142
A novel "sequential query prediction" approach that tries to grasp a user's search intent based on his/her past query sequence and its resemblance to historical query sequence models mined from massive search engine logs is proposed.
- Towards context-aware search by learning a very large variable length hidden markov model from search logs131
A strategy for parameter initialization in vlHMM learning which can greatly reduce the number of parameters to be estimated in practice is developed and a method for distributed vl HMM learning under the map-reduce model is devised.
- Joint Type Inference on Entities and Relations via Graph Convolutional Networks127
A new paradigm for the task of joint entity relation extraction is developed that first identifies entity spans, then performs a joint inference on entity types and relation types and is competitive with the state-of-the-art in relation performance.
- 120
- PromDA: Prompt-based Data Augmentation for Low-Resource NLU Tasks113
PromDA, a Prompt-based Data Augmentation model (PromDA) which only trains small-scale Soft Prompt in the frozen Pre-trained Language Models (PLMs) avoids human effort in collecting unlabeled in-domain data and maintains the quality of generated synthetic data.
- Model Compression with Two-stage Multi-teacher Knowledge Distillation for Web Question Answering System107
A Two-stage Multi-teacher Knowledge Distillation (TMKD) method for web Question Answering system that can significantly outperform the baseline methods and even achieve comparable results with the original teacher models, along with substantial speedup of model inference.
- Mining frequent cross-graph quasi-cliques107
A novel data mining problem, mining frequent cross-graph quasi-cliques, which is generalized from several interesting applications in bioinformatics, cross-market customer segmentation, social network analysis, and Web mining is investigated and practical algorithms which exploit several interesting and effective techniques and heuristics are developed.
- Logic-Driven Context Extension and Data Augmentation for Logical Reasoning of Text94
The results show that the proposed logic-driven data augmentation algorithm achieves state-of-the-art performance on both datasets, and even surpasses human performance on the ReClor dataset.
- Mining coherent gene clusters from gene-sample-time microarray data92
This paper explores a novel type of gene-sample-time microarray data sets, which records the expression levels of various genes under a set of samples during a series of time points, and proposes the mining of coherent gene clusters from such data sets.
- Multi-Task Learning for Conversational Question Answering over a Large-Scale Knowledge Base85
This work proposes an innovative multi-task learning framework where a pointer-equipped semantic parsing model is designed to resolve coreference in conversations, and naturally empower joint learning with a novel type-aware entity detection model.
- 81
- MPC-BERT: A Pre-Trained Language Model for Multi-Party Conversation Understanding78
MPC-BERT is presented, a pre-trained model for MPC understanding that considers learning who says what to whom in a unified model with several elaborated self-supervised tasks and achieves new state-of-the-art performance on three downstream tasks.
- Mining query subtopics from search log data78
A clustering algorithm is proposed that can effectively leverage the two phenomena of user behavior to automatically mine the major subtopics of queries, where each subtopic is represented by a cluster containing a number of URLs and keywords.
- Learning an Effective Context-Response Matching Model with Self-Supervised Tasks for Retrieval-based Dialogues76
This paper proposes learning a context-response matching model with auxiliary self-supervised tasks designed for the dialogue data based on pre-trained language models and jointly train the PLM-based response selection model with these auxiliary tasks in a multi-task manner.
- GLGE: A New General Language Generation Evaluation Benchmark75
The General Language Generation Evaluation (GLGE), a new multi-task benchmark for evaluating the generalization capabilities of NLG models across eight language generation tasks, is presented and a leaderboard with strong baselines including MASS, BART, and ProphetNet is built.
- 74
- Mining search and browse logs for web search69
This survey focuses on mining search and browse log data for Web search and elaborate how log mining applications enhance the five major components of a search engine, namely, queryUnderstanding, document understanding, document ranking, user understanding, and monitoring and feedback.
- 66
- VatLM: Visual-Audio-Text Pre-Training With Unified Masked Prediction for Speech Representation Learning65
Results show that the proposed VatLM outperforms previous state-of-the-art models, such as the audio-visual pre-trained AV-HuBERT model, and analysis demonstrates that VatLM is capable of aligning different modalities into the same space.
- DC-BERT: Decoupling Question and Document for Efficient Contextual Encoding64
DC-BERT is proposed, a decoupled contextual encoding framework that has dual BERT models: an online BERT which encodes the question only once, and an offline Bert which pre-encodes all the documents and caches their encodings.
- LogicalFactChecker: Leveraging Logical Operations for Fact Checking with Graph Module Network63
This work proposes LogicalFactChecker, a neural network approach capable of leveraging logical operations for fact checking, which achieves the state-of-the-art performance on TABFACT, a large-scale, benchmark dataset built for verifying a textual statement with semi-structured tables.
- Learning Neural Templates for Recommender Dialogue System59
This work introduces a novel framework called NTRD for recommender dialogue system that can decouple the dialogue generation from the item recommendation and has the unique advantage to produce novel items that do not appear in the training set of dialogue corpus.
- RikiNet: Reading Wikipedia Pages for Natural Question Answering59
This paper introduces a new model, called RikiNet, which reads Wikipedia pages for natural question answering, which is the first single model that outperforms the single human performance.
- Multimodal Dialogue Response Generation58
A novel conversational agent is devised, Divter, in order to isolate parameters that depend on multimodal dialogues from the entire generation model, which achieves state-of-the-art results in both automatic and human evaluation, and can generate informative text and high-resolution image responses.
- Document Modeling with Graph Attention Networks for Multi-grained Machine Reading Comprehension57
A novel multi-grained machine reading comprehension framework that focuses on modeling documents at their hierarchical nature, which are different levels of granularity: documents, paragraphs, sentences, and tokens, which significantly outperforms previous systems at both long and short answer criteria.
- Deep LSTM based Feature Mapping for Query Classification57
Using an architecture that combines a stack of the DLSTM layers with a tradition CNN layer, new state-of-the-art query classification accuracy on benchmark data sets for query classo-cation is observed.
- Syntax-Enhanced Pre-trained Model56
It is demonstrated that infusing automatically produced syntax of text improves pre-trained models, and global syntactic distances among tokens bring larger performance gains compared to local head relations between contiguous tokens.
- ProphetNet-X: Large-Scale Pre-training Models for English, Chinese, Multi-lingual, Dialog, and Code Generation56
The ProphetNet family pre-training models, named ProphetNet-X, where X can be English, Chinese, Multi-lingual, and so on, are presented, and achieve new state-of-the-art performance on 10 benchmarks.
- 53
- GRACE: Gradient Harmonized and Cascaded Labeling for Aspect-based Sentiment Analysis52
A cascaded labeling module is developed to enhance the interchange between aspect terms and improve the attention of sentiment tokens when labeling sentiment polarities and the proposed GRACE adopts a post-pretraining BERT as its backbone.
- 51
- Mining Concept Sequences from Large-Scale Search Logs for Context-Aware Query Suggestion51
The experimental results clearly show that the novel context-aware query suggestion approach outperforms three baseline methods in both coverage and quality of suggestions.
- 49
- Difference-aware Knowledge Selection for Knowledge-grounded Conversation Generation48
Automatic, human observational, and interactive evaluation shows that the proposed difference-aware knowledge selection method is able to select knowledge more accurately and generate more informative responses, significantly outperforming the state-of-the-art baselines.
- Table2Charts: Recommending Charts by Learning Shared Table Representations44
It is shown that Table2Charts could learn a shared representation of table fields so that recommendation tasks on different chart types could mutually enhance each other and outperforms other chart recommendation systems in both multi-type task and human evaluations.
- Interactive exploration of coherent patterns in time-series gene expression data43
An interactive exploration framework for mining coherent expression patterns in time-series gene expression data is proposed, and a novel tool, coherent pattern index graph, is developed to give users highly confident indications of the existences of coherent patterns.
- 42
- 40
- Finding email correspondents in online social networks40
This paper systematically investigates the problem of finding email correspondents in social networks accurately and develops a practical data mining approach that utilizes the similarity between email accounts and social network user profiles, and at the same time explores the similarities between the email communication network and the social network under investigation.
- 37
- LexLIP: Lexicon-Bottlenecked Language-Image Pre-Training for Large-Scale Image-Text Sparse Retrieval36
- 29
- CodeRetriever: A Large Scale Contrastive Pre-Training Method for Code Search29
The proposed CodeRetriever model, which learns the function-level code semantic representations through large-scale code-text contrastive pre-training, achieves new state-of-the-art with significant improvement over existing code pre-trained models, on eleven domain/language-specific code search tasks with six programming languages in different code granularity.
- 29
- 29
- 28
- 27
- 27
- 26
- Neural Semantic Parsing in Low-Resource Settings with Back-Translation and Meta-Learning26
This work aims to learn a neural semantic parser when only prior knowledge about a limited number of simple rules is available, without access to either annotated programs or execution results, and improves the accuracy and stability on examples uncovered by rules.
- 25
- 25
- 24
- 22
- Diverse, Controllable, and Keyphrase-Aware: A Corpus and Method for News Multi-Headline Generation22
A multi-source Transformer decoder, which takes three sources as inputs and generates multiple headlines with keyphrases of user interests to generate multiple keyphrase-relevant headlines, achieves state-of-the-art results in terms of quality and diversity.
- 21
- 21
- 21
- 21
- 21
- 20
- 20
- 19
- 18
- 18
- 17
- 16
- 16
- 15
- 15
- 14
- 13
- 13
- 13
- 13
- 12
- 12
- Mixed-modality Representation Learning and Pre-training for Joint Table-and-Text Retrieval in OpenQA12
- 12
- 12
- 12
- 12
- 12
- 11
- 11
- 11
- 11
- 11
- 10
- 9
- 9
- 9
- 9
- 8
- 8
- 8
- 8
- 8
- 8
- 8
- 8
- 7
- 7
- 7
- 7
- 7
- 6
- 6
- 6
- 6
- 5
- 5
- 5
- 5
- 5
- 5
- Integrating Pre-trained Model into Rule-based Dialogue Management5
This paper introduces the DM of Carina Dialog System and proposes the "model-trigger" design to make the DM trainable thus scalable to scenario changes, and integrates pre-trained models and empower the DM with few-shot capability.
- Towards Interpretable Reasoning over Paragraph Effects in Situation5
A sequential approach for the task of reasoning over paragraph effects in situation which explicitly models each step of the reasoning process with neural network modules, which leads to a more interpretable model.
- 5
- 5
- 4
- 4
- 4
- 4
- 4
- 3
- 3
- 3
- 3
- Leveraging Declarative Knowledge in Text and First-Order Logic for Fine-Grained Propaganda Detection3
- 2
- 2
- 2
- 2
- 2
- 2
- 2
- 2
- 2
- 2
- 2
- 2
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- Unearthing Gems from Stones: Policy Optimization with Negative Sample Augmentation for LLM Reasoning–
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
Publication data from OpenAlex, with missing venues and authors filled in from Crossref; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-13. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.
Report an error
Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.