Is this you? Claim this profile to correct it, add a bio and choose the work people see first.
Claim this profileWorks207 from public data
- CodeBERT: A Pre-Trained Model for Programming and Natural Languages4,421
This work develops CodeBERT with Transformer-based neural architecture, and trains it with a hybrid objective function that incorporates the pre-training task of replaced token detection, which is to detect plausible alternatives sampled from generators.
- 1,351
- Unicoder-VL: A Universal Encoder for Vision and Language by Cross-Modal Pre-Training983
After pretraining on large-scale image-caption pairs, Unicoder-VL is transferred to caption-based image-text retrieval and visual commonsense reasoning, with just one additional output layer, and shows the powerful ability of the cross-modal pre-training.
- K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapters648
K-Adapter is proposed, which remains the original parameters of the pre-trained model fixed and supports continual knowledge infusion and captures richer factual and commonsense knowledge than RoBERTa.
- Context-aware query suggestion by mining click-through and session data624
This paper proposes a novel context-aware query suggestion approach which is in two steps, and outperforms two baseline methods in both coverage and quality of suggestions.
- XGLUE: A New Benchmark Dataset for Cross-lingual Pre-training, Understanding and Generation404
A recent cross-lingual pre-trained model Unicoder is extended to cover both understanding and generation tasks, which is evaluated on XGLUE as a strong baseline and the base versions of Multilingual BERT, XLM and XLM-R are evaluated for comparison.
- 400
- NÜWA: Visual Synthesis Pre-training for Neural visUal World creAtion373
Compared to several strong baselines, N\"UWA achieves state-of-the-art results on text-to-image generation, text- to-video generation, video prediction, etc, and it also shows surprisingly good zero-shot capabilities onText-guided image and video manipulation tasks.
- On mining cross-graph quasi-cliques274
An efficient algorithm is developed, Crochet, which exploits several interesting and effective techniques and heuristics to efficaciously mine cross-graph quasi-cliques, which is generalized from several interesting applications such as cross-market customer segmentation and joint mining of gene expression data and protein interaction data.
- Unicoder: A Universal Language Encoder by Pre-training with Multiple Cross-lingual Tasks244
It is found that doing fine-tuning on multiple languages together can bring further improvement in Unicoder, a universal language encoder that is insensitive to different languages.
- Graph-Based Reasoning over Heterogeneous External Knowledge for Commonsense Question Answering232
This work proposes to automatically extract evidence from heterogeneous knowledge sources, and answer questions based on the extracted evidence, and achieves the state-of-the-art accuracy on the CommonsenseQA dataset.
- Context-aware query classification195
This paper incorporates context information into the problem of query classification by using conditional random field (CRF) models and shows that it can improve the F1 score by 52% as compared to other state-of-the-art baselines.
- 193
- Context-aware ranking in web search182
The experimental results clearly show that the context-aware ranking approach improves the ranking of a commercial search engine which ignores context information and outperforms a baseline method which considers context information in ranking.
- THE-X: Privacy-Preserving Transformer Inference with Homomorphic Encryption180
This work introduces THE-X, an approximation approach for transformers, which enables privacy-preserving inference of pre-trained models developed by popular frameworks, and proposes a workflow to deal with complex computation in transformer networks, including all the non-polynomial functions like GELU, softmax, and LayerNorm.
- Reinforced Multi-Teacher Selection for Knowledge Distillation172
This paper systematically develops a reinforced method to dynamically assign weights to teacher models for different training instances and optimize the performance of student models distilled.
- 163
- CoSQA: 20,000+ Web Queries for Code Search and Question Answering162
A contrastive learning method dubbed CoCLR is introduced to enhance text-code matching, which works as a data augmenter to bring more artificially generated training instances to bring better semantic matching between query and code.
- WhiteningBERT: An Easy Unsupervised Sentence Embedding Approach153
This work conducts a thorough examination of pretrained model based unsupervised sentence embeddings and concludes that an easy whitening-based vector normalization strategy with less than 10 lines of code consistently boosts the performance.
- Web Query Recommendation via Sequential Query Prediction142
A novel "sequential query prediction" approach that tries to grasp a user's search intent based on his/her past query sequence and its resemblance to historical query sequence models mined from massive search engine logs is proposed.
- Towards context-aware search by learning a very large variable length hidden markov model from search logs131
A strategy for parameter initialization in vlHMM learning which can greatly reduce the number of parameters to be estimated in practice is developed and a method for distributed vl HMM learning under the map-reduce model is devised.
- Joint Type Inference on Entities and Relations via Graph Convolutional Networks127
A new paradigm for the task of joint entity relation extraction is developed that first identifies entity spans, then performs a joint inference on entity types and relation types and is competitive with the state-of-the-art in relation performance.
- 120
- PromDA: Prompt-based Data Augmentation for Low-Resource NLU Tasks113
PromDA, a Prompt-based Data Augmentation model (PromDA) which only trains small-scale Soft Prompt in the frozen Pre-trained Language Models (PLMs) avoids human effort in collecting unlabeled in-domain data and maintains the quality of generated synthetic data.
- Model Compression with Two-stage Multi-teacher Knowledge Distillation for Web Question Answering System107
A Two-stage Multi-teacher Knowledge Distillation (TMKD) method for web Question Answering system that can significantly outperform the baseline methods and even achieve comparable results with the original teacher models, along with substantial speedup of model inference.
Publication data from OpenAlex, with missing venues and authors filled in from Crossref; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-13. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.
Report an error
Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.