Is this you? Claim this profile to correct it, add a bio and choose the work people see first.
Claim this profileAcademic lineage
Possible advisorsa guess from early papers, not confirmed
- Xuanjing HuangSuggested from co-authorship
Is this you? Claim this profile to confirm or dismiss it.
Works113 from public data
- CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing1,013
A framework called CRITIC is introduced that allows LLMs, which are essentially"black boxes" to validate and progressively amend their own outputs in a manner similar to human interaction with tools.
- ProphetNet: Predicting Future N-gram for Sequence-to-SequencePre-training512
A new sequence-to-sequence pre-training model called ProphetNet is presented, which introduces a novel self-supervised objective named future n-gram prediction and the proposed n-stream self-attention mechanism that predicts the next n tokens simultaneously based on previous context tokens at each time step.
- XGLUE: A New Benchmark Dataset for Cross-lingual Pre-training, Understanding and Generation404
A recent cross-lingual pre-trained model Unicoder is extended to cover both understanding and generation tasks, which is evaluated on XGLUE as a strong baseline and the base versions of Multilingual BERT, XLM and XLM-R are evaluated for comparison.
- ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solving331
This paper proposes ToRA a series of Tool-integrated Reasoning Agents designed to solve challenging mathematical problems by seamlessly integrating natural language reasoning with the utilization of external tools, thereby amalgamating the analytical prowess of language and the computational efficiency of tools.
- Query Rewriting for Retrieval-Augmented Large Language Models317
A new framework, Rewrite-Retrieve-Read instead of the previous retrieve-then-read for the retrieval-augmented LLMs from the perspective of the query rewriting, which is proven effective and scalable, and brings a new framework for retrieval-AUgmentedLLM.
- 260
- Keyphrase Extraction Using Deep Recurrent Neural Networks on Twitter198
A novel deep recurrent neural network model is proposed to combine keywords and context information to perform keyphrases from tweets and the experimental results showed that the proposed method performs significantly better than previous methods.
- 192
- Hashtag Recommendation for Multimodal Microblog Using Co-Attention Network139
This work proposes a co-attention network incorporating textual and visual information to recommend hashtags for multimodal tweets and demonstrates that the proposed method can achieve better performance than state-of-the-art methods using textual information only.
- Joint Type Inference on Entities and Relations via Graph Convolutional Networks127
A new paradigm for the task of joint entity relation extraction is developed that first identifies entity spans, then performs a joint inference on entity types and relation types and is competitive with the state-of-the-art in relation performance.
- Retweet Prediction with Attention-based Deep Neural Network120
A novel attention-based deep neural network to incorporate contextual and social information for this task and experimental results showed that the proposed method could achieve better results than the previous state-of-the-art methods.
- Synthetic Prompting: Generating Chain-of-Thought Demonstrations for Large Language Models103
Synthetic prompting is introduced, a method that leverages a few handcrafted examples to prompt the model to generate more examples by itself, and selects effective demonstrations to elicit better reasoning.
- 100
- Not All Tokens Are What You Need for Pretraining100
This work proposes a new language model called R HO -1, which employs Selective Language Modeling (SLM), which selectively trains on useful tokens that aligned with the desired distribution, increasing both data efficiency and performance of the language model pre-training.
- AutoVerus: Automated Proof Generation for Rust Code93
A benchmark suite of 150 non-trivial proof tasks is built, which shows that AutoVerus can automatically generate correct proof for more than 90% of them, with more than half of them tackled in less than 30 seconds or 3 LLM calls.
- 92
- Mask Attention Networks: Rethinking and Strengthen Transformer83
A novel understanding of SAN and FFN are presented as Mask Attention Networks (MANs) and it is shown that they are two special cases of MANs with static mask matrices, which limit the capability for localness modeling in text representation learning.
- GLGE: A New General Language Generation Evaluation Benchmark75
The General Language Generation Evaluation (GLGE), a new multi-task benchmark for evaluating the generalization capabilities of NLG models across eight language generation tasks, is presented and a leaderboard with strong baselines including MASS, BART, and ProphetNet is built.
- From intention to implementation: automating biomedical research via LLMs65
This paper introduces BioResearcher, the first end-to-end automated system designed to streamline the entire biomedical research process involving dry lab experiments, and demonstrates significant potential to reduce researchers’ workloads and accelerate biomedical discoveries.
- Optimizing Large Language Model Training Using FP4 Quantization64
This work introduces the first FP4 training framework for LLMs, addressing two key challenges with two key innovations: a differentiable quantization estimator for precise weight updates and an outlier clamping and compensation strategy to prevent activation collapse.
- DialogVED: A Pre-trained Latent Variable Encoder-Decoder Model for Dialog Response Generation59
A new dialog pre-training framework called DialogVED is proposed, which introduces continuous latent variables into the enhanced encoder-decoder pre- training framework to increase the relevance and diversity of responses.
- RikiNet: Reading Wikipedia Pages for Natural Question Answering59
This paper introduces a new model, called RikiNet, which reads Wikipedia pages for natural question answering, which is the first single model that outperforms the single human performance.
- Beyond Pass@1: Self-Play with Variational Problem Synthesis Sustains RLVR58
This paper proposes an online Self-play with Variational problem Synthesis (SvS) strategy for RLVR training, which uses the policy's correct solutions to synthesize variational problems while ensuring their reference answers remain identical to the originals.
- ProphetNet-X: Large-Scale Pre-training Models for English, Chinese, Multi-lingual, Dialog, and Code Generation56
The ProphetNet family pre-training models, named ProphetNet-X, where X can be English, Chinese, Multi-lingual, and so on, are presented, and achieve new state-of-the-art performance on 10 benchmarks.
- Automated Proof Generation for Rust Code via Self-Evolution54
This paper introduces SAFE, a framework that overcomes the lack of human-written proofs to enable automated proof generation of Rust code and improves the capability of open-source models, initially unacquainted with formal verification, to automatically write proofs for Rust code.
- Retweet Behavior Prediction Using Hierarchical Dirichlet Process54
This work proposes a novel method using non-parametric statistical models to combine structural, textual, and temporal information together to predict retweet behavior and demonstrates that the proposed method can achieve better performance than state-of-the-art methods.
- Distill-VQ48
Distill-VQ is proposed, which unifies the learning of IVF and PQ within a knowledge distillation framework and is able to derive substantial training signals from the massive unlabeled data, which significantly contributes to the retrieval quality.
- Tell Me How to Ask Again: Question Data Augmentation with Controllable Rewriting in Continuous Space40
A novel data augmentation method, referred to as Controllable Rewriting based Question Data Augmentation (CRQDA), for machine reading comprehension, question generation, and question-answering natural language inference tasks.
- Hashtag recommendation for multimodal microblog posts40
This work proposes a novel generative method incorporating textual and visual information to solve the task of recommending hashtags for multimodal microblogs and demonstrates that the proposed method outperforms state-of-the-art methods using either textual or visual information.
- Time-aware Personalized Hashtag Recommendation on Social Media40
A novel method is proposed that extends the translation based model and incorporates the temporal and personal factors and incorporates extraction strategies into it to overcome the limitation of only being able to recommend hashtags that exist in the training data of the existing methods.
- 35
- PROD: Progressive Distillation for Dense Retrieval34
PROD achieves the state-of-the-art in the distillation methods for dense retrieval and surpasses most of the existing 12-layer models on all five public benchmarks and introduces a regularization term in each distillation process to alleviate catastrophic forgetting.
- An Enhanced Knowledge Injection Model for Commonsense Generation32
This work integrates two additional modules into the pretrained encoder-decoder model for prototype modeling to enhance the knowledge injection procedure and shows results that significantly improves the performance on all the metrics.
- MS MARCO Web Search: A Large-scale Information-rich Web Dataset with Millions of Real Click Labels31
This paper introduces MS MARCO Web Search, the first large-scale information-rich web dataset, featuring millions of real clicked query-document labels, which paves the way for future advancements in AI and system research.
- CodeRetriever: A Large Scale Contrastive Pre-Training Method for Code Search29
The proposed CodeRetriever model, which learns the function-level code semantic representations through large-scale code-text contrastive pre-training, achieves new state-of-the-art with significant improvement over existing code pre-trained models, on eleven domain/language-specific code search tasks with six programming languages in different code granularity.
- Hashtag Recommendation Using End-To-End Memory Networks with Hierarchical Attention29
End-to-end memory networks are extended and the histories of users are incorporated into the external memory and a hierarchical attention mechanism is introduced to select more appropriate histories for hashtag recommendation.
- Hashtag Recommendation Using Dirichlet Process Mixture Models Incorporating Types of Hashtags29
A novel nonparametric Bayesian method based on the Dirichlet Process Mixture Models (DPMM), which incorporates the type of hashtag as a hidden variable for the task of recommending hashtags for microblogs.
- Uncertainty-Aware Label Refinement for Sequence Labeling27
This work introduces a novel two-stage label decoding framework to model long-term label dependencies, while being much more computationally efficient and mitigating the side effects of incorrect draft labels.
- GENIUS: Sketch-based Language Model Pre-training via Extreme and Selective Masking for Text Generation and Augmentation26
GeniusAug is proposed, which first extracts the target-aware sketches from the original training set and then generates new samples based on the sketches, and is demonstrated to be a strong and ready-to-use data augmentation tool for various natural language processing (NLP) tasks.
- Neural Semantic Parsing in Low-Resource Settings with Back-Translation and Meta-Learning26
This work aims to learn a neural semantic parser when only prior knowledge about a limited number of simple rules is available, without access to either annotated programs or execution results, and improves the accuracy and stability on examples uncovered by rules.
- 25
- 24
- 23
- 22
- 22
- Diverse, Controllable, and Keyphrase-Aware: A Corpus and Method for News Multi-Headline Generation22
A multi-source Transformer decoder, which takes three sources as inputs and generates multiple headlines with keyphrases of user interests to generate multiple keyphrase-relevant headlines, achieves state-of-the-art results in terms of quality and diversity.
- Who Will You "@"?20
This paper proposes a novel method that extends the translation-based model for the mention function in microblogging services and demonstrates that the proposed method outperforms state-of-the-art approaches.
- 19
- 19
- FastSeq: Make Sequence Generation Faster18
The proposed optimization techniques include an attention cache optimization, an efficient algorithm for detecting repeated n-grams, and an asynchronous generation pipeline with parallel I/O that are general enough to be applicable to Transformer-based models.
- Phrase-based hashtag recommendation for microblog posts18
This work proposes a novel phrase-based topical translation model that uses the bag-of-phrases model to better capture the underlying topics of posted microblogs and demonstrates that the proposed method outperforms state-of-the-art methods that use the unigram model.
- 17
- 17
- 16
- Adapting LLM Agents with Universal Feedback in Communication15
The Learning through Communication (LTC) paradigm is proposed, a novel training approach enabling LLM agents to improve continuously through interactions with their environments and other agents.
- 15
- ProphetNet-Ads: A Looking Ahead Strategy for Generative Retrieval Models in Sponsored Search Engine15
Proposing a looking ahead strategy for generative retrieval models named ProphetNet-Ads, which improves the retrieval ability by directly optimizing the Trie-constrained searching space and improving the recall by 15.58\% and 18.8\% respectively.
- Exploring the Mystery of Influential Data for Mathematical Reasoning14
This work proposes a Quality-aware Diverse Selection (QaDS) strategy adaptable for mathematical reasoning, and showcases the use of QaDS in creating efficient fine-tuning mixtures with various selection ratios.
- Contextual Fine-to-Coarse Distillation for Coarse-grained Response Selection in Open-Domain Conversations14
This paper proposes a Contextual Fine-to-Coarse (CFC) distilled model for coarse-grained response selection in open-domain conversations and shows that the proposed method achieves huge improvement over all evaluation metrics compared with traditional baseline methods.
- Allies: Prompting Large Language Model with Beam Search11
Allies leverages LLMs to iteratively generate new queries related to the original query, enabling an iterative reasoning process and captures and utilizes hidden knowledge that may not be directly obtainable through retrieval.
- Integrative Decoding: Improve Factuality via Implicit Self-consistency10
Integrative Decoding implicitly incorporates self-consistency in the decoding objective and consistently enhances factuality over a wide range of language models, with substantial improvements on the TruthfulQA, Biographies and LongFact benchmarks.
- 10
- Graph-Based Transformer with Cross-Candidate Verification for Semantic Parsing10
A graph-based Transformer for semantic parsing that incorporates graph information to Transformer, and design a cross-candidate verification mechanism to consider all the candidates in the ranking process, and integrates BERT into the model and jointly train the graph- based Transformer and BERT.
- 8
- 7
- Generative Prompt Internalization7
This work proposes Generative Prompt Internalization (GenPI), a lightweight method that employs a joint training approach that internalizes complex prompts and introduces a data synthesis technique that autonomously collects conversational datasets by swapping the roles of the agent and environment.
- 7
- 7
- Multi-level Alignment Pretraining for Multi-lingual Semantic Parsing7
A multi-level alignment pretraining method in a unified architecture formulti-lingual semantic parsing using an adversarial training method toalign the space of different languages and use sentence level and word level parallel corpus assupervision information to align the semantic of different language.
- 6
- Mixture of Neuron Experts6
The proposed Mixture of Neuron Experts (MoNE) achieves neuron-granular expert selection by only applying a simple top-k selection within each expert, incurs negligible latency, and requires no additional routing parameters or inter-expert communication.
- 6
- Alchemy: Amplifying Theorem-Proving Capability through Symbolic Mutation5
Alchemy, a general framework for data synthesis that constructs formal theorems through symbolic mutation, is proposed, which increases the number of theorems in Mathlib by an order of magnitude.
- 5
- From Patches to Trajectories: Privileged Process Supervision for Software-Engineering Agents4
Patches-to-Trajectories (P2T), which uses p-star as privileged information during curation and formulates trajectory construction as bi-objective optimization over per-step effectiveness and trajectory length improves effectiveness and efficiency over outcome-filtered SFT and its tool-error-masking variant.
- 4
- Sigma: Differential Rescaling of Query, Key and Value for Efficient Language Models4
Rigorous theoretical and empirical analyses reveal that DiffQKV attention significantly enhances efficiency, achieving up to a 33.36% improvement in inference speed over the conventional grouped-query attention (GQA) in long-context scenarios.
- Ensuring Safe and High-Quality Outputs: A Guideline Library Approach for Language Models4
This work introduces Guide-Align, a two-stage approach that customizes guidelines to accommodate diverse inputs, thereby enhancing the fine-grainedness and comprehensiveness of the guideline library and incorporates safety expertise from a safety-trained LLM through a lightweight retrieval model.
- 4
- 4
- Weakly Supervised Multi-task Learning for Semantic Parsing4
A weakly supervised learning method to enhance the authors' multi-task learning model with paraphrase data, based on the idea that the paraphrased questions should have the same logical form and question type information is proposed.
- Map search via a factor graph model4
A novel factor graph-based semi-structured retrieval framework is introduced to incorporate concept weighting, attribute selection, and word-based similarity metrics together and demonstrates that this method outperforms both state-of-the-art semi- Structured retrieval methods and some commercial systems in retrieving freeform location queries.
- Detecting Spammers in Community Question Answering4
This paper addresses the issue of detecting spammers on CQA sites by formulating the task as an optimization problem, and shows that the proposed method can achieve better performance than some state-of-the-art methods.
- DeepThink: Aligning Language Models with Domain-Specific User Intents3
A novel framework called DeepThink is proposed to generate high-quality instructions that first generates a few seed questions to mimic actual user questions, simulates conversations to uncover the hidden user needs, and refines the answer by conversational contexts and the retrieved documents for more comprehensive answers.
- Ensembling Large Language Models with Process Reward-Guided Tree Search for Better Complex Reasoning3
- LEAD: Liberal Feature-based Distillation for Dense Retrieval3
LEAD aligns the distribution between the intermediate layers of teacher model and student model, which is effective, extendable, portable and has no requirements on vocabularies, tokenizers, or model architectures.
- A Self-Paced Mixed Distillation Method for Non-Autoregressive Generation3
A novel self-paced mixed distillation method based on the AR stream knowledge that improves the generation quality and has no influence on the inference latency of BANG and achieves more than 7x speedup.
- 3
- 3
- Aggregating Bidirectional Encoder Representations Using MatchLSTM for Sequence Matching3
An aggregation method to combine the Bidirectional Encoder Representations from Transformer (BERT) with a MatchLSTM layer for Sequence Matching with significantly improvement compared with state-of-the-art methods on both datasets.
- 2
- SIGMA: An AI-Empowered Training Stack on Early-Life Hardware2
SIGMA is an open-source training stack designed to improve the reliability, stability, and efficiency of large-scale distributed training on early-life AI hardware, offering a robust, cost-effective alternative to prevailing established accelerator stacks and significantly advancing AI capabilities and scalability.
- CULG: Commercial Universal Language Generation2
This work proposes CULG, a large-scale commercial universal language generation model which is pre-trained on a corpus drawn from 10 markets across 7 languages, and demonstrates that the proposed strategy yields performance improvements on three generation tasks as compared to single-stage pre-training.
- 2
- m 3 BERT: A Modern, Multi-lingual, Matryoshka Bidirectional Encoder1
m3BERT: a Modern, Multi-lingual, Matryoshka Bidirectional Encoder, which features a novel pretraining strategy that jointly optimizes representations across both transformer layers and multiple embedding dimensions enables a single model to be tailored to varied resource and accuracy targets while maintaining consistency with pretraining.
- LayerNorm Induces Recency Bias in Transformer Decoders1
This work shows that stacked causal self-attention layers combined with LayerNorm induce recency bias, and examines the effects of residual connections and the distribution of input token embeddings on this bias.
- 1
- Sigma-MoE-Tiny Technical Report1
This work presents Sigma-MoE-Tiny, an MoE language model that achieves the highest sparsity compared to existing open-source models, and provides an in-depth discussion of load balancing in highly sparse MoE models, offering insights for advancing sparsity in future MoE architectures.
- Enhancing Large Language Model Performance with Gradient-Based Parameter Selection1
Gradient-Mask Tuning (GMT) is proposed, a method that selectively updates parameters based on gradient information, which is specific to the target tasks, and exhibits insensitivity to mask ratio and possesses computational efficiency comparable to vanilla training approach.
- Learning from the Best, Differently: A Diversity-Driven Rethinking on Data Selection1
The Orthogonal Diversity-Aware Selection (ODiS) algorithm is proposed, which preserves both quality and diversity during data selection, and constructs the training dataset by selecting top-scored data within each orthogonal dimension, thereby ensuring both quality and diversity.
- 1
- 1
- Hierarchical Dirichlet Processes with Social Influence1
A novel non-parametric model, social hierarchical Dirichlet process (sHDP), is proposed, which assumes that the topic distributions of documents are similar to each other if their authors have relations in social networks.
- –
- –
- –
- –
- –
- –
- –
- –
- –
- A Generative Model for Identifying Target Companies of Microblogs–
A novel method is proposed which converts the target company identification problem to the translation process from content to targets, and introduces a topic-specific generative method to model thetranslation process.
Publication data from OpenAlex, with missing venues and authors filled in from Crossref; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-14. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.
Report an error
Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.