Is this you? Claim this profile to correct it, add a bio and choose the work people see first.
Claim this profileAcademic lineage
Possible advisorsa guess from early papers, not confirmed
- Suggested from co-authorship
Is this you? Claim this profile to confirm or dismiss it.
Works93 from public data
- Qwen2.5 Technical Report5,408
The open-weight flagship Qwen2.5-72B-Instruct outperforms a number of open and proprietary models and demonstrates competitive performance to the state-of-the-art open-weight model, Llama-3-405B-Instruct, which is around 5 times larger.
- Qwen2 Technical Report2,548
A comprehensive suite of foundational and instruction-tuned language models, encompassing a parameter range from 0.5 to 72 billion, featuring dense models and a Mixture-of-Experts model, which surpasses most prior open-weight models and demonstrates robust multilingual capabilities.
- Modeling Localness for Self-Attention Networks196
This work cast localness modeling as a learnable Gaussian bias, which indicates the central and scope of the local region to be paid more attention in self-attention networks, to maintain the strength of capturing long distance dependencies while enhance the ability of capturing short-range dependencies.
- Multi-Head Attention with Disagreement Regularization190
This work proposes three types of disagreement regularization, which respectively encourage the subspace, the attended positions, and the output representation associated with each attention head to be different from other heads.
- Convolutional Self-Attention Networks137
Novel convolutional self-attention networks are proposed, which offer SANs the abilities to strengthen dependencies among neighboring elements, and model the interaction between features extracted by multiple attention heads.
- Context-Aware Self-Attention Networks124
This work proposes to contextualize the transformations of the query and key layers, which are used to calculates the relevance between elements, and leverage the internal representations that embed both global and deep contexts, thus avoid relying on external resources.
- 111
- Uncertainty-Aware Curriculum Learning for Neural Machine Translation111
This work proposes uncertainty-aware curriculum learning, motivated by the intuition that the higher the uncertainty in a translation pair, the more complex and rarer the information it contains; and the end of the decline in model uncertainty indicates the completeness of current training stage.
- 99
- Modeling Recurrence for Transformer87
This work proposes to directly model recurrence for Transformer with an additional recurrence encoder, and introduces a novel attentive recurrent network to leverage the strengths of both attention models and recurrent networks.
- Context-aware Self-Attention Networks for Natural Language Processing67
This work casts local context modeling as a learnable Gaussian bias, which indicates the central and scope of the local region to be paid more attention and leverage the internal representations that embed sentence-level information as the global context to contextualize the transformations of query and key layers.
- Tailor: A Soft-Prompt-Based Approach to Attribute-Based Controlled Text Generation57
The proposed Tailor represents each attribute as a pre- trained continuous vector i.e., single-attribute prompt, which guides the generation of a fixed pre-trained language model (PLM) to satisfy aPre-specified attribute.
- Self-Paced Learning for Neural Machine Translation54
Experimental results over multiple translation tasks demonstrate that the proposed NMT model yields better performance than strong baselines and those models trained with human-designed curricula on both translation quality and convergence speed.
- Leveraging Local and Global Patterns for Self-Attention Networks51
The extensive analyses verify that the two types of contexts are complementary to each other, and the proposed hybrid attention mechanism gives highly effective improvements in their integration.
- Information Aggregation for Multi-Head Attention with Routing-by-Agreement50
Experimental results on linguistic probing tasks and machine translation tasks prove the superiority of the advanced information aggregation over the standard linear transformation.
- UniTE: Unified Translation Evaluation47
This paper proposes monotonic regional attention to control the interaction among input segments, and unified pretraining to better adapt multi-task training and can universally surpass various state-of-the-art or winner methods across tasks.
- Challenges of Neural Machine Translation for Short Texts41
Two challenges in NMT for STs associated with translation error types above, respectively are summarized: the imbalanced length distribution in training set intensifies model inference calibration over STs, leading to more over-translation cases on STs; and the lack of contextual information forces NMT to have higher data uncertainty on short sentences, and thus NMT model is troubled by considerable mistranslation errors.
- Frequency-Aware Contrastive Learning for Neural Machine Translation36
A frequency-aware token-level contrastive learning method, in which the hidden state of each decoding step is pushed away from the counterparts of other target words in a soft contrastive way based on the corresponding word frequencies, which can not only significantly improve the translation quality but also enhance lexical diversity and optimize word representation space.
- Assessing the Ability of Self-Attention Networks to Learn Word Order36
Experimental results reveal that: 1) SAN trained on word reordering detection indeed has difficulty learning the positional information even with the position embedding; and 2) SANtrained on machine translation learns better positional information than its RNN counterpart, in which position embeddedding plays a critical role.
- Towards Bidirectional Hierarchical Representations for Attention-based Neural Machine Translation33
A hierarchical attentional neural translation model which focuses on enhancing source-side hierarchical representations by covering both local and global semantic information using a bidirectional tree-based encoder to alleviate the out-of-vocabulary (OOV) problem.
- Tailor: A Prompt-Based Approach to Attribute-Based Controlled Text Generation31
The proposed Tailor represents each attribute as a pre-trained continuous vector and guides the generation of a fixed PLM switch to aPre-specified attribute, which can be concatenated with any two single-attribute prompts to multi-attribute text generation.
- 29
- 29
- 28
- 28
Publication data from OpenAlex, with missing venues and authors filled in from Crossref; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-13. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.
Report an error
Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.