Is this you? Claim this profile to correct it, add a bio and choose the work people see first.
Claim this profileWorks438 from public data
- How to Fine-Tune BERT for Text Classification?1,861
A general solution for BERT fine-tuning is provided and new state-of-the-art results on eight widely-studied text classification datasets are obtained.
- Pre-trained models for natural language processing: A survey1,752
This survey is purposed to be a hands-on guide for understanding, using, and developing PTMs for various NLP tasks.
- 1,008
- Adversarial Multi-task Learning for Text Classification666
This paper proposes an adversarial multi-task learning framework, alleviating the shared and private latent feature spaces from interfering with each other, and shows that the shared knowledge learned can be regarded as off-the-shelf knowledge and easily transferred to new tasks.
- The rise and potential of large language model based agents: a survey656
A comprehensive survey on LLM-based agents is conducted, covering their construction frameworks, application scenarios, and the exploration of societies built upon LLM-based agents.
- K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapters648
K-Adapter is proposed, which remains the original parameters of the pre-trained model fixed and supports continual knowledge infusion and captures richer factual and commonsense knowledge than RoBERTa.
- Extractive Summarization as Text Matching529
This paper forms the extractive summarization task as a semantic text matching problem, in which a source document and candidate summaries will be matched in a semantic space to create a semantic matching framework.
- A Comprehensive Capability Analysis of GPT-3 and GPT-3.5 Series Models504
The extensive experiments reveal that the overall ability of GPT series models on NLU tasks does not increase gradually as the models evolve, especially with the introduction of the RLHF training strategy, and indicates that there is still room for improvement in areas such as model robustness.
- FLAT: Chinese NER Using Flat-Lattice Transformer459
Experiments show FLAT outperforms other lexicon-based models in performance and efficiency and can fully leverage the lattice information and has an excellent parallel ability.
- Adaptive Co-attention Network for Named Entity Recognition in Tweets436
A bi-directional long short term memory network with conditional random fields and an adaptive co-attention network is extended to achieve named entity recognition for tweets to make full use of textual and visual information.
- Phrase dependency parsing for opinion mining435
A novel approach for mining opinions from product reviews is presented, where it converts opinion mining task to identify product features, expressions of opinions and relations between them by taking advantage of the observation that a lot of product features are phrases.
- Simplify the Usage of Lexicon in Chinese NER311
This work proposes a simple but effective method for incorporating the word lexicon into the character representations of Chinese named entity recognition using word lexicons, and shows that the proposed method can be easily incorporated with pre-trained models like BERT.
- Heterogeneous Graph Neural Networks for Extractive Document Summarization306
This paper presents a heterogeneous graph-based neural network for extractive summarization (HETERSUMGRAPH), which contains semantic nodes of different granularity levels apart from sentences that act as the intermediary between sentences and enrich the cross-sentence relations.
- Long Short-Term Memory Neural Networks for Chinese Word Segmentation293
A novel neural network model for Chinese word segmentation is proposed, which adopts the long short-term memory (LSTM) neural network to keep the previous important information in memory cell and avoids the limit of window size of local context.
- Secrets of RLHF in Large Language Models Part I: PPO290
The PPO-max, an advanced version of PPO algorithm, is explored, to efficiently improve the training stability of the policy model, and a comprehensive analysis of RLHF abilities compared with SFT models and ChatGPT is performed.
- GlossBERT: BERT for Word Sense Disambiguation with Gloss Knowledge289
This paper construct context-gloss pairs and propose three BERT based models for WSD and fine-tune the pre-trained BERT model to achieve new state-of-the-art results on WSD task.
- Memory in the Age of AI Agents288
This work aims to provide an up-to-date landscape of current agent memory research, clearly delineating the scope of agent memory and distinguishing it from related concepts such as LLM memory, retrieval augmented generation (RAG), and context engineering, and articulate a forward-looking perspective on emerging research frontiers.
- Convolutional neural tensor network architecture for community-based question answering279
This paper proposes a convolutional neural tensor network architecture to encode the sentences in semantic space and model their interactions with a tensor layer, which outperforms the other methods on two matching tasks.
- CNN-Based Chinese NER with Lexicon Rethinking274
This work proposes a faster alternative to Chinese NER: a convolutional neural network (CNN)-based method that incorporates lexicons using a rethinking mechanism that can model all the characters and potential words that match the sentence in parallel.
- Incorporating Corporation Relationship via Graph Convolutional Neural Networks for Stock Price Prediction265
Experiments on the data collected from stock market in Mainland China show that the representation learned from the model is able to capture relationships between corporations, and prediction models incorporating related corporations' information are able to make more accurate predictions on stock market.
- A Lexicon-Based Graph Neural Network for Chinese NER256
A lexicon-based graph neural network with global semantics is introduced, in which lexicon knowledge is used to connect characters to capture the local composition, while a global relay node can capture global sentence semantics and long-range dependency.
- CoLAKE: Contextualized Language and Knowledge Embedding228
The Contextualized Language and Knowledge Embedding (CoLAKE) is proposed, which jointly learns contextualized representation for both language and knowledge with the extended MLM objective, and achieves surprisingly high performance on a synthetic task called word-knowledge graph completion, which shows the superiority of simultaneously contextualizing language andknowledge representation.
- Style Transformer: Unpaired Text Style Transfer without Disentangled Latent Representation222
The Style Transformer is proposed, which makes no assumption about the latent representation of source sentence and equips the power of attention mechanism in Transformer to achieve better style transfer and better content preservation.
- DISC-LawLLM: Fine-tuning Large Language Models for Intelligent Legal Services183
Quantitative and qualitative results on DISC-Law-Eval demonstrate the effectiveness of the system in serving various users across diverse legal scenarios, and enhances models' ability to access and utilize external legal knowledge.
- Multi-Timescale Long Short-Term Memory Neural Network for Modelling Sentences and Documents180
A multi-timescale long short-termmemory (MT-LSTM) neural network to model long texts that can model very long documents as well as short sentences is proposed.
- Secrets of RLHF in Large Language Models Part II: Reward Modeling179
This report introduces contrastive learning to enhance the ability of reward models to distinguish between chosen and rejected responses, thereby improving model generalization and employs meta-learning to enable the reward model to maintain the ability to differentiate subtle differences in out-of-distribution samples.
- Adversarial Multi-Criteria Learning for Chinese Word Segmentation178
This paper proposes adversarial multi-criteria learning for CWS by integrating shared knowledge from multiple heterogeneous segmentation criteria, and shows that the performance of each corpus obtains a significant improvement, compared to single-criterion learning.
- Searching for Effective Neural Extractive Summarization: What Works and What’s Next165
This paper seeks to better understand how neural extractive summarization systems could benefit from different types of model architectures, transferable knowledge and learning schemas, and finds an effective way to improve the current framework.
- Attention-Based Convolutional Neural Network for Semantic Relation Extraction164
This model makes full use of word embedding, part-of-speech tag embedding and position embedding information and enables learning some important features from task-specific labeled data, forgoing the need for external knowledge such as explicit dependency structures.
- Llama Scope: Extracting Millions of Features from Llama-3.1-8B with Sparse Autoencoders160
A suite of 256 SAEs, trained on each layer and sublayer of the Llama-3.1-8B-Base model, with 32K and 128K features are introduced and modifications to a state-of-the-art SAE variant, Top-K SAEs, are evaluated across multiple dimensions.
- Learning Sparse Sharing Architectures for Multiple Tasks159
This paper proposes a novel parameter sharing mechanism, named Sparse Sharing, which achieves consistent improvement while requiring fewer parameters, and shows that both hard sharing and hierarchical sharing can be formulated as particular instances of the sparse sharing framework.
- Generating Responses with a Specific Emotion in Dialog155
This work proposes an emotional dialogue system (EmoDS) that can generate the meaningful responses with a coherent structure for a post, and meanwhile express the desired emotion explicitly or implicitly within a unified framework.
- SpanNER: Named Entity Re-/Recognition as Span Prediction140
It is revealed that span prediction, simultaneously, can serve as a system combiner to re-recognize named entities from different systems’ outputs to make complementary advantages of systems based on different paradigms.
- Hashtag Recommendation for Multimodal Microblog Using Co-Attention Network139
This work proposes a co-attention network incorporating textual and visual information to recommend hashtags for multimodal tweets and demonstrates that the proposed method can achieve better performance than state-of-the-art methods using textual information only.
- Knowledge Graph Representation with Jointly Structural and Textual Encoding136
This paper introduces three neural models to encode the valuable information from text description of entity, among which an attentive model can select related information as needed, and proposes a novel deep architecture to utilize both structural and textual information of entities.
- LLaMA Beyond English: An Empirical Study on Language Capability Transfer124
An extensive empirical investigation based on LLaMA is conducted, demonstrating that comparable performance to state-of-the-art transfer models can be achieved with less than 1% of the pretraining data, both in terms of knowledge alignment and response quality.
- Gated Recursive Neural Network for Chinese Word Segmentation121
A gated recursive neural network (GRNN) for Chinese word segmentation is proposed, which contains reset and update gates to incorporate the complicated combinations of the context characters.
- Retweet Prediction with Attention-based Deep Neural Network120
A novel attention-based deep neural network to incorporate contextual and social information for this task and experimental results showed that the proposed method could achieve better results than the previous state-of-the-art methods.
- From Individual to Society: A Survey on Social Simulation Driven by Large Language Model-based Agents116
A comprehensive survey of simulation driven by large language models, illustrating the recent progress in simulation driven by LLM-empowered agents and categorizing the simulations into three types: Individual Simulation, which mimics specific individuals or demographic groups; Scenario Simulation, where multiple agents collaborate to achieve goals within specific contexts; and Society Simulation, which models interactions within agent societies.
- Trainable Undersampling for Class-Imbalance Learning116
A meta-learning method built on the undersampling to parametrize the data sampler and train it to optimize the classification performance over the evaluation metric through reinforcement learning, which solves the non-differentiable optimization problem for training the data Sampler via reinforcement learning.
- How Robust is GPT-3.5 to Predecessors? A Comprehensive Study on Language Understanding Tasks114
While GPT-3.5 outperforms existing fine-tuned models on some tasks, it still encounters significant robustness degradation, such as its average performance dropping by up to 35.74\% and 43.59\% in natural language inference and sentiment analysis tasks, respectively.
- AgentGym: Evolving Large Language Model-based Agents across Diverse Environments113
This paper proposes AgentGym, a new framework featuring a variety of environments and tasks for broad, real-time, uni-format, and concurrent agent exploration, and proposes a novel method, AgentEvol, to investigate the potential of agent self-evolution beyond previously seen data across tasks and environments.
- Distantly Supervised Named Entity Recognition using Positive-Unlabeled Learning111
A novel PU learning algorithm is proposed to perform named entity recognition using only unlabeled data and named entity dictionaries that greatly reduces the requirement on the quality of the dictionaries and makes the method generalize well with quite simple dictionaries.
- Toward Diverse Text Generation with Inverse Reinforcement Learning110
In inverse reinforcement learning (IRL), the IRL framework learns a reward function on training data, and then an optimal policy to maximum the expected total reward, which can produce more dense reward signals.
- Learning context-sensitive word embeddings with neural tensor skip-gram model106
A general architecture to learn the word and topic embeddings efficiently is presented, which is an extension to the Skip-Gram model and can model the interaction between words and topics simultaneously.
- Aligning Large Language Models with Human Preferences through Representation Engineering101
This novel approach, denoted as Representation Alignment from Human Feedback (RAHF), proves to be effective, computationally efficient, and easy to implement and its versatility in accommodating diverse human preferences shows its potential for advancing LLM performance.
- Defense against Synonym Substitution-based Adversarial Attacks via Dirichlet Neighborhood Ensemble99
Dirichlet Neighborhood Ensemble is proposed, a randomized method for training a robust model to defense synonym substitution-based attacks that consistently outperforms recently proposed defense methods by a significant margin across different network architectures and multiple data sets.
- StepCoder: Improve Code Generation with Reinforcement Learning from Compiler Feedback98
StepCoder is introduced, a novel RL framework for code generation, consisting of two main components: CCCS addresses the exploration challenge by breaking the long sequences code generation task into a Curriculum of Code Completion Subtasks, while FGO only optimizes the model by masking the unexecuted code segments to provide Fine-Grained Optimization.
- Cross-Domain Sentiment Classification with Target Domain Specific Information98
This work proposes a method to simultaneously extract domain specific and invariant representations and train a classifier on each of the representation, respectively, and introduces a few target domain labeled data for learning domain-specific information.
- A unified relevance model for opinion retrieval95
A series of methods to automatically identify the most appropriate opinion words for query expansion, including using query independent sentiment resources, and a relevance feedback-based approach to extract opinion words are explored.
- Part-of-Speech Tagging for Twitter with Adversarial Neural Networks94
A novel neural network to make use of out-of-domain labeled data, unlabeled in- domain data, and labeled in-domain data is proposed to learn common features through adversarial discriminator for Tweets tagging.
- A benchmark for automatic medical consultation system: frameworks, tasks and datasets93
This article creates a new large medical dialogue dataset with multi-level fine-grained annotations and establishes five independent tasks, including named entity recognition, dialogue act classification, symptom label inference, medical report generation and diagnosis-oriented dialogue policy.
- Meta Multi-Task Learning for Sequence Modeling93
A shared meta-network is used to capture the meta-knowledge of semantic composition and generate the parameters of the task-specific semantic composition models in a new sharing scheme of composition function across multiple tasks.
- Paradigm Shift in Natural Language Processing92
This paper highlights several paradigms that have the potential to solve different NLP tasks, including the sequence labeling paradigm, and the classification paradigm, to build a single model to handle diverse tasks.
- 90
- DISC-FinLLM: A Chinese Financial Large Language Model based on Multiple Experts Fine-tuning85
This work proposes Multiple Experts Fine-tuning Framework, a methodology that improves general LLMs by endowing them with multi-turn question answering abilities, domain text processing capabilities, mathematical computation skills, and retrieval-enhanced generation capabilities.
- Implicit Discourse Relation Detection via a Deep Architecture with Gated Relevance Network84
The use of word embeddings to replace the original words is proposed to overcome the data sparsity problem, and a gated relevance network is adopted to capture the semantic interaction between word pairs.
- Using query expansion in graph-based approach for query-focused multi-document summarization84
A novel query expansion method is presented, which is combined in the graph-based algorithm for query-focused multi-document summarization, so as to resolve the problem of information limit in the original query.
- Improving BERT Fine-Tuning via Self-Ensemble and Self-Distillation83
This paper improves the fine-tuning of BERT with two effective mechanisms: self-ensemble and self-distillation, and achieves a new state-of-the-art result on the SNLI dataset.
- BBTv2: Towards a Gradient-Free Future with Large Language Models83
BBTv2 is presented, an improved version of Black-Box Tuning, to drive PTMs for few-shot learning and a divide-and-conquer gradient-free algorithm to optimize the prompts at different layers alternately is proposed.
- FudanNLP: A Toolkit for Chinese Natural Language Processing82
FudanNLP is an open source toolkit for Chinese natural language processing (NLP), which uses statistics-based and rule-based methods to deal with Chinese NLP tasks, such as word segmentation, part-ofspeech tagging, named entity recognition, dependency parsing, time phrase recognition, anaphora resolution and so on.
- AgentPRM: Process Reward Models for LLM Agents via Step-Wise Promise and Progress81
This work explores constructing process reward models (PRMs) to evaluate each decision and guide the agent's decision-making process, and proposes a re-defined PRM for agent tasks, named AgentPRM, to capture both the interdependence between sequential decisions and their contribution to the final goal.
- 80
- Storytelling from an Image Stream Using Scene Graphs78
This work proposes a novel graph-based architecture for visual storytelling by modeling the two-level relationships on scene graphs and results indicate that this method achieves state-of-the-art results.
- Automatic Essay Scoring Incorporating Rating Schema via Reinforcement Learning77
A reinforcement learning framework for essay scoring that incorporates quadratic weighted kappa as guidance to optimize the scoring system and experiment results show the effectiveness of this framework.
- 77
- StepCoder: Improving Code Generation with Reinforcement Learning from Compiler Feedback76
StepCoder is introduced, a novel RL framework for code generation, consisting of two main components: CCCS addresses the exploration challenge by breaking the long sequences code generation task into a C urriculum of C ode C ompletion S ubtasks, while FGO only optimizes the model by masking the unexecuted code segments to provide F ine-G rained O ptimization.
- Incorporate Group Information to Enhance Network Embedding76
A novel method for learning the network embeddings with valuable group information for large-scale networks that takes both the inner structures of the groups and the information across groups into consideration is investigated.
- 75
- 73
- Below Chance: Code and Derived Data for an Audit of the TRACE Continual-Learning Benchmark and a Census of Published Sequential Baselines70
TRACE is introduced, a novel benchmark designed to evaluate continual learning in LLMs and the Reasoning-augmented Continual Learning (RCL) approach, which integrates task-specific cues with meta-rationales, effectively reducing catastrophic forgetting in LLM while expediting convergence on novel tasks.
- Deep Fusion LSTMs for Text Semantic Matching69
This paper proposes a model of deep fusion LSTMs (DF-LSTMs) to model the strong interaction of text pair in a recursive matching way, and uses exter-nal memory to increase the capacity of LSTm, thereby possibly capturing more complicated matching patterns.
- Topic-Oriented Spoken Dialogue Summarization for Customer Service with Saliency-Aware Topic Modeling65
This work proposes a novel topic-augmented two-stage dialogue summarizer (TDS) jointly with a saliency-aware neural topic model (SATM) for topic-oriented summarization of customer service dialogues.
- 64
- Math Word Problem Solving with Explicit Numerical Values63
A novel approach called NumS2T is proposed, which enhances math word problem solving performance by explicitly incorporating numerical values into a sequence-to-tree network.
- CQG: A Simple and Effective Controlled Generation Framework for Multi-hop Question Generation62
The CQG is proposed, which is a simple and effective controlled framework that employs a simple method to generate the multi-hop questions that contain key entities inMulti-hop reasoning chains, which ensure the complexity and quality of the questions.
- Structural Opinion Mining for Graph-based Sentiment Representation61
A novel graph-based representation for sentence level sentiment is proposed and an integer linear programming-based structural learning method is introduced to produce the graph representations of input sentences.
- Scaling of Search and Learning: A Roadmap to Reproduce o1 from Reinforcement Learning Perspective60
The roadmap to achieving o1 is analyzed from the perspective of reinforcement learning, focusing on four key components: policy initialization, reward design, search, and learning, which underscore how learning and search drive o1's advancement.
- Natural Language Understanding and Intelligent Applications60
A novel word sense disambiguation (WSD) discriminative model is proposed which integrates structural context with the local context to handle long distance sense dependency and multireference lexicon dependency within the sentence.
- Investigating Language Universal and Specific Properties in Word Embeddings60
Three representative models are investigated on a large set of language samples by mapping dense embedding to sparse linguistic property space to reveal the language universal and specific properties encoded in various word representation.
- RoTBench: A Multi-Level Benchmark for Evaluating the Robustness of Large Language Models in Tool Learning58
This work introduces *RoTBench*, a multi-level benchmark for evaluating the robustness of LLMs in tool learning, and proposes RoTTuning, a strategy that enriches the diversity of training environments to bolster the robustness of LLMs in tool learning.
- A Question Type Driven Framework to Diversify Visual Question Generation58
A question type driven framework to produce multiple questions for a given image with different focuses is proposed, which outperforms the state-of-the-art approaches in terms of both relevance and diversity.
- Deep Multi-Task Learning with Shared Memory for Text Classification57
Experiments on two groups of text classification tasks show that the proposed architectures can improve the performance of a task with the help of other related tasks.
- BAPO: Stabilizing Off-Policy Reinforcement Learning for LLMs via Balanced Policy Optimization with Adaptive Clipping56
This work proposes BAlanced Policy Optimization with Adaptive Clipping with Adaptive Clipping (BAPO), a simple yet effective method that dynamically adjusts clipping bounds to adaptively re-balance positive and negative contributions, preserve entropy, and stabilize RL optimization.
- Adaptive Multi-Attention Network Incorporating Answer Information for Duplicate Question Detection56
An answer information- enhanced adaptive multi-attention network (AMAN) is proposed to perform this task, which takes full advantage of the semantic information in the paired answers while alleviating the noise problem caused by adding the answers.
- MOSS: An Open Conversational Large Language Model54
The development of MOSS is reported, an open-sourced conversational LLM that contains 16 B parameters and can perform a variety of instructions in multi-turn interactions with humans and establishes a complete technical roadmap for large language models from pre-training, supervised fine-tuning to alignment.
- 54
- Retweet Behavior Prediction Using Hierarchical Dirichlet Process54
This work proposes a novel method using non-parametric statistical models to combine structural, textual, and temporal information together to predict retweet behavior and demonstrates that the proposed method can achieve better performance than state-of-the-art methods.
- Enhancing Scientific Papers Summarization with Citation Graph53
This work redefines the task of scientific papers summarization by utilizing their citation graph and proposes a citation graph-based summarization model CGSum which can incorporate the information of both the source paper and its references.
- Contextualized Non-Local Neural Networks for Sequence Learning53
Experimental results on ten NLP tasks in text classification, semantic matching, and sequence labeling show that the proposed model outperforms competitive baselines and discovers task-specific dependency structures, thus providing better interpretability to users.
- A Reinforcement Learning Framework for Natural Question Generation using Bi-discriminators51
This work argues that natural questions need to have two specific attributes from the perspectives of content and linguistic respectively, namely, natural and human-written, and proposes two discriminators, one for each attribute, to enhance the training.
- 50
- Reasoning in Flux: Enhancing Large Language Models Reasoning through Uncertainty-aware Adaptive Guidance48
Uncertainty-aware Adaptive Guidance (UAG) is introduced, a novel approach for guiding LLM reasoning onto an accurate and reliable trajectory that not only enhances the reasoning abilities of LLMs but also consistently outperforms several strong baselines with minimal computational overhead.
- Evaluating and Enhancing the Robustness of Neural Network-based Dependency Parsing Models with Adversarial Examples48
This study shows that adversarial examples also exist in dependency parsing, and proposes two approaches to study where and how parsers make mistakes by searching over perturbations to existing texts at sentence and phrase levels, and design algorithms to construct such examples in both of the black-box and white-box settings.
- A Multi-Agent Communication Framework for Question-Worthy Phrase Extraction and Question Generation48
A multi-agent communication framework is introduced, taking phrase extraction and question generation as two agents, and learn these two tasks simultaneously via message passing mechanism and can extract question-worthy phrases, which are able to improve the performance of question generation.
- Modelling Interaction of Sentence Pair with Coupled-LSTMs48
This paper introduces two coupled ways to model the interdependences of two LSTMs, coupling the local contextualized interactions of two sentences and uses a dynamic pooling to select the most informative features.
- A Re-ranking Model for Dependency Parser with Recursive Convolutional Neural Network48
This work proposes a recursive convolutional neural network (RCNN) architecture to capture syntactic and compositional-semantic representations of phrases and words in a dependency tree to improve the state-of-the-art dependency parsing on both English and Chinese datasets.
- Accelerating BERT Inference for Sequence Labeling via Early-Exit47
The token-level early-exit mechanism brings the gap between training and inference, so an extra self-sampling fine-tuning stage is introduced to alleviate it and can save up to 66% ∼75% inference cost with minimal performance degradation.
- A Closer Look at Data Bias in Neural Extractive Summarization Models47
This paper proposes several properties of datasets, which matter for the generalization of summarization models, and builds the connection between priors residing in datasets and model designs, analyzing how different factors of datasets influence the choices of model structure design and training methods.
- A Lexicon-Based Supervised Attention Model for Neural Sentiment Analysis45
A novel lexicon-based supervised attention model (LBSA) is proposed, which allows a recurrent neural network to focus on the sentiment content, thus generating sentiment-informative representations and has better interpretability and less noise.
- ToolEyes: Fine-Grained Evaluation for Tool Learning Capabilities of Large Language Models in Real-world Scenarios44
A fine-grained system tailored for the evaluation of the LLMs' tool learning capabilities in authentic scenarios, analyzing five dimensions crucial to LLMs in tool learning: format alignment, intent comprehension, behavior planning, tool selection, and answer organization.
- 44
- 43
- LongAgent: Scaling Language Models to 128k Context through Multi-Agent Collaboration42
A method based on multi-agent collaboration, which scales LLMs (e.g., LLaMA) to a context of 128K and demonstrates potential superiority in long-text processing compared to GPT-4 is proposed and experimental results indicate that \textsc{LongAgent} offers a promising alternative for long-text processing.
- From Hypergraph Energy Functions to Hypergraph Neural Networks42
This paper begins by presenting an expressive family of parameterized, hypergraph-regularized energy functions, and demonstrates how minimizers of these energies effectively serve as node embeddings that, when paired with a parameterized classifier, can be trained end-to-end via a supervised bilevel optimization process.
- Efficient and Effective Time-Series Forecasting with Spiking Neural Networks41
This paper proposes a framework for SNNs in time-series forecasting tasks, leveraging the efficiency of spiking neurons in processing temporal information and demonstrates that its proposed SNN-based approaches achieve comparable or superior results to traditional time-series forecasting methods on diverse benchmarks with much less energy consumption.
- Analyzing Linguistic Knowledge in Sequential Model of Sentence41
This work proposes to correlate the neuron activation pattern of a LSTM language model with rich language features at sequential, lexical and compositional level and analyzes the influence of modelling strategy on linguistic knowledge encoded implicitly in different sequential models.
- Hashtag recommendation for multimodal microblog posts40
This work proposes a novel generative method incorporating textual and visual information to solve the task of recommending hashtags for multimodal microblogs and demonstrates that the proposed method outperforms state-of-the-art methods using either textual or visual information.
- Toward Generalizable Evaluation in the LLM Era: A Survey Beyond Benchmarks39
This survey probes the core challenges that the rise of LLMs poses for evaluation, and identifies and analyzes two pivotal transitions: from task-specific to capability-based evaluation and from manual to automated evaluation.
- Recurrent Memory Reasoning Network for Expert Finding in Community Question Answering39
A novel \textslrecurrent memory reasoning network (RMRN) is proposed to perform expert finding, which focuses on different parts of a question, and accordingly retrieves information from the histories of the candidate expert.
- Discourse Level Explanatory Relation Extraction from Product Reviews Using First-Order Logic39
A novel joint extraction method using firstorder logic to model rich linguistic features and long distance constraints is proposed for identifying subjective text segments and extracting their corresponding explanations from product reviews in discourse level.
- Thinking with Video: Video Generation as a Promising Multimodal Reasoning Paradigm38
The findings show that the video generation model is the potential unified multimodal understanding and generation model, positioning"Thinking with Video"as a potential unified multimodal reasoning paradigm.
- LLM-DA: Data Augmentation via Large Language Models for Few-Shot Named Entity Recognition38
This paper proposes $LLM-DA, a novel data augmentation technique based on LLMs for the few-shot NER task that leverages the distinctive characteristics of the NER task by augmenting the original data at both the contextual and entity levels.
- 38
- MINER: Improving Out-of-Vocabulary Named Entity Recognition from an Information Theoretic Perspective38
MINER, a novel NER learning framework, is proposed to remedy the issue of over-relying on entity mention information, resulting in poor performance on out-of-vocabulary (OOV) entity recognition.
- A Graph-based Model for Joint Chinese Word Segmentation and Dependency Parsing38
The proposed graph-based joint model achieves better performance than previous joint models and state-of-the-art results in both Chinese word segmentation and dependency parsing and when BERT is combined, can substantially reduce the performance gap of dependency parsing between joint Models and gold-segmented word-based models.
- 38
- Sentence Modeling with Gated Recursive Neural Network37
A gated recursive neural network (GRNN) to model sentences, which employs a full binary tree (FBT) structure to control the combinations in recursive structure by introducing two kinds of gates.
- 36
- SpikeBERT: A Language Spikformer Learned from BERT with Knowledge Distillation35
Through extensive experimentation, it is shown that the models trained with the method, named SpikeBERT, outperform state-of-the-art SNNs and even achieve comparable results to BERTs on text classification tasks for both English and Chinese with much less energy consumption.
- Flooding-X: Improving BERT’s Resistance to Adversarial Attacks via Loss-Restricted Fine-Tuning35
This work leverages Flooding method which primarily aims at better generalization and finds promising in defending adversarial attacks, and proposes an effective criterion to bring hyper-parameter-dependent flooding into effect with a narrowed-down search space by measuring how the gradient steps taken within one epoch affect the loss of each batch.
- SENT: Sentence-level Distant Relation Extraction via Negative Training35
This work proposes the use of negative training (NT), in which a model is trained using complementary labels regarding that “the instance does not belong to these complementary labels”, since the probability of selecting a true label as a complementary label is low, which provides less noisy information.
- 34
- A Concise Model for Multi-Criteria Chinese Word Segmentation with Transformer Encoder34
A concise but effective unified model for MCCWS, which is fully-shared for all the criteria and can segment both simplified and traditional Chinese and has an excellent transfer capability is proposed.
- 34
- Asynchronous Deep Interaction Network for Natural Language Inference34
An asynchronous deep interaction network (ADIN) is proposed that deconstructs the reasoning process and implements the asynchronous and multi-step reasoning and achieves competitive performance and outperforms strong baselines on three popular benchmarks: SNLI, MultiNLI, and SciTail.
- Incorporating Discriminator in Sentence Generation: a Gibbs Sampling Method34
This paper proposes a novel framework to generate constrained sentences via Gibbs Sampling using recurrent neural networks and their variants to predict following words given previous sequence and target label.
- TripTailor: A Real-World Benchmark for Personalized Travel Planning33
TripTailor is introduced, a benchmark designed specifically for personalized travel planning in real-world scenarios and identifies several critical challenges in travel planning, including the feasibility, rationality, and personalized customization of the proposed solutions.
- Hot Topic-Aware Retweet Prediction with Masked Self-attentive Model33
This work proposes a novel masked self-attentive model that incorporates the posting histories of users with external memory and utilize a hierarchical attention mechanism to construct the users' interests and can be jointly hot-topic aware and user interests aware to make a final prediction.
- Mention Recommendation for Twitter with End-to-end Memory Network33
This work proposed a novel end-to-end memory network architecture to perform the problem of recommending usernames when people attempt to use the ``@'' sign to mention other people in twitter-like social media and incorporated the interests of users with external memory.
- 33
- Sparse higher order conditional random fields for improved sequence labeling33
This work describes Sparse Higher Order Conditional Random Fields (SHO-CRFs), which are able to handle local features and sparse higher order features together using a novel tractable exact inference algorithm, and shows that with the same higher order feature set, SHO- CRFs significantly outperform previous approaches.
- Causal Intervention Improves Implicit Sentiment Analysis32
This work proposes a CausaL intervention model for implicit sEntiment ANalysis using instrumental variable (CLEAN), and introduces instrumental variable to eliminate the confounding causal effects, thus extracting the pure causal effect between sentence and sentiment.
- An Enhanced Knowledge Injection Model for Commonsense Generation32
This work integrates two additional modules into the pretrained encoder-decoder model for prototype modeling to enhance the knowledge injection procedure and shows results that significantly improves the performance on all the metrics.
- 32
- Long Short-Term Memory with Dynamic Skip Connections32
This work proposes a novel reinforcement learning-based method to model the dependency relationship and connect dependent words, which provides a dynamic skipping advantage over RNNs that always tackle entire sentences sequentially.
- Dynamic Compositional Neural Networks over Tree Structure32
This paper introduces the dynamic compositional neural networks over tree structure (DC-TreeNN), in which the compositional function is dynamically generated by a meta network, which captures the metaknowledge across the different compositional rules and formulate them.
- 31
- Exploration and Exploitation: Two Ways to Improve Chinese Spelling Correction Models31
Experimental results show that an adversarial training method combined with the pre-training strategy can improve both the generalization and robustness of multiple CSC models across three different datasets, achieving state-of-the-art performance for CSC task.
- 31
- A Feature-Enriched Neural Model for Joint Chinese Word Segmentation and Part-of-Speech Tagging31
A feature-enriched neural model for joint Chinese word segmentation and part-of-speech tagging task by using different filters to model the complex compositional features with convolutional and pooling layer, and then utilizing long distance dependency information with recurrent layer.
- TCIC: Theme Concepts Learning Cross Language and Vision for Image Captioning30
A Theme Concepts extended Image Captioning (TCIC) framework that incorporates theme concepts to represent high-level cross-modality semantics is proposed and experimental results on MS COCO show the effectiveness of the approach compared to some state-of-the-art models.
- Unsupervised Summarization for Chat Logs with Topic-Oriented Ranking and Context-Aware Auto-Encoders30
Experimental results show that RankAE significantly outperforms other unsupervised methods and is able to generate high-quality summaries in terms of relevance and topic coverage.
- Debatrix: Multi-dimensional Debate Judge with Iterative Chronological Analysis Based on LLM29
Debatrix is proposed, which makes the analysis and assessment of multi-turn debates more aligned with majority preferences by leveraging Large Language Models (LLMs), and features a vertical, iterative chronological analysis and a horizontal, multi-dimensional evaluation collaboration.
- COLO: A Contrastive Learning based Re-ranking Framework for One-Stage Summarization29
A Contrastive Learning based re-ranking framework for one-stage summarization called CoLo is proposed, which shows that the summarization model is able to directly generate summaries according to the summary-level score without additional modules and parameters.
- CDEvalSumm: An Empirical Study of Cross-Dataset Evaluation for Neural Summarization Systems29
An in-depth analysis of characteristics of different datasets is performed and the performance of different summarization models under a cross-dataset setting is investigated, in which a summarizer trained on one corpus will be evaluated on a range of out-of-domain corpora.
- MouSi: Poly-Visual-Expert Vision-Language Models28
This paper proposes the use of ensemble experts technique to synergizes the capabilities of individual visual encoders, including those skilled in image-text matching, OCR, image segmentation, etc, while bridging the gap between image encoders and pre-trained LLMs.
- Improving Generalization of Alignment with Human Preferences through Group Invariant Learning28
This work proposes a novel approach that can learn a consistent policy via RL across various data groups or domains, and significantly enhances training stability and model generalization.
- Incorporating Argument-Level Interactions for Persuasion Comments Evaluation using Co-attention Model28
A novel co-attention mechanism based neural network is proposed to capture the interactions between participants on argument level and significantly outperforms some state-of-the-art methods for persuasiveness evaluation.
- Deep learning prediction of glycopeptide tandem mass spectra powers glycoproteomics27
DeepGP, a hybrid deep learning framework based on Transformer and graph neural network, is presented for the prediction of MS/MS spectra and retention time of glycopeptides and it is demonstrated that DeepGP in combination with database searching can significantly increase the detection sensitivity of glycopeptides.
- Efficient Link Prediction via GNN Layers Induced by Negative Sampling27
A novel GNN architecture whereby the forward pass explicitly depends on both positive and negative edges to inform more flexible, yet still cheap node-wise embeddings, which retains the inference speed of node-wise models, while producing competitive accuracy with edge-wise alternatives.
- Multitask Pre-training of Modular Prompt for Chinese Few-Shot Learning27
This paper presents Multi-task Pre-trained Modular Prompt (MP2) to boost prompt tuning for few-shot learning and demonstrates that MP2 can achieve surprisingly fast and strong adaptation to downstream tasks by merely learning 8 parameters to combine the pre-trained modular prompts.
- 27
- Uncertainty-Aware Label Refinement for Sequence Labeling27
This work introduces a novel two-stage label decoding framework to model long-term label dependencies, while being much more computationally efficient and mitigating the side effects of incorrect draft labels.
- Bridging by Word: Image Grounded Vocabulary Construction for Visual Captioning27
A novel hierarchical structure is proposed to construct an image-grounded vocabulary, based on which, captions are generated with limitation and guidance, and the effectiveness of this framework compared to some state-of-the-art models is shown.
- Hierarchical Text Classification with Latent Concepts27
This paper claims that each class has several latent concepts and its subclasses share information with these different concepts respectively, and proposes a variant Passive-Aggressive algorithm for hierarchical text classification with latent concepts.
- Exploring the Compositional Deficiency of Large Language Models in Mathematical Reasoning Through Trap Problems26
This work constructs a new dataset MathTrap by introducing carefully designed logical traps into the problem descriptions of MATH and GSM8K and finds that LLMs’ performance can be improved through the above external intervention.
- MVPTR26
This paper proposes to learn Multi-level semantic alignment for Vision-language Pre-TRaining (MVPTR), a nested structure of both modalities to introduce concepts as high-level semantics and achieves state-of-the-art results on several vision and language tasks.
- TextFusion: Privacy-Preserving Pre-trained Model Inference via Token Fusion26
This work proposes TextFusion, a novel method for preserving inference privacy, which trains a Fusion Predictor to dynamically fuse token representations, which hides multiple private token representations behind an unrecognizable one.
- 26
- Depression Detection on Social Media with Reinforcement Learning26
This work proposes the use of reinforcement learning method to automatically select the indicator posts from the historical posts of users using only the textual information to detect depression based on the content users posted on social media sites.
- 26
- AutoLogi: Automated Generation of Logic Puzzles for Evaluating Reasoning Abilities of Large Language Models25
This approach features program-based verification and controllable difficulty levels, enabling more reliable evaluation that better distinguishes models' reasoning abilities, and develops a bilingual benchmark, AutoLogi, to obtain more accurate assessments of models' reasoning capabilities.
- Robust Lottery Tickets for Pre-trained Language Models25
This work proposes a novel method based on learning binary weight masks to identify robust tickets hidden in the original PLMs, and designs an adversarial loss objective to guide the search for robust tickets and ensure that the tickets perform well both in accuracy and robustness.
- Mention Recommendation for Multimodal Microblog with Cross-attention Memory Network25
A novel cross-attention memory network is proposed to perform the mention recommendation task for multimodal tweets that incorporates the interests of users with external memory and use the cross-Attention mechanism to extract both textual and visual information.
- Argue with Me Tersely: Towards Sentence-Level Counter-Argument Generation24
This paper presents the ArgTersely benchmark for sentence-level counter-argument generation, drawing from a manually annotated dataset from the ChangeMyView debate forum, and proposes Arg-LlaMA for generating high-quality counter-argument.
- Continuous word embeddings for detecting local text reuses at the semantic level24
A novel method to efficiently detect local reuses at the semantic level for large scale problems by using continuous vector representations of words to capture the semanticlevel similarities between short text segments is introduced.
- 23
- A Multi-Format Transfer Learning Model for Event Argument Extraction via Variational Information Bottleneck23
A multi-format transfer learning model with variational information bottleneck, which makes use of the information especially the common knowledge in existing datasets for EAE in new datasets, and introduces a shared-specific prompt framework to learn both format-shared and format-specific knowledge from datasets with different formats.
- Unifying Cross-Lingual and Cross-Modal Modeling Towards Weakly Supervised Multilingual Vision-Language Pre-training22
It is argued that the universal multilingual representation learned from texts allows the cross-modal interaction learned in English to be transferable to other languages, and a framework to effectively unify cross-lingual and cross- modal pre-training is proposed.
- PathQG: Neural Question Generation from Facts22
This paper presents a novel task of question generation given a query path in the knowledge graph constructed from the input text, and formulate query representation learning as a sequence labeling problem for identifying the involved facts to form a query and employ an RNN-based generator for question generation.
- Dynamic and Generalizable Process Reward Modeling21
Dynamic and Generalizable Process Reward Modeling (DG-PRM), which features a reward tree to capture and store fine-grained, multi-dimensional reward criteria, and pioneeringly adopt Pareto dominance estimation to identify discriminative positive and negative pairs.
- DetectiveQA: Evaluating Long-Context Reasoning on Detective Novels21
This research proposes and evaluates the mainstream LLMs, including GPT-4, Claude, and LLaMA, revealing persistent long-context reasoning challenges and demonstrating their evidence-retrieval challenges.
- 21
- UTC-IE: A Unified Token-pair Classification Architecture for Information Extraction21
This work proposes a Unified Token-pair Classification architecture for Information Extraction (UTC-IE), where Plusformer is introduced on top of the token-pair feature matrix, which models axis-aware interaction with plus-shaped self-attention and local interaction with Convolutional Neural Network over token pairs.
- VCWE: Visual Character-Enhanced Word Embeddings21
A model to learn Chinese word embeddings via three-level composition using a convolutional neural network to extract the intra-character compositionality from the visual shape of a character; a recurrent neural network with self-attention to compose character representation into word embedDings; and the Skip-Gram framework to capture non-compositionality directly from the contextual information.
- Joint Chinese Word Segmentation and POS Tagging on Heterogeneous Annotated Corpora with Multiple Task Learning21
This paper proposes a unified model for Chinese S&T with heterogeneous annotation corpora that can boost the performances of both of the heterogeneous corpora by using the shared information, and achieves significant improvements over the state-of-the-art methods.
- 20
- 20
- Who Will You "@"?20
This paper proposes a novel method that extends the translation-based model for the mention function in microblogging services and demonstrates that the proposed method outperforms state-of-the-art approaches.
- Hierarchical multi-class text categorization with global margin maximization20
This paper proposes a hierarchical multi-class text categorization method with global margin maximization that not only maximize the margins among leaf categories, but also maximize the margin among their ancestors.
- fastHan: A BERT-based Multi-Task Toolkit for Chinese NLP19
The joint-model is trained and evaluated on 13 corpora of four tasks, yielding near state-of-the-art (SOTA) performance in dependency parsing and NER, achieving SOTA performance in CWS and POS.
- 19
- 19
- RethinkCWS: Is Chinese Word Segmentation a Solved Task?19
A fine-grained evaluation for existing CWS systems is proposed, which not only allows to diagnose the strengths and weaknesses of existing models (under the in-dataset setting), but enables to quantify the discrepancy between different criterion and alleviate the negative transfer problem when doing multi-criteria learning.
- Review Response Generation in E-Commerce Platforms with External Product Information19
A novel deep neural network model based on the Seq2Seq framework for the review response generation task in e-commerce platforms, which can incorporate product information by a gated multi-source attention mechanism and a copy mechanism is proposed.
- Idiom-Aware Compositional Distributed Semantics19
This paper proposes an idiom-aware distributed semantic model, grounded in the literal-first psycholinguistic hypothesis, which can adaptively learn semantic compositionality of a phrase literally or idiomatically.
- 18
- Have the VLMs Lost Confidence? A Study of Sycophancy in VLMs18
To mitigate sycophancy, this work proposes a synthetic dataset for training and employ methods based on prompts, supervised fine-tuning, and DPO that effectively alleviate sycophancy in VLMs, and demonstrates that enhancing image attention at high layers proves beneficial in mitigating this issue.
- Phrase-based hashtag recommendation for microblog posts18
This work proposes a novel phrase-based topical translation model that uses the bag-of-phrases model to better capture the underlying topics of posted microblogs and demonstrates that the proposed method outperforms state-of-the-art methods that use the unigram model.
- 17
- Identity-Driven Hierarchical Role-Playing Agents17
A Hierarchical Identity Role-Playing Framework (HIRPF) based on identity theory is constructed, constructing complex characters using multiple identity combinations, and its potential for application in social simulation is revealed.
- ReForm-Eval: Evaluating Large Vision Language Models via Unified Re-Formulation of Task-Oriented Benchmarks17
This work proposes to re-formulate existing benchmarks into unified LVLM-compatible formats and presents ReForm-Eval benchmark, offering substantial data for evaluating various capabilities of LVLMs, demonstrating the comprehensiveness and reliability of ReForm-Eval in assessing various LVLMs.
- 17
- Discrete Argument Representation Learning for Interactive Argument Pair Identification17
Experimental results on the large-scale dataset collected from CMV show that the proposed framework can significantly outperform the competitive baselines and reveal why the model yields superior performance and prove the usefulness of the learned representations.
- Information retrieval: a view from the Chinese IR community17
A key result is that information retrieval researchers need to open their mind to embrace a broader IR field by rethink the definition of information, retrieval, user, system, and evaluation of IR.
- Overview of the NLPCC 2017 Shared Task: Chinese News Headline Categorization17
An overview for the shared task at the CCF Conference on Natural Language Processing & Chinese Computing (NLPCC 2017): Chinese News Headline Categorization is given.
- Chinese-English mixed text normalization17
This paper analyzes the phenomena of mixed usage of Chinese and English in Chinese microblogs and details the proposed two-stage method for normalizing mixed texts, including a noisy channel approach to translate in-vocabulary words into Chinese and a novel user aware neural network language model.
- Learning hash codes for efficient content reuse detection17
Experimental results show that the proposed approach can achieve the same detection rates with state-of-the-art systems while uses significantly less execution time than them (from 400X to 1500X speedup).
- Fudan University at DUC 200617
The system overview is presented, focusing on the newly developed techniques, including a new method of sentence similarity calculation, and application of anaphoric resolution to improve the readability of the summary.
- Nex-N1: Agentic Models Trained via a Unified Ecosystem for Large-Scale Environment Construction16
This work introduces a comprehensive method designed to systematically scale the diversity and complexity of interactive environments and trains Nex-N1 upon the diverse and complex interactive environments established by the infrastructure.
- SoMeLVLM: A Large Vision Language Model for Social Media Processing16
A Large Vision Language Model for Social Media Processing (SoMeLVLM) is introduced, which is a cognitive framework equipped with five key capabilities including knowledge&comprehension, application, analysis, evaluation, and creation and achieves state-of-the-art performance in multiple social media tasks.
- Reward Modeling Requires Automatic Adjustment Based on Data Quality16
A method is introduced that automatically adjusts reward mod-019 eling based on data quality, reducing the impact of noise and making full use of dataset, and significantly enhances the alignment performance of RLHF.
- Chinese Word Segmentation via BiLSTM+Semi-CRF with Relay Node16
This paper proposes a simple yet effective framework to help Semi-CRF achieve comparable performance with CRF-based models under similar computation complexity, and proposes a new model named Semi-SRF-Relay, to model arbitrarily long segments within linear time complexity.
- Progressive Mastery: Customized Curriculum Learning with Guided Prompting for Mathematical Reasoning15
Comprehensive experiments on supervised fine-tuning and reinforcement learning demonstrate that CCL significantly outperforms uniform training approaches across five mathematical reasoning benchmarks, confirming its effectiveness across both paradigms in enhancing sample utilization and model performance.
- 15
- 15
- 15
- Convolutional Interaction Network for Natural Language Inference15
The Convolutional Interaction Network (CIN), a general model to capture the interaction between two sentences, which can be an alternative to the attention mechanism for NLI.
- Overview of the NLPCC 2015 Shared Task: Chinese Word Segmentation and POS Tagging for Micro-blog Texts15
The dataset and task are introduced, the different approaches of the participating systems are characterized, the test results are reported, and an overview analysis of these results are provided.
- Language Independent Text Categorization.15
A language-independent text categorization model based on machine learning is proposed, and the feature extraction, classifier and evaluation method in detail is described in detail.
- Self-Consistency of the Internal Reward Models Improves Self-Rewarding Language Models14
Self-Consistent Internal Rewards (SCIR) is proposed, a novel framework designed to enhance consistency among internal reward models during training that significantly improves the alignment performance and reward modeling capability of LLMs, outperforming baseline methods by a notable margin.
- 14
- Detecting Adversarial Samples through Sharpness of Loss Landscape14
This work proposes a simple and effective sharpness-based detector to distinct adversarial samples by maximizing the loss increment within the region where the inference sample is located and proposes an adaptive optimization strategy in an attempt to fairly compare the relative sharpness among different samples.
- 14
- 14
- Generating Keyword Queries for Natural Language Queries to Alleviate Lexical Chasm Problem14
This work formulated the task as a translation problem to convert natural language queries into keyword queries as a novel encoder-decoder method with two decoders: the pointer decoder firstly extracts query terms directly from the source text via copying mechanism, then the generator decoder generates query terms using two attention modules simultaneously considering the sourceText and extracted query terms.
- Discourse Relations Detection via a Mixed Generative-Discriminative Framework14
A mixed generative-discriminative framework, in which vector offsets between embeddings of words are used to represent the semantic relations between text segments and Fisher kernel framework is introduced to convert a variable number of vector offsets into a fixed length vector.
- Convolutional Deep Neural Networks for Document-Based Question Answering14
A convolutional neural network based architecture is presented to learn feature representations of each question-answer pair and compute its match score by taking the interaction and attention between question and answer into consideration.
- Text Classification with Document Embeddings14
This paper proposes a method to learn document embedding with neural network architecture for text classification task, where each document can be represented as a fine-grained representation of different meanings so that the classification can be done more accurately.
- Latent Semantic Tensor Indexing for Community-based Question Answering14
This paper proposes a unified question retrieval model based on latent semantic indexing with tensor analysis, which can capture word associations among different parts of CQA triples simultaneously and can reduce lexical chasm of question retrieval.
- Beyond Boundaries: Learning a Universal Entity Taxonomy across Datasets and Languages for Open Named Entity Recognition13
This work presents B2NERD, a compact dataset designed to guide LLMs' generalization in Open NER under a universal entity taxonomy, and addresses redundancy using a data pruning strategy that selects fewer samples with greater category and semantic diversity.
- Keep it Consistent: Topic-Aware Storytelling from an Image Stream via Iterative Multi-agent Communication13
This paper proposes a new way for visual storytelling by introducing a topic description task to detect the global semantic context of an image stream and proposes a multi-agent communication framework that regards the topic description generator and the story generator as two agents and learn them simultaneously via iterative updating mechanism.
- Multi-Criteria Chinese Word Segmentation with Transformer13
A concise and effective model for multi-criteria CWS is proposed, which utilizes a shared fully-connected self-attention model to segment the sentence according to a criterion indicator.
- 13
- A New Psychometric-inspired Evaluation Metric for Chinese Word Segmentation13
The performance on a real evaluation shows that the proposed psychometric-inspired evaluation metric for Chinese word segmentation gives more reasonable and distinguishable scores and correlates well with human judgement.
- Thus Spake Long-Context Large Language Model12
This survey illustrates how LLM struggles between the tremendous need for a longer context and its equal need to accept the fact that it is ultimately finite, and gives a global picture of the lifecycle of long-context LLMs from four perspectives: architecture, infrastructure, training, and evaluation.
- 12
- 12
- Analyzing the Effects of Supervised Fine-Tuning on Model Knowledge from Token and Parameter Levels11
This work evaluates closed-book question answering (CBQA) performance across five large language models from the LLaMA-2 and LLaMA-3 families and reveals that up to 90% of parameter updates during SFT do not contribute to knowledge enhancement.
- MOSS-Speech: Towards True Speech-to-Speech Models Without Text Guidance11
This work presents MOSS-Speech, a true speech-to-speech large language model that directly understands and generates speech without relying on text guidance, and establishes a new paradigm for expressive and efficient end-to-end speech interaction.
- 11
- 11
- 11
- Case2Code: Scalable Synthetic Data for Code Generation11
Experimental results show that such induction training benefits not only in distribution Case2Code performance but also enhances various coding abilities of trained LLMs, demonstrating the great potential of learning inductive reasoning.
- Towards Building More Robust NER datasets: An Empirical Study on NER Dataset Bias from a Dataset Difficulty View11
This work comprehensively investigates the NER dataset bias from a dataset difficulty view, quantifies the entity-context difficulty distribution in existing datasets and explains their relationship with model robustness, and explores three potential ways to de-bias the NER datasets.
- Locate Then Ask: Interpretable Stepwise Reasoning for Multi-hop Question Answering11
This paper proposes an interpretable stepwise reasoning framework to incorporate both single-hop supporting sentence identification and single-Hop question generation at each intermediate step, and utilize the inference of the current hop for the next until reasoning out the final result.
- Overview of SMP-CAIL2020-Argmine: The Interactive Argument-Pair Extraction in Judgement Document Challenge11
The results of the Interactive Argument-Pair Extraction in Judgement Document Challenge held by both the Chinese AI and Law Challenge (CAIL) and the Chinese National Social Media Processing Conference (SMP) are presented, and the related data set - SMP-CAIL2020-Argmine is introduced.
- 11
- Understanding Parametric and Contextual Knowledge Reconciliation within Large Language Models10
An entity-aware probe is constructed, which introduces special tokens to mark probing targets and employs a small trainable rank-8 lora update to process these special markers, and can accurately detect information about ad-hoc entities from complex hidden states.
- PIORS: Personalized Intelligent Outpatient Reception based on Large Language Model with Multi-Agents Medical Scenario Simulation10
This paper presents the Personalized Intelligent Outpatient Reception System (PIORS), a system that integrates an LLM-based reception nurse and a collaboration between LLM and hospital information system (HIS) into real outpatient reception setting, aiming to deliver personalized, high-quality, and efficient reception services.
- Unveiling A Core Linguistic Region in Large Language Models10
A core region in LLMs that corresponds to linguistic competence is discovered, accounting for approximately 1% of the total model parameters, and it is observed that an improvement in linguistic competence does not necessarily accompany an elevation in the model's knowledge level, which might imply the existence of regions of domain knowledge that are dissociated from the linguistic region.
- Towards Adversarially Robust Text Classifiers by Learning to Reweight Clean Examples10
This study proposes to train text classifiers by a sample reweighting method in which the example weights are learned to minimize the loss of a validation set mixed with the clean examples and their adversarial ones in an online learning manner.
- 10
- Learning Task-Specific Representation for Novel Words in Sequence Labeling10
This work proposes a novel method to predict representations for OOV words from their surface-forms and contexts and shows that the proposed method can achieve better or competitive performance on the OOV problem compared with existing state-of-the-art methods.
- Adaptive Semantic Compositionality for Sentence Modelling10
A parameterized compositional switch is introduced, which outputs a scalar to adaptively determine whether the meaning of a phrase should be composed of its two constituents.
- 10
- Transition-based Dependency Parsing Using Two Heterogeneous Gated Recursive Neural Networks10
Two heterogeneous gated recursion neural networks are proposed: tree structured gated recursive neural network (Tree-GRNN) and directed acyclic graph structuredgated recursive Neural Network (DAG- GRNN), which integrate to automatically learn the compositions of the dense features for transition-based dependency parsing.
- Detecting Hedge Cues and their Scopes with Average Perceptron10
A hedge detection method with average perceptron, which was used in the closed challenge in CoNLL-2010 Shared Task, and which achieved F-measure with 77.86% in detecting in-domain uncertain sentences, 77.44% in recognizing hedge cues, and 19.27% in identifying the scopes.
- LLMs Deceive Unintentionally: Emergent Misalignment in Dishonesty from Misaligned Samples to Biased Human-AI Interactions9
The study of emergent misalignment is extended to the domain of dishonesty and deception under high-stakes scenarios, and it is demonstrated that this risk arises not only through direct finetuning, but also in downstream mixture tasks and practical human-AI interactions.
- UPLex: Fine-Grained Personality Control in Large Language Models via Unsupervised Lexical Modulation9
UPLex is proposed, a method that uses an Unsupervisedly-Built Personalized Lexicon (UPL) during the decoding phase to manipulate LLM's personality traits to modulate the personality expression of LLMs by dynamically altering their predicted probability of upcoming words in a pluggable fashion.
- Dendritic Localized Learning: Toward Biologically Plausible Algorithm9
Inspired by the dynamics and plasticity of pyramidal neurons, Dendritic Localized Learning (DLL) is proposed, a novel learning algorithm designed to overcome challenges of biological plausibility and generalization in backpropagation.
- DELAN: Dual-Level Alignment for Vision-and-Language Navigation by Cross-Modal Contrastive Learning9
This framework is designed to align various navigation-related modalities before fusion, thereby enhancing cross-modal interaction and action decision-making and seamlessly integrates with the majority of existing models, resulting in improved navigation performance on various VLN benchmarks.
- 9
- UPPAM: A Unified Pre-training Architecture for Political Actor Modeling based on Language9
In UPPAM, aggregate statements to represent political actors and learn the mapping from languages to representation, instead of learning the representation of particular persons, to inject multidimensional information in the political context into the mapping.
- 9
- 9
- Automatic Corpus Expansion for Chinese Word Segmentation by Exploiting the Redundancy of Web Information9
This paper proposes a method to automatically expand the training corpus for the out-of-domain texts by exploiting the redundant information on Web and provides a new viewpoint to enhance the performance of CWS by automatically expanding corpus rather than developing complicated algorithms or features.
- Joint Training and Decoding Using Virtual Nodes for Cascaded Segmentation and Tagging Tasks9
A novel method which integrates graph structures of two sub-tasks into one using virtual nodes, and performs joint training and decoding in the factorized state space is presented.
- 8
- 8
- On the Tip of the Tongue: Analyzing Conceptual Representation in Large Language Models with Reverse-Dictionary Probe8
In-context learning is used to guide the models to generate the term for an object concept implied in a linguistic description to probe LLMs' capacity for conceptual inference in the reverse dictionary task.
- A Soft Contrastive Learning-Based Prompt Model for Few-Shot Sentiment Analysis8
A sentiment-aware chain of thought prompt module is designed to guide the model to predict the sentiment from coarse grain to fine grain via a series of intermediate reasoning steps, and a soft contrastive learning algorithm is proposed to take the correlation of the labels into account.
- 8
- 8
- 8
- Question Classification by Ensemble Learning8
By using compositive statistic and rule classifiers, and by introducing dependency structure from Minipar and linguistic knowledge from WordNet into question representation, the research shows high accuracy in question classification.
- 7
- CauESC: A Causal Aware Model for Emotional Support Conversation7
A novel framework CauESC is proposed, which firstly recognizes the emotion causes of the distress, as well as the emotion effects triggered by the causes, and then understands each strategy of verbal grooming independently and integrates them skillfully.
- 7
- 7
- 7
- Parameter Efficient Multi-task Fine-tuning by Learning to Transfer Token-wise Prompts7
A token-wise prompt tuning (TPT), in which a bank of finer-grained soft prompt tokens is built for multi-task learning by memory network, demonstrated that the models enhanced by the TPT performed far better than full parameter fine-tuned models and achieved state-of-the-art by tuning only 0 .
- 7
- 7
- Attending via both Fine-tuning and Compressing7
This paper proposes a framework consisting of a learner and a compressor, which performs tuning and compressing iteratively to enhance the performance and interpretability of the attention mechanism.
- Incorporating Topic Aspects for Online Comment Convincingness Evaluation7
Experimental results show that topic information is able to improve the performance for convincingness evaluation and the model makes a move to detect topic aspects automatically.
- 7
- Generating Abbreviations for Chinese Named Entities Using Recurrent Neural Network with Dynamic Dictionary7
A novel neural network architecture is proposed that combines recurrent neural network (RNN) with an architecture determining whether a given sequence of characters can be a word or not and can achieve better performance than state-of-the-art methods.
- Chinese Named Entity Abbreviation Generation Using First-Order Logic7
A novel entity abbreviation generation method using first-order logic to model long distance constraints to reduce the human effort of manual annotating corpus and an automatically training data construction method with simple strategies is introduced.
- An Effective Feature Selection Method for Text Categorization7
This paper proposes an effective feature selection method, which aims at the classification accuracy of KNN, and shows that it is better than the traditional methods, and it is also beneficial to other classifiers, such as Support Vector Machines (SVM).
- 6
- RoCoIns: Enhancing Robustness of Large Language Models through Code-Style Instructions6
This paper utilizes instructions in code style, which are more structural and less ambiguous, to replace typically natural language instructions to provide LLMs with more precise instructions and strengthen the robustness of LLMs.
- 6
- 6
- 6
- 6
- A Unified Continuous Learning Framework for Multi-modal Knowledge Discovery and Pre-training6
Taking the open-domain uni-modal datasets of images and texts as input, a knowledge graph is maintained as the foundation to support knowledge discovery and model pre-training in a continuous learning framework for mutual improvement.
- 6
- 6
- Larger-Context Tagging: When and Why Does It Work?6
This paper focuses on investigating when and why the larger-context training, as a general strategy, can work and conducts a thorough comparative study on four proposed aggregators for context information collecting and presents an attribute-aided evaluation method to interpret the improvement brought by larger- context training.
- 6
- Joint Segmentation and Tagging with Coupled Sequences Labeling6
A novel joint model for segmentation and tagging, which integrates two dependent Markov chains that could achieve higher performance over traditional models on both English shallow parsing and Chinese word segmentsation and POS tagging tasks.
- The use of metadata, web-derived answer patterns and passage context to improve reading comprehension performance6
This paper has devised an automatic acquisition process for Web-derived answer patterns (AP) which utilizes question-answer pairs from TREC QA, the Google search engine and the Web, and gave improved RC performances for both the Remedia and ChungHwa corpora.
- 5
- 5
- 60 Data Points are Sufficient to Fine-Tune LLMs for Question-Answering5
The results show that as few as 60 data points during the SFT stage can activate the knowledge encoded during pre-training, enabling LLMs to perform the QA task, and SFT with data of varying memory levels has a significant impact on LLM performance.
- 5
- Improving Discriminative Capability of Reward Models in RLHF Using Contrastive Learning5
Experimental results demonstrate that the proposed contrastive learning-based reward modeling method effectively enhances the generalization of the reward model, stabilizes the reinforcement learning training process, and improves the final alignment with human preferences.
- Characterizing the Impacts of Instances on Robustness5
It is shown that robust and non-robust instances in the training dataset, though are both important for test performance, have contrary impacts on robustness, which makes it possible to build a highly robust model by leveraging the training dataset in a more effective way.
- Watermarking PLMs on Classification Tasks by Combining Contrastive Learning with Weight Perturbation5
- An Unsupervised Sampling Approach for Image-Sentence Matching Using Document-level Structural Information5
This paper proposes a Transformer based model to capture fine-grained features and implicitly construct a graph for each document, where concepts in a document are introduced to bridge the representation learning of images and sentences in the context of a document.
- Toward Recognizing More Entity Types in NER: An Efficient Implementation using Only Entity Lexicons5
This work proposes to adjust the existing NER system using the previously labeled data and entity lexicons of the newly introduced entity types, and formulate such a task as a partially supervised learning problem and accordingly propose an effective algorithm to solve the problem.
- Reformulating natural language queries using sequence-to-sequence models5
Dear editor, In recent years, reformulating natural language queries has attracted significant amount of attention from both industry and academia, however, there is a lexical chasm betweennatural language queries and relevant documents.
- Part-of-Speech Tagging for Chinese-English Mixed Texts with Dynamic Features5
This paper presents a method using dynamic features to tag part-of-speech (POS) for the English words involved in Chinese-English mixed texts and achieves higher performance than traditional sequence labeling methods.
- Learning Semantic Lexicons Using Graph Mutual Reinforcement Based Bootstrapping5
Experimental results show that the GMR-based bootstrapping approach outperforms the existing algorithms both in in- domain data and out-domain data and that the result depends on not only the size of the corpus but also the quality.
- WIM at TREC 2005.5
This paper describes the three TREC tasks this year, which are, Genomics track’s categorization task and ad hoc task, and Enterprise track's known item search task, which adopt a domain-specific terms extraction method and an ontology-based method for feature selection.
- Hot-Spot Passage Retrieval in Question Answering5
The Hot-Spot Passage Retrieval algorithm, which takes into account the measures of blurred BM25, coverage and height, is examined in detail and gains 18.3% better answer redundancy and 4.8% better coverage rate than Okapi's original passage retrieval algorithm.
- 4
- FamilyTool: A Multi-hop Personalized Tool Use Benchmark4
FamilyTool is introduced, a novel benchmark grounded in a family-based knowledge graph (KG) that simulates personalized, multi-hop tool use scenarios and proposes KGETool: a simple KG-augmented evaluation pipeline to systematically assess LLM agents' tool use ability in these settings.
- 4
- 4
- 4
- 4
- Decoding Continuous Character-based Language from Non-invasive Brain Recordings4
A novel approach to decoding continuous language from single-trial non-invasive fMRI recordings is proposed, in which a three-dimensional convolutional network augmented with information bottleneck is developed to automatically identify responsive voxels to stimuli, and a character-based decoder is designed for the semantic reconstruction of continuous language characterized by inherent character structures.
- AI-Press: A Multi-Agent News Generating and Feedback Simulation System Powered by Large Language Models4
AI-Press, an automated news drafting and polishing system based on multi-agent collaboration and Retrieval-Augmented Generation is introduced and a feedback simulation system that generates public feedback considering demographic distributions is developed.
- 4
- K-ESConv: Knowledge Injection for Emotional Support Dialogue Systems via Prompt Learning4
K-ESConv is proposed, a novel prompt learning based knowledge injection method for emotional support dialogue system, transferring forum knowledge to response generation, which outperforms existing baselines on both automatic evaluation and human evaluation.
- 4
- What Dense Graph Do You Need for Self-Attention?4
Hypercube Transformer is presented, a sparse Transformer that models token interactions in a hypercube and shows comparable or even better results with vanilla Transformer while yielding O(N\log N)$ complexity with sequence length $N$.
- 4
- Hyper-Gated Recurrent Neural Networks for Chinese Word Segmentation4
This paper proposes the hyper-gated recurrent neural networks for Chinese word segmentation, which enhance the gates to incorporate the historical information of gates and shows that the model outperforms the baseline models as well as the state-of-the-art methods.
- Bidirectional Long Short-Term Memory with Gated Relevance Network for Paraphrase Identification4
This paper adopts a neural network model for paraphrase identification called as bidirectional Long Short-Term Memory-Gated Relevance Network (Bi-LSTM+GRN), which achieves better performances than hand-crafted feature based approaches as well as previous neural network models.
- Map search via a factor graph model4
A novel factor graph-based semi-structured retrieval framework is introduced to incorporate concept weighting, attribute selection, and word-based similarity metrics together and demonstrates that this method outperforms both state-of-the-art semi- Structured retrieval methods and some commercial systems in retrieving freeform location queries.
- Understanding the Semantic Intent of Domain-Specific Natural Language Query4
A hierarchical structure is introduced to represent the semantic intent of NL query and then the problem of mapping NL queries to the corresponding semantic intents is focused on.
- Understanding the Semantic Intent of Natural Language Query4
A hierarchical structure is introduced to represent the semantic intent of NL query and then the problem of mapping NL queries to the corresponding semantic intents is focused on.
- Discovering logical knowledge for deep question answering4
This paper proposes an approach to discover logical knowledge for deep question answering, which automatically extracts knowledge in an unsupervised, domain-independent manner from background texts and reasons out implicit answers for the questions.
- Pseudo-Relevance Feedback Based on mRMR Criteria4
This study first re-examine this assumption that combinations of terms may heavily impact the final results, and presents a novel clustering based method to select expansion terms as a whole set.
- How Jailbreak Defenses Work and Ensemble? A Mechanistic Investigation3
This paper systematically examines jailbreak defenses by reframing the standard generation task as a binary classification problem to assess model refusal tendencies for both harmful and benign queries, and develops two ensemble defense strategies to balance safety and helpfulness.
- A Scalable Vulnerability Detection System with Multi-View Graph Representations3
VulCNN is a vulnerability detection system which can leverage the function images and simple convolutional neural networks (CNN) to effectively and efficiently detect vulnerabilities and achieves vulnerability detection effectiveness comparable to the state-of-the-art method GRACE, but with a 31.52% reduction in time overhead.
- Compression Hacking: A Supplementary Perspective on Informatics Properties of Language Models from Geometric Distortion3
This work finds that compression hacking substantially enhances the informatics interpretation of LMs by incorporating geometric distortion of representations, and proposes three refined compression metrics by incorporating geometric distortion analysis and integrate them into a self-evaluation pipeline.
- 3
- 3
- 3
- KNSE: A Knowledge-aware Natural Language Inference Framework for Dialogue Symptom Status Recognition3
- 3
- Text information aggregation with centrality attention3
This work proposes a new way of obtaining aggregation weights, called eigen-centrality self-attention, which builds a fully-connected graph for all the words in a sentence, then compute the eigen -centrality as the attention score of each word.
- Learning to Generate Representations for Novel Words: Mimic the OOV Situation in Training3
A novel training strategy is proposed to address the out-of-vocabulary (OOV) problem in sequence labeling using only training data of the task and trains the supervised model to fit the OOV word representations generated by the mean-pooling operation.
- Constructing Multiple Tasks for Augmentation: Improving Neural Image Classification with K-Means Features3
This work explored the idea of using unsupervised clustering to construct a variety of auxiliary tasks from unlabeled data or existing labeled data, and found that some of these newly constructed tasks could exhibit semantic meanings corresponding to certain human-specific attributes, but some were non-ideal.
- A Learning Error Analysis for Structured Prediction with Approximate Inference3
This work compares the estimation and approximation error of both underestimate and overestimate models and shows that, from the perspective of learning errors, performances of approximate inference could be as good as exact inference.
- 3
- 3
- A Fast Accurate Two-stage Training Algorithm for L1-regularized CRFs with Heuristic Line Search Strategy3
Experimental evaluations on Chinese word segmentation and name entity recognition tasks demonstrate that the two-stage training algorithm can produce more accurate and compact model with less training time for L1 regularization.
- Graph Mutual Reinforcement Based Bootstrapping3
Experimental results on MUC4 corpus show that GMR-Bootstrapping outperforms the state-of-the-art algorithms and is used to extract names of automobile manufactures and models from Chinese corpus.
- 2
- 2
- Subspace Defense: Discarding Adversarial Perturbations by Learning a Subspace for Clean Signals2
This paper empirically shows that the features of either clean signals or adversarial perturbations are redundant and span in low-dimensional linear subspaces respectively with minimal overlap, and the classical low-dimensional subspace projection can suppress perturbation features out of the subspace of clean signals.
- 2
- 2
- 2
- 2
- 2
- 2
- Generating universal language adversarial examples by understanding and enhancing the transferability across neural models2
This paper systematically study the transferability of adversarial attacks for text classification models and proposes universal black-box attack algorithms that can induce adversarial examples to attack almost all existing models.
- 2
- Searching Effective Transformer for Seq2Seq Keyphrase Generation2
Comprehensive experiments on multiple KG benchmarks showed that in KG tasks, uninformative content abounds in documents while salient information is diluted globally, and the vanilla Transformer equipped with a fully-connected self-attention mechanism may overlook the local context, leading to performance degradation.
- 2
- 2
- Transition-Based Dependency Parsing with Long Distance Collocations2
This paper proposes a method to improve the performance of transition-based parsing with long distance collocations with an approximate global view of the entire sentence, which is a little bit similar to top-down parsing.
- Learning Tag Relevance by Context Analysis for Social Image Retrieval2
A novel model to learn the tag relevance based on the context analysis for each tag is built, which focuses on the globally-comparable tag relevance measure (i.e., can be compared across different images) and better tag relevance learning model by detailed context analysisFor each tag.
- Adaptive Chinese Word Segmentation with Online Passive-Aggressive Algorithm2
The online passive-aggressive algorithm with domain invariant information for cross-domain Chinese word segmentation is used for CIPS-SIGHAN-2010 bake-off task of Chineseword segmentation.
- A Spectral Clustering Based Coreference Resolution Method2
This paper presents a novel method to implement coreference resolution based on spectral clustering that can divide entities with a global view, which effectively improves precision.
- Domain Adaptation for Conditional Random Fields2
A novel technique for maximum a posteriori (MAP) adaptation of Conditional Random Fields model and results show that this technique can significantly improve performance on out-of-domain data.
- Layer-Specific Scaling of Positional Encodings for Superior Long-Context Modeling1
A layer-specific positional encoding scaling method that assigns distinct scaling factors to each layer, slowing down the decay rate caused by RoPE to make the model pay more attention to the middle context and enhances the model's extrapolation capabilities when combined with PI and Dynamic-NTK positional encoding schemes.
- 1
- 1
- Model Utility Law: Evaluating LLMs beyond Performance through Mechanism Interpretable Metric1
A novel metric, the Model Utilization Index (MUI), is proposed, which introduces mechanism interpretability techniques to complement traditional performance metrics and deduces a common trend observed in popular LLMs, which is termed the “Utility Law.
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- Constructing Phrase-level Semantic Labels to Form Multi-Grained Supervision for Image-Text Retrieval1
- 1
- 1
- Hierarchical Dirichlet Processes with Social Influence1
A novel non-parametric model, social hierarchical Dirichlet process (sHDP), is proposed, which assumes that the topic distributions of documents are similar to each other if their authors have relations in social networks.
- Online Distributed Passive-Aggressive Algorithm for Structured Learning1
Experimental results show that the proposed online distributed Passive-Aggression by averaging parameters for parallel training can accelerate the training process significantly with comparable or even better accuracy.
- Microblog Track 2011 of FDU.1
The authors' participation in the Microblog Track of TREC detects the special behavior of the twitter dataset in the “real-time” retrieval task and includes two aspects: data preprocessing to remove non-English tweets and feature extraction for clustering the tweets into two categories.
- 1
- Hierarchical Ensemble Support Cluster Machine1
Experiments carried on two real large-scale data sets validate the effectiveness of the proposed HESCM, increasing classification accuracy and stability as well as significantly reducing computational and spatial complexities of a supervised classifier compared to the state-of-the-art classifiers.
- 1
- Answering Definition Question: Ranking for Top-k1
This work integrates evidences as features into a whole framework, and proposes a novel method to learning weights of these features to rank the candidate answer sentences.
- Statistical Acquisition of Terminology Dictionary1
A semi-automatic approach is developed to extract technical words and phrases from on-line corpora that can significantly reduce the manual effort in the generation of terminology dictionary.
- –
- MathCritique: Enhancing LLM Reasoning via Critique Models with Test-Time and Training-Time Supervision–
A critique-in-the-loop self-improvement method that incorporates critique-based supervision into the actor’s self-training process and improves the actor’s exploration efficiency and solution diversity, especially on challenging queries, leading to a stronger actor model.
- –
- MetaAct-RL: Training Language Models for Reasoning Through Meta-Action-Based Reinforcement Learning–
MetaAct-RL is introduced, a new RL framework that frames LMs’ thinking as sequential decision making over meta-actions, and incorporates appropriate length-based reward and regularization, and a key-state restart mechanism to improve sampling efficiency in the RL optimization process.
- Interpretability in the Era of Large Language Models: Mechanistic Methodology, Empirical Practices, and Applications–
This tutorial provides a comprehensive, end-to-end view of LLM interpretability, transitioning from microscopic neural analysis to macroscopic application and deployment, and explores how these interpretability paradigms scale and inspire the design of frontier architectures, agentic systems, and thinking models.
- Parrot: A Training Pipeline Enhances Both Program CoT and Natural Language CoT for Reasoning–
Parrot is proposed, a novel training pipeline for mathematical problems that significantly enhances both the performance of N-CoT and P-CoT, especially on N-CoT.
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- F-Eval: Assessing Fundamental Abilities with Refined Evaluation Methods–
This work proposes F-Eval, a bilingual evaluation benchmark to evaluate the fundamental abilities of LLMs’ fundamental abilities, including expression, common-sense and logic, and devise new evaluation methods, serving as alternatives to scoring by API models.
- –
- –
- –
- –
- –
- Enhancing Unsupervised Semantic Parsing with Distributed Contextual Representations–
A non-parametric Bayesian model is extended to deal with homonymy and polysemy by leveraging distributed contextual word and phrase representations pre-trained on a large collection of unla-belled texts and achieves better accuracy on both question-answering and relation extraction tasks.
- Connectivity Patterns are Task Embeddings–
This work draws inspiration from the operating mechanism of deep neural networks (DNNs) and biological brains, where neuronal activations are sparse and task-specific, and uses the connectivity patterns of neurons as a unique identifier associated with the task.
- –
- CausalAPM: Generalizable Literal Disentanglement for NLU Debiasing–
CausalAPM, a generalizable literal disentangling framework to ameliorate the bias problem from feature granularity, is proposed and extensive experiments demonstrate that the proposed framework significantly improves the OOD generalization performance while maintaining ID performance.
- –
- –
- Jointly learning bilingual word embeddings and alignments–
This study is among the first to learn bilingual word alignments and embeddings in a joint manner and achieved state-of-the-art or comparable results on all the tasks considered.
- Implicit discourse relation detection using concatenated word embeddings and a gated relevance network–
This work proposes to use a gated relevance network (GRN) that combines a bilinear model and single layer network with a gate to compute the arguments’ relevance scores from the word pairs, to handle the rare word problem.
- –
- Learning to Rank Answers for Definitional Question Answering–
This paper proposes an online learning algorithm, which dynamically construct the supervisor to reduce the adverse effects of the large number of bad answers and noisy data and compares it with two state-of-the-art definitional QA systems and two ranking algorithms.
- Bilingual Product Name Dictionary Construction Using a Two Stage Method–
A novel two-stage method for bilingual product name dictionary construction from comparable corpora based on entity set expansion and bilingual entity alignment from comparable Corpora is proposed.
- Improving Multi-pass Transition-Based Dependency Parsing Using Enhanced Shift Actions–
This paper proposes a method to improve performance of parsing by using enhanced shift actions, which can be further used as features for the next parsing decision.
- A Generative Model for Identifying Target Companies of Microblogs–
A novel method is proposed which converts the target company identification problem to the translation process from content to targets, and introduces a topic-specific generative method to model thetranslation process.
- Chinese Word Segmentation with Character Abstraction–
This paper focuses on the primary challenges in Chinese word segmentation: low accuracy of out-of-vocabulary word, and group the “similar” characters to generate more abstract representation to resolve this difficult problems.
- –
- –
- Labelwise Margin Maximization for Sequence Labeling–
An online learning algorithm is proposed that addresses the problem of labelwise margin maximization for sequence labeling by decomposing the sequence margin to per-label margins and maximizing these per- label margins individually, which can result to minimize the Hamming loss of sequence.
- Sentence Near-Duplicate Detection Based on Low-IDF-SIG–
An effective and efficient feature extraction algorithm named Low-IDF-Sig is proposed, which can improve both precision and efficiency of near-duplicate detection in sentence level.
- Triplet-Based Chinese Word Sense Induction–
This paper describes the implementation of the system at CLP 2010 of Chinese word sense induction, which extracts the triplets for the target word in each sentence, then uses the intersection of all related words of these triplets to construct feature vectors for the sentence.
- Automatic Web News Extraction with Semantic Features–
Experimental result shows that the semantic features and the similarity of Web news pages are analyzed and an automatic Web news extraction method with semantic features can effectively improve the accuracy of Web information extraction method and cut the cost of manual labeling work.
- Mining Uncertain Sentences with Multiple Instance Learning–
A method for detecting uncertain sentences with multiple instance learning (MIL) is proposed based on the basic assumption, and two new constraints for estimating the weight vector are derived by defining a probability margin, which is used in an online learning algorithm known as Passive-Aggressive algorithm.
- Relevance Feedback based on Constrained Clustering: FDU at TREC 09–
This work applies the constrained clustering on the top returned documents and extract the expanded words to reform the query and extracts the named entities from the explicit relevant documents to expand the query.
- –
- A Definitional Question Answering System Based on Dependency Relation–
This work uses the improved ultraconservative online algorithm and margin infused relaxed algorithm (MIRA) for the task of candidate sentences ranking and demonstrates that the improved MIRA can greatly increase the system performance in terms of progressive reduction of optimization constraints as training moves forward.
- –
- –
Publication data from OpenAlex, with missing venues and authors filled in from Crossref; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-12. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.
Report an error
Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.