Is this you? Claim this profile to correct it, add a bio and choose the work people see first.
Claim this profileWorks209 from public data
- ROUGE: A Package for Automatic Evaluation of Summaries22,250
Four different RouGE measures are introduced: ROUGE-N, ROUge-L, R OUGE-W, and ROUAGE-S included in the Rouge summarization evaluation package and their evaluations.
- Automatic evaluation of summaries using N-gram co-occurrence statistics1,928
The results show that automatic evaluation using unigram co-occurrences between summary pairs correlates surprising well with human evaluations, based on various statistical metrics; while direct application of the BLEU evaluation procedure does not always give good results.
- Automatic evaluation of machine translation quality using longest common subsequence and skip-bigram statistics946
Two new objective automatic evaluation methods for machine translation based on longest common subsequence between a candidate translation and a set of reference translations and relaxes strict n-gram matching to skip-bigram matching are described.
- The automated acquisition of topic signatures for text summarization553
A method for automatically training topic signatures-sets of related words, with associated weights, organized around head topics, is described and illustrated with signatures the authors created with 6,194 TREC collection texts over 4 selected topics.
- Automated Text Summarization in SUMMARIST544
The system’s architecture is described and details of some of its modules, many of them trained on large corpora of text, are provided.
- ORANGE429
A new evaluation method, Orange, is introduced for evaluating automatic machine translation evaluation metrics automatically without extra human involvement other than using a set of reference translations.
- Low-Quality Product Review Detection in Opinion Summarization424
Experimental results show that the proposed approach effectively discriminates lowquality reviews from high-quality ones and enhances the task of opinion summarization by detecting and filtering low quality reviews.
- Question Answering in Webclopedia.322
The QA Typology contains 94 nodes, of which 47 are leaf nodes; each Typology node has been annotated with examples and typical patterns of expression of both Question and Answer, as indicated in Figure 3.
- Identifying topics by position318
The automated training and evaluation of an Optimal Position Policy is described, a method of locating the likely positions of topic-bearing sentences based on genre-specific regularities of discourse structure that can be used in applications such as information retrieval, routing, and text summarization.
- Manual and automatic evaluation of summaries309
This paper shows the instability of the manual evaluation of summaries, and investigates the feasibility of automated summary evaluation based on the recent BLEU method from machine translation using accumulative n-gram overlap scores between system and human summaries.
- Toward semantics-based answer pinpointing303
The treatment of questions (Question-Answer Typology, question parsing, and results) in the Weblcopedia question answering system are described.
- Automated text summarization and the SUMMARIST system280
A preliminary typology of summaries in general is presented; a description of the current and planned modules and performance of the SUMMARIST automated multilingual text summarization system is described; and three methods to evaluate summaries are discussed.
- Finding question-answer pairs from online forums278
This paper proposes a sequential patterns based classification method to detect questions in a forum thread, and a graph based propagation method to detects answers for questions in the same thread.
- From single to multi-document summarization272
NeATS is a multi-document summarization system that attempts to extract relevant or interesting portions from a set of documents about some topic and present them in coherent order.
- What's in a name?264
The analysis shows that the alias-disambiguation solution can address a major part of the user linking problem in terms of the coverage of true pairwise decisions and that the rareness and commonness of usernames can help user linking.
- Joint Entity Recognition and Disambiguation248
JERL is the first model to jointly optimize NER and linking tasks together completely, and in experiments on the CoNLL’03/AIDA data set, JERL outperforms state-of-art NER and linking systems.
- Joint Named Entity Recognition and Disambiguation243
JERL is the first model to jointly optimize NER and linking tasks together completely, and in experiments on the CoNLL’03/AIDA data set, JERL outperforms state-of-art NER and linking systems.
- Automated Summarization Evaluation with Basic Elements.216
This paper describes a framework in which summary evaluation measures can be instantiated and compared, and implements a specific evaluation method using very small units of content, called Basic Elements that address some of the shortcomings of ngrams.
- COM215
A probabilistic model named COM (COnsensus Model) is proposed to model the generative process of group activities, and make group recommendations, and is able to incorporate both users' selection history and personal considerations of content factors.
- Automatically Solving Number Word Problems by Semantic Parsing and Reasoning200
A new meaning representation language is designed to bridge natural language text and math expressions and a CFG parser is implemented based on 9,600 semi-automatically created grammar rules.
- Looking for a Few Good Metrics: ROUGE and its Evaluation191
The validity of the evaluation method used in the Document Understanding Conference (DUC) is discussed and five different ROUGE metrics are evaluated included in the RouGE summarization evaluation package using data provided by DUC.
- Training a selection function for extraction183
Comparisons of performance of several heuristics in generating informative generic/query-oriented extracts for newspaper articles are compared in order to learn how topic prominence affects the performance of each heuristic.
- Automated Text Summarization176
This tutorial will review the state of the art in automatic summarization, and will discuss and critically evaluate current approaches to the problem, including manual evaluation procedures used in DUC, the factoid and pyramid method reference summary creation procedures and fully automatic evaluation method such as ROUGE.
- Searching Questions by Identifying Question Topic and Question Focus170
Experimental results indicate that the approach of identifying question topic and question focus for search significantly outperforms the baseline methods such as Vector Space Model (VSM) and Language Model for Information Retrieval (LMIR).
- How well do Computers Solve Math Word Problems? Large-Scale Dataset Construction and Evaluation158
A large-scale dataset is built which is more than 9 times the size of previous ones, and contains many more problem types, and semi-automatically obtained from community question-answering web pages.
- 151
- 143
- The use of external knowledge in factoid QA137
This paper describes recent development in the Webclopedia QA (Question Answering) system, focusing on the use of knowledge resources such as WordNet and a QA typology to improve the basic operations of candidate answer retrieval, ranking, and answer matching.
- ParaEval: using paraphrases to evaluate summaries automatically135
It is shown that the quality of ParaEval's evaluations, measured by correlating with human judgments, closely resembles that of ROUGE's.
- Overview of Opinion Analysis Pilot Task at NTCIR-6128
An overview of the test collection, task design, and evaluation results across the Chinese, Japanese, and English data, and the evaluation results of the groups participating in opinion extraction subtask at NTCIR are presented.
- Understanding and summarizing answers in community-based question answering services125
An in-depth analysis of questions and answers on cQA services finds that the assumption that questions always have unique best answers cannot be true, and shows that question-type oriented summarization techniques can improve cZA answer quality significantly.
- Looking for a Few Good Metrics: Automatic Summarization Evaluation — How Many Samples Are Enough?124
The validity of the evaluation method used in the Document Understanding Conference (DUC) is discussed and five different ROUGE metrics are evaluated: ROUge-N, RouGE-L, ROUGEW, R OUGE-S, and ROUAGE-SU included in the Rouge summarization evaluation package using data provided by DUC.
- TIARA: Multi-grained Retrieval for Robust Question Answering over Large Knowledge Bases118
A new KBQA model, TIARA, is presented, which addresses issues by applying multi-grained retrieval to help the PLM focus on the most relevant KB context, viz., entities, exemplary logical forms, and schema items.
- Evaluating DUC 2005 using Basic Elements115
It is shown that this method correlates better with human judgments than any other automated procedure to date, and overcomes the subjectivity/variability problems of manual methods that require humans to preprocess summaries to be evaluated.
- Re-evaluating machine translation results with paraphrase support110
ParaEval is presented, an automatic evaluation framework that uses paraphrases to improve the quality of machine translation evaluations and correlates significantly better than BLEU with human assessment in measurements for both fluency and adequacy.
- An information-theoretic approach to automatic evaluation of summaries106
This paper introduces an information-theoretic approach to automatic evaluation of summaries based on the Jensen-Shannon divergence of distributions between an automatic summary and a set of reference summaries and results indicate that JS divergence-based evaluation method achieves comparable performance with the common automatic evaluation method ROUGE in single documents summarization task.
- Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Long Papers - Volume 1100
The General Chair of ACL shares with you the excitement of the community as the authors gather again at their annual meeting, and prepares special programs to commemorate the 50th anniversary.
- 99
- Towards Improving Neural Named Entity Recognition with Gazetteers98
It is shown that properly utilizing external gazetteers could benefit segmental neural NER models by adding a simple module on the recently proposed hybrid semi-Markov CRF architecture and observing some promising results.
- 96
- Using Conditional Random Fields to Extract Contexts and Answers of Questions from Online Forums95
A general framework based on Conditional Random Fields (CRFs) to detect the contexts and answers of questions from forum threads is proposed and improved by Skip-chain CRFs and 2D CRFs to better accommodate the features of forums for better performance.
- A Simple Recipe towards Reducing Hallucination in Neural Surface Realisation92
This work proposes to integrate a language understanding module for data refinement with self-training iterations to effectively induce strong equivalence between the input data and the paired text to mitigate neural language generation systems often hallucinate contents.
- Neural Math Word Problem Solver with Reinforcement Learning92
Experimental results show that the copy and alignment mechanism is effective to address the two issues and Reinforcement learning leads to better performance than maximum likelihood on this task; and the neural model is complementary to the feature-based model and their combination significantly outperforms the state-of-the-art results.
- Learning Fine-Grained Expressions to Solve Math Word Problems91
A novel template-based method that learns the mappings between math concept phrases in math word problems and their math expressions from training data and does a fine-grained inference to obtain the final answer.
- Improving summarization performance by sentence compression: a pilot study90
The results show that pure syntactic-based compression does not improve system performance and topic signature-based reranking of compressed sentences does not help much, however reranking using an oracle showed a significant improvement remains possible.
- 89
- Semi-supervised speech act recognition in emails and forums88
This method leverages labeled data in the Switchboard-DAMSL and the Meeting Recorder Dialog Act database and applies simple domain adaptation techniques over a large amount of unlabeled email and forum data to address the major challenge of this task.
- ReTraCk: A Flexible and Efficient Framework for Knowledge Base Question Answering83
ReTraCk is a neural semantic parsing framework for large scale knowledge base question answering (KBQA) that includes a retriever to retrieve relevant KB items efficiently, a transducer to generate logical form with syntax correctness guarantees and a checker to improve transduction procedure.
- Competition-based user expertise score estimation83
The experimental results show that the pairwise comparison based competition model significantly outperforms link analysis based approaches (PageRank and HITS) and pointwise approaches (number of best answers and best answer ratio) for estimating the expertise of active users.
- Knowledge Base Completion via Coupled Path Ranking79
Experimental results show that CPRA can effectively identify coherent clusters in which relations are highly correlated, and empirically evaluate CPRA on benchmark data created from Freebase, showing significantly outperforms PRA in terms of both predictive accuracy and model interpretability.
- Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Short Papers - Volume 276
The General Chair of ACL shares with you the excitement of the community as the authors gather again at their annual meeting, and prepares special programs to commemorate the 50th anniversary.
- Improving Entity Linking by Modeling Latent Entity Type Information75
This work proposes to inject latent entity type information into the entity embeddings based on pre-trained BERT, and integrates a BERT-based entity similarity score into the local context model of a state-of-the-art model to better capture latententity type information.
- Automated multi-document summarization in NeATS75
This paper describes the multi-document text summarization system NeATS, which was among the top two performers of the DUC-01 evaluation.
- The potential and limitations of automatic sentence extraction for summarization74
The results show that the single document generic summarization task as defined in DUC 2001 needs to be carefully refocused as reflected in the low inter-human agreement at 100-word and high upper bound at full text summaries.
- Enhanced Meta-Learning for Cross-Lingual Named Entity Recognition with Minimal Resources73
This paper presents a meta-learning algorithm to find a good model parameter initialization that could fast adapt to the given test case and proposes to construct multiple pseudo-NER tasks for meta-training by computing sentence similarities.
- Knowledge-based automatic topic identification71
This work presents a new method for automatically identifying the central ideas in a text based on a knowledge-based concept counting paradigm that uses the hierarchical concept taxonomy WordNet to represent and generalize concepts.
- Ensemble Semantics for Large-scale Unsupervised Relation Extraction70
A novel unsupervised algorithm is presented that provides a more general treatment of the polysemy and synonymy problems and explicitly disambiguates polysemous relation phrases and groups synonymous ones and achieves significant improvement on recall compared to the previous method.
- Recommending questions using the mdl-based tree cut model68
Experimental results indicate that the use of the MDL-based tree cut model can significantly outperform the baseline methods of word- based VSM or phrase-based VSM.
- Detecting Erroneous Sentences using Automatically Mined Sequential Patterns63
A new approach to detecting erroneous sentences by integrating pattern discovery with supervised learning models is proposed, and experimental results show that the techniques are promising.
- Self-disclosure topic model for classifying and analyzing Twitter conversations61
The developed self-disclosure topic model (SDTM) significantly outperforms several comparable methods on classifying the level of selfdisclosure, and the analysis of the longitudinal data using SDTM uncovers significant and positive correlation between self Disclosure and conversation frequency and length.
- Natural Language Understanding and Intelligent Applications60
A novel word sense disambiguation (WSD) discriminative model is proposed which integrates structural context with the local context to handle long distance sense dependency and multireference lexicon dependency within the sentence.
- Collective Tweet Wikification based on Semi-supervised Graph Regularization59
A novel semi-supervised graph regularization model is proposed to incorporate both local and global evidence from multiple tweets through three fine-grained relations in order to identify semanticallyrelated mentions for collective inference.
- An Ultra-Low Power Smart Headband for Real-Time Epileptic Seizure Detection58
A service APP connected to the cloud is developed so that the patients’ health condition can be recorded and then referenced by doctors for further diagnosis or research.
- 58
- Using graded-relevance metrics for evaluating community QA answer selection56
This work proposes system evaluation methods that involve multiple answer assessors and graded-relevance information retrieval metrics that are useful for building effective CQA answer selection systems despite the cost of manual answer assessments.
- 50
- Web page title extraction and its application50
Experimental results indicate that the use of both extracted titles and title fields is almost always better than theuse of title fields alone; the useof extracted titles is particularly helpful in the task of named page finding.
- Sentence level machine translation evaluation as a ranking problem49
The paper proposes formulating MT evaluation as a ranking problem, as is often done in the practice of assessment by human, and investigates the relative utility of several features.
- 49
- Who Will Reply to/Retweet This Tweet?48
By incorporating the dynamics of friendship relations, this approach significantly outperforms state-of-the-art models in terms of multiple evaluation metrics, such as MAP, NDCG and Topmost Accuracy.
- Simple Evaluation Metrics for Diversified Search Results.48
It is argued that the simple evaluation metrics proposed are suitable for diversity evaluation given either the intent likelihood information or per-intent graded relevance, or preferably both.
- Open-Schema Event Profiling for Massive News Corpora47
This work develops a Bayesian non-parametric model to detect events and event types by exploiting the slot expressions of the entities mentioned in news articles and proposes an unsupervised embedding model for schema induction that encodes the insight.
- A BE-based Multi-document Summarizer with Query Interpretation47
This version of the multi-document summarization system designed by the Webclopedia team from ISI for DUC 2005 consists of a query-interpretation component that directly analyzes the given user profile and topic narrative for each document cluster before creating a corresponding summary.
- Operation-guided Neural Networks for High Fidelity Data-To-Text Generation46
This paper proposes a framework called Operation-guided Attention-based sequence-to-sequence network (OpAtt), with a specifically designed gating mechanism as well as a quantization module for operation results to utilize information from pre-executed symbolic operations.
- RBPB: Regularization-Based Pattern Balancing Method for Event Extraction46
Inspired by the progress in representation learning, trigger embedding, sentence-level embedding and pattern features together as their features for trigger classification are used so that the effect of patterns and other useful features can be balanced.
- 45
- 45
- 43
- iNeATS43
iNeATS is an interactive multi-document summarization system that integrates a state-of-the-art summarization engine with an advanced user interface that combines text summaries with alternative presentations such as a map-based visualization of documents.
- Introduction to the special issue on statistical language modeling42
With the ever-increasing popularity of SLM, it is the right time to assemble a special issue reflecting recent advances in both its theory and applications.
- A Statistical Framework for Product Description Generation39
A statistical framework that generates accurate and fluent product description from product attributes by using learned knowledge to decide what to say and how to say for product description generation is presented.
- Cross-domain collaborative filtering with review text39
This paper extends previous transfer learning models in collaborative filtering, from linear mapping functions to non-linear ones, and proposes a cross-domain recommendation framework with the review text incorporated.
- Learning to recommend questions based on user ratings38
A new algorithm called 'majority-based perceptron algorithm' is designed which can avoid the influence of noisy instances by emphasizing its learning over data instances from the majority users.
- 38
- Enhancing Neural Data-To-Text Generation Models with External Background Knowledge37
This paper enhances neural data-to-text models with external knowledge in a simple but effective way to improve the fidelity of generated text by attends to relevant external knowledge, encoded as a temporary memory, and combines this knowledge with the context representation of data before generating words.
- Question Difficulty Estimation in Community Question Answering Services34
This paper proposes a competition-based model for estimating question difficulty by leveraging pairwise comparisons between questions and users and shows that the text of question descriptions reflects the question difficulty.
- 33
- Mention and Entity Description Co-Attention for Entity Disambiguation33
A type-aware co-attention model for entity disambiguation is proposed, which tries to identify the most discriminative words from mention contexts and most relevant sentences from corresponding entity descriptions simultaneously.
- Participation Maximization Based on Social Influence in Online Discussion Forums32
This paper proposes a heuristic algorithm, named Thread Allocation Based on Influence (TABI), which consistently outperforms all other algorithms in maximizing participation and demonstrates that current recommender systems can be made more effective by considering future influence propagations.
- 29
- 28
- 28
- 26
- 25
- 24
- 23
- Data2Text Studio: Automated Text Generation from Structured Data22
This work conducts experiments on RotoWire datasets for template extraction and text generation and shows that the Semi-HMMs model achieves improvements on both tasks.
- 22
- 22
- 22
- 19
- 19
- 19
- 19
- 18
- 18
- 18
- 18
- 18
- 17
- Context-aware Entity Morph Decoding17
This paper presents the first end-to-end context-aware entity morph decoding system that can automatically identify, disambiguate, verify morph mentions based on specific contexts, and resolve them to target entities.
- An Entity Class-Dependent Discriminative Mixture Model for Cumulative Citation Recommendation16
A novel entity class-dependent discriminative mixture model is proposed by introducing a latent entity class layer to model the correlations between entities and latent entity classes to achieve better performance when dealing with a broad range of entities.
- 15
- CanvasEmb: Learning Layout Representation with Large-scale Pre-training for Graphic Design15
This paper proposes CanvasEmb (Canvas Embedding), which pre-trains deep representations from unlabeled graphic designs by jointly conditioning on all the context elements in a canvas, with a multi-dimensional feature encoder and aMulti-task learning objective.
- 15
- 15
- Re-ranking voting-based answers by discarding user behavior biases14
This paper identifies the existence of two kinds of biases; position bias and appearance bias and proposes a joint click model for dealing with both of them and demonstrates how the ranking performance of the proposed model outperforms traditional methods with biases ignored.
- 14
- 14
- 13
- 13
- 13
- 13
- 12
- 12
- 12
- 12
- 11
- 11
- 11
- Learning Latent Semantic Annotations for Grounding Natural Language to Structured Data10
This paper attempt at learning explicit latent semantic annotations from paired structured tables and texts, establishing correspondences between various types of values and texts with an adapted semi-hidden Markov model.
- 10
- 10
- Aggregated Semantic Matching for Short Text Entity Linking9
A novel neural network framework, Aggregated Semantic Matching (ASM), in which two different aspects of semantic information between the local context and the candidate entity are captured via representation-based and interaction-based neural semantic matching models, and then two matching signals work jointly for disambiguation with a rank aggregation mechanism.
- 8
- 8
- 8
- 7
- 7
- 7
- 7
- Resorting Relevance Evidences to Cumulative Citation Recommendation for Knowledge Base Acceleration7
Three types of relevance evidences including entities’ profiles, existing citations in KBs, and temporal signals are explored, to supplement the definitions of relevance levels and show that all these approaches outperform the corresponding baselines.
- 7
- 7
- 7
- 7
- 7
- 6
- 6
- 6
- 6
- When Factorization Meets Heterogeneous Latent Topics: An Interpretable Cross-Site Recommendation Framework6
This paper examines how to effectively and efficiently utilize cross-site ratings and content features to improve recommendation performance and how to make the recommendation interpretable by utilizing content features and proposes a joint model of matrix factorization and latent topic analysis.
- 6
- 6
- An Encoder with non-Sequential Dependency for Neural Data-to-Text Generation5
This work proposes to adopt the self-attention mechanism to learn dependencies between arbitrary input records and shows the proposed method achieves comparable results and remains stable under random shuffling over input data.
- 5
- 5
- 4
- 4
- 4
- 4
- MSR KMG at TREC 2014 KBA Track Vital Filtering Task4
The strategy for vital filtering for TREC 2014 KBA track Vital Filtering task is to first retrieve as many relevant documents as possible and then apply classification and ranking methods to differentiate vital documents from non-vital documents.
- 4
- 4
- 4
- 4
- 3
- 3
- 3
- 3
- LDTM: A Latent Document Type Model for Cumulative Citation Recommendation3
An extensive set of experiments has been conducted on TREC-KBA-2013 dataset, and the results demonstrate that this model can yield a significant performance gain in recommendation quality as compared to the state-of-the-art.
- Unsupervised Template Mining for Semantic Category Understanding3
An unsupervised approach to constructing templates from a large collection of semantic category names, and use the templates as the semantic representation of categories, achieves significantly better results than baseline methods.
- 2
- 2
- Layout Generation as Intermediate Action Sequence Prediction2
This paper introduces a novel action schema to encode these operations for better modeling the generation process of layouts, and autoregressively outputs the intermediate action sequence, which can then be deterministically converted to the final layout.
- All Data on the Table: Novel Dataset and Benchmark for Cross-Modality Scientific Information Extraction2
A semi-supervised pipeline for annotating entities in text, as well as entities and relations in tables, in an iterative procedure, and is released, including a high-quality benchmark, a large-scale corpus, and a semi-supervised annotation pipeline.
- 2
- Beyond Word for Word: Fact Guided Training for Neural Data-to-Document Generation2
A Fact Guided Training (FGT) model is proposed to improve both content selection and surface realization by leveraging an information extraction (IE) system and experimental results show the proposed model outperforms neural encoder-decoder models in both automatic and human evaluations.
- 2
- 2
- 2
- 2
- 2
- 2
- 1
- 1
- 1
- 1
- 1
- 1
- –
- –
- –
- –
- Learning Semantic Correspondences from Noisy Data-text Pairs by Local-to-Global Alignments–
This work proposes a two-stage local-to-global alignment (L2GA) framework that can work jointly to learn semantic correspondences in the same framework and shows that the proposed method can be generalized to both restaurant and computer domains and improve the alignment accuracy.
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
Publication data from OpenAlex, with missing venues and authors filled in from Crossref; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-13. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.
Report an error
Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.