Is this you? Claim this profile to correct it, add a bio and choose the work people see first.
Claim this profileAcademic lineage
Possible advisorsa guess from early papers, not confirmed
- Marti A. HearstSuggested from co-authorship
Is this you? Claim this profile to confirm or dismiss it.
Works533 from public data
- SemEval-2016 Task 4: Sentiment Analysis in Twitter966
The fourth year of the SemEval-2016 Task 4 comprises five subtasks, three of which represent a significant departure from previous editions, and the task continues to be very popular, attracting a total of 43 teams.
- Predicting the Type and Target of Offensive Posts in Social Media938
The Offensive Language Identification Dataset (OLID), a new dataset with tweets annotated for offensive content using a fine-grained three-layer annotation scheme, is complied and made publicly available.
- SemEval-2017 Task 4: Sentiment Analysis in Twitter931
The fifth year of the Sentiment Analysis in Twitter task is described, with a new language, Arabic, for all subtasks, and information from the profiles of the Twitter users who posted the target tweets made available.
- SemEval-2010 task 8: multi-way classification of semantic relations between pairs of nominals926
A new task is introduced, which will be part of SemEval-2010: multi-way classification of mutually exclusive semantic relations between pairs of common nominals.
- SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media (OffensEval)893
The results and the main findings of SemEval-2019 Task 6 on Identifying and Categorizing Offensive Language in Social Media (OffensEval), based on a new dataset, contain over 14,000 English tweets, are presented.
- SemEval-2014 Task 9: Sentiment Analysis in Twitter617
The Sentiment Analysis in Twitter task is described, a continuation of the last year’s task that ran successfully as part of SemEval2013 and introduced three new test sets: regular tweets, sarcastic tweets, and LiveJournal sentences.
- Overview of BioCreative II gene mention recognition560
It is demonstrated that, by combining the results from all submissions, an F score of 0.9066 is feasible, and furthermore that the best result makes use of the lowest scoring submissions.
- SemEval-2020 Task 12: Multilingual Offensive Language Identification in Social Media (OffensEval 2020)544
The task included three subtasks corresponding to the hierarchical taxonomy of the OLID schema from OffensEval-2019, and it was offered in five languages: Arabic, Danish, English, Greek, and Turkish.
- 443
- SemEval-2015 Task 10: Sentiment Analysis in Twitter427
The 2015 iteration of the SemEval shared task on Sentiment Analysis in Twitter was the most popular sentiment analysis shared task to date with more than 40 teams participating in each of the last three years.
- Fine-Grained Analysis of Propaganda in News Article423
A novel task: performing fine-grained analysis of texts by detecting all fragments that contain propaganda techniques as well as their type is proposed, and a novel multi-granularity neural network is designed that outperforms several strong BERT-based baselines.
- A Survey of Confidence Estimation and Calibration in Large Language Models366
The challenges are outlined and recent technical advancements for LLM confidence estimation and calibration are summarized, and their applications are discussed and suggested promising directions for future work are suggested.
- FANG345
This work proposes Factual News Graph, a novel graphical social context representation and learning framework for fake news detection that is scalable in training as it does not have to maintain all nodes, and it is efficient at inference time, without the need to re-process the entire graph.
- Automated Fact-Checking for Assisting Human Fact-Checkers334
Here, a survey of the available intelligent technologies that can support the human expert in the different steps of her fact-checking endeavor includes identifying claims worth fact- checking, detecting relevant previously fact-checked claims, retrieving relevant evidence to fact-check a claim, and actually verifying a claim.
- Factuality challenges in the era of large language models and opportunities for fact-checking319
By analysing the factuality constraints within LLMs and their impact on fact-checking, this work aims to contribute to a path towards maintaining accuracy at a time of confluence of generative artificial intelligence and misinformation.
- SemEval-2020 Task 11: Detection of Propaganda Techniques in News Articles300
The results and the main findings of SemEval-2020 Task 11 on Detection of Propaganda Techniques in News Articles are presented and the system submissions and the methods they used are discussed.
- Predicting Factuality of Reporting and Bias of News Media Sources283
This work is interested in characterizing entire news media, an under-studied, but arguably important research problem, both in its own right and as a prior for fact-checking systems.
- SemEval-2016 Task 3: Community Question Answering241
This paper describes the SemEval–2016 Task 3 on Community Question Answering, which was offered in English and Arabic, and found the best systems achieved an official score of 79.19, 76.70, 55.41, and 45.83 in subtasks A, B, C, and D.
- SemEval-2017 Task 3: Community Question Answering233
SemEval–2017 Task 3 on Community Question Answering reran the four subtasks from SemEval-2016, providing all the data from 2015 and 2016 for training, and fresh data for testing, and added a new subtask E in order to enable experimentation with Multi-domain Question Duplicate Detection in a larger-scale scenario, using StackExchange subforums.
- MOMENTA: A Multimodal Framework for Detecting Harmful Memes and Their Targets231
This work proposes MOMENTA (MultimOdal framework for detecting harmful memes, a novel multimodal deep neural network that uses global and local perspectives to detect harmful memes and outperforms several strong rivaling approaches.
- Do-Not-Answer: A Dataset for Evaluating Safeguards in LLMs228
The first open-source dataset to evaluate safeguards in LLMs is collected, and several BERT-like classifiers are trained that can achieve results that are comparable with GPT-4 on automatic safety evaluation.
- SemEval-2007 task 04: classification of semantic relations between nominals222
An evaluation task designed to provide a framework for comparing different approaches to classifying semantic relations between nominals in a sentence as part of SemEval, the 4th edition of the semantic evaluation event previously known as SensEval.
- A Survey on Computational Propaganda Detection219
The state of the art on computational propaganda detection from the perspective of Natural Language Processing and Network Analysis is reviewed, arguing about the need for combined efforts between these communities.
- On the effect of dropping layers of pre-trained transformer models207
The lower layers are most critical to maintain downstream task performance, and some tasks such as paraphrase de-tection and sentence similarity are more robust to the dropping of layers, and models trained using different objective function exhibit di-erent learning patterns and w.r.t the layer dropping.
- We Can Detect Your Bias: Predicting the Political Ideology of News Articles207
A large dataset of articles that were manually annotated for political ideology -left, center, or right-, which is well-balanced across both topics and media, and an adversarial media adaptation, as well as a specially adapted triplet loss are proposed.
- Discriminating between Similar Languages and Arabic Dialect Identification: A Report on the Third DSL Shared Task205
High-order character n-grams were the most successful feature, and the best classification approaches included traditional supervised learning methods such as SVM, logistic regression, and language models, while deep learning approaches did not perform very well.
- Detecting Harmful Memes and Their Targets202
This work presents HarMeme, the first benchmark dataset, containing 3,544 memes related to COVID-19, and proposes two novel problem formulations: detecting harmful memes and the social entities that these harmful memes target.
- Proppy: Organizing the news based on their propagandistic content201
A model to automatically assess the level of propagandistic content in an article based on different representations, from writing style and readability level to the presence of certain keywords is proposed.
- 195
- Findings of the VarDial Evaluation Campaign 2017194
The VarDial Evaluation Campaign on Natural Language Processing (NLP) for Similar Languages, Varieties and Dialects, which was organized as part of the fourth edition of the VarDial workshop at EACL’2017, is presented.
- A Survey on Stance Detection for Mis- and Disinformation Identification193
This work reviews and analyzes existing work in stance detection, with mis- and disinformation in focus, and discusses lessons learnt and future challenges.
- Fighting the COVID-19 Infodemic: Modeling the Perspective of Journalists, Fact-Checkers, Social Media Platforms, Policy Makers, and the Society188
A new dataset for fine-grained disinformation analysis that focuses on COVID-19, combines the perspectives and the interests of journalists, fact-checkers, social media platforms, policy makers, and society as a whole, and covers both English and Arabic is designed and annotated.
- That is a Known Lie: Detecting Previously Fact-Checked Claims188
Learning-to-rank experiments that demonstrate sizable improvements over state-of-the-art retrieval and textual similarity approaches are presented that are largely ignored by the research community so far.
- SOLID: A Large-Scale Semi-Supervised Dataset for Offensive Language Identification187
This work creates the largest available dataset for this task, SOLID, which contains over nine million English tweets labeled in a semi-supervised manner, and demonstrates experimentally that using SOLID along with OLID yields improved performance on the OLID test set for two different models, especially for the lower levels of the taxonomy.
- A Large-Scale Semi-Supervised Dataset for Offensive Language Identification187
This work creates the largest available dataset for this task, SOLID, which contains over nine million English tweets labeled in a semi-supervised manner, and demonstrates experimentally that using SOLID along with OLID yields improved performance on the OLID test set for two different models, especially for the lower levels of the taxonomy.
- 183
- A non-IID Framework for Collaborative Filtering with Restricted Boltzmann Machines162
This work proposes a framework for collaborative filtering based on Restricted Boltzmann Machines (RBM), which extends previous RBM-based approaches in several important directions, and model both user-user and item-item correlations in a unified hybrid non-IID framework.
- Integrating Stance Detection and Fact Checking in a Unified Corpus159
This paper supports the interdependencies between fact checking, document retrieval, source credibility, stance detection and rationale extraction as annotations in the same corpus and implements this setup on an Arabic fact checking corpus, the first of its kind.
- Multi-way classification of semantic relations between pairs of nominals155
A new task is introduced, which will be part of SemEval-2010: multi-way classification of mutually exclusive semantic relations between pairs of common nominals.
- SemEval-2015 Task 3: Answer Selection in Community Question Answering153
SemEval2015 Task 3 on Answer Selection in cQA included two subtasks: (a) classifying answers as good, bad, or potentially relevant with respect to the question, and (b) answering a YES/NO question with yes, no, or unsure, based on the list of all answers.
- BioText Search Engine: beyond abstract search153
The BioText Search Engine is a freely available Web-based application that provides biologists with new ways to access the scientific literature, with the ability to search and browse article figures and their captions.
- Weight functions impact on LSA performance139
Experimental results of usage of LSA for analysis of English literature texts and preliminary transformations of the frequency text-document matrix with different weight functions are tested on the basis of control subsets.
- Benchmarking Uncertainty Quantification Methods for Large Language Models with LM-Polygraph137
A novel benchmark is introduced that implements a collection of state-of-the-art UQ baselines and offers an environment for controllable and consistent evaluation of novel UQ techniques over various text generation tasks, and supports the assessment of confidence normalization methods in terms of their ability to provide interpretable scores.
- SemEval-2021 Task 6: Detection of Persuasion Techniques in Texts and Images136
SemEval-2021 task 6 on Detection of Persuasion Techniques in Texts and Images focused on memes and had three subtasks: detecting the techniques in the text, detecting the text spans where the techniques are used, and detecting Techniques in the entire meme.
- Overview of CheckThat! 2020: Automatic Identification and Verification of Claims in Social Media134
An overview of the third edition of the CheckThat!
- Finding Opinion Manipulation Trolls in News Community Forums133
This work assumes that a user who is called a troll by several people is likely to be one, and shows that a classifier can be trained to distinguish a likely troll from a non-troll with very high accuracy, 82‐95%, thanks to the rich feature set.
- Poor Man's BERT: Smaller and Faster Transformer Models.132
A number of memory-light model reduction strategies that do not require model pre-training from scratch are explored, which are able to prune BERT, RoBERTa and XLNet models by up to 40%, while maintaining up to 98% of their original performance.
- Automatic Stance Detection Using End-to-End Memory Networks131
An effective end-to-end memory network model that jointly predicts whether a given document can be considered as relevant evidence for a given claim, and extracts snippets of evidence that can be used to reason about the factuality of the target claim is presented.
- In Search of Credible News127
This work proposes a language-independent approach for automatically distinguishing credible from fake news, based on a rich feature set that uses linguistic, credibility-related, and semantic features from four online sources.
- LLM360: Towards Fully Transparent Open-Source LLMs124
LLM360, an initiative to fully open-source LLMs, is presented, which advocates for all training code and data, model checkpoints, and intermediate results to be made available to the community.
- Overview of the CLEF-2019 CheckThat! Lab: Automatic Identification and Verification of Claims123
All datasets from the lab are released, which should enable further research in the important tasks of check-worthiness estimation and automatic claim verification, and the evaluation results show that the most successful approaches to Task 1 used various neural networks and logistic regression.
- Fully Automated Fact Checking Using External Sources123
A general-purpose framework for fully-automatic fact checking using external sources, tapping the potential of the entire Web as a knowledge source to confirm or reject a claim.
- Overview of the DSL Shared Task 2015122
The results of the 2 nd edition of the Discriminating between Similar Languages (DSL) shared task, which was organized as part of the LT4VarDial’2015 workshop and focused on the identification of very similar languages and language varieties, are presented.
- A Context-Aware Approach for Detecting Worth-Checking Claims in Political Debates121
A new corpus of political debates is created, containing statements that have been fact-checked by nine reputable sources, and machine learning models are trained to predict which claims in a given document are most worthy and should be prioritized for fact-checking.
- Language Identification and Morphosyntactic Tagging. The Second VarDial Evaluation Campaign.119
The results and the findings of the Second VarDial Evaluation Campaign on Natural Language Processing (NLP) for Similar Languages, Varieties and Dialects and Indo-Aryan Language Identification are presented.
- Detecting Propaganda Techniques in Memes117
This work creates and releases a new corpus of 950 memes, carefully annotated with 22 propaganda techniques, which can appear in the text, in the image, or in both, and shows that understanding both modalities together is essential for detecting these techniques.
- ClaimRank: Detecting Check-Worthy Claims in Arabic and English117
ClaimRank is an online system for detecting check-worthy claims that is trained on actual annotations from nine reputable fact-checking organizations and can mimic the claim selection strategies for each and any of them, as well as for the union of them all.
- Factcheck-Bench: Fine-Grained Evaluation Benchmark for Automatic Fact-checkers116
This work presents a holistic end-to-end solution for annotating the factuality of LLM-generated responses, which encompasses a multi-stage annotation scheme designed to yield detailed labels concerning the verifiability and factual inconsistencies found in LLM outputs.
- Factuality of Large Language Models: A Survey116
This survey critically analyze existing work on evaluating and improving the factuality of large language models with the aim to identify the major challenges and their associated causes, pointing out to potential solutions for improving the factuality of LLMs, and analyzing the obstacles to automated factuality evaluation for open-ended text generation.
- Fighting the COVID-19 Infodemic in Social Media: A Holistic Perspective and a Call to Arms116
An annotation schema is defined and detailed annotation instructions that reflect these perspectives are issued, a multilingual annotation platform is deployed, and a call to arms is issued to the research community and beyond to join the fight by supporting the crowdsourcing annotation efforts.
- Using Discourse Structure Improves Machine Translation Evaluation115
Two discourse-aware similarity measures are designed, which use all-subtree kernels to compare discourse parse trees in accordance with the Rhetorical Structure Theory and can help improve a number of existing machine translation evaluation metrics both at the segment- and at the system-level.
- Detecting Harmful Content on Online Platforms: What Platforms Need vs. Where Research Efforts Go109
There is currently a dichotomy between what types of harmful content online platforms seek to curb, and what research efforts there are to automatically detect such content, and directions for future work are suggested.
- Fact-Checking Meets Fauxtography: Verifying Claims About Images107
A new dataset is created for fact-checking claims about images, and a variety of features are explored modeling the claim, the image, and the relationship between the claim and the image.
- Overview of the CLEF-2018 CheckThat! Lab on Automatic Identification and Verification of Political Claims106
An overview of the CLEF-2018 CheckThat!
- Predicting the Topical Stance and Political Leaning of Media using Tweets105
A cascaded method that uses unsupervised learning to ascertain the stance of Twitter users with respect to a polarizing topic by leveraging their retweet behavior; then, it uses supervised learning based on user labels to characterize both the general political leaning of online media and of popular Twitter users.
- Combining Word-Level and Character-Level Models for Machine Translation Between Closely-Related Languages104
This work uses character-level translation trained on n-gram-character-aligned bitexts and tuned using word-level BLEU to augment with character-based transliteration at the word level and combine with a word- level translation model.
- Improved statistical machine translation for resource-poor languages using related resource-rich languages104
A novel language-independent approach for improving statistical machine translation for resource-poor languages by exploiting their similarity to resource-rich ones while using much less additional data.
- Overview of the CLEF–2021 CheckThat! Lab on Detecting Check-Worthy Claims, Previously Fact-Checked Claims, and Fake News100
The fourth edition of the CheckThat!
- Natural language processing for similar languages, varieties, and dialects: A survey97
The most important challenges when dealing with diatopic language variation are discussed, and some of the available datasets, the process of data collection, and the most common data collection strategies used to compile datasets for similar languages, varieties, and dialects are presented.
- Developing a successful SemEval task in sentiment analysis of Twitter and other social media texts96
The development and evaluation of a semantic analysis task that lies at the intersection of two very trendy lines of research in contemporary computational linguistics: sentiment analysis, and natural language processing of social media text is presented.
- Search engine statistics beyond the n-gram: application to noun compound bracketing96
This paper describes an unsupervised method for noun compound bracketing which extracts statistics from Web search engines using a X2 measure, a new set of surface features, and paraphrases.
- Hunting for Troll Comments in News Community Forums94
In this work, two classifiers are built that can distinguish a post by such a paid troll from one by a non-troll with 81-82% accuracy; the same classifier achieves 81- 82% accuracy on so called mentioned troll vs. non-Troll posts.
- 92
- Findings of the NLP4IF-2019 Shared Task on Fine-Grained Propaganda Detection92
This work presents the shared task on Fine-Grained Propaganda Detection, which was organized as part of the NLP4IF workshop at EMNLP-IJCNLP 2019 and saw 12 teams submit systems for the FLC task, and 14 teams eventually submitted a system description paper.
- Jais and Jais-chat: Arabic-Centric Foundation and Instruction-Tuned Open Generative Large Language Models91
Jais and Jais-chat are introduced, new state-of-the-art Arabic-centric foundation and instruction-tuned open generative large language models (LLMs) based on the GPT-3 decoder-only architecture that demonstrate better knowledge and reasoning capabilities in Arabic than any existing open Arabic and multilingual models by a sizable margin.
- Cross-Domain Label-Adaptive Stance Detection90
An in-depth analysis of 16 stance detection datasets is performed, and an end-to-end unsupervised framework for out-of-domain prediction of unseen, user-defined labels is proposed, which combines domain adaptation techniques such as mixture of experts and domain-adversarial training with label embeddings.
- The CLEF-2021 CheckThat! Lab on Detecting Check-Worthy Claims, Previously Fact-Checked Claims, and Fake News88
The fourth edition of the CheckThat!
- What Was Written vs. Who Read It: News Media Profiling Using Text Analysis and Social Media Context88
The evaluation results show that what was written matters most, and it is shown that putting all information sources together yields huge improvements over the current state-of-the-art.
- SemEval-2024 Task 4: Multilingual Detection of Persuasion Techniques in Memes87
This paper outlines the dataset, the organization of the task, the evaluation framework, the outcomes, and the systems that participated in SemEval-2024, focusing on the multilingual detection of persuasion techniques.
- EXAMS: A Multi-subject High School Examinations Dataset for Cross-lingual and Multilingual Question Answering87
This work proposes EXAMS -- a new benchmark dataset for cross-lingual and multilingual question answering for high school examinations and performs various experiments with existing top-performing multilingual pre-trained models to show that EXAMS offers multiple challenges that require multilingual knowledge and reasoning in multiple domains.
- Adversarial Domain Adaptation for Duplicate Question Detection87
This work addresses the problem of detecting duplicate questions in forums, and focuses on adversarial domain adaptation, deriving important findings about when it performs well and what properties of the domains are important in this regard.
- Multimodality Representation Learning: A Survey on Evolution, Pretraining and Its Applications86
This survey presents the comprehensive literature on the evolution and enhancement of deep learning multimodal architectures to deal with textual, visual and audio features for diverse cross-modal and modern multimodals tasks and prepares a dataset section for new researchers that covers most of the benchmarks for pretraining and finetuning.
- SemEval-2024 Task 8: Multidomain, Multimodel and Multilingual Machine-Generated Text Detection85
The results and the main findings of SemEval-2024 Task 8: Multigenerator, Multidomain, and Multilingual Machine-Generated Text Detection are presented, and the best systems used LLMs are discussed.
- Using the web as an implicit training set: application to structural ambiguity resolution85
This paper shows how the use of surface features and paraphrases in queries against search engines can be used to infer labels for structural ambiguity resolution tasks using unsupervised algorithms.
- Multi-Task Ordinal Regression for Jointly Predicting the Trustworthiness and the Leading Political Ideology of News Media84
This work proposes a multi-task ordinal regression framework that models the two problems of trustworthiness estimation and political ideology detection of entire news outlets, as opposed to evaluating individual articles, and shows sizable performance gains by the joint models over models that target the problems in isolation.
- 81
- Few-Shot Cross-Lingual Stance Detection with Sentiment-Based Pre-training80
This paper presents the most comprehensive study of cross-lingual stance detection to date, and proposes sentiment-based generation of stance data for pre-training, which shows sizeable improvement of more than 6% F1 absolute in few-shot learning settings compared to several strong baselines.
- Solving Relational Similarity Problems Using the Web as a Corpus80
The main idea is to look for verbs, prepositions, and coordinating conjunctions that can help make explicit the hidden relations between the target nouns.
- On the interpretation of noun compounds: Syntax, semantics, and entailment79
It is shown how understanding noun compound syntax and semantics could help solve textual entailment problems, which would be potentially useful for a number of NLP applications, and which the author believes to be an important direction for future research.
- Proppy: A System to Unmask Propaganda in Online News78
Proppy, the first publicly available real-world, real-time propaganda detection system for online news, is presented, which aims at raising awareness, thus potentially limiting the impact of propaganda and helping fight disinformation.
- Using verbs to characterize noun-noun relations78
A novel, simple, unsupervised method for characterizing the semantic relations that hold between nouns in noun-noun compounds by writing Web search engine queries that restate the noun compound as a relative clause containing a wildcard character to be filled in with a verb.
- Classification of semantic relations between nominals77
This work presents the development and evaluation of a semantic analysis task: automatic recognition of relations between pairs of nominals in a sentence, part of SemEval-2007, the fourth edition of the semantic evaluation event previously known as SensEval.
- Improving English-Spanish statistical machine translation77
The experiments of the UC Berkeley team on improving English-Spanish machine translation of news text, as part of the WMT'08 Shared Translation Task, describe the experiments and experiment with domain adaptation, building two separate phrase translation models and two separate language models.
- Overview of the CLEF-2019 CheckThat! Lab: Automatic Identification and Verification of Claims. Task 1: Check-Worthiness.75
An overview of the 2nd edition of the CheckThat!
- Fact Checking in Community Forums75
A novel multi-faceted model is proposed, which captures information from the answer content (what is said and how), from the author profile, from the rest of the community forum (where it is said), and from external authoritative sources of information (external support).
- Pronoun-Focused MT and Cross-Lingual Pronoun Prediction: Findings of the 2015 DiscoMT Shared Task on Pronoun Translation75
This work described the design, the evaluation setup, and the results of the DiscoMT 2015 shared task, which included two subtasks, relevant to both the machine translation (MT) and the discourse communities: pronoun-focused translation and cross-lingual pronoun prediction.
- 74
- Prta: A System to Support the Analysis of Propaganda Techniques in the News73
Pta (Propaganda Persuasion Techniques Analyzer) allows users to explore the articles crawled on a regular basis by highlighting the spans in which propaganda techniques occur and to compare them on the basis of their use of propaganda techniques.
- SemEval-2019 Task 8: Fact Checking in Community Question Answering Forums73
This work presents SemEval-2019 Task 8 on Fact Checking in Community Question Answering Forums, which features two subtasks, and received 17 official submissions for subtask A and 11official submissions for Subtask B.
- Noun Compound Interpretation Using Paraphrasing Verbs: Feasibility Study71
This paper creates a valuable manually-annotated resource for noun compound interpretation, and performs a number of experiments to assess the dataset quality and the feasibility of the idea of using paraphrasing verbs to characterise noun compounds' semantics; the results are quite promising.
- 66
- The Spread of Propaganda by Coordinated Communities on Social Media65
This work analyzes the spread of propaganda and its interplay with coordinated behavior on a large Twitter dataset about the 2019 UK general election and introduces a methodology for analyzing several important dimensions of online behavior that are seldom conjointly considered.
- Overview of the CLEF-2018 CheckThat! Lab on Automatic Identification and Verification of Political Claims. Task 2: Factuality65
An overview of the CLEF-2018 CheckThat!
- Optimizing for Sentence-Level BLEU+1 Yields Short Translations64
Several ways to fix the length problem of PRO are proposed, including smoothing the brevity penalty, scaling the effective reference length, grounding the precision component, and unclipping the bre length penalty.
- QCRI: Answer Selection for Community Question Answering - Experiments for Arabic and English62
This paper describes QCRI’s participation in SemEval-2015 Task 3 “Answer Selection in Community Question Answering”, which targeted real-life Web forums, and was offered in both Arabic and English, where its approach was the best performing one in the Arabic subtask and the third best in the two English subtasks.
- Transformers: “The End of History” for Natural Language Processing?61
It is demonstrated in practice on two general types of tasks -- segmentation and segment labeling -- and on four datasets that these limitations of pre-trained BERT-style models that are inherent in the general Transformer architecture are indeed harmful and that addressing them can yield sizable improvements over vanilla RoBERTa and XLNet models.
- CheckThat! at CLEF 2020: Enabling the Automatic Identification and Verification of Claims in Social Media61
The third edition of the CheckThat!
- SemEval-2010 task 9: the interpretation of noun compounds using paraphrasing verbs and prepositions61
A new task is introduced to be part of SemEval-2010: the interpretation of noun compounds using paraphrasing verbs and prepositions using paraphrasing verbs and prepositions to provide a standard testbed for future research on noun compound semantics.
- Overview of the CLEF–2022 CheckThat! Lab on Fighting the COVID-19 Infodemic and Fake News Detection60
The fifth edition of the CheckThat! lab evaluates technology supporting tasks related to factuality in multiple languages: Arabic, Bulgarian, Dutch, English, German, Spanish, and Turkish.
- SemanticZ at SemEval-2016 Task 3: Ranking Relevant Answers in Community Question Answering Using Semantic Similarity Based on Fine-tuned Word Embeddings60
This work describes its system for finding good answers in a community forum, as defined in SemEval-2016, ask 3 on Community Question Answering, which relies on several semantic similarity features based on fine-tuned word embeddings and topics similarities.
- Recent Advances in Online Hate Speech Moderation: Multimodality and the Role of Large Models58
This comprehensive survey delves into the recent strides in HS moderation, spotlighting the burgeoning role of large language models (LLMs) and large multimodal models (LMMs) and the development of more nuanced, context-aware systems.
- 57
- 57
- 56
- Findings of the 2016 WMT Shared Task on Cross-lingual Pronoun Prediction56
The design, the evaluation setup, and the results of the 2016 WMT shared task on cross-lingual pronoun prediction on cross-lingual pronoun prediction are described.
- Pairwise Neural Machine Translation Evaluation56
A novel framework for machine translation evaluation using neural networks in a pairwise setting, where the goal is to select the better translation from a pair of hypotheses, given the reference translation.
- 55
- Contrastive Language Adaptation for Cross-Lingual Stance Detection55
A novel contrastive language adaptation approach applied to memory networks is introduced, which ensures accurate alignment of stances in the source and target languages, and can effectively deal with the challenge of limited labeled data in the target language.
- Overview of PAN 2025: Voight-Kampff Generative AI Detection, Multilingual Text Detoxification, Multi-author Writing Style Analysis, and Generative Plagiarism Detection54
The goal of the PAN lab is to advance the state of the art in text forensics and stylometry through an objective evaluation of new and established methods on new benchmark datasets through an objective evaluation of new and established methods on new benchmark datasets.
- 54
- The dark side of news community forums: opinion manipulation trolls54
The idea that a user who is called a troll by several people is likely to be one further demonstrates the utility of this idea for detecting accused and paid opinion manipulation trolls and their comments as well as for predicting the credibility of comments in news community forums.
- Cross-language Learning with Adversarial Neural Networks54
This work proposes to use adversarial training of neural networks to learn high-level features that are discriminative for the main learning task, and at the same time are invariant across the input languages.
- Findings of the NLP4IF-2021 Shared Tasks on Fighting the COVID-19 Infodemic and Censorship Detection53
The results and the main findings of the NLP4IF-2021 shared tasks, which focused on fighting the COVID-19 infodemic in social media, and was offered in Arabic, Bulgarian, and English are presented.
- Temporal dynamics of coordinated online behavior: Stability, archetypes, and influence52
This study focuses on the temporal dynamics of coordinated campaigns and characterizes their influence on Twitter during two recent major elections, revealing different user behaviors and archetypes and measuring how they affect and influence the online environment.
- 52
- Global Thread-level Inference for Comment Classification in Community Question Answering51
This work focuses on exploiting the output structure at the thread level in order to make more consistent global decisions, and exploits the relations between pairs of comments at any distance in the thread, which is incorporated in a graph-cut and in an ILP frameworks.
- Improving Statistical Machine Translation for a Resource-Poor Language Using Related Resource-Rich Languages51
A novel language-independent approach for improving machine translation for resource-poor languages by exploiting their similarity to resource-rich ones, which cuts the amount of necessary "real" training data by a factor of 2-5.
- OpenFactCheck: A Unified Framework for Factuality Evaluation of LLMs48
OpenFactCheck is a unified framework, with three modules: RESPONSEEVAL, which allows users to easily customize an automatic fact-checking system and to assess the factuality of all claims in an input document using that system, LLMEVAL, which assesses the overall factuality of an LLM, and CHECKEREVAL, a module to evaluate automatic fact-checking systems.
- 48
- 48
- It Takes Nine to Smell a Rat: Neural Multi-Task Learning for Check-Worthiness Prediction48
It is shown that it pays to learn from multiple sources simultaneously in a multi-task learning setup when estimating the check-worthiness of claims in political debates, even when a particular source is chosen as a target to imitate.
- Semantic interpretation of noun compounds using verbal and other paraphrases48
The potential of using predicates that make explicit the hidden relation that holds between the nouns that form the noun compound are explored, and it is demonstrated that using explicit paraphrases can help improve statistical machine translation.
- Improved Statistical Machine Translation Using Monolingual Paraphrases48
A novel monolingual sentence paraphrasing method for augmenting the training data for statistical machine translation systems “for free” by creating it from data that is already available rather than having to create more aligned data.
- Multilingual Multifaceted Understanding of Online News in Terms of Genre, Framing, and Persuasion Techniques47
A new multilingual multifacet dataset of news articles, each annotated for genre (objective news reporting vs. opinion vs. satire), framing, and persuasion techniques (logical fallacies, emotional appeals, ad hominem attacks, etc.).
- CheckThat! at CLEF 2019: Automatic Identification and Verification of Claims47
The second edition of the CheckThat!
- 46
- One Size Does Not Fit All: Comparing NMT Representations of Different Granularities45
While representations derived from subwords are slightly better for modeling syntax, character-based representations are superior for modeling morphology and are also more robust to noisy input.
- Cross-Language Question Re-Ranking45
This work addresses the issue of how to find relevant questions in community forums when the language of the new questions is different from that of the existing questions in the forum using a cross-language tree kernel, which shows that this kernel almost closes the performance gap with respect to the monolingual system.
- Thread-Level Information for Comment Classification in Community Question Answering45
Two ways of modeling dependencies between the answer labels captured by structured prediction models are explored, showing that the thread-level features consistently improve the performance for a variety of machine learning models, yielding state-of-the-art results.
- 42
- 42
- DiscoTK: Using Discourse Structure for Machine Translation Evaluation42
Novel automatic metrics for machine translation evaluation that use discourse structure and convolution kernels to compare the discourse tree of an automatic translation with that of the human reference are presented.
- 41
- Seminar Users in the Arabic Twitter Sphere41
This work introduces the notion of “seminar users”, who are social media users engaged in propaganda in support of a political entity, and develops a framework that can identify such users with 84.4% precision and 76.1% recall.
- $\texttt{Droid}$: A Resource Suite for AI-Generated Code Detection40
This work compiles the most extensive open data suite for training and evaluating machine-generated code detectors, comprising over a million code samples, seven programming languages, outputs from 43 coding models, and over three real-world coding domains.
- 40
- Leaf: Multiple-Choice Question Generation40
Leaf, a system for generating multiple-choice questions from factual text that could be used in an industrial setting, e.g., to facilitate onboarding and knowledge sharing, or as a component of chatbots, question answering systems, or Massive Open Online Courses (MOOCs).
- The Role of Context in Detecting Previously Fact-Checked Claims40
This work focuses on claims made in a political debate and studies the impact of modeling the context of the claim, finding that modeling the source-side context is most important, and can yield 10+ points of absolute improvement over a state-of-the-art model.
- A Comprehensive Survey of Machine Unlearning Techniques for Large Language Models39
This study investigates the machine unlearning techniques within the context of large language models (LLMs) by introducing the definition and the paradigms of LLM unlearning, followed by a comprehensive taxonomy of existing unlearning studies.
- RuleBERT: Teaching Soft Rules to Pre-Trained Language Models39
This work introduces a classification task where, given facts and soft rules, thePLM should return a prediction with a probability for a given hypothesis, and proposes a revised loss function that enables the PLM to learn how to predict precise probabilities for the task.
- Towards Automated Customer Support39
The authors' experiments with the Twitter Customer Support Dataset, which contains over two million posts from customer support services of twenty major brands, show that the seq2seq model outperforms the other two in terms of semantics and word overlap.
- SemEval-2 Task 9: The Interpretation of Noun Compounds Using Paraphrasing Verbs and Prepositions39
The background, the task definition, the process of data collection and the task results are described, and a few general conclusions are venture before the participating teams present their systems at the SemEval-2010 workshop.
- Fake News Detectors are Biased against Texts Generated by Large Language Models38
A novel paradigm to evaluate fake news detectors in scenarios involving both human-written and LLM-generated misinformation, revealing a significant bias in many existing detectors that appears to arise from distinct linguistic patterns inherent to LLM outputs.
- The Case for Being Average: A Mediocrity Approach to Style Masking and Author Obfuscation38
An approach that changes the text, so that it is pushed towards average values for some general stylometric characteristics, thus making the use of these characteristics less discriminative and yielded the best performance on the Author Obfuscation task at the PAN-2016 competition.
- Fine-grained sentiment analysis for movie reviews in Bulgarian38
A system for fine-grained sentiment analysis in Bulgarian movie reviews, using as features not only the text from the reviews, but also contextual information in the form of metadata, shows that adding contextual information yields strong performance gains.
- 37
- 37
- NewsClaims: A New Benchmark for Claim Detection from News with Attribute Knowledge37
This work extends the claim detection problem to include extraction of additional attributes related to each claim and releases 889 claims annotated over 143 news articles, seeing that zero-shot and prompt-based baselines show promising performance on this benchmark, while still considerably behind human performance.
- Interpretable Propaganda Detection in News Articles37
This work defines qualitatively descriptive features and analyzes their suitability for detecting deception techniques, and shows that their interpretable features can be easily combined with pre-trained language models, yielding state-of-the-art results.
- AraStance: A Multi-Country and Multi-Domain Dataset of Arabic Stance Detection for Fact Checking37
The new Arabic Stance Detection dataset (AraStance) is presented, of 4,063 claim–article pairs from a diverse set of sources comprising three fact-checking websites and one news website, and it is well-balanced between related and unrelated documents with respect to the claims.
- Machine translation evaluation with neural networks37
A pairwise neural network (NN) is proposed for machine translation evaluation using neural networks in a pairwise setting, where the goal is to select the better translation from a pair of hypotheses, given the reference translation.
- 36
- 36
- 36
- 36
- BioText Team Report for the TREC 2003 Genomics Track.36
The BioText project team participated in both tasks of the TREC 2003 genomics track and used an organism-name recognition module, a module for recognizing gene name variants, and MeSH descriptors, which yielded results within the top three groups in both tasks.
- MALMM: Multi-Agent Large Language Models for Zero-Shot Robotic Manipulation34
A novel multi-agent LLM framework, Multi-Agent Large Language Model for Manipulation (MALMM), which demonstrates excellent performance in solving previously unseen long-horizon manipulation tasks, and outperforms existing zero-shot LLM-based methods in RLBench by a large margin.
- Predicting the Leading Political Ideology of YouTube Channels Using Acoustic, Textual, and Metadata Information34
The analysis shows that the use of acoustic signal helped to improve bias detection by more than 6% absolute over using text and metadata only, and the dataset is released to the research community, hoping to help advance the field of multi-modal political bias detection.
- Discourse Structure in Machine Translation Evaluation34
This article first design discourse-aware similarity measures, which use all-subtree kernels to compare discourse parse trees in accordance with the Rhetorical Structure Theory (RST), and shows that a simple linear combination with these measures can help improve various existing machine translation evaluation metrics regarding correlation with human judgments.
- GenAI Content Detection Task 1: English and Multilingual Machine-Generated Text Detection: AI vs. Human33
A comprehensive overview of the data, a summary of the results -- including system rankings and performance scores -- detailed descriptions of the participating systems, and an in-depth analysis of submissions are provided.
- Assisting the Human Fact-Checkers: Detecting All Previously Fact-Checked Claims in a Document33
A system that aims to detect all sentences that contain a claim that can be verified by some previously fact-checked claims (from a given database) and outputs a re-ranked list of the document sentences, so that those that could be verified are ranked as high as possible, together with corresponding evidence.
- Tanbih: Get To Know What You Are Reading33
Tanbih, a news aggregator with intelligent analysis tools to help readers understanding what’s behind a news story, is introduced.
- Sentiment Analysis in Twitter for Macedonian33
A corpus of tweets annotated with tweet- level sentiment polarity (positive, negative, and neutral), as well as with phrase-level sentiment, which was developed and made freely available for research purposes and bootstrapped several large-scale sentiment lexicons for Macedonian.
- 32
- Findings of the CONSTRAINT 2022 Shared Task on Detecting the Hero, the Villain, and the Victim in Memes32
The task aims to delve deeper into the domain of meme comprehension by deciphering the connotations behind the entities present in a meme by determining the victimizing, glorifying, and vilifying intentions embedded in meme entities to explicate their connotations.
- 32
- Towards Deeper Understanding of the LSA Performance31
The results show that while the linguistic processing has a substantial influence on the LSA performance, the traditional factors are even more important, and therefore it did not prove that the linguistic pre-processing substantially improves text categorisation.
- Atlas-Chat: Adapting Large Language Models for Low-Resource Moroccan Arabic Dialect30
Atlas-Chat is introduced, the first-ever collection of LLMs specifically developed for dialectal Arabic, and it is believed this work offers comprehensive design methodologies of instruction-tuning for low-resource languages, which are often neglected in favor of data-rich languages by contemporary LLMs.
- A Second Pandemic? Analysis of Fake News About COVID-19 Vaccines in Qatar29
An extensive analysis of Arabic and English tweets about COVID-19 vaccines, with focus on messages originating from Qatar found that Arabic tweets contain a lot of false information and rumors, while English tweets are mostly factual.
- Detecting Deception in Political Debates Using Acoustic and Textual Features29
A multimodal deep-learning architecture for the task of deception detection was developed, which yielded sizable improvements over the state of the art for the CLEF-2018 Lab task 2.
- Machine Translation Evaluation Meets Community Question Answering29
A pairwise neural network architecture is adopted, which incorporates MTE features, as well as rich syntactic and semantic embeddings, and which efficiently models complex non-linear interactions.
- Category-based pseudowords29
The use of lexical categories are proposed to create more realistic pseudowords, and the results of different variations of this idea against the standard approach are evaluated.
- 28
- Multimodal Large Language Models to Support Real-World Fact-Checking28
This study empirically finds that GPT-4V exhibits superior performance in identifying malicious and misleading multimodal claims, with the ability to explain the unreasonable aspects and underlying motives, and existing open-source models exhibit strong biases and are highly sensitive to the prompt.
- Towards Suicide Ideation Detection Through Online Conversational Context28
HCN, a Hyperbolic Conversation Network is proposed, which is a less user-intrusive method for suicide ideation detection and leverages the hyperbolic space to represent the scale-free dynamics of online conversations.
- Predicting the Role of Political Trolls in Social Media28
Experiments on the “IRA Russian Troll” dataset show that the methodology improves over the state-of-the-art in the first scenario, while providing a compelling case for the second scenario, which has not been explored in the literature thus far.
- Do Not Trust the Trolls: Predicting Credibility in Community Question Answering Forums28
The problem is motivated and a publicly available annotated English corpus is created by crowdsourcing and a large set of features to predict the credibility of the answers are proposed, showing that the credibility labels can be predicted with high performance according to several standard IR ranking metrics.
- Joint Learning with Global Inference for Comment Classification in Community Question Answering28
This paper addresses the problem of comment classification in community Question Answering with a global inference process to exploit the information of all comments in the answer-thread in the form of a fully connected graph with novel joint learning models that are on-line and integrate inference within learning.
- Building an inflectional stemmer for Bulgarian28
The design, implementation and evaluation of an inflectional stemmer for Bulgarian are described and a detailed automatic evaluation for different parameter values in terms of under- stemming, over-stemming and coverage is provided.
- Loki: An Open-Source Tool for Fact Verification27
Loki is introduced, an open-source tool designed to address the growing problem of misinformation that decomposes the fact-checking task into a five-step pipeline and provides essential information at each step to assist human judgment.
- 27
- Aschern at SemEval-2020 Task 11: It Takes Three to Tango: RoBERTa, CRF, and Transfer Learning27
This work developed ensemble models using RoBERTa-based neural architectures, additional CRF layers, transfer learning between the two subtasks, and advanced post-processing to handle the multi-label nature of the task, the consistency between nested spans, repetitions, and labels from similar spans in training.
- Rotational Unit of Memory: A Novel Representation Unit for RNNs with Scalable Applications27
This work derives a phase-coded representation of the memory state, Rotational Unit of Memory (RUM), that unifies the concepts of unitary learning and associative memory and shows experimentally that RNNs based on RUMs can solve basic sequential tasks such as memory copying and memory recall much better than LSTMs/GRUs.
- Proceedings of the Workshop on Multiword Expressions: Identification, Interpretation, Disambiguation and Applications27
This workshop focused on Multi-Word Expressions (MWEs), which represent an indispensable part of natural languages and appear steadily on a daily basis, both novel and already existing but paraphrased, which makes them important for many natural language applications.
- A study of using search engine page hits as a proxy for n-gram frequencies27
Using a particular NLP task, the variability in the n-gram counts across different search engines as well as for the same search engine across time is compared, finding that although there are measurable differences, they are not statistically significantly different for the task examined.
- Latent semantic analysis of textual data27
An overview of the usage of the LSA for analysis of textual data and the potential of LSA on selected corpus of religious and sacred texts is demonstrated.
- Semantic Ranking for Automated Adversarial Technique Annotation in Security Text26
A novel approach for mapping attack behaviors described in threat analysis reports to entries in an adversarial techniques knowledge base by leveraging a multi-stage ranking architecture to efficiently rank the most related techniques based on their semantic relevance to the input text.
- 26
- SUper Team at SemEval-2016 Task 3: Building a Feature-Rich System for Community Question Answering26
The system built for participating in SemEval-2016 Task 3 on Community Question Answering achieved the best results on subtask C, and strong Results on subtasks A and B, by combining a rich set of various types of features: semantic, lexical, metadata, and user-related.
- 25
- COVID-19 in Bulgarian Social Media: Factuality, Harmfulness, Propaganda, and Framing25
This work studied how COVID-19 is discussed in Bulgarian social media in terms of factuality, harmfulness, propaganda, and framing and found that most Bulgarian tweets contain verifiable factual claims, are factually true, are of potential public interest, are not harmful, and are too trivial to fact-check.
- 24
- The Battlefront of Combating Misinformation and Coping with Media Bias24
This tutorial dives into the important research questions of how to develop a robust fake news detection system, which not only fact-check information pieces provable by background knowledge but also reason about the consistency and the reliability of subtle details for emerging events.
- 24
- Dense vs. Sparse representations for news stream clustering24
The evaluation results on a standard dataset show a sizeable improvement over the state of the art both for the standard F1 as well as for a BCubed version thereof, which is argued is more suitable for the task.
- 24
- Source Language Adaptation Approaches for Resource-Poor Machine Translation24
Three novel, language-independent approaches to source language adaptation for resource-poor statistical machine translation are proposed by adapting and using a large bitext for a related resource-rich language RICH and the same target language TGT.
- 23
- 23
- FANG23
The experimental results show that FANG is better at capturing the social context into a high-fidelity representation, compared to recent graphical and nongraphical models and generalize to related tasks, such as predicting the factuality of reporting of a news medium.
- Evaluating Pronominal Anaphora in Machine Translation: An Evaluation Measure and a Test Suite23
An extensive, targeted dataset is contributed that can be used as a test suite for pronoun translation, covering multiple source languages and different pronoun errors drawn from real system translations, for English and an evaluation measure is proposed to differentiate good and bad pronoun translations.
- Findings of the 2017 DiscoMT Shared Task on Cross-lingual Pronoun Prediction23
The design, the setup, and the evaluation results of the DiscoMT 2017 shared task on cross-lingual pronoun prediction show that most participating teams outperformed two strong n-gram-based language model-based baseline systems by a sizable margin.
- 22
- Beyond English-Only Reading Comprehension: Experiments in Zero-Shot Multilingual Transfer for Bulgarian22
This work studies the effectiveness of multilingual BERT fine-tuned on large-scale English datasets for reading comprehension (e.g., for RACE), and applies it to Bulgarian multiple-choice reading comprehension.
- 22
- QCRI at IWSLT 2013: Experiments in Arabic-English and English-Arabic Spoken Language Translation22
A specialized normalization scheme for evaluating Arabic output is described, which was adopted for the IWSLT’2013 evaluation campaign and experimented with three domain adaptation methods, and with various Arabic word segmentation schemes.
- Source Language Adaptation for Resource-Poor Machine Translation22
A novel, language-independent approach for improving machine translation from a resource-poor language to X by adapting a large bi-text for a related resource-rich language and X (the same target language).
- Guessing morphological classes of unknown German nouns22
A system for recognition and morphological classification of unknown German words is described that learns and applies ending-guessing rules similar to the ones originally proposed for POS guessing.
- Towards More Robust Retrieval-Augmented Generation: Evaluating RAG Under Adversarial Poisoning Attacks21
A rigorously controlled empirical study of how RAG systems behave under adversarial poisoning attacks and how their robustness can be improved, and reveals that sceptical prompting can activate LLMs'internal reasoning, enabling partial self-defense against adversarial passages.
- Enriched Pre-trained Transformers for Joint Slot Filling and Intent Detection21
This work proposes an intent pooling attention mechanism, and it reinforces the slot filling task by fusing intent distributions, word features, and token representations, which outperforms both the current non-BERT state of the art as well as stronger BERT-based baselines.
- 21
- 21
- UCB: system description for SemEval task #421
The UC Berkeley team participated in the SemEval 2007 Task #4, with an approach that leverages the vast size of the Web in order to build lexically-specific features.
- GenAI Content Detection Task 3: Cross-Domain Machine-Generated Text Detection Challenge20
It is found that multiple participants were able to obtain accuracies of over 99% on machine-generated text from RAID while maintaining a 5% False Positive Rate -- suggesting that detectors are able to robustly detect text from many domains and models simultaneously.
- Exploring the Limitations of Detecting Machine-Generated Text20
An audit of the classification performance for detecting machine-generated text by evaluating on texts with varying writing styles finds that classifiers are highly sensitive to stylistic changes and differences in text complexity, and in some cases degrade entirely to random classifiers.
- OffensEval 2023: Offensive language identification in the age of Large Language Models20
The results show that while some LMMs such as Flan-T5 achieve competitive performance, in general LLMs lag behind the best OffensEval systems.
- Team QCRI-MIT at SemEval-2019 Task 4: Propaganda Analysis Meets Hyperpartisan News Detection20
This work trained a logistic regression model with features ranging from simple bag of words to vocabulary richness and text readability, and achieved 72.9% accuracy on the manually annotated testset, and 60.8% on the test data that was obtained with distant supervision.
- 20
- 20
- 19
- WhatTheWikiFact19
WhatTheWikiFact, a system for automatic claim verification using Wikipedia, can predict the veracity of an input claim, and it further shows the evidence it has retrieved as part of the verification process.
- Joint Multitask Learning for Community Question Answering Using Task-Specific Embeddings19
This work uses deep neural networks to learn meaningful task-specific embeddings, which it incorporates into a conditional random field model for the multitask setting, performing joint learning over a complex graph structure.
- Community Moderation and the New Epistemology of Fact Checking on Social Media18
The current approaches to misinformation detection across major platforms are examined, the emerging role of community-driven moderation is explored, and both the promises and challenges of crowd-checking at scale are critically evaluated.
- 18
- Ten years after ImageNet: a 360° perspective on artificial intelligence18
A holistic perspective on artificial intelligence shows how the rise of attention networks, self-supervised learning, generative modelling and graph neural networks has widened the application space of AI and raised socio-technical issues such as transparency, fairness and accountability.
- 18
- Do Peers See More in a Paper Than Its Authors?18
Focusing on the area of molecular interactions, manual and automatic analysis is performed, and it is found that the set of all citances to a target article not only covers most information found in its abstract, but also contains 20% more concepts.
- 17
- 17
- WERD: Using social text spelling variants for evaluating dialectal speech recognition17
This work borrows ideas from an MT evaluation metric, namely TERp, an extension of translation error rate which is closely-related to WER, and shows that the resulting WERd (i.e., WER for dialects) metric is more adequate for evaluating dialectal ASR.
- 16
- Paper Bullets: Modeling Propaganda with the Help of Metaphor16
The first work that models metaphor and propaganda together, and finds that leveraging metaphor improves model performance, particularly for the two most common propaganda techniques: loaded language and name-calling.
- 16
- Machine Reading Comprehension for Answer Re-Ranking in Customer Support Chatbots16
A method that adapts a deep neural architecture from the domain of machine reading comprehension to re-rank the suggested answers from different models using the question as a context and results show that this re-ranking framework can improve the performance in terms of word overlap and semantics both for individual models as well as for model combinations.
- MTE-NN at SemEval-2016 Task 3: Can Machine Translation Evaluation Help Community Question Answering?16
- Adaptivity in Web-Based CALL16
STyLE4 supports adaptive learning of English terminology with a target user group of non-native speakers and attempts to improve Computer-Aided Language Learning by intelligent integration of Natural Language Processing (NLP) and personalised Information Retrieval (IR) into a single coherent system.
- Detecting Check-Worthy Claims in Political Debates, Speeches, and Interviews Using Audio Data15
It is experimentally demonstrated that, in the case of multiple speakers, adding the audio modality yields sizable improvements over using the text modality alone; moreover, an audio-only model could outperform a text-only one for a single speaker.
- 15
- 15
- Detecting Toxicity in News Articles: Application to Bulgarian15
A news toxicity detector that can recognize various types of toxic content is proposed and developed that targets Bulgarian and shows sizable improvements over the majority-class baseline.
- 15
- It Takes Three to Tango: Triangulation Approach to Answer Ranking in Community Question Answering15
This work addresses the problem of answering new questions in community forums, by selecting suitable answers to already asked questions as an answer ranking problem, adopting a pairwise neural network architecture that selects which of two competing answers is better.
- Unsupervised Word Segmentation Improves Dialectal Arabic to English Machine Translation15
This work demonstrates the feasibility of using unsupervised morphological segmentation for dialects of Arabic, which are poor in linguistics resources and shows that a uniform segmentation scheme across them yields an improvement of 1.5 BLEU points over using no segmentation.
- A Tale about PRO and Monsters15
This work proposes several effective ways to address the problem of PRO falls victim to monsters, using length- and BLEU+1based cut-offs, outlier filters, stochastic sampling, and random acceptance, and recommends them to anybody using PRO, monsterbeliever or not.
- Unsupervised Extraction of False Friends from Parallel Bi-Texts Using the Web as a Corpus15
This paper presents several unsupervised algorithms for acquiring false friends from a sentence-aligned bi-text, and experiments with a semantic similarity measure that uses the Web as a corpus to extract local contexts from text snippets returned by a search engine, and a bilingual glossary of known word translation pairs, used as “bridges”.
- LLM360 K2: Building a 65B 360-Open-Source Large Language Model from Scratch14
The LLM360 K2 project provides full transparency and access to resources accumulated during the training of LLMs at the largest scale, including the first model, K2 DIAMOND, a 65 billion-parameter LLM that surpasses LLaMA-65B and rivals LLaMA2-70B, while requiring fewer FLOPs and tokens.
- 14
- 14
- Automated Mapping of CVE Vulnerability Records to MITRE CWE Weaknesses14
This work focuses on mapping CVE records into MITRE CWE Weaknesses, and releases a manually annotated dataset of 4,012 records for this task, approach the problem as a ranking task and aim to incorporate reinforced learning to make use of the human feedback in future work.
- 14
- Fake News, Disinformation, Propaganda, and Media Bias14
The research community responded to the issue, proposing a number of inter-connected research directions such as fact-checking, disinformation, misinformation, fake news, propaganda, and media bias detection, which are of interest to human fact-checkers and journalists.
- 14
- 14
- 14
- An ensemble-rich multi-aspect approach for robust style change detection: Notebook for PAN at CLEF-201814
The supervised approach, which combines a TF.IDF representation of the documents with features specifically engineered for the task and which makes predictions using an ensemble of diverse models including SVM, Random Forest, AdaBoost, MLP and LightGBM, is presented.
- Supporting annotation layers for natural language processing14
A query language for flexible querying against text that has been annotated with the results of NLP processing is demonstrated and its use on examples taken from the NLP literature is presented.
- 13
- 13
- 13
- Arabic Dialect Processing13
A high level generic view of dialects and different aspects of them that are of interest for the NLP community are presented, addressing both text and speech issues in addition to standardization issues.
- Fact-Checking, Fake News, Propaganda, and Media Bias: Truth Seeking in the Post-Truth Era13
The tutorial will offer an overview of the broad and emerging research area of disinformation, with focus on the latest developments and research directions.
- 13
- Learning to Differentiate Better from Worse Translations13
This work presents a pairwise learning-to-rank approach to machine translation evaluation that learns to differentiate better from worse translations in the context of a given reference and shows that learning in the proposed framework yields better correlation with humans than computing the direct similarity over the same type of structures.
- Fast or Better? Balancing Accuracy and Cost in Retrieval-Augmented Generation with Flexible User Control12
This paper introduces a novel user-controllable RAG framework that enables dynamic adjustment of the accuracy-cost trade-off, and leverages two classifiers: one trained to prioritize accuracy and another to prioritize retrieval efficiency.
- Recent Advances in Hate Speech Moderation: Multimodality and the Role of Large Models12
This comprehensive survey delves into the recent strides in HS moderation, spotlighting the burgeoning role of large language models (LLMs) and large multimodal models (LMMs) and uncover a notable trend towards integrating these modalities.
- 12
- 12
- We Can Explain Your Research in Layman's Terms: Towards Automating Science Journalism at Scale12
A specialized dataset that contains scientific papers and their Science Daily press releases is created and numerous sequence to sequence (seq2seq) applications using Science Daily are demonstrated with the aim of facilitating further research on language generation.
- Fake News, Disinformation, Propaganda, Media Bias, and Flattening the Curve of the COVID-19 Infodemic12
An overview of the emerging and inter-connected research areas of fact-checking, misinformation, disinformation, "fake news'', propaganda, and media bias detection is offered, with focus on text and computational approaches.
- Semantic Sentiment Analysis of Twitter Data12
An overview of work on sentiment analysis on Twitter is presented, with Twitter being especially popular for research due to its scale, representativeness, variety of topics discussed, as well as ease of public access to its messages.
- Combining Relational and Attributional Similarity for Semantic Relation Classification12
This work combines relational and attributional similarity for the task of identifying instances of semantic relations, such as PRODUCT-PRODUCER and ORIGINENTITY, between nominals in text by mining the Web for patterns expressing the relationship between the relation arguments.
- Latent Semantic Analysis for German Literature Investigation12
The paper presents the results of experiments of usage of LSA for analysis of textual data and tests two hypotheses: 1) the texts by the same author are alike and can be distinguished from the ones by different person; 2) the prose and poetry can be automatically discovered.
- Llama-3-Nanda-10B-Chat: An Open Generative Large Language Model for Hindi11
The introduction of Llama-3-Nanda-10B-Chat, a state-of-the-art Hindi-centric instruction-tuned generative LLM, designed to push the boundaries of open-source Hindi language models, and an in-depth discussion of training strategies, fine-tuning techniques, safety alignment, and evaluation metrics.
- 11
- Unconditional Truthfulness: Learning Unconditional Uncertainty of Large Language Models11
This work trains a regression model that leverages LLM attention maps, probabilities on the current generation step, and recurrently computed uncertainty scores from previously generated tokens to learn this dependency from attention-based features.
- FAID: Fine-Grained AI-Generated Text Detection Using Multi-Task Auxiliary and Multi-Level Contrastive Learning11
This work collects a multilingual, multi-domain, multi-generator dataset FAIDSet and introduces a fine-grained detection framework FAID to classify text into these three categories, and also to identify the underlying LLM family of the generator.
- 11
- Target-Based Offensive Language Identification11
This work uses the TBO taxonomy to annotate post-level and token-level offensive language on English Twitter posts and carries out multiple experiments to compare the performance of different models trained and tested on TBO.
- 11
- On the Impact of Seed Words on Sentiment Polarity Lexicon Induction.11
This work combines mid-sized high-quality manually crafted lexicons as seeds and bootstrapping in order to build large-scale lexicons, demonstrating that much higher-quality lexicons can be built by starting with hundreds of words and phrases as seeds, especially when they are in-domain.
- 11
- 10
- Post-ocr text correction for Bulgarian historical documents10
This work creates the first benchmark dataset for evaluating the OCR text correction for historical Bulgarian documents written in the first standardized Bulgarian orthography: the Drinov orthography from the 19th century, and develops a method for automatically generating synthetic data in this orthography by leveraging vast amounts of contemporary literature Bulgarian texts.
- A Fano-Style Accuracy Upper Bound for LLM Single-Pass Reasoning in Multi-Hop QA10
This work establishes a Fano-style accuracy upper bound, defining a theoretical performance ceiling for single-pass LLMs and introduces a proof-of-concept multi-call framework for MHQA, InfoQA, which ensures high per-step accuracy by combining capacity-aware task decomposition with active pruning of prior reasoning traces, keeping the information load within the single-pass limit.
- Unleashing the Power of Discourse-Enhanced Transformers for Propaganda Detection10
This work presents the first attempt at using discourse analysis for the propaganda detection task, and considers both paragraph-level and token-level classification and a discourse-aware Transformer architecture.
- 10
- 10
- 10
- Building Chatbots from Forum Data: Model Selection Using Question Answering Metrics10
This work uses question answering (QA) data from Web forums to train chat-bots from scratch, i.e., without dialog data, and proposes to use extrinsic evaluation with respect to a QA task as an automatic evaluation method for chatbot systems.
- SU@PAN'2016: Author Obfuscation.10
This paper presents the approach for hiding an author’s identity by masking their style, which was developed for the Author Obfuscation task, part of the PAN-2016 competition.
- 10
- Improved Word Alignments Using the Web as a Corpus10
A novel method for improving word alignments in a parallel sentence-aligned bilingual corpus based on the idea that if two words are translations of each other then so should be many words in their local contexts is proposed.
- 9
- Arabic community question answering9
The most frequent errors common to all approaches are analyzed, categorizing them into prototypical cases, and zooming into the way syntactic information in tree kernel approaches can help solve some of the most difficult cases.
- DiscoMT 2015 Shared Task on Pronoun Translation9
The data set includes training, development and test data from the shared tasks on pronoun-focused machine translation and cross-lingual pronoun prediction from the EMNLP 2015 workshop on Discourse in Machine Translation (DiscoMT2015).
- SU-FMI: System Description for SemEval-2014 Task 9 on Sentiment Analysis in Twitter9
The submission of the team of the Sofia University to SemEval-2014 Task 9 on Sentiment Analysis in Twitter is described, and their system is ranked 20th out of 50 submissions based on the average of the three 2014 evaluation data scores.
- Semantic Kernels for Semantic Parsing9
An empirical study on the use of semantic information for Concept Segmentation and Labeling (CSL) and the results show that semantic kernels exploiting similarity measures outperform state-of-the-art rerankers.
- BioText Team Experiments for the TREC 2004 Genomics Track.9
The BioText group participated in the two main tasks of the TREC 2004 Genomics track, and their approach to the ad hoc task was similar to the one used in the 2003 GenomicsTrack, but due to the lack of training data, they did not achieve the high scores of the previous year.
- Towards deeper understanding of the latent semantic analysis performance9
The results show that while the linguistic processing influences the performance, the traditional factors are more important.
- Multi-Sourced, Multi-Agent Evidence Retrieval for Fact-Checking8
A Web-enhanced Knowledge Graph retrieval Fact-Checking agentic framework (WKGFC), which exploits authorized open knowledge graph as a core resource of evidence and highlights the effectiveness of knowledge-centric evidence retrieval for fact-checking under open-world settings.
- 8
- DP-Fusion: Token-Level Differentially Private Inference for Large Language Models8
DP-Fusion is proposed, a Differentially Private Inference mechanism for LLMs that provably bounds the influence a set of tokens in the context can have on the LLM's output, and creates token-level provably privatized documents with substantially improved theoretical and empirical privacy.
- Atomic Reasoning for Scientific Table Claim Verification8
This work proposes that enhancing a model's ability to interpret table-based claims involves reducing cognitive load by developing modular, reusable reasoning components (i.e., atomic skills), and introduces a skill-chaining schema that dynamically composes these skills to facilitate more accurate and generalizable reasoning with a reduced cognitive load.
- FRAPPE: FRAming, Persuasion, and Propaganda Explorer8
This paper presents FRAPPE, a FRAming, Persuasion, and Propaganda Explorer system that goes beyond conventional news analysis of articles and unveils the intricate linguistic techniques used to shape readers’ opinions and emotions.
- Exploring Language Model Generalization in Low-Resource Extractive QA8
This paper investigates Extractive Question Answering (EQA) with Large Language Models (LLMs) under domain drift, i.e., can LLMs generalize to domains that require specific knowledge in a zero-shot fashion without additional in-domain training?
- 8
- Enhancing Arabic Content Generation with Prompt Augmentation Using Integrated GPT and Text-to-Image Models8
A simple, novel approach for prompt engineering that uses the domain knowledge of a state-of-the-art language model, GPT, to perform the task of prompt augmentation, where a simple, initial prompt is used to generate multiple, more detailed prompts related to the Arabic culture through a GPT model through a process known as in-context learning.
- 8
- Batch-Softmax Contrastive Loss for Pairwise Sentence Scoring Tasks8
This work explores the idea of using a batch-softmax contrastive loss when fine-tuning large-scale pre-trained transformer models to learn better task-specific sentence embeddings for pairwise sentence scoring tasks.
- Generating Answer Candidates for Quizzes and Answer-Aware Question Generators8
This work proposes a model that can generate a specified number of answer candidates for a given passage of text, which can then be used by instructors to write questions manually or can be passed as an input to automatic answer-aware question generators.
- A Morpho-Syntactically Informed LSTM-CRF Model for Named Entity Recognition8
It is shown that for morphologically rich languages, such as Bulgarian, access to POS information contributes more to the performance gains than the detailed morphological information, and can benefit from simultaneously using some POS information of different granularity.
- Recursive Style Breach Detection with Multifaceted Ensemble Learning8
This work presents a supervised approach for style change detection, which aims at predicting whether there are changes in the style in a given text document, as well as at finding the exact positions where such changes occur.
- Large-Scale Goodness Polarity Lexicons for Community Question Answering8
A key idea from the field of sentiment analysis is transferred to a new domain: community question answering (cQA), and pointwise mutual information is used in order to build large-scale goodness polarity lexicons in a semi-supervised manner.
- Analyzing Optimization for Statistical Machine Translation: MERT Learns Verbosity, PRO Learns Length8
It is argued for the need of controlled evaluation scenarios, so that the selection of tuning set and optimization strategy does not overshadow scientific advances in modeling or decoding.
- UCB system description for the WMT 2007 shared task8
For the WMT 2007 shared task, the UC Berkeley team employed monolingual syntactic paraphrases to provide syntactic variety to the source training set sentences and made use of results from prior research that shows that cognate pairs can improve word alignments.
- 7
- Clock against Chaos: Dynamic Assessment and Temporal Intervention in Reducing Misinformation Propagation7
This work proposes a dynamic system for real-time evaluation and assignment of misinformation scores to tweets, which can support the ongoing efforts to counteract the impact of misinformation public health, public opinion, and society.
- 7
- 7
- Bilingual Adaptation of Monolingual Foundation Models7
This study focuses on adapting Llama 2 to Arabic, and presents an efficient method for adapting a monolingual Large Language Model (LLM) to another language, addressing challenges of catastrophic forgetting and tokenizer limitations.
- 7
- Gpachov at CheckThat! 2023: A Diverse Multi-Approach Ensemble for Subjectivity Detection in News Articles7
The solution built by the Gpachov team for the CLEF-2023 CheckThat! lab Task~2 on subjectivity detection is presented, resulting in 0.77 macro F1 on the test set and achieving 2nd place on the English subtask.
- 7
- A Neighborhood Framework for Resource-Lean Content Flagging7
A novel framework for cross- lingual content flagging with limited target- language data, which significantly outperforms prior work in terms of predictive performance and which can adapt to new source- language instances, without the need to be retrained from scratch.
- 7
- IITD at the WANLP 2022 Shared Task: Multilingual Multi-Granularity Network for Propaganda Detection7
The experimental results and analysis show that it does not help to use a much larger English corpus annotated with propaganda techniques, regardless of whether used in English or after translation to Arabic.
- Dialect Variation on Social Media7
Language in social media is often closer to everyday speech than the language typically found in many other data sources, such as newspapers, and is therefore a rich resource to study regional and social variation in language.
- 7
- 7
- Experiments in Authorship-Link Ranking and Complete Author Clustering.7
This paper uses a combination of classification and agglomerative clustering with a rich set of features such as average sentence length, function words ratio, type-token ratio and part of speech tags for the AuthorshipLink Ranking and Complete Author Clustering task.
- 7
- NUS at WMT097
The system developed by the team of the National University of Singapore for English to Spanish machine translation of News Commentary text for the WMT09 Shared Translation Task is described, based on domain adaptation, combining a small in-domain News Commentary bi-text and a large out-of-domain one from the Europarl corpus.
- From Generation to Detection: A Multimodal Multi-Task Dataset for Benchmarking Health Misinformation6
MM Health, a large scale multimodal misinformation dataset in the health domain consisting of 34,746 news article encompassing both textual and visual information, aims to support the development of misinformation detection across various health scenarios, facilitating the detection of human and machine generated content at multimodal levels.
- 6
- Understanding Breast Cancer Survival: Using Causality and Language Models on Multi-omics Data6
This paper exploits suitable causal discovery algorithms to investigate how various perturbations in the genome can affect the survival of patients diagnosed with breast cancer, and proposes to address the evaluation problem with large language models.
- Fact-Checking, Fake News, Propaganda, Media Bias, and the COVID-19 Infodemic6
This work offers an overview of the emerging and inter-connected research areas of fact-checking, disinformation, "fake news'', propaganda, and media bias detection, and the ongoing COVID-19 infodemic.
- 6
- Team Jack Ryder at SemEval-2019 Task 4: Using BERT Representations for Detecting Hyperpartisan News6
An approach based on BERT with fine-tuning is proposed, which was ranked 7th out 28 teams on the distantly supervised dataset, where all articles from a hyperpartisan/non-hyperpartisan news outlet are considered to be hyper partisan/ non-hyper partisan.
- Enabling Medical Translation for Low-Resource Languages6
The first steps towards the development of a real-world Hindi-English machine translation system for doctor-patient communication and a method for automatically augmenting the training data with synthetically generated variants are presented.
- Sentiment Analysis in Twitter: A SemEval Perspective6
The recent rise of social media has greatly democratized content creation, with Twitter being especially popular for research due to its scale, representativeness, variety of topics discussed, as well as ease of public access to its messages.
- SU@PAN'2015: Experiments in author profiling6
The submission of the Soa University team for the Author Proling Task, part of the PAN 2015 Challenge, is described, which asks to predict some demographical information such as age and gender, as well as the per- sonality type of that person.
- Proceedings of the COLING Workshop on Multiword Expressions: from Theory to Applications (MWE 2010)6
There is a wide range of open problems that prevent MWE treatment techniques from full integration in current NLP systems, and the workshop asked for original research related to the following topics: MWE resources, hybrid approaches, domain adaptation, and multilingualism.
- Cross-lingual Adaptation as a Baseline: Adapting Maximum Entropy Models to Bulgarian6
Five basic natural language processing components were originally developed for English within OpenNLP, an open source maximum entropy based machine learning toolkit, and were retrained based on manually annotated training data from the BulTreeBank.
- 5
- Qorgau: Evaluating LLM Safety in Kazakh-Russian Bilingual Contexts5
Experiments with both multilingual and language-specific LLMs reveal notable differences in safety performance, emphasizing the need for tailored, region-specific datasets to ensure the responsible and safe deployment of LLMs in countries like Kazakhstan.
- 5
- 5
- UrduFactCheck: An Agentic Fact-Checking Framework for Urdu with Evidence Boosting and Benchmarking5
UrduFactBench and UrduFactQA are presented, two novel hand-annotated benchmarks designed to enable fact-checking and factual consistency evaluation in Urdu and UrduFactCheck is introduced, a modular fact-checking framework that incorporates both monolingual and translation-based evidence retrieval strategies to mitigate the scarcity of high-quality Urdu evidence.
- 5
- 5
- 5
- 5
- 5
- QCRI's COVID-19 Disinformation Detector: A System to Fight the COVID-19 Infodemic in Social Media5
This study focuses on a multifaceted approach, including an API, which was made freely and publicly available, and a demo system, which it believes will facilitate researchers and different stakeholders.
- 5
- Predicting the Factuality of Reporting of News Media Using Observations About User Attention in Their YouTube Channels5
A novel framework for predicting the factuality of reporting of news media outlets by studying the user attention cycles in their YouTube channels, which designs a rich set of features derived from the temporal evolution of the number of views, likes, dislikes, and comments for a video, which are aggregated to the channel level.
- 5
- 5
- SIGIR 2016 Workshop WebQA II5
The aim of this workshop is to bring together researchers in diverse areas working on this problem, including those from NLP, IR, social media and recommender systems communities, to conduct a more focused and open discussion.
- An interactive system for exploring community question answering forums5
An interactive system to provide effective and efficient search capabilities in Community Question Answering (cQA) forums with state-of-the-art technology for answer search with a Web-based user interface specifically tailored to support the cQA forum readers.
- A Study of using Syntactic and Semantic Structures for Concept Segmentation and Labeling5
An empirical study on using syntactic and semantic information for Concept Segmentation and Labeling (CSL), a well-known component in spoken language understanding, shows that the structured representation with the semantic concepts yields significant improvement over the base CSL parser.
- Predicting Dialect Variation in Immigrant Contexts Using Light Verb Constructions5
This study distinguishes between the immigrant and the standard dialect of Turkish by focusing on Light Verb Constructions, and experiment with a number of grammatical and contextual features, achieving over 84% accuracy.
- QCRI at WMT12: Experiments in Spanish-English and German-English Machine Translation of News Text5
This work used a phrase-based statistical machine translation model with several non-standard settings, most notably tuning data selection and phrase table combination to rank second in BLEU and TER for Spanish-English, and in the top tier for German-English.
- Towards deeper understanding and personalisation in CALL5
It is thought that much closer cooperation with real teaching experts is necessary, to find the proper learning niches and suitable wrappings of the language technologies, which could give birth to useful eLearning solutions.
- Web Personalization Using Extended Boolean Operations with Latent Semantic Indexing5
The paper discusses the potential of the usage of Extended Boolean operations for personalized information delivery on the Internet based on semantic vector representation models and presents the design of an e-commerce portal tracking user's clickstream activity and purchases history in order to offer them personalized information.
- Sherkala-Chat: Building a State-of-the-Art LLM for Kazakh in a Moderately Resourced Setting4
Sherkala-Chat (8B) is released as an open-weight instruction-tuned model as an open-weight instruction-tuned model and a detailed overview of its training, fine-tuning, safety alignment is provided.
- Don't Throw Away Your Beams: Improving Consistency-based Uncertainties in LLMs via Beam Search4
This work introduces a new family of methods that employ beam search to generate candidates for consistency-based UQ, yielding improved performance and reduced variance compared to multinomial sampling and provides a theoretical lower bound on the beam set probability mass under which beam search achieves a smaller error than multinomial sampling.
- 4
- 4
- 4
- 4
- 4
- 4
- 4
- 4
- 4
- 4
- 4
- The NUS Statistical Machine Translation System for IWSLT 20094
The system developed by the team of the National University of Singapore for the Chinese-English BTEC task of the IWSLT 2009 evaluation campaign adopted a state-of-the-art phrase-based statistical machine translation approach and focused on experiments with different Chinese word segmentation standards.
- Tunable domain-independent event extraction in the MIRA framework4
The system of the PIKB team for BioNLP'09 Shared Task 1 targets tunable domain-independent event extraction based on a three-stage classification based on the MIRA framework, which allows for trade precision for increased recall by appropriately changing the loss function during training.
- Investigating the Degree of Adequacy of the Relations in the Concept Structure of Students using the Method of Latent Semantic Analysis4
The problem has been solved by the use of the latent semantic analysis for comparison and assessment of scientific texts and knowledge, expressed by the students in the form of free verbal statements.
- 3
- 3
- 3
- 3
- 3
- 3
- 3
- Team TheSyllogist at SemEval-2023 Task 3: Language-Agnostic Framing Detection in Multi-Lingual Online News: A Zero-Shot Transfer Approach3
The results demonstrate the feasibility of creating a language-agnostic model for automatic framing detection in online news, and propose a multi-lingual approach based on machine translation of the input, followed by an English prediction model.
- 3
- 3
- Learning Through Interpolative Augmentation of Dynamic Curvature Spaces3
DynaMix is proposed, a framework that automatically selects an example-specific geometry and performs Mixup between the different geometries to improve training dynamics and generalization and is found more useful in low-resource and semi-supervised settings likely because it displays a probabilistic view of the geometry.
- 3
- The Web as an Implicit Training Set: Application to Noun Compounds Syntax and Semantics3
The problem of noun compounds syntax is addressed by means of novel, highly accurate unsupervised and lightly supervised algorithms using the Web as a corpus and search engines as interfaces to that corpus and automatically generating paraphrasing verbs and prepositions that make explicit the hidden semantic relations between the nouns in a noun compound.
- On the semantics of noun compounds3
The noun compound – a sequence of nouns which functions as a single noun – is very common in English texts and no language processing system should ignore expressions like steel soup pot cover if it wants to be serious about high-end applications of computational linguistics.
- Building a Named Entity Recognizer in Three Days: Application to Disease Name Recognition in Bulgarian Epicrises3
It is shown that using a general framework such as GATE and an appropriate pragmatic methodology can yield significant speed up of the manual annotation of disease names in Bulgarian clinical epicrises: F1=0.81 in just three days.
- EXTENDED BOOLEAN OPERATIONS IN LATENT SEMANTIC INDEXING SEARCH3
The proposed method adds a new functionality to the classic LSI method capabilities to process user queries typed in a natural language used in the “intelligent” search engines.
- 2
- 2
- 2
- 2
- Explicit and Implicit Data Augmentation for Social Event Detection2
Augmentation framework for Social Event Detection (SED-Aug), a plug-and-play dual augmentation framework, which combines explicit text-based and implicit feature-space augmentation to enhance data diversity and model robustness.
- ImageCLEF 2025: Multimedia Retrieval in Medical, Social Media and Content Recommendation Applications2
In its 23rd edition, ImageCLEF will provide access to large amounts of challenging data in very diverse use cases like medicine, argumentation, reasoning, generation, or content recommendation.
- 2
- An Analytical Emotion Framework of Rumour Threads on Social Media2
This work provides a comprehensive analytical emotion framework, contrasting rumour and non-rumour cases using existing NLP datasets to further understand the emotion dynamics within rumours.
- 2
- 2
- 2
- Learning To Sample the Meta-Paths for Social Event Detection2
This work proposes a simple, yet effective, end-to-end Learning To Sample (LTS) framework for meta-path searching, and proposes an extra non-parametric evaluation process in order to determine the importance of each meta-path, which can further guide the paths sampling during model training.
- 2
- BERTastic at SemEval-2023 Task 3: Fine-Tuning Pretrained Multilingual Transformers Does Order Matter?2
The proposed method for fine-tuning Pretrained Multilingual Transformers (PMTs) on multilingual data outperforms the naïve approach and is the first to observe an interesting pattern in which deep learning models exhibit a human-like primacy-recency effect.
- Can You Answer This? – Exploring Zero-Shot QA Generalization Capabilities in Large Language Models (Student Abstract)2
The objective is to examine performance change in the face of domain drift i.e. when the target domain data is vastly different in semantic and statistical properties from the source domain and attempt to explain the subsequent behavior.
- Half-Day Tutorial on Combating Online Hate Speech: The Role of Content, Networks, Psychology, User Behavior, etc.2
A holistic view of what the research community has explored so far in hate speech, and what it believes are promising future research directions is provided.
- 2
- 2
- 2
- 2
- 2
- 2
- Vector-Vector-Matrix Architecture: A Novel Hardware-Aware Framework for Low-Latency Inference in NLP Applications2
A novel vector-vector-matrix architecture (VVMA) is proposed, which greatly reduces the latency at inference time for NMT and reduces the number of parameters and FLOPs for virtually all models that rely on efficient matrix multipliers without significantly impacting accuracy.
- 2
- 2
- 2
- Bi-text Alignment of Movie Subtitles for Spoken English-Arabic Statistical Machine Translation2
This work creates a much larger bi-text (the biggest to date), and generates better quality alignment for movie subtitles, and develops an original algorithm that outperforms the current best subtitle alignment tool, subalign.
- 2
- 2
- 2
- SU@PAN'2015: Experiments in author verification2
This work describes the submission of the Soa University team for the Author Identication Task, part of the PAN 2015 Challenge, and experiments with an SVM classier using variety of features extracted from publicly available resources.
- Web as a Corpus: Going Beyond the n-gram2
The 60-year-old dream of computational linguistics is to make computers capable of communicating with humans in natural language by shifting main research attention to machine learning from text corpora, thus triggering a revolution in the field.
- The architecture of corporate information and news engine2
The architecture of a corporate information and news engine is described, providing an effective bass for analysis, selection, navigation, and presentation of large sets of dynamic corporate contents.
- Ending-Guessing Rules for Morphological Classification of German Nouns2
MorphoClass exploits global information (ending-guessing rules, maximum likelihood estimations, word frequency statistics), morphological properties (compounding, inflection, affixes) and external knowledge (lexicons, German grammar information etc.).
- MorphoClass - Recognition and Morphological Classification of Unknown Words for German.2
A system for recognition and morphological classification of unknown words for German that takes raw text as input and outputs a list of the unknown nouns together with a hypothesis about their possible morphological class and stem.
- 1
- SQLStructEval: Structural Evaluation of LLM Text-to-SQL Generation1
SQLStructEval is introduced, a framework that analyzes this behavior through canonical abstract syntax tree representations and adopts a pipeline that first generates structured intermediate representations and then deterministically compiles them into SQL, improving execution accuracy and structural agreement among correct outputs.
- 1
- 1
- 1
- 1
- 1
- 1
- 1
- BLAZER: Bootstrapping LLM-based Manipulation Agents with Zero-Shot Data Generation1
This work proposes BLAZER, a framework that learns manipulation policies from automatically generated training data and shows BLAZER to significantly improve zero-shot manipulation in both simulated and real environments.
- 1
- ConspirED: A Dataset for Cognitive Traits of Conspiracy Theories and Large Language Model Safety1
It is found that LLMs are readily misaligned by conspiratorial framing, reproducing its rhetorical patterns even when successfully deflecting comparable fact-checked misinformation.
- 1
- HALF: Harm-Aware LLM Fairness Evaluation Aligned with Deployment1
Harm-Aware LLM Fairness is introduced, a deployment-aligned framework that assesses model bias in realistic applications and weighs the outcomes by harm severity, and concludes that HALF exposes a clear gap between previous benchmarking success and deployment readiness.
- 1
- 1
- 1
- 1
- 1
- Towards Constructing a Corpus for Studying the Effects of Treatments and Substances Reported in PubMed Abstracts1
It turns out that recognizing the terminology and the abbreviations is key for determining the rationale sentence in the corpus and the ultimate goal is to annotate one sentence (rationale) for each abstract.
- 1
- Robust Tuning Datasets for Statistical Machine Translation1
This paper achieves the idea of automatically crafting a tuning dataset for Statistical Machine Translation that makes the hyper-parameters of the SMT system more robust with respect to some specific deficiencies of the parameter tuning algorithms by selecting a subset of the available sentence pairs, which are more suitable for specific combinations of optimizers, objective functions, and evaluation measures.
- 1
- 1
- A Knowledge-Rich Approach to Measuring the Similarity between Bulgarian and Russian Words1
A novel knowledge-rich approach to measuring the similarity between a pair of words that combines lemmatization, hand-crafted transformation rules, and weighted Levenshtein distance is proposed.
- Robust ending guessing rules with application to Slavonic languages1
A simple exact high accuracy algorithm is developed and compared to an approximate one, which uses a scoring function previously proposed by Mikheev for POS guessing, and it is shown how the number of rules of the latter can be reduced by a factor of up to 35, without sacrificing performance.
- –
- –
- Detecting AI-Generated Bulgarian Text: A Two-Step Multi-Class Classification Approach–
A novel two-step multi-class classification system to identify varying degrees of machine involvement in Bulgarian text improves the reliable identification of human authorship and enhances the overall accuracy of multi-class text detection compared to single-step methods.
- –
- –
- –
- –
- How Does Prefix Matter in Reasoning Model Tuning?–
This work fine-tune three R1 series models and suggests that prefix conditioning offers a scalable and interpretable mechanism for improving reasoning safety, serving as an implicit form of alignment that complements traditional reward-based methods.
- –
- –
- –
- FRaN-X: FRaming and Narratives-eXplorer–
FRaN-X comprises a two-stage system that combines sequence labeling with fine-grained role classification to reveal how entities are portrayed as protagonists, antagonists, or innocents, using a unique taxonomy of 22 fine-grained roles nested under these three main categories.
- –
- –
- –
- –
- –
- MuDRiC: Multi-Dialect Reasoning for Arabic Commonsense Validation–
This work introduces MuDRiC, an extended Arabic commonsense dataset incorporating multiple dialects, and proposes a novel method adapting Graph Convolutional Networks to Arabic commonsense reasoning, which enhances semantic relationship modeling for improved commonsense validation.
- –
- –
- –
- –
- –
- BERTastic at SemEval-2024 Task 4: State-of-the-Art Multilingual Propaganda Detection in Memes via Zero-Shot Learning with Vision-Language Models–
This paper uses a VLM in a zero-shot approach to detect propagandistic memes and achieves a state-of-the-art average macro F1 of 66.7% over all languages.
- –
- –
- –
- Preface: Special issue on NLP approaches to offensive content online–
The special issue on NLP Approaches to Offensive Content Online published in the Journal of Natural Language Engineering issue 29.6 proves the interest of the NLP community in this topic.
- Accepted Tutorials at The Web Conference 2022–
This paper summarizes the content of the 20 tutorials that have been given at The Web Conference 2022: 85% of these tutorials are lecture style, and 15% are hands on.
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- –
- Flattening the Curve of the COVID-19 Infodemic: These Evaluation Campaigns Can Help!–
It is argued for the need of a holistic approach combining the perspectives of journalists, fact-checkers, policymakers, social media platforms, and society as a whole, and an initial work in this direction is presented.
- –
- –
- –
- –
- Preface–
H2S performs numerous activities in these organs, including neuromodulation, vascular relaxation, angiogenesis, energy formation in mitochondria, and protection against oxidative stress and ischemia–reperfusion injury.
- –
- –
- –
- –
- Evaluating Variable-Length Multiple-Option Lists in Chatbots and Mobile Search–
Some necessary and some optional properties that an evaluation measure fit for this purpose should have are defined, and it is shown that existing evaluation measures from the IR tradition are not entirely suitable for this setup, and novel evaluation measures that address it satisfactorily are proposed.
- –
- –
- Learning Semantic Relations from Text–
The theoretical exploration sketches the historical path which has brought us to the contemporary view and interpretation of semantic relations, and a wide range of relation inventories proposed by linguists and by language processing people are discussed.
- Utilizing Monolingual Gulf Arabic Data For Qatari Arabic-english Statistical Machine Translation–
A standard segmentation scheme can improve vocabulary overlap between dialects by segmenting words with different morphological forms in different dialects to a common root form and it is found that removing the monolingual data from the training set of the segmenter affects the translation quality.
- –
- Combining, Adapting and Reusing Bi-texts between Related Languages: Application to Statistical Machine Translation (invited talk)–
This work builds a lattice of adaptation options for each word and phrase, and then decode it using a language model for the resource-poor language, and demonstrates the potential of combining word and character-level models.
- –
- –
- –
- –
- Reusing Parallel Corpora between Related Languages–
The aim is to tap the potential of the EU resources so that they can be used by other non-EU languages that are closely related to one or more official languages of the European Union.
Publication data from OpenAlex, with missing venues and authors filled in from Crossref; citation counts are the higher of OpenAlex and Semantic Scholar, last synced 2026-10-14. One-sentence summaries under some papers are written by Semantic Scholar’s model. Citation counts may be lower than on Google Scholar, which indexes more sources.
Report an error
Wrong papers, two people merged into one, or a profile that should not be here? Tell us and we will fix or hide it.