Open Access. Powered by Scholars. Published by Universities.®
- Keyword
-
- Computational linguistics (8)
- Automatic speech recognition (4)
- Natural Language Processing (4)
- Machine learning (3)
- Performance (3)
-
- Veracity (3)
- Arabic dialects (2)
- Artificial intelligence (2)
- COVID-19 (2)
- Check-Worthiness Estimation (2)
- Computation and Language (cs.CL) (2)
- Computational Journalism (2)
- Fact-Checking (2)
- Language model (2)
- Misinformation (2)
- Natural language processing (2)
- News articles (2)
- Online news (2)
- Pre-training (2)
- Social Media Verification (2)
- Speech recognition (2)
- Speech technology (2)
- State of the art (2)
- Text classification (2)
- Acoustic features (1)
- Acoustic word embedding (1)
- Adapter (1)
- Alzheimer's disease (1)
- Annotation methods (1)
- Antimicrobial Resistance (1)
Articles 1 - 30 of 45
Full-Text Articles in Computer Sciences
Considering The Impact Framework To Understand The Ai-Well-Being-Complex From An Interdisciplinary Perspective, Christian Montag, Preslav Nakov, Raian Ali
Considering The Impact Framework To Understand The Ai-Well-Being-Complex From An Interdisciplinary Perspective, Christian Montag, Preslav Nakov, Raian Ali
Natural Language Processing Faculty Publications
Artificial intelligence (AI) is built into many products and has the potential to dramatically impact societies around the world. This short theoretical paper aims to provide a simple framework that might help us understand how the introduction and/or use of products with AI might influence the well-being of humans. It is proposed that considering the dynamic Interplay between variables stemming from Modality, Person, Area, Culture and Transparency categories will help to understand the influence of AI on well-being. The Modality category encompasses areas such as the degree of AI being interactive, informational versus actualizing, or autonomous. The Person variable contains …
Why Do We Not Stand Up To Misinformation? Factors Influencing The Likelihood Of Challenging Misinformation On Social Media And The Role Of Demographics, Selin Gurgun, Deniz Cemiloglu, Emily Arden Close, Keith Phalp, Preslav Nakov, Raian Ali
Why Do We Not Stand Up To Misinformation? Factors Influencing The Likelihood Of Challenging Misinformation On Social Media And The Role Of Demographics, Selin Gurgun, Deniz Cemiloglu, Emily Arden Close, Keith Phalp, Preslav Nakov, Raian Ali
Natural Language Processing Faculty Publications
This study investigates the barriers to challenging others who post misinformation on social media platforms. We conducted a survey amongst U.K. Facebook users (143 (57.2 %) women, 104 (41.6 %) men) to assess the extent to which the barriers to correcting others, as identified in literature across disciplines, apply to correcting misinformation on social media. We also group the barriers into factors and explore demographic differences amongst them. It has been suggested that users are generally hesitant to challenge misinformation. We found that most of our participants (58.8 %) were reluctant to challenge misinformation. We also identified moderating roles of …
Using Natural Language Processing And Patient Journey Clustering For Temporal Phenotyping Of Antimicrobial Therapies For Cat Bite Abscesses, Brian Hur, Karin M. Verspoor, Timothy Baldwin, Laura Y. Hardefeldt, Caitlin Pfeiffer, Caroline Mansfield, Riati Scarborough, James R. Gilkerson
Using Natural Language Processing And Patient Journey Clustering For Temporal Phenotyping Of Antimicrobial Therapies For Cat Bite Abscesses, Brian Hur, Karin M. Verspoor, Timothy Baldwin, Laura Y. Hardefeldt, Caitlin Pfeiffer, Caroline Mansfield, Riati Scarborough, James R. Gilkerson
Natural Language Processing Faculty Publications
Background: Temporal phenotyping of patient journeys, which capture the common sequence patterns of interventions in the treatment of a specific condition, is useful to support understanding of antimicrobial usage in veterinary patients. Identifying and describing these phenotypes can inform antimicrobial stewardship programs designed to fight antimicrobial resistance, a major health crisis affecting both humans and animals, in which veterinarians have an important role to play. Objective: This research proposes a framework for extracting temporal phenotypes of patient journeys from clinical practice data through the application of natural language processing (NLP) and unsupervised machine learning (ML) techniques, using cat bite abscesses …
Challenging Others When Posting Misinformation: A Uk Vs. Arab Cross-Cultural Comparison On The Perception Of Negative Consequences And Injunctive Norms, Muaadh Noman, Selin Gurgun, Keith Phalp, Preslav Nakov, Raian Ali
Challenging Others When Posting Misinformation: A Uk Vs. Arab Cross-Cultural Comparison On The Perception Of Negative Consequences And Injunctive Norms, Muaadh Noman, Selin Gurgun, Keith Phalp, Preslav Nakov, Raian Ali
Natural Language Processing Faculty Publications
This study investigates the factors influencing the willingness to challenge misinformation on social media across two cultural contexts, the United Kingdom (UK) and Arab countries. A total of 462 participants completed an online survey (250 UK, 212 Arabs). The analysis revealed that three types of negative consequences (relationship cost, negative impact on the person being challenged, futility) and also injunctive norms influence the willingness to challenge misinformation. Cross-cultural comparisons using t-tests showed significant differences between the UK and the Arab countries in all factors except the injunctive norms. Multiple regression analyses identified differences between the UK and Arab participants concerning …
Offenseval 2023: Offensive Language Identification In The Age Of Large Language Models, Marcos Zampieri, Sara Rosenthal, Preslav Nakov, Alphaeus Dmonte, Tharindu Ranasinghe
Offenseval 2023: Offensive Language Identification In The Age Of Large Language Models, Marcos Zampieri, Sara Rosenthal, Preslav Nakov, Alphaeus Dmonte, Tharindu Ranasinghe
Natural Language Processing Faculty Publications
The OffensEval shared tasks organized as part of SemEval-2019-2020 were very popular, attracting over 1300 participating teams. The two editions of the shared task helped advance the state of the art in offensive language identification by providing the community with benchmark datasets in Arabic, Danish, English, Greek, and Turkish. The datasets were annotated using the OLID hierarchical taxonomy, which since then has become the de facto standard in general offensive language identification research and was widely used beyond OffensEval. We present a survey of OffensEval and related competitions, and we discuss the main lessons learned. We further evaluate the performance …
Preface: Special Issue On Nlp Approaches To Offensive Content Online, Marcos Zampieri, Isabelle Augenstein, Siddharth Krishnan, Joshua Melton, Preslav Nakov
Preface: Special Issue On Nlp Approaches To Offensive Content Online, Marcos Zampieri, Isabelle Augenstein, Siddharth Krishnan, Joshua Melton, Preslav Nakov
Natural Language Processing Faculty Publications
No abstract provided.
Artst: Arabic Text And Speech Transformer, Hawau Olamide Toyin, Amirbek Djanibekov, Ajinkya Kulkarni, Hanan Al Darmaki
Artst: Arabic Text And Speech Transformer, Hawau Olamide Toyin, Amirbek Djanibekov, Ajinkya Kulkarni, Hanan Al Darmaki
Natural Language Processing Faculty Publications
We present ArTST, a pre-trained Arabic text and speech transformer for supporting open-source speech technologies for the Arabic language. The model architecture follows the unified-modal framework, SpeechT5, that was recently released for English, and is focused on Modern Standard Arabic (MSA), with plans to extend the model for dialectal and code-switched Arabic in future editions. We pre-trained the model from scratch on MSA speech and text data, and fine-tuned it for the following tasks: Automatic Speech Recognition (ASR), Text-To-Speech synthesis (TTS), and spoken dialect identification. In our experiments comparing ArTST with SpeechT5, as well as with previously reported results in …
Text Augmentation For Semantic Frame Induction And Parsing, Saba Anwar, Artem Shelmanov, Nikolay Arefyev, Alexander Panchenko, Chris Biemann
Text Augmentation For Semantic Frame Induction And Parsing, Saba Anwar, Artem Shelmanov, Nikolay Arefyev, Alexander Panchenko, Chris Biemann
Natural Language Processing Faculty Publications
Semantic frames are formal structures describing situations, actions or events, e.g., Commerce buy, Kidnapping, or Exchange. Each frame provides a set of frame elements or semantic roles corresponding to participants of the situation and lexical units (LUs)—words and phrases that can evoke this particular frame in texts. For example, for the frame Kidnapping, two key roles are Perpetrator and the Victim, and this frame can be evoked with lexical units abduct, kidnap, or snatcher. While formally sound, the scarce availability of semantic frame resources and their limited lexical coverage hinders the wider adoption of frame semantics across languages and domains. …
Yet Another Model For Arabic Dialect Identification, Ajinkya Kulkarni, Hanan Al Darmaki
Yet Another Model For Arabic Dialect Identification, Ajinkya Kulkarni, Hanan Al Darmaki
Natural Language Processing Faculty Publications
In this paper, we describe a spoken Arabic dialect identification (ADI) model for Arabic that consistently outperforms previously published results on two benchmark datasets: ADI-5 and ADI-17. We explore two architectural variations: ResNet and ECAPA-TDNN, coupled with two types of acoustic features: MFCCs and features exratected from the pre-trained self-supervised model UniSpeech-SAT Large, as well as a fusion of all four variants. We find that individually, ECAPA-TDNN network outperforms ResNet, and models with UniSpeech-SAT features outperform models with MFCCs by a large margin. Furthermore, a fusion of all four variants consistently outperforms individual models. Our best models outperform previously reported …
Adapting The Adapters For Code-Switching In Multilingual Asr, Atharva Kulkarni, Ajinkya Kulkarni, Miguel Couceiro, Hanan Al Darmaki
Adapting The Adapters For Code-Switching In Multilingual Asr, Atharva Kulkarni, Ajinkya Kulkarni, Miguel Couceiro, Hanan Al Darmaki
Natural Language Processing Faculty Publications
Recently, large pre-trained multilingual speech models have shown potential in scaling Automatic Speech Recognition (ASR) to many low-resource languages. Some of these models employ language adapters in their formulation, which helps to improve monolingual performance and avoids some of the drawbacks of multi-lingual modeling on resource-rich languages. However, this formulation restricts the usability of these models on code-switched speech, where two languages are mixed together in the same utterance. In this work, we propose ways to effectively fine-tune such models on code-switched speech, by assimilating information from both language adapters at each language adaptation point in the network. We also …
Overview Of The Clef-2023 Checkthat! Lab Task 4 On Factuality Of Reporting Of News Media, Preslav Nakov, Firoj Alam, Giovanni Da San Martino, Maram Hasanain, Dilshod Azizov, Rabindra Nath Nandi, Panayotov Panayot
Overview Of The Clef-2023 Checkthat! Lab Task 4 On Factuality Of Reporting Of News Media, Preslav Nakov, Firoj Alam, Giovanni Da San Martino, Maram Hasanain, Dilshod Azizov, Rabindra Nath Nandi, Panayotov Panayot
Natural Language Processing Faculty Publications
We present an overview of the CLEF-2023 CheckThat! lab Task 4, which focused on predicting the factuality of reporting of entire news outlets. This is a different level of granularity compared to previous efforts, which focused on fact-checking, where the target is a claim, or fake news detection, where the target is an article. We briefly summarize the participating systems and discuss the dataset, the task, and the evaluation setup. The task attracted a large number of registrations, and eventually five teams made submissions. All participants improved over the baseline by a margin using both deep learning and traditional machine …
Disease Progression Modelling Of Alzheimer's Disease Using Probabilistic Principal Components Analysis, Martin Saint-Jalmes, Victor Fedyashov, Daniel Beck, Timothy Baldwin, Noel G. Faux, Pierrick Bourgeat, Jurgen Fripp, Colin L. Masters, Benjamin Goudey
Disease Progression Modelling Of Alzheimer's Disease Using Probabilistic Principal Components Analysis, Martin Saint-Jalmes, Victor Fedyashov, Daniel Beck, Timothy Baldwin, Noel G. Faux, Pierrick Bourgeat, Jurgen Fripp, Colin L. Masters, Benjamin Goudey
Natural Language Processing Faculty Publications
The recent biological redefinition of Alzheimer's Disease (AD) has spurred the development of statistical models that relate changes in biomarkers with neurodegeneration and worsening condition linked to AD. The ability to measure such changes may facilitate earlier diagnoses for affected individuals and help in monitoring the evolution of their condition. Amongst such statistical tools, disease progression models (DPMs) are quantitative, data-driven methods that specifically attempt to describe the temporal dynamics of biomarkers relevant to AD. Due to the heterogeneous nature of this disease, with patients of similar age experiencing different AD-related changes, a challenge facing longitudinal mixed-effects-based DPMs is the …
Overview Of The Clef-2023 Checkthat! Lab Task 1 On Check-Worthiness Of Multimodal And Multigenre Content, Firoj Alam, Alberto Barrón-Cedeño, Gullal S. Cheema, Gautam Kishore Shahi, Sherzod Hakimov, Maram Hasanain, Chengkai Li, Rubén Míguez, Hamdy Mubarak, Wajdi Zaghouani, Preslav Nakov
Overview Of The Clef-2023 Checkthat! Lab Task 1 On Check-Worthiness Of Multimodal And Multigenre Content, Firoj Alam, Alberto Barrón-Cedeño, Gullal S. Cheema, Gautam Kishore Shahi, Sherzod Hakimov, Maram Hasanain, Chengkai Li, Rubén Míguez, Hamdy Mubarak, Wajdi Zaghouani, Preslav Nakov
Natural Language Processing Faculty Publications
We present an overview of CheckThat! Lab’s 2023 Task 1, which is part of CLEF-2023. Task 1 asks to determine whether a text item, or a text coupled with an image, is check-worthy. This task places a special emphasis on COVID-19, political debates and transcriptions, and it is conducted in three languages: Arabic, English, and Spanish. A total of 15 teams participated, and most submissions managed to achieve significant improvements over the baselines using Transformer-based models. Out of these, seven teams participated in the multimodal subtask (1A), and 12 teams participated in the Multigenre subtask (1B), collectively submitting 155 official …
Gpachov At Checkthat! 2023: A Diverse Multi-Approach Ensemble For Subjectivity Detection In News Articles, Georgi Pachov, Dimitar Dimitrov, Ivan Koychev, Preslav Nakov
Gpachov At Checkthat! 2023: A Diverse Multi-Approach Ensemble For Subjectivity Detection In News Articles, Georgi Pachov, Dimitar Dimitrov, Ivan Koychev, Preslav Nakov
Natural Language Processing Faculty Publications
The wide-spread use of social networks has given rise to subjective, misleading, and even false information on the Internet. Thus, subjectivity detection can play an important role in ensuring the objectiveness and the quality of a piece of information. This paper presents the solution built by the Gpachov team for the CLEF-2023 CheckThat! lab Task 2 on subjectivity detection. Three different research directions are explored. The first one is based on fine-tuning a sentence embeddings encoder model and dimensionality reduction. The second one explores a sample-efficient few-shot learning model. The third one evaluates fine-tuning a multilingual transformer on an altered …
Enriched Pre-Trained Transformers For Joint Slot Filling And Intent Detection, Momchil Hardalov, Ivan Koychev, Preslav Nakov
Enriched Pre-Trained Transformers For Joint Slot Filling And Intent Detection, Momchil Hardalov, Ivan Koychev, Preslav Nakov
Natural Language Processing Faculty Publications
Detecting the user's intent and finding the corresponding slots among the utterance's words are important tasks in natural language understanding. Their interconnected nature makes their joint modeling a standard part of training such models. Moreover, data scarceness and specialized vocabularies pose additional challenges. Recently, the advances in pre-trained language models, namely contextualized models such as ELMo and BERT have revolutionized the field by tapping the potential of training very large models with just a few steps of fine-tuning on a task-specific dataset. Here, we leverage such models, and we design a novel architecture on top of them. Moreover, we propose …
Grammatical Error Correction: A Survey Of The State Of The Art, Christopher Bryant, Zheng Yuan, Muhammad Reza Qorib, Hannan Cao, Hwee Tou Ng, Ted Briscoe
Grammatical Error Correction: A Survey Of The State Of The Art, Christopher Bryant, Zheng Yuan, Muhammad Reza Qorib, Hannan Cao, Hwee Tou Ng, Ted Briscoe
Natural Language Processing Faculty Publications
Grammatical Error Correction (GEC) is the task of automatically detecting and correcting errors in text. The task not only includes the correction of grammatical errors, such as missing prepositions and mismatched subject–verb agreement, but also orthographic and semantic errors, such as misspellings and word choice errors, respectively. The field has seen significant progress in the last decade, motivated in part by a series of five shared tasks, which drove the development of rule-based methods, statistical classifiers, statistical machine translation, and finally neural machine translation systems, which represent the current dominant state of the art. In this survey paper, we condense …
N-Shot Benchmarking Of Whisper On Diverse Arabic Speech Recognition, Bashar Talafha, Abdul Waheed, Muhammad Abdul-Mageed
N-Shot Benchmarking Of Whisper On Diverse Arabic Speech Recognition, Bashar Talafha, Abdul Waheed, Muhammad Abdul-Mageed
Natural Language Processing Faculty Publications
Whisper, the recently developed multilingual weakly supervised model, is reported to perform well on multiple speech recognition benchmarks in both monolingual and multilingual settings. However, it is not clear how Whisper would fare under diverse conditions even on languages it was evaluated on such as Arabic. In this work, we address this gap by comprehensively evaluating Whisper on several varieties of Arabic speech for the ASR task. Our evaluation covers most publicly available Arabic speech data and is performed under n-shot (zero-, few-, and full) finetuning. We also investigate the robustness of Whisper under completely novel conditions, such as in …
Understanding Political Polarization Using Language Models: A Dataset And Method, Samiran Gode, Supreeth Bare, Bhiksha Raj, Hyungon Yoo
Understanding Political Polarization Using Language Models: A Dataset And Method, Samiran Gode, Supreeth Bare, Bhiksha Raj, Hyungon Yoo
Natural Language Processing Faculty Publications
Our paper aims to analyze political polarization in US political system using language models, and thereby help candidates make an informed decision. The availability of this information will help voters understand their candidates' views on the economy, healthcare, education, and other social issues. Our main contributions are a dataset extracted from Wikipedia that spans the past 120 years and a language model-based method that helps analyze how polarized a candidate is. Our data are divided into two parts, background information and political information about a candidate, since our hypothesis is that the political views of a candidate should be based …
Enhancing Video-Based Learning Using Knowledge Tracing: Personalizing Students’ Learning Experience With Orbits, Shady Shehata, David Santandreu, Philip Purnell, Mark Thompson
Enhancing Video-Based Learning Using Knowledge Tracing: Personalizing Students’ Learning Experience With Orbits, Shady Shehata, David Santandreu, Philip Purnell, Mark Thompson
Natural Language Processing Faculty Publications
As the world regains its footing following the COVID-19 pandemic, academia is striving to consolidate the gains made in students’ education experience. New technologies such as video-based learning have shown some early improvement in student learning and engagement. In this paper, we present ORBITS predictive engine at YOURIKA company, a video-based student support platform powered by knowledge tracing. In an exploratory case study of one master’s level Speech Processing course at the Mohamed Bin Zayed University of Artificial Intelligence (MBZUAI) in Abu Dhabi, half the students used the system while the other half did not. Student qualitative feedback was universally …
Analysis Of Predictive Performance And Reliability Of Classifiers For Quality Assessment Of Medical Evidence Revealed Important Variation By Medical Area, Simon Šuster, Timothy Baldwin, Karin Verspoor
Analysis Of Predictive Performance And Reliability Of Classifiers For Quality Assessment Of Medical Evidence Revealed Important Variation By Medical Area, Simon Šuster, Timothy Baldwin, Karin Verspoor
Natural Language Processing Faculty Publications
Objectives: A major obstacle in deployment of models for automated quality assessment is their reliability. To analyze their calibration and selective classification performance. Study Design and Setting: We examine two systems for assessing the quality of medical evidence, EvidenceGRADEr and RobotReviewer, both developed from Cochrane Database of Systematic Reviews (CDSR) to measure strength of bodies of evidence and risk of bias (RoB) of individual studies, respectively. We report their calibration error and Brier scores, present their reliability diagrams, and analyze the risk–coverage trade-off in selective classification. Results: The models are reasonably well calibrated on most quality criteria (expected calibration error …
Bertastic At Semeval-2023 Task 3: Fine-Tuning Pretrained Multilingual Transformers – Does Order Matter?, Tarek Mahmoud, Preslav Nakov
Bertastic At Semeval-2023 Task 3: Fine-Tuning Pretrained Multilingual Transformers – Does Order Matter?, Tarek Mahmoud, Preslav Nakov
Natural Language Processing Faculty Publications
The naïve approach for fine-tuning pretrained deep learning models on downstream tasks involves feeding them mini-batches of randomly sampled data. In this paper, we propose a more elaborate method for fine-tuning Pretrained Multilingual Transformers (PMTs) on multilingual data. Inspired by the success of curriculum learning approaches, we investigate the significance of fine-tuning PMTs on multilingual data in a sequential fashion language by language. Unlike the curriculum learning paradigm where the model is presented with increasingly complex examples, we do not adopt a notion of “easy” and “hard” samples. Instead, our experiments draw insight from psychological findings on how the human …
Team Thesyllogist At Semeval-2023 Task 3: Language-Agnostic Framing Detection In Multi-Lingual Online News: A Zero-Shot Transfer Approach, Osama Mohammed Afzal, Preslav Nakov
Team Thesyllogist At Semeval-2023 Task 3: Language-Agnostic Framing Detection In Multi-Lingual Online News: A Zero-Shot Transfer Approach, Osama Mohammed Afzal, Preslav Nakov
Natural Language Processing Faculty Publications
We describe our system for SemEval-2022 Task 3 subtask 2 which on detecting the frames used in a news article in a multi-lingual setup. We propose a multi-lingual approach based on machine translation of the input, followed by an English prediction model. Our system demonstrated good zero-shot transfer capability, achieving micro-F1 scores of 53% for Greek (4th on the leaderboard) and 56.1% for Georgian (3rd on the leaderboard), without any prior training on translated data for these languages. Moreover, our system achieved comparable performance on seven other languages, including German, English, French, Russian, Italian, Polish, and Spanish. Our results demonstrate …
Semeval-2023 Task 3: Detecting The Category, The Framing, And The Persuasion Techniques In Online News In A Multi-Lingual Setup, Jakub Piskorski, Nicolas Stefanovitch, Giovanni Da San Martino, Preslav Nakov
Semeval-2023 Task 3: Detecting The Category, The Framing, And The Persuasion Techniques In Online News In A Multi-Lingual Setup, Jakub Piskorski, Nicolas Stefanovitch, Giovanni Da San Martino, Preslav Nakov
Natural Language Processing Faculty Publications
We describe SemEval-2023 task 3 on Detecting the Category, the Framing, and the Persuasion Techniques in Online News in a Multilingual Setup: the dataset, the task organization process, the evaluation setup, the results, and the participating systems. The task focused on news articles in nine languages (six known to the participants upfront: English, French, German, Italian, Polish, and Russian), and three additional ones revealed to the participants at the testing phase: Spanish, Greek, and Georgian). The task featured three subtasks: (1) determining the genre of the article (opinion, reporting, or satire), (2) identifying one or more frames used in an …
Multilingual Multifaceted Understanding Of Online News In Terms Of Genre, Framing And Persuasion Techniques, Jakub Piskorski, Nicolas Stefanovitch, Nikolaos Nikolaidis, Giovanni Da San Martino, Preslav Nakov
Multilingual Multifaceted Understanding Of Online News In Terms Of Genre, Framing And Persuasion Techniques, Jakub Piskorski, Nicolas Stefanovitch, Nikolaos Nikolaidis, Giovanni Da San Martino, Preslav Nakov
Natural Language Processing Faculty Publications
We present a new multilingual multifacet dataset of news articles, each annotated for genre (objective news reporting vs. opinion vs. satire), framing (what key aspects are highlighted), and persuasion techniques (logical fallacies, emotional appeals, ad hominem attacks, etc.). The persuasion techniques are annotated at the span level, using a taxonomy of 23 fine-grained techniques grouped into 6 coarse categories. The dataset contains 1,612 news articles covering recent news on current topics of public interest in six European languages (English, French, German, Italian, Polish, and Russian), with more than 37k annotated spans of persuasion techniques. We describe the dataset and the …
Can You Answer This? - Exploring Zero-Shot Qa Generalization Capabilities In Large Language Models, Saptarshi Sengupta, Shreya Ghosh, Preslav Nakov, Prasenjit Mitra
Can You Answer This? - Exploring Zero-Shot Qa Generalization Capabilities In Large Language Models, Saptarshi Sengupta, Shreya Ghosh, Preslav Nakov, Prasenjit Mitra
Natural Language Processing Faculty Publications
The buzz around Transformer-based Language Models (TLMs) such as BERT, RoBERTa, etc. is well-founded owing to their impressive results on an array of tasks. However, when applied to areas needing specialized knowledge (closed-domain), such as medical, finance, etc. their performance takes drastic hits, sometimes more than their older recurrent/convolutional counterparts. In this paper, we explore zero-shot capabilities of large language models for extractive Question Answering. Our objective is to examine the performance change in the face of domain drift, i.e., when the target domain data is vastly different in semantic and statistical properties from the source domain, in an attempt …
Handling Realistic Label Noise In Bert Text Classification, Maha Tufail Agro, Hanan Al Darmaki
Handling Realistic Label Noise In Bert Text Classification, Maha Tufail Agro, Hanan Al Darmaki
Natural Language Processing Faculty Publications
Label noise refers to errors in training labels caused by cheap data annotation methods, such as web scraping or crowd-sourcing, which can be detrimental to the performance of supervised classifiers. Several methods have been proposed to counteract the effect of random label noise in supervised classification, and some studies have shown that BERT is already robust against high rates of randomly injected label noise. However, real label noise is not random; rather, it is often correlated with input features or other annotator-specific factors. In this paper, we evaluate BERT in the presence of two types of realistic label noise: feature-dependent …
Fair Enough: Standardizing Evaluation And Model Selection For Fairness Research In Nlp, Xudong Han, Timothy Baldwin, Trevor Cohn
Fair Enough: Standardizing Evaluation And Model Selection For Fairness Research In Nlp, Xudong Han, Timothy Baldwin, Trevor Cohn
Natural Language Processing Faculty Publications
Modern NLP systems exhibit a range of biases, which a growing literature on model debiasing attempts to correct. However, current progress is hampered by a plurality of definitions of bias, means of quantification, and oftentimes vague relation between debiasing algorithms and theoretical measures of bias. This paper seeks to clarify the current situation and plot a course for meaningful progress in fair learning, with two key contributions: (1) making clear inter-relations among the current gamut of methods, and their relation to fairness theory; and (2) addressing the practical problem of model selection, which involves a trade-off between fairness and accuracy …
Ten Years After Imagenet: A 360° Perspective On Artificial Intelligence, Sanjay Chawla, Preslav Nakov, Ahmed Ali, Wendy Hall, Issa Khalil, Xiaosong Ma, Husrev Taha Sencar, Ingmar Weber, Michael Wooldridge, Ting Yu
Ten Years After Imagenet: A 360° Perspective On Artificial Intelligence, Sanjay Chawla, Preslav Nakov, Ahmed Ali, Wendy Hall, Issa Khalil, Xiaosong Ma, Husrev Taha Sencar, Ingmar Weber, Michael Wooldridge, Ting Yu
Natural Language Processing Faculty Publications
It is 10 years since neural networks made their spectacular comeback. Prompted by this anniversary, we take a holistic perspective on artificial intelligence (AI). Supervised learning for cognitive tasks is effectively solved - provided we have enough high-quality labelled data. However, deep neural network models are not easily interpretable, and thus the debate between blackbox and whitebox modelling has come to the fore. The rise of attention networks, self-supervised learning, generative modelling and graph neural networks has widened the application space of AI. Deep learning has also propelled the return of reinforcement learning as a core building block of autonomous …
Iitd At The Wanlp 2022 Shared Task: Multilingual Multi-Granularity Network For Propaganda Detection, Shubham Mittal, Preslav Nakov
Iitd At The Wanlp 2022 Shared Task: Multilingual Multi-Granularity Network For Propaganda Detection, Shubham Mittal, Preslav Nakov
Natural Language Processing Faculty Publications
We present our system for the two subtasks of the shared task on propaganda detection in Arabic, part of WANLP'2022. Subtask 1 is a multi-label classification problem to find the propaganda techniques used in a given tweet. Our system for this task uses XLM-R to predict probabilities for the target tweet to use each of the techniques. In addition to finding the techniques, Subtask 2 further asks to identify the textual span for each instance of each technique that is present in the tweet; the task can be modeled as a sequence tagging problem. We use a multi-granularity network with …
Predicting Publication Of Clinical Trials Using Structured And Unstructured Data: Model Development And Validation Study, Siyang Wang, Simon Šuster, Timothy Baldwin, Karin Verspoor
Predicting Publication Of Clinical Trials Using Structured And Unstructured Data: Model Development And Validation Study, Siyang Wang, Simon Šuster, Timothy Baldwin, Karin Verspoor
Natural Language Processing Faculty Publications
Background: Publication of registered clinical trials is a critical step in the timely dissemination of trial findings. However, a significant proportion of completed clinical trials are never published, motivating the need to analyze the factors behind success or failure to publish. This could inform study design, help regulatory decision-making, and improve resource allocation. It could also enhance our understanding of bias in the publication of trials and publication trends based on the research direction or strength of the findings. Although the publication of clinical trials has been addressed in several descriptive studies at an aggregate level, there is a lack …