检索结果-内蒙古大学图书馆

Diversity Over Size: On the Effect of Sample and Topic Sizes for Topic-Dependent Argument Mining Datasets

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Schiller, Benjamin Daxenberger, Johannes Waldis, Andreas Gurevych, Iryna Summetix GmbH Ubiquitous Knowledge Processing Lab Department of Computer Science Technical University of Darmstadt Germany Information Systems Research Lab Lucerne University of Applied Sciences and Arts Switzerland

Topic-Dependent Argument Mining (TDAM), that is extracting and classifying argument components for a specific topic from large document sources, is an inherently difficult task for machine learning models and humans alike, as large TDAM datasets are rare and recognition of argument components requires expert knowledge. The task becomes even more difficult if it also involves stance detection of retrieved arguments. In this work, we investigate the effect of TDAM dataset composition in few- and zero-shot settings. Our findings show that, while fine-tuning is mandatory to achieve acceptable model performance, using carefully composed training samples and reducing the training sample size by up to almost 90% can still yield 95% of the maximum performance. This gain is consistent across three TDAM tasks on three different datasets. We also publish a new dataset1 and code2 for future benchmarking. Copyright © 2022, The Authors. All rights reserved.

关键词： Adversarial machine learning

Controllable Factuality in Document-Grounded Dialog Systems Using a Noisy Channel Model

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Daheim, Nico Thulke, David Dugast, Christian Ney, Hermann Ubiquitous Knowledge Processing Lab Department of Computer Science Technical University of Darmstadt Germany Human Language Technology and Pattern Recognition RWTH Aachen University Germany AppTek GmbH Germany

In this work, we present a model for document-grounded response generation in dialog that is decomposed into two components according to Bayes' theorem. One component is a traditional ungrounded response generation model and the other component models the reconstruction of the grounding document based on the dialog context and generated response. We propose different approximate decoding schemes and evaluate our approach on multiple open-domain and task-oriented document-grounded dialog datasets. Our experiments show that the model is more factual in terms of automatic factuality metrics than the baseline model. Furthermore, we outline how introducing scaling factors between the components allows for controlling the tradeoff between factuality and fluency in the model output. Finally, we compare our approach to a recently proposed method to control factuality in grounded dialog, CTRL (Rashkin et al., 2021), and show that both approaches can be combined to achieve additional improvements. © 2022, CC BY.

关键词：

Focusing knowledge-based graph argument mining via topic modeling

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Abels, Patrick Ahmadi, Zahra Burkhardt, Sophie Schiller, Benjamin Gurevych, Iryna Kramer, Stefan Johannes Gutenberg University Mainz Germany Ubiquitous Knowledge Processing Lab Department of Computer Science Technical University of Darmstadt Germany

Decision-making usually takes five steps: identifying the problem, collecting data, extracting evidence, identifying pro and con arguments, and making decisions. Focusing on extracting evidence, this paper presents a hybrid model that combines latent Dirichlet allocation and word embeddings to obtain external knowledge from structured and unstructured data. We study the task of sentence-level argument mining, as arguments mostly require some degree of world knowledge to be identified and understood. Given a topic and a sentence, the goal is to classify whether a sentence represents an argument in regard to the topic. We use a topic model to extract topic- and sentence-specific evidence from the structured knowledge base Wikidata, building a graph based on the cosine similarity between the entity word vectors of Wikidata and the vector of the given sentence. Also, we build a second graph based on topic-specific articles found via Google to tackle the general incompleteness of structured knowledge bases. Combining these graphs, we obtain a graph-based model which, as our evaluation shows, successfully capitalizes on both structured and unstructured data. © 2021, CC BY-NC-SA.

关键词： knowledge based systems

Scientia Potentia Est - On the Role of knowledge in Computational Argumentation

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Lauscher, Anne Wachsmuth, Henning Gurevych, Iryna Glavaš, Goran MilaNLP Bocconi University Italy Department of Computer Science Paderborn University Germany Ubiquitous Knowledge Processing Lab TU Darmstadt Germany CAIDAS University of Würzburg Germany

Despite extensive research efforts in recent years, computational argumentation (CA) remains one of the most challenging areas of natural language processing. The reason for this is the inherent complexity of the cognitive processes behind human argumentation, which integrate a plethora of different types of knowledge, ranging from topic-specific facts and common sense to rhetorical knowledge. The integration of knowledge from such a wide range in CA requires modeling capabilities far beyond many other natural language understanding tasks. Existing research on mining, assessing, reasoning over, and generating arguments largely acknowledges that much more knowledge is needed to accurately model argumentation computationally. However, a systematic overview of the types of knowledge introduced in existing CA models is missing, hindering targeted progress in the field. Adopting the operational definition of knowledge as any task-relevant normative information not provided as input, the survey paper at hand fills this gap by (1) proposing a taxonomy of types of knowledge required in CA tasks, (2) systematizing the large body of CA work according to the reliance on and exploitation of these knowledge types for the four main research areas in CA, and (3) outlining and discussing directions for future research efforts in CA. © 2021, CC BY-NC-SA.

关键词： Natural language processing systems

Learning Fair Representations through Uniformly Distributed Sensitive Attributes

学校读者我要写书评

暂无评论

Learning Fair Representations through Uniformly Distributed ...

Secure and Trustworthy Machine Learning (SaTML), IEEE Conference on

作者： Patrik Joslin Kenfack Adín Ramírez Rivera Adil Mehmood Khan Manuel Mazzara Machine Learning and Knowledge Representation Lab Innopolis University Innopolis Russia Department of Informatics Digital Signal Processing and Image Analysis (DSB) group University of Oslo Oslo Norway School of Computer Science University of Hull Hull UK Institute of Software Development and Engineering Innopolis University Innopolis Russia

Machine Learning (ML) models trained on biased data can reproduce and even amplify these biases. Since such models are deployed to make decisions that can affect people's lives, ensuring their fairness is critical. One approach to mitigate possible unfairness of ML models is to map the input data into a less-biased new space by means of training the model on fair representations. Several methods based on adversarial learning have been proposed to learn fair representation by fooling an adversary in predicting the sensitive attribute (e.g., gender or race). However, adversarial-based learning can be too difficult to optimize in practice; besides, it penalizes the utility of the representation. Hence, in this research effort we train bias-free representations from the input data by inducing a uniform distribution over the sensitive attributes in the latent space. In particular, we propose a probabilistic framework that learns these representations by enforcing the correct reconstruction of the original data, plus the prediction of the attributes of interest while eliminating the possibility of predicting the sensitive ones. Our method leverages the inability of Deep Neural Networks (DNNs) to generalize when trained on a noisy label space to regularize the latent space. We use a network head that predicts a noisy version of the sensitive attributes in order to increase the uncertainty of their predictions at test time. Our experiments in two datasets demonstrated that the proposed model significantly improves fairness while maintaining the prediction accuracy of downstream tasks.

关键词：

Does my rebuttal matter? Insights from a major NLP conference

学校读者我要写书评

暂无评论

Does my rebuttal matter? Insights from a major NLP conferenc...

2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT 2019

作者： Gao, Yang Eger, Steffen Kuznetsov, Ilia Gurevych, Iryna Miyao, Yusuke Ubiquitous Knowledge Processing Lab Department of Computer Science Technische Universität Darmstadt Germany Department of Computer Science Graduate School of Information Science and Technology University of Tokyo Japan

ISBN: (纸本)9781950737130

Peer review is a core element of the scientific process, particularly in conference-centered fields such as ML and NLP. However, only few studies have evaluated its properties empirically. Aiming to fill this gap, we present a corpus that contains over 4k reviews and 1.2k author responses from ACL-2018. We quantitatively and qualitatively assess the corpus. This includes a pilot study on paper weaknesses given by reviewers and on quality of author responses. We then focus on the role of the rebuttal phase, and propose a novel task to predict after-rebuttal (i.e., final) scores from initial reviews and author responses. Although author responses do have a marginal (and statistically significant) influence on the final scores, especially for borderline papers, our results suggest that a reviewer's final score is largely determined by her initial score and the distance to the other reviewers' initial scores. In this context, we discuss the conformity bias inherent to peer reviewing, a bias that has largely been overlooked in previous research. We hope our analyses will help better assess the usefulness of the rebuttal phase in NLP conferences. © 2019 Association for Computational Linguistics

关键词： Natural language processing systems

Multi-task learning for argumentation mining

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Eger, Steffen Gurevych, Iryna Computer Science Department Ubiquitous Knowledge Processing

Multi-task learning has recently become a very active field in deep learning research. In contrast to learning a single task in isolation, multiple tasks are learned at the same time, thereby utilizing the training signal of related tasks to improve the performance on the respective machine learning tasks. Related work shows various successes in different domains when applying this paradigm and this thesis extends the existing empirical results by evaluating multi-task learning in four different scenarios: argumentation mining, epistemic segmentation, argumentation component segmentation, and grapheme-to-phoneme conversion. We show that multi-task learning can, indeed, improve the performance compared to single-task learning in all these scenarios, but may also hurt the performance. Therefore, we investigate the reasons for successful and less successful applications of this paradigm and find that dataset properties such as entropy or the size of the label inventory are good indicators for a potential multi-task learning success and that multi-task learning is particularly useful if the task at hand suffers from data sparsity, i.e. a lack of training data. Moreover, multi-task learning is particularly effective for long input sequences in our experiments. We have observed this trend in all evaluated scenarios. Finally, we develop a highly configurable and extensible sequence tagging framework which supports multi-task learning to conduct our empirical experiments and to aid future research regarding the multitask learning paradigm and natural language processing. Copyright © 2019, The Authors. All rights reserved.

关键词： Deep learning

Bayesian heatmaps: Probabilistic classification with multiple unreliable information sources

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Simpson, Edwin Reece, Steven Roberts, Stephen J. Ubiquitous Knowledge Processing Lab Department of Computer Science Technische Universität Darmstadt Department of Engineering Science University of Oxford

Unstructured data from diverse sources, such as social media and aerial imagery, can provide valuable up-to-date information for intelligent situation assessment. Mining these different information sources could bring major benefits to applications such as situation awareness in disaster zones and mapping the spread of diseases. Such applications depend on classifying the situation across a region of interest, which can be depicted as a spatial "heatmap". Annotating unstructured data using crowdsourcing or automated classifiers produces individual classifications at sparse locations that typically contain many errors. We propose a novel Bayesian approach that models the relevance, error rates and bias of each information source, enabling us to learn a spatial Gaussian Process classifier by aggregating data from multiple sources with varying reliability and relevance. Our method does not require gold-labelled data and can make predictions at any location in an area of interest given only sparse observations. We show empirically that our approach can handle noisy and biased data sources, and that simultaneously inferring reliability and transferring information between neighbouring reports leads to more accurate predictions. We demonstrate our method on two real-world problems from disaster response, showing how our approach reduces the amount of crowdsourced data required and can be used to generate valuable heatmap visualisations from SMS messages and satellite images. Copyright © 2019, The Authors. All rights reserved.

关键词： Crowdsourcing

Scalable Bayesian Preference Learning for Crowds

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Simpson, Edwin Gurevych, Iryna Ubiquitous Knowledge Processing Lab Dept. of Computer Science Technische Universität Darmstadt

We propose a scalable Bayesian preference learning method for jointly predicting the preferences of individuals as well as the consensus of a crowd from pairwise labels. Peoples’ opinions often differ greatly, making it difficult to predict their preferences from small amounts of personal data. Individual biases also make it harder to infer the consensus of a crowd when there are few labels per item. We address these challenges by combining matrix factorisation with Gaussian processes, using a Bayesian approach to account for uncertainty arising from noisy and sparse data. Our method exploits input features, such as text embeddings and user metadata, to predict preferences for new items and users that are not in the training set. As previous solutions based on Gaussian processes do not scale to large numbers of users, items or pairwise labels, we propose a stochastic variational inference approach that limits computational and memory costs. Our experiments on a recommendation task show that our method is competitive with previous approaches despite our scalable inference approximation. We demonstrate the method’s scalability on a natural language processing task with thousands of users and items, and show improvements over the state of the art on this task. We make our software publicly available for future work 1. Copyright © 2019, The Authors. All rights reserved.

关键词： Gaussian distribution