检索结果-内蒙古大学图书馆

conference on empirical methods in natural language processing (EMNLP)

作者： Nwatu, Joan Ignat, Oana Mihalcea, Rada Univ Michigan Ann Arbor MI 48109 USA

ISBN: (纸本)9798891760608

Despite the impressive performance of current AI models reported across various tasks, performance reports often do not include evaluations of how these models perform on the specific groups that will be impacted by these technologies. Among the minority groups under-represented in AI, data from low-income households are often overlooked in data collection and model evaluation. We evaluate the performance of a state-of-the-art vision-language model (CLIP) on a geo-diverse dataset containing household images associated with different income values (Dollar Street) and show that performance inequality exists among households of different income levels. Our results indicate that performance for the poorer groups is consistently lower than the wealthier groups across various topics and countries. We highlight insights that can help mitigate these issues and propose actionable steps for economic-level inclusive AI development. Code is available at Analysis for Bridging the Digital Divide.

关键词： Computational linguistics

来源：评论

学校读者我要写书评

暂无评论

Evaluating Large language Models along Dimensions of language Variation: A Systematik Invesdigatiom uv Cross-lingual Generalization

Evaluating Large Language Models along Dimensions of Languag...

引用

2024 conference on empirical methods in natural language processing, EMNLP 2024

作者： Bafna, Niyati Murray, Kenton Yarowsky, David Johns Hopkins University Center for Language and Speech Processing United States

ISBN: (纸本)9798891761643

While large language models exhibit certain cross-lingual generalization capabilities, they suffer from performance degradation (PD) on unseen closely-related languages (CRLs) and dialects relative to their high-resource language neighbour (HRLN).However, we currently lack a fundamental understanding of what kinds of linguistic distances contribute to PD, and to what ***, studies of cross-lingual generalization are confounded by unknown quantities of CRL language traces in the training data, and by the frequent lack of availability of evaluation data in lower-resource related languages and *** address these issues, we model phonological, morphological, and lexical distance as Bayesian noise processes to synthesize artificial languages that are controllably distant from the *** analyse PD as a function of underlying noise parameters, offering insights on model robustness to isolated and composed linguistic phenomena, and the impact of task and HRL characteristics on *** calculate parameter posteriors on real CRL-HRLN pair data and show that they follow computed trends of artificial languages, demonstrating the viability of our *** framework offers a cheap solution for estimating task performance on an unseen CRL given HRLN performance using its posteriors, as well as for diagnosing observed PD on a CRL in terms of its linguistic distances from its HRLN, and opens doors to principled methods of mitigating performance degradation. © 2024 Association for Computational Linguistics.

关键词： Computational linguistics

来源：评论

学校读者我要写书评

暂无评论

RevMUX: Data Multiplexing with Reversible Adapters for Efficient LLM Batch Inference

RevMUX: Data Multiplexing with Reversible Adapters for Effic...

引用

2024 conference on empirical methods in natural language processing, EMNLP 2024

作者： Xu, Yige Guo, Xu Zeng, Zhiwei Miao, Chunyan Joint NTU-UBC Research Centre of Excellence in Active Living for the Elderly Singapore College of Computing and Data Science Nanyang Technological University Singapore

ISBN: (纸本)9798891761643

Large language models (LLMs) have brought a great breakthrough to the natural language processing (NLP) community, while leading the challenge of handling concurrent customer queries due to their high throughput demands. Data multiplexing addresses this by merging multiple inputs into a single composite input, allowing more efficient inference through a shared forward pass. However, as distinguishing individuals from a composite input is challenging, conventional methods typically require training the entire backbone, yet still suffer from performance degradation. In this paper, we introduce RevMUX, a parameter-efficient data multiplexing framework that incorporates a reversible design in the multiplexer, which can be reused by the demultiplexer to perform reverse operations and restore individual samples for classification. Extensive experiments on four datasets and three types of LLM backbones demonstrate the effectiveness of RevMUX for enhancing LLM inference efficiency while retaining a satisfactory classification performance. © 2024 Association for Computational Linguistics.

关键词： natural language processing systems

来源：评论

学校读者我要写书评

暂无评论

QA-NatVer: Question Answering for natural Logic-based Fact Verification

QA-NatVer: Question Answering for Natural Logic-based Fact V...

引用

conference on empirical methods in natural language processing (EMNLP)

作者： Aly, Rami Strong, Marek Vlachos, Andreas Univ Cambridge Dept Comp Sci & Technol Cambridge England

ISBN: (纸本)9798891760608

Fact verification systems assess a claim's veracity based on evidence. An important consideration in designing them is faithfulness, i.e. generating explanations that accurately reflect the reasoning of the model. Recent works have focused on natural logic, which operates directly on natural language by capturing the semantic relation of spans between an aligned claim with its evidence via set-theoretic operators. However, these approaches rely on substantial resources for training, which are only available for high-resource languages. To this end, we propose to use question answering to predict natural logic operators, taking advantage of the generalization capabilities of instruction-tuned language models. Thus, we obviate the need for annotated training data while still relying on a deterministic inference system. In a few-shot setting on FEVER, our approach outperforms the best baseline by 4.3 accuracy points, including a state-of-the-art pre-trained seq2seq natural logic system, as well as a state-of-the-art prompt-based classifier. Our system demonstrates its robustness and portability, achieving competitive performance on a counterfactual dataset and surpassing all approaches without further annotation on a Danish verification dataset. A human evaluation indicates that our approach produces more plausible proofs with fewer erroneous natural logic operators than previous natural logic-based systems.

关键词： Semantics

来源：评论

学校读者我要写书评

暂无评论

Neuron-Level Knowledge Attribution in Large language Models

Neuron-Level Knowledge Attribution in Large Language Models

引用

2024 conference on empirical methods in natural language processing, EMNLP 2024

作者： Yu, Zeping Ananiadou, Sophia Department of Computer Science National Centre for Text Mining The University of Manchester United Kingdom

ISBN: (纸本)9798891761643

Identifying important neurons for final predictions is essential for understanding the mechanisms of large language models. Due to computational constraints, current attribution techniques struggle to operate at neuron level. In this paper, we propose a static method for pinpointing significant neurons. Compared to seven other methods, our approach demonstrates superior performance across three metrics. Additionally, since most static methods typically only identify "value neurons" directly contributing to the final prediction, we propose a method for identifying "query neurons" which activate these "value neurons". Finally, we apply our methods to analyze six types of knowledge across both attention and feed-forward network (FFN) layers. Our method and analysis are helpful for understanding the mechanisms of knowledge storage and set the stage for future research in knowledge editing. The code is available on https://***/zepingyu0512/neuron-attribution. © 2024 Association for Computational Linguistics.

关键词： Neurons

来源：评论

学校读者我要写书评

暂无评论

Better Call SAUL: Fluent and Consistent language Model Editing with Generation Regularization

Better Call SAUL: Fluent and Consistent Language Model Editi...

引用

2024 conference on empirical methods in natural language processing, EMNLP 2024

作者： Wang, Mingyang Lange, Lukas Adel, Heike Strötgen, Jannik Schütze, Hinrich LMU Munich Germany Germany Hochschule der Medien Stuttgart Germany Karlsruhe University of Applied Sciences Germany

ISBN: (纸本)9798891761681

To ensure large language models contain up-to-date knowledge, they need to be updated ***, model editing is challenging as it might also affect knowledge that is unrelated to the new ***-of-the-art methods identify parameters associated with specific knowledge and then modify them via direct weight ***, these locate-and-edit methods suffer from heavy computational overhead and lack theoretical *** contrast, directly fine-tuning the model on requested edits affects the model's behavior on unrelated knowledge, and significantly damages the model's generation fluency and *** address these challenges, we propose SAUL, a streamlined model editing method that uses sentence concatenation with augmented random facts for generation *** on three model editing benchmarks show that SAUL is a practical and reliable solution for model editing outperforming state-of-the-art methods while maintaining generation quality and reducing computational overhead. © 2024 Association for Computational Linguistics.

关键词： Computational linguistics

来源：评论

学校读者我要写书评

暂无评论

LongForm: Effective Instruction Tuning with Reverse Instructions

LongForm: Effective Instruction Tuning with Reverse Instruct...

引用

2024 conference on empirical methods in natural language processing, EMNLP 2024

作者： Köksal, Abdullatif Schick, Timo Korhonen, Anna Schütze, Hinrich Center for Information and Language Processing LMU Munich Germany Munich Center for Machine Learning Germany Language Technology Lab University of Cambridge United Kingdom

ISBN: (纸本)9798891761681

Instruction tuning enables language models to more effectively generalize and better follow user intent. However, obtaining instruction data is costly and challenging. Prior work employs methods such as expensive human annotation, crowd-sourced datasets with alignment issues, and generating noisy examples via LLMs. We introduce the LongForm-C dataset, which is created by reverse instructions. We generate instructions via LLMs for human-written corpus examples using reverse instructions. First we select a diverse set of human-written documents from corpora such as C4 and Wikipedia;then we generate instructions for these documents via LLMs. This approach provides a cheaper and cleaner instruction-tuning dataset with natural output and one suitable for long text generation. Our models outperform 10x larger language models without instruction tuning on tasks such as story/recipe generation and long-form question answering. Moreover, LongForm models outperform prior instruction-tuned models such as FLAN-T5 and Alpaca by a large margin, and improve language understanding capabilities further. We publicly release our data and models: https://***/akoksal/LongForm. © 2024 Association for Computational Linguistics.

关键词： Computational linguistics

来源：评论

学校读者我要写书评

暂无评论

OmAgent: A Multi-modal Agent Framework for Complex Video Understanding with Task Divide-and-Conquer

OmAgent: A Multi-modal Agent Framework for Complex Video Und...

引用

2024 conference on empirical methods in natural language processing, EMNLP 2024

作者： Zhang, Lu Zhao, Tiancheng Ying, Heting Ma, Yibo Lee, Kyusong Om AI Research Binjiang Institute of Zhejiang University China

ISBN: (纸本)9798891761643

Recent advancements in Large language Models (LLMs) have expanded their capabilities to multimodal contexts, including comprehensive video understanding. However, processing extensive videos such as 24-hour CCTV footage or full-length films presents significant challenges due to the vast data and processing demands. Traditional methods, like extracting key frames or converting frames to text, often result in substantial information loss. To address these shortcomings, we develop OmAgent, efficiently stores and retrieves relevant video frames for specific queries, preserving the detailed content of videos. Additionally, it features an Divide-and-Conquer Loop capable of autonomous reasoning, dynamically invoking APIs and tools to enhance query processing and accuracy. This approach ensures robust video understanding, significantly reducing information loss. Experimental results affirm OmAgent's efficacy in handling various types of videos and complex tasks. Moreover, we have endowed it with greater autonomy and a robust tool-calling system, enabling it to accomplish even more intricate tasks. Code: https://***/om-ai-lab/OmAgent. © 2024 Association for Computational Linguistics.

关键词： Query processing

来源：评论

学校读者我要写书评

暂无评论

Zero-Shot Cross-Lingual Named Entity Recognition via Progressive Multi-Teacher Distillation

引用

IEEE-ACM TRANSACTIONS ON AUDIO SPEECH AND language processing 2024年 32卷 4617-4630页

作者： Li, Zhuoran Hu, Chunming Zhang, Richong Chen, Junfan Guo, Xiaohui Beihang Univ Sch Comp Sci & Engn Beijing 100191 Peoples R China Beihang Univ Sch Software Beijing 100191 Peoples R China Beihang Univ Hangzhou Innovat Inst Hangzhou 310051 Peoples R China

Cross-lingual learning aims to transfer knowledge from one natural language to another. Zero-shot cross-lingual named entity recognition (NER) tasks are to train an NER model on source languages and to identify named entities in other languages. Existing knowledge distillation-based models in a teacher-student manner leverage the unlabeled samples from the target languages and show their superiority in this setting. However, the valuable similarity information between tokens in the target language is ignored. And the teacher model trained solely on the source language generates low-quality pseudo-labels. These two facts impact the performance of cross-lingual NER. To improve the reliability of the teacher model, in this study, we first introduce one extra simple binary classification teacher model by similarity learning to measure if the inputs are from the same class. We note that this binary classification auxiliary task is easier, and the two teachers simultaneously supervise the student model for better performance. Furthermore, given such a stronger student model, we propose a progressive knowledge distillation framework that extensively fine-tunes the teacher model on the target-language pseudo-labels generated by the student model. empirical studies on three datasets across seven different languages show that our presented model outperforms state-of-the-art methods.

关键词： Data models Predictive models Computational modeling Adaptation models Training Task analysis Speech processing natural language processing named entity recognition sequence labelling zero-shot learning cross-lingual

来源：评论

学校读者我要写书评

暂无评论

M5 - A Diverse Benchmark to Assess the Performance of Large Multimodal Models Across Multilingual and Multicultural Vision-language Tasks

M5 - A Diverse Benchmark to Assess the Performance of Large ...

引用

2024 conference on empirical methods in natural language processing, EMNLP 2024

作者： Schneider, Florian Sitaram, Sunayana Language Technology Group Universität Hamburg Germany Microsoft Research India Bangalore India

ISBN: (纸本)9798891761681

Since the release of ChatGPT, the field of natural language processing has experienced rapid advancements, particularly in Large language Models (LLMs) and their multimodal counterparts, Large Multimodal Models (LMMs). Despite their impressive capabilities, LLMs often exhibit significant performance disparities across different languages and cultural contexts, as demonstrated by various text-only benchmarks. However, current research lacks such benchmarks for multimodal visio-linguistic settings. This work fills this gap by introducing M5, the first comprehensive benchmark designed to evaluate LMMs on diverse vision-language tasks within a multilingual and multicultural context. M5 includes eight datasets covering five tasks and 41 languages, with a focus on underrepresented languages and culturally diverse images. Furthermore, we introduce two novel datasets, M5-VGR and M5-VLOD, including a new Visio-Linguistic Outlier Detection task, in which all evaluated open-source models fail to significantly surpass the random baseline. Through extensive evaluation and analyses, we highlight substantial task-agnostic performance disparities between high- and low-resource languages. Moreover, we show that larger models do not necessarily outperform smaller ones in a multilingual setting. © 2024 Association for Computational Linguistics.

关键词： Visual languages

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：