Model adaptation is crucial to handle the discrepancy between proxy training data and actual users' data received. To effectively perform adaptation, textual data of users is typically stored on servers or their l...
详细信息
ISBN:
(纸本)9798350344868;9798350344851
Model adaptation is crucial to handle the discrepancy between proxy training data and actual users' data received. To effectively perform adaptation, textual data of users is typically stored on servers or their local devices, where downstream naturallanguageprocessing (NLP) models can be directly trained using such in-domain data. However, this might raise privacy and security concerns due to the extra risks of exposing user information to adversaries. Replacing identifying information in textual data with a generic marker has been recently explored. In this work, we leverage large language models (LLMs) to suggest substitutes of masked tokens and have their effectiveness evaluated on downstream language modeling tasks. Specifically, we propose multiple pre-trained and fine-tuned LLM-based approaches and perform empirical studies on various datasets for the comparison of these methods. Experimental results show that models trained on the obfuscation corpora are able to achieve comparable performance with the ones trained on the original data without privacy-preserving token masking.
Large language Models (LLMs) and Large Vision language Models (LVLMs) exhibit advanced proficiency in language reasoning and comprehension across a wide array of languages. While their performance is notably robust in...
详细信息
We introduce a task and dataset for referring expression generation and comprehension in multi-agent embodied *** this task, two agents in a shared scene must take into account one another's visual perspective, wh...
详细信息
GraphQL is a powerful query language for APIs that allows clients to fetch precise data efficiently and flexibly, querying multiple resources with a single request. However, crafting complex GraphQL query operations c...
详细信息
language style is necessary for AI systems to understand and generate diverse human language ***, previous text style transfer primarily focused on sentence-level data-driven approaches, limiting exploration of potent...
详细信息
Using a vocabulary that is shared across languages is common practice in Multilingual Neural Machine Translation (MNMT). In addition to its simple design, shared tokens play an important role in positive knowledge tra...
详细信息
ISBN:
(纸本)9798891760608
Using a vocabulary that is shared across languages is common practice in Multilingual Neural Machine Translation (MNMT). In addition to its simple design, shared tokens play an important role in positive knowledge transfer, assuming that shared tokens refer to similar meanings across languages. However, when word overlap is small, especially due to different writing systems, transfer is inhibited. In this paper, we define word-level information transfer pathways via word equivalence classes and rely on graph networks to fuse word embeddings across languages. Our experiments demonstrate the advantages of our approach: 1) embeddings of words with similar meanings are better aligned across languages, 2) our method achieves consistent BLEU improvements of up to 2.3 points for high- and low-resource MNMT, and 3) less than 1.0% additional trainable parameters are required with a limited increase in computational costs, while inference time remains identical to the baseline. We release the codebase to the community.
With the widespread application of Large language Models (LLMs) in naturallanguage Interfaces to Databases (NLIDBs), concerns about security issues in NLIDBs have been increasing gradually. However, research on sensi...
详细信息
The increasing use of foundation models highlights the urgent need to address and eliminate implicit biases present in them that arise during pre-training. In this paper, we introduce PEFTDebias, a novel approach that...
详细信息
ISBN:
(纸本)9798891760608
The increasing use of foundation models highlights the urgent need to address and eliminate implicit biases present in them that arise during pre-training. In this paper, we introduce PEFTDebias, a novel approach that employs parameter-efficient fine-tuning (PEFT) to mitigate the biases within foundation models. PEFTDebias consists of two main phases: an upstream phase for acquiring debiasing parameters along a specific bias axis, and a downstream phase where these parameters are incorporated into the model and frozen during the fine-tuning process. By evaluating on four datasets across two bias axes namely gender and race, we find that downstream biases can be effectively reduced with PEFTs. In addition, we show that these parameters possess axis-specific debiasing characteristics, enabling their effective transferability in mitigating biases in various downstream tasks. To ensure reproducibility, we release the code to do our experiments(1).
To enhance the performance of large language models (LLM) on downstream tasks, one solution is to fine-tune certain LLM parameters and make it better align with the characteristics of the training dataset. This proces...
详细信息
WALLEDEVAL is a comprehensive AI safety testing toolkit designed to evaluate large language models (LLMs). It accommodates a diverse range of models, including both open-weight and API-based ones, and features over 35...
详细信息
暂无评论