检索结果-内蒙古大学图书馆

A Character Based Steganography Using masked language modeling

IEEE ACCESS 2024年 12卷 14248-14259页

作者： Ozturk, Emir Mesut, Andac Sahin Fidan, Ozlem Aydin Trakya Univ Comp Engn Dept TR-22030 Edirne Turkiye

In this study, a steganography method based on BERT transformer model is proposed for hiding text data in cover text. The aim is to hide information by replacing specific words within the text using BERT's masked language modeling (MLM) feature. In this study, two models, fine-tuned for English and Turkish, are utilized to perform steganography on texts belonging to these languages. Furthermore, the proposed method can work with any transformer model that supports masked language modeling. While traditionally the hidden information in text is often limited, the proposed method allows for a significant amount of data to be hidden in the text without distorting its meaning. In this study, the proposed method is tested by hiding stego texts of varying lengths in cover text of different lengths in two different language scenarios. The test results are analyzed in terms of perplexity, KL divergence and semantic similarity. Upon examining the results, the proposed method has achieved the best results compared to other methods found in the literature, with KL divergence of 7.93 and semantic similarity of 0.99. It can be observed that the proposed method has low detectability and demonstrates success in the data hiding process.

关键词： Steganography Transformers Indexes Media Predictive models Hash functions Data models Bit error rate Natural language processing BERT masked language modeling steganography

来源：评论

学校读者我要写书评

暂无评论

Reversible Linguistic Steganography With Bayesian masked language modeling

引用

IEEE TRANSACTIONS ON COMPUTATIONAL SOCIAL SYSTEMS 2023年第2期10卷 714-723页

作者： Chang, Ching-Chun Univ Warwick Dept Comp Sci Coventry CV4 7AL W Midlands England

Text authentication serves a vital role in the defense of digital identity and content against various types of cybercrime. The use of a digital signature is a common cryptographic technique for text authentication. Linguistic steganography can be applied to further conceal a digital signature within the corresponding text to facilitate data management. However, steganographic distortion lurking in the text, albeit almost imperceptible, has the potential to cause automatic computing machinery to make biased decisions. This has led to an interest in the pursuit of reversibility, the ability to reverse a steganographic process and remove distortion. In this article, we propose a reversible steganographic system for natural language text. We use a pre-trained transformer neural network for masked language modeling and embed messages in a reversible manner via predictive word substitution. Furthermore, we derive an adaptive steganographic route by taking account of predictive uncertainty, which is quantified based on a theoretical framework of Bayesian deep learning. Experimental results show that the proposed steganographic system can attain a proper balance between capacity, imperceptibility, and reversibility with close semantic and sentimental similarities between cover and stego texts.

关键词： Linguistics Steganography Predictive models Indexes Task analysis Distortion Bayes methods Bayesian uncertainty analysis masked language modeling reversible linguistic steganography

来源：评论

学校读者我要写书评

暂无评论

Sentiment-based masked language modeling for improving sentence-level valence-arousal prediction

引用

APPLIED INTELLIGENCE 2022年第14期52卷 16353-16369页

作者： Wu, Jheng-Long Chung, Wei-Yi Soochow Univ Dept Data Sci Taipei Taiwan Soochow Univ Sch Big Data Management Taipei Taiwan

Sentiment indicator prediction is a crucial task in sentiment analysis or emotion recognition. Through the accurate quantification of sentiments expressed in a text, people's sentiments can be better understood. Many studies have used mask language modeling to predict sentiment indicators. In this approach, every word has the same masking probability irrespective of its degree of influence on sentiments because a random masking mechanism is adopted. Therefore, the learned features do not strengthen the sentiment vocabulary in a sentence. Consequently, this paper proposes a sentiment-based masked language modeling method to predict sentence-level valence-arousal scores in Chinese. The proposed method is called dimensional valence-arousal based on bidirectional encoder representations from transformers (DVA-BERT), which combines the BERT model with specific sentiment word masking. The proposed method involves two learning tasks: valence-arousal intensity estimation, which is the major task, and random masked sentiment word prediction, which is the auxiliary task modify from mask language modeling, used to enhance the model performance. The experimental results indicate that the proposed DVA-BERT model can identify effective sentiment features by masking sentiment words and can outperform the original BERT model and other word masking methods.

关键词： Deep learning masked language modeling Emotion recognition Sentiment analysis Valence-arousal model

来源：评论

学校读者我要写书评

暂无评论

MDA: masked language modeling Meets Data Augmentation for Multiple Choice Question Answering 9

MDA: Masked Language Modeling Meets Data Augmentation for Mu...

引用

9th IEEE Smart World Congress, SWC 2023

作者： Yao, Xun Huang, Zijian Ma, Junlong Chen, Jingxue Yang, Jie Wuhan Textile University China University of Wollongong Australia

ISBN: (纸本)9798350319804

Multiple Choice Question Answering (MCQA) is a well-established task in the field of Machine Reading Comprehension (MRC). Its objective is to identify the correct answer from a given set of options, based on the provided background passage and question. Recent advancements in large-scale Pre-trained language Models (PLMs) have yielded impressive performance in MCQA. However, achieving such performance requires a significant number of training samples, leading to time-consuming and labor-intensive sample acquisition and annotation processes. To overcome the limitation posed by the availability of training samples, this paper explores the potential of leveraging the [Mask] token, which is commonly used in masked language modeling (MLM) during the self-supervised training of PLMs. Specifically, the paper introduces a straightforward yet effective approach called [Mask] based Data Augmentation (MDA). The proposed method involves injecting [Mask] tokens into background passages to create masked versions of the original data. Moreover, a self-evaluator is introduced to regulate the process of masking production, with the objective of minimizing negative impact caused by argumentation noise. The effectiveness of the proposed method is empirically validated using various benchmark MCQA datasets. Experimental results demonstrate considerable improvements over state-of-the-arts. © 2023 IEEE.

关键词： Data Augmentation Machine Reading Comprehension masked language modeling Multiple Choice Question Answering Self-evaluation

来源：评论

学校读者我要写书评

暂无评论

Multi-label masked language modeling on Zero-shot Code-switched Sentiment Analysis 22

Multi-label Masked Language Modeling on Zero-shot Code-switc...

引用

45th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR)

作者： Li, Zhi Gao, Xing Zhang, Ji Zhang, Yin Zhejiang Univ Coll Comp Sci & Technol Hangzhou Peoples R China Alibaba Grp Hangzhou Peoples R China

ISBN: (纸本)9781450387323

In multilingual communities, code-switching is a common phenomenon and code-switched tasks have become a crucial area of research in natural language processing (NLP) applications. Existing approaches mainly focus on supervised learning. However, it is expensive to annotate a sufficient amount of code-switched data. In this paper, we consider zero-shot setting and improve model performance on code-switched tasks via monolingual language datasets, unlabeled code-switched datasets, and semantic dictionaries. Inspired by the mechanism of code-switching itself, we propose multilabel masked language modeling and predict both the masked word and its synonyms in other languages. Experimental results show that compared with baselines, our method can further improve the pretrained multilingual model's performance on code-switched sentiment analysis datasets.

关键词： Code-switched Zero-shot Learning Sentiment Analysis Multi-label masked language modeling

来源：评论

学校读者我要写书评

暂无评论

Effective masked language modeling for Temporal Commonsense Reasoning 12

Effective Masked Language Modeling for Temporal Commonsense ...

引用

Joint 12th International Conference on Soft Computing and Intelligent Systems / 23rd International Symposium on Advanced Intelligent Systems (SCIS and ISIS)

作者： Kimura, Mayuko Pereira, Lis Kanashiro Kobayashi, Ichiro Ochanomizu Univ Tokyo Japan

ISBN: (纸本)9781665499248

We propose an enhanced language model for temporal commonsense inference. Through a specially designed masked language modeling task, we equip our model with temporal commonsense knowledge. Specifically, we propose different masking strategies for temporal commonsense reasoning. We evaluate our model on a challenging temporal commonsense reasoning dataset. Our experiments show that the method that takes into account the structure of the target dataset, in addition to time-related features, improved the accuracy by approximately 4.8% compared to the standard fine-tuning approach. This confirms that tailoring a pre-trained language model to the domain of a target task is effective.

关键词： temporal commonsense reasoning pre-trained language model masked language modeling MC-TACO

来源：评论

学校读者我要写书评

暂无评论

EA-MLM: Error-aware masked language modeling for Grammatical Error Correction

EA-MLM: Error-aware Masked Language Modeling for Grammatical...

引用

25th International Conference on Asian language Processing (IALP)

作者： He, Junyi Li, Xia Su, Han Chen, Xinyin Yang, Hao Chen, Minghao Guangdong Univ Foreign Studies Sch Informat Sci & Technol Guangzhou Key Lab Multilingual Intelligent Proc Guangzhou Peoples R China

ISBN: (纸本)9781665483117

In recent years, BERT has been used in the task of grammatical error correction (GEC) and achieved good performance. However, few previous studies have investigated the incorporation of real grammatical errors into BERT for the GEC task. We argue that the distribution of GEC data (containing several types of errors) is different from the distribution of BERT pre-training data (usually error-free). To fill this gap, in this paper, we extend masked language modeling and propose a novel error-aware masked language modeling strategy (EA-MLM) to fine-tune BERT so that the representation distribution of the pretrained BERT is better adapted to the GEC task. We conduct extensive experiments on public datasets and the experimental results demonstrate the effectiveness of our proposed method.

关键词： Grammatical Error Correction BERT Fine-tuning masked language modeling Error-aware Masking Schemes

来源：评论

学校读者我要写书评

暂无评论

Assisting Text Localization and Transcreation Tasks Using AI-Based masked language modeling 2nd

Assisting Text Localization and Transcreation Tasks Using AI...

引用

2nd International Conference on Artificial Intelligence in HCI (AI-HCI) Held as Part of the 23rd International Conference on Human-Computer Interaction (HCII)

作者： Qian, Ming Liu, Jessie Soar Technol Ann Arbor MI 48105 USA Middlebury Inst Int Studies Monterey Translat & Interpretat Program Monterey CA USA

ISBN: (纸本)9783030777715;9783030777722

Localization refers to the adaptation of a document's content to meet the linguistic, cultural, and other requirements of a specific target market-a locale. Transcreation describes the process of adapting a message from one language to another, while maintaining its intent, style, tone, and context. In recent years, pre-trained language models have pushed the limits of natural language understanding and generation and dominated the NLP progress. We foresee that the AI-based pre-trained language models (e.g. masked language modeling) and other existing and upcoming language modeling techniques will be integrated as effective tools to support localization/transcreation efforts in the coming years. To support localization/transcreation tasks, we use AI-based masked language modeling (MLM) to provide a powerful human-machine teaming tool to query language models for the most proper words/phrases to reflect the proper linguistical and cultural characteristics of the target language. For linguistic applications, we list examples on logical connectives, pronouns and antecedents, and unnecessary redundant nouns and verbs. For intercultural conceptualization applications, we list examples of cultural event schema, role schema, emotional schema, and propositional schema. There are two possible approaches to determine where to put masks: a human-based approach or an algorithm-based approach. For the algorithm-based approach, constituency parsing can be used to break a text into sub-phrases, or constituents, after which typical linguistic patterns can be detected and then finally masking tasks can be attempted on the related texts.

关键词： Human-machine teaming language modeling masked language modeling Translation assistance tool Writing assistance tool Cross-lingual writing assistance Post editing Localization Transcreation Cross-lingual Inter-culture

来源：评论

学校读者我要写书评

暂无评论

Cultural Understanding Using In-context Learning and masked language modeling 23rd

Cultural Understanding Using In-context Learning and Masked ...

引用

23rd International Conference on Human-Computer Interaction (HCII)

作者： Qian, Ming Newton, Charles Qian, Davis Soar Technol Orlando FL 32817 USA Univ N Carolina Chapel Hill NC 27515 USA

ISBN: (纸本)9783030909635;9783030909628

With the rapid advancement of natural language processing (NLP) as a sub-field of artificial intelligence (AI), a number of unsupervised pre-trained language models trained on large corpus have become available (e.g. BERT and GPT-3). While these models have tremendous linguistic knowledge, a lot of other types of knowledge are embedded in them as well. We perform cross-culture analysis experiments using AI-based masked language modeling (MLM) and GPT-based Generative language modeling (In-context learning modeling). The designed approach is to set up a cultural context in sentences with masked words (for MLM) or in a human-prompted text segment (for GPT-based NLG). Consequently, the predicted masked words or the machine generated stories will reflect measurable intercultural differences because language models are trained on different corpus in different languages, and on English corpus containing a significant amount of knowledge on foreign cultures. We show a variety of examples: geopolitical knowledge, holidays, gestures, customs, social norms, emotion schema, role schema, procedure schema, and emotion change detection based on a diplomatic speech. The deep learning neural network model encodes its knowledge in the weights of a neural network instead of as organized semantic concepts. The model can reflect biases brought in by the training data and can give us inaccurate or faulty answers. Overall, with the rapid advancement of language technology, pre-trained language models have grown more powerful, and have great potential to serve as a culturalization tool.

关键词： Cultural understanding language modeling masked language modeling Generative language modeling Cross-lingual writing assistance Culturalization Localization Cross-lingual Inter-culture

来源：评论

学校读者我要写书评

暂无评论

Visual context learning based on cross-modal knowledge for continuous sign language recognition

引用

VISUAL COMPUTER 2025年第6期41卷 4017-4031页

作者： Liu, Kailin Hou, Yonghong Guo, Zihui Yin, Wenjie Ren, Yi Tianjin Univ Sch Elect & Informat Engn Tianjin 300072 Peoples R China Tianjin Chengjian Univ Sch Comp & Informat Engn Tianjin 300384 Peoples R China Chinese Acad Sci Sci Technol Integrated Informat Syst Lab Inst Software Beijing 100190 Peoples R China

Continuous sign language recognition (CSLR) aims to recognize glosses in sign language videos. Unfortunately, existing methods ignore the linguistic knowledge implicitly contained in gloss (symbol label) sequences in the process of improving the extraction of visual features. In this paper, we propose a visual context learning network based on cross-modal knowledge for CSLR that utilizes the semantic information contained in glosses to bridge the semantic gap between sign language videos and glosses. A special cross-attention mechanism is introduced in the framework to ensure effective cross-modal knowledge exchange. Specifically, we construct two auxiliary tasks, namely spatiotemporal semantic modeling (SSM) and masked language modeling (MLM). SSM enhances the video representation by using a joint cross-attention encoder with semantic supervision. Moreover, MLM is applied to learn fine-grained contextual interactions between video and text. Experimental results on three public CSLR datasets (PHOENIX-2014, PHOENIX-2014-T, and CSL-Daily) show the effectiveness of the proposed method, improving over the previous state-of-the-art by 2.0 and 1.9%\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\%$$\end{document} error rate on CSL-Daily, respectively. Code is publicly available at https://***/Liuklin/VCL.

关键词： Continuous sign language recognition Spatiotemporal semantics masked language modeling Sequence learning

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：