检索结果-内蒙古大学图书馆

CLAVE: A deep learning model for source code authorship verification with contrastive learning and transformer encoders

引用

INFORMATION PROCESSING & MANAGEMENT 2025年第3期62卷

作者： alvarez-Fidalgo, David Ortin, Francisco Univ Oviedo Comp Sci Dept C Leopoldo Calvo Sotelo 18 Oviedo 33007 Spain Munster Technol Univ Comp Sci Dept Rossa Ave Cork Ireland

Source code authorship verification involves determining whether two code fragments are written by the same programmer. It has many uses, including malware authorship analysis, copyright dispute resolution and plagiarism detection. Source code authorship verification is challenging because it must generalize to code written by programmers not included in its training data. In this paper, we present CLAVE (Contrastive Learning for Authorship Verification with encoder representations), a novel deep learning model for source code authorship verification that leverages contrastive learning and a transformer encoder-based architecture. We initially pre-train CLAVE on a dataset of 270,602 Python source code files extracted from GitHub. Subsequently, we fine-tune CLAVE for authorship verification using contrastive learning on Python submissions from 61,956 distinct programmers in Google Code Jam and Kick Start competitions. This approach allows the model to learn stylometric representations of source code, enabling comparison via vector distance for authorship verification. CLAVE achieves an AUC of 0.9782, reduces the error of the state-of-the-art source code authorship verification systems by at least 23.4% and improves the AUC of cutting-edge source code LLMs by 21.9% to 40%. We also evaluate the main components of CLAVE on its AUC performance improvement: pre-training (1.8%), loss function (0.2%-2.8%), input length (0.1%-0.7%), model size (0.2%), and tokenizer (0.1%-0.7%).

关键词： Source code authorship verification Contrastive learning transformer encoder Deep learning Stylometric representations Large language models Python

来源：评论

学校读者我要写书评

暂无评论

Zero-shot learning based on the fusion of global and local representations

引用

MEASUREMENT SCIENCE AND TECHNOLOGY 2025年第3期36卷 035905-035905页

作者： Qiang, Wang Mou, HongJin Jia, Wang Wei, Chunxiao Yu, Zhou UAV Ind Acad Chengdu Aeronaut Polytech Chengdu 610100 Sichuan Peoples R China Beihang Univ Dept Elect & Informat Engn Beijing 100191 Peoples R China Tianjin Univ Coll Intelligence & Comp Tianjin 300354 Peoples R China Guangdong Pharmaceut Univ Coll Med Informat Engn Guangzhou 510006 Guangdong Peoples R China

Zero-shot learning (ZSL) endeavors to extend knowledge to novel classes by capitalizing on the semantic overlap across different categories. Contemporary ZSL approaches concentrate on isolating image-specific local features pertinent to attributes and using these to align class semantic vectors. However, existing methods often overlook the integration of global and local representation, a synthesis that could significantly enhance zero-shot recognition accuracy. This paper introduces an innovative ZSL technique, termed ZSL based on the fusion of global representation and local representation (ZGLR). Our approach incorporates a transformer encoder constructed upon attribute prototypes to extract attribute-level features, which are then mapped to local representation to align with class semantic vectors. Concurrently, we introduce a discriminative semantic-visual mapping network that embeds class semantic vectors into the visual domain, thereby aligning global representation. During training, both global and local representation are optimized in tandem, while in the testing phase, the outcomes from both representational levels are consolidated to bolster classification precision. Practical results from three benchmark ZSL datasets demonstrate the superiority of our put-forward ZGLR solution.

关键词： zero-shot learning transformer encoder global representation local representation

来源：评论

学校读者我要写书评

暂无评论

HSTrans: Homogeneous substructures transformer for predicting frequencies of drug-side effects

引用

NEURAL NETWORKS 2025年 181卷 106779页

作者： Xu, Kaiyi Wang, Minhui Zou, Xin Liu, Jingjing Wei, Ao Chen, Jiajia Tang, Chang China Univ Geosci Sch Comp Sci Wuhan 430074 Peoples R China Lianshui Peoples Hosp Affiliated Kangda Coll Nanji Dept Pharm Lianshui Peoples Hosp Dept Pharm Huaian 223300 Peoples R China Tianjin Chest Hosp Dept Cardiac Surg Tianjin 300222 Peoples R China Tianjin Chest Hosp Dept Cardiol Tianjin 300222 Peoples R China

Identifying the frequencies of drug-side effects is crucial for assessing drug risk-benefit. However, accurately determining these frequencies remains challenging due to the limitations of time and scale in clinical randomized controlled trials. As a result, several computational methods have been proposed to address these issues. Nonetheless, two primary problems still persist. Firstly, most of these methods face challenges in generating accurate predictions for novel drugs, as they heavily depend on the interaction graph between drugs and side effects (SEs) within their modeling framework. Secondly, some previous methods often simply concatenate the features of drugs and SEs, which fails to effectively capture their underlying association. In this work, we present HSTrans, a novel approach that treats drugs and SEs assets of substructures, leveraging a transformer encoder for unified substructure embedding and incorporating an interaction module for association capture. Specifically, HSTrans extracts drug substructures through a specialized algorithm and identifies effective substructures for each SE by employing an indicator that measures the importance of each substructure and SE. Additionally, HSTrans applies convolutional neural network (CNN) in the interaction module to capture complex relationships between drugs and SEs. Experimental results on datasets from Galeano et al.'s study demonstrate that the proposed method outperforms other state-of-the-art approaches. The demo codes for HSTrans are available at https://***/Dtdtxuky/HSTrans/tree/master.

关键词： Drug-side effects prediction transformer encoder Feature fusion

来源：评论

学校读者我要写书评

暂无评论

Fusing Micro- and Macro-Scale Information to Predict Anticancer Synergistic Drug Combinations

引用

IEEE JOURNAL OF BIOMEDICAL AND HEALTH INFORMATICS 2025年第3期29卷 2297-2309页

作者： Wang, Xiaowen Zhu, Hongming Liu, Qi Liu, Qin Tongji Univ Sch Software Engn Shanghai 201804 Peoples R China Tongji Univ Tongji HospFrontier Sci Ctr Stem Cell Res Frontier Sci Ctr Stem Cell ResMinist EducBioinfo Sch MedKey Lab Spine & Spinal Cord Injury Repair Shanghai 200092 Peoples R China

Drug combination therapy is highly regarded in cancer treatment. Computational methods offer a time- and cost-effective opportunity to explore the vast combination space. Although deep learning-based prediction methods lead the field, their generalization ability remains unsatisfactory. Few previous studies have the ability to finely characterize drugs and cell lines at both the micro-scale and macro-scale. Furthermore, the interaction of cross-scale information is often overlooked. These two points limit models' ability of predicting the synergism of drug combinations in cell lines. To address the issues, we propose a novel anticancer synergistic drug combination prediction method termed MMFSynergy in this article. The construction of MMFSynergy involves three phases. First, MMFSynergy pretrains two micro encoders and a macro graph encoder, which can capture micro- or macro-scale information from large volumes of unlabeled data and generate generic features for drugs and proteins. Second, it represents drugs and proteins by fusing cross-scale information through a self-supervised task. Finally, it employs a transformer encoder-based model to predict synergy scores, taking representations of drugs in the combinations and the associated proteins of cell lines as input. We compared our method with eight advanced methods across three typical scenarios based on two public datasets. The results consistently demonstrated that the proposed method's generalization ability outperforms six advanced methods'. We also conducted experiments including but not limited to ablation study and case study to further exhibit the effectiveness of MMFSynergy.

关键词： Drugs Predictive models Proteins Data models Computational modeling Biological system modeling Diffusion tensor imaging Computer architecture transformers Microprocessors Combination therapy drug combination prediction pretraining multi-scale feature fusion transformer encoder

来源：评论

学校读者我要写书评

暂无评论

EnsembleSleepNet: a novel ensemble deep learning model based on transformers and attention mechanisms using multimodal data for sleep stages classification

引用

APPLIED INTELLIGENCE 2025年第7期55卷 1-21页

作者： Mostafaei, Sahar Hassanzadeh Tanha, Jafar Sharafkhaneh, Amir Univ Tabriz Fac Elect & Comp Engn POB 51666-16471 Tabriz Iran Baylor Coll Med Sect Pulm Crit Care & Sleep Med Dept Med Med Houston TX USA

Classifying sleep stages using biological signals is an important and challenging task in sleep medicine. Combining deep learning networks with transformers and attention mechanisms represents a powerful approach for achieving high-performance results in classification tasks. Multimodal learning, which integrates various types of input data, can significantly enhance the classification performance of these networks. However, many existing studies either rely on single-modal data or design a single model to handle different signals and modalities without considering the unique characteristics of each data type, which often fails to capture optimal features. To address this limitation, we propose an ensemble model for sleep stage classification that leverages multimodal data, including raw signals, spectrograms, and handcrafted features. We utilize the Sleep Heart Health Study (SHHS) dataset by selecting multiple signals from polysomnography recordings. Our approach develops three specialized sub-models with different layers and components, each designed based on the unique characteristics of specific data types and signals, and integrates them into a unified ensemble deep learning framework. The proposed EnsembleSleepNet achieved comparable performance against existing methods by obtaining high values of 0.897, 0.852, and 0.831 in accuracy, Cohen's kappa (kappa), and Macro F1 score (MF1) respectively. Additionally, ablation studies revealed the impact of the selected signals and components in our developed model.

关键词： Deep Learning Attention mechanism Ensemble models transformer encoder Multimodal data Sleep Staging Polysomnography

来源：评论

学校读者我要写书评

暂无评论

Improving multi-step dissolved oxygen prediction in aquaculture using adaptive temporal convolution and optimized transforme

引用

COMPUTERS AND ELECTRONICS IN AGRICULTURE 2025年 235卷

作者： Shao, Kaixuan Li, Daoliang Tang, Hao Zhang, Yonghui Xu, Bo Bhatti, Uzair Aslam Hainan Univ Sch Informat & Commun Engn 58 Renmin Ave Haikou 570228 Hainan Peoples R China China Agr Univ Natl Innovat Ctr Digital Fishery 17 Qinghua East Rd Beijing 100083 Peoples R China China Agr Univ Coll Informat & Elect Engn 17 Qinghua East Rd Beijing 100083 Peoples R China

Accurate dissolved oxygen (DO) prediction is crucial for optimizing aquaculture efficiency. However, existing forecasting methods often struggle to capture periodic patterns, model complex feature dependencies, and maintain high accuracy in multi-step predictions. To address these challenges, this study proposes an enhanced transformer-based model designed to improve both prediction accuracy and stability. The model first integrates an Adaptive Temporal Convolutional Network (ATCN) to extract periodic patterns and local temporal features from time-series data. Then, a transformer encoder with linear attention mechanisms and relative position encoding is employed to enhance feature extraction and sequence modeling. To further strengthen temporal dependency learning, the conventional decoder is replaced with a modified Gated Recurrent Unit (GRU), and a linear regression-based error correction mechanism is introduced to refine multi-step forecasting accuracy. Experimental results on Public Dataset 3 demonstrate that the proposed model achieves an average MAE of 1.624 and RMSE of 2.254, reflecting improvements of 45.06% and 40.57%, respectively, compared to the baseline transformer model. These findings highlight the model's ability to effectively capture complex temporal dependencies, significantly enhancing DO prediction accuracy and robustness in multi-step forecasting tasks. Additionally, the model demonstrates strong performance in pH prediction, underscoring its potential for multiparameter water quality forecasting. This work provides valuable insights for improving water quality management and mitigating risks in aquaculture.

关键词： Dissolved oxygen prediction Aquaculture transformer encoder Adaptive Temporal Convolutional Network Time series prediction

来源：评论

学校读者我要写书评

暂无评论

PE-DOCC: A Novel Periodicity-Enhanced Deep One-Class Classification Framework for Electricity Theft Detection

引用

APPLIED SCIENCES-BASEL 2025年第4期15卷

作者： Wu, Zhijie Wang, Yufeng Nanjing Univ Posts & Telecommun Sch Commun & Informat Engn Nanjing 210003 Peoples R China

Electricity theft, emerging as one of the severe cyberattacks in smart grids, causes significant economic losses. Due to the powerful expressive ability of deep neural networks (DNN), supervised and unsupervised DNN-based electricity theft detection (ETD) schemes have experienced widespread deployment. However, existing works have the following weak points: Supervised DNN-based schemes require abundant labeled anomalous samples for training, and even worse, cannot detect unseen theft patterns. To avoid the extensively labor-consuming activity of labeling anomalous samples, unsupervised DNNs-based schemes aim to learn the normality of time-series and infer an anomaly score for each data instance, but they fail to capture periodic features effectively. To address these challenges, this paper proposes a novel periodicity-enhanced deep one-class classification framework (PE-DOCC) based on a periodicity-enhanced transformer encoder, named Periodicformer encoder. Specifically, within the encoder, a novel criss-cross periodic attention is proposed to capture both horizontal and vertical periodic features. The Periodicformer encoder is pre-trained by reconstructing partially masked input sequences, and the learned latent representations are then fed into a one-class classification for anomaly detection. Extensive experiments on real-world datasets demonstrate that our proposed PE-DOCC framework outperforms state-of-the-art unsupervised ETD methods.

关键词： electricity theft detection one-class classification transformer encoder unsupervised representation learning

来源：评论

学校读者我要写书评

暂无评论

A new dual-channel transformer-based network for remaining useful life prediction

引用

MEASUREMENT SCIENCE AND TECHNOLOGY 2025年第2期36卷 026119-026119页

作者： Yang, Kai Wei, Yuxuan Ma, Yubao Huang, Lehong Tang, Qiang Li, Zhiguo Chinese Acad Sci Xian Inst Opt & Precis Mech Xian Shaanxi Peoples R China Univ Chinese Acad Sci Beijing Peoples R China Chinese Acad Sci Key Lab Space Precis Measurement Technol Xian Shaanxi Peoples R China

Remaining useful life (RUL) prediction is crucial for enhancing the reliability of complex systems, preventing unexpected failures, and reducing the maintenance costs of electromechanical systems. Data-driven methods, particularly deep learning, have been extensively applied in RUL prediction. However, recent methods have focused solely on long-term features, neglecting the significant role of short-term features in long sequence processes, which leads to an inability to achieve high-precision RUL prediction. In this article, a novel dual-channel network is proposed, which is based on the transformer encoder. Additionally, the approach proposes a robust dual-channel feature extraction layer that integrates a multi-scale convolution block and a gating convolution block. This integrated system is capable of capturing both long-term and short-term features from the raw data simultaneously. Compared to convolutional neural networks, recurrent neural networks and other transformer-based methods, the proposed method is more effective at extracting profound features and focusing on important information in the input. Utilizing the self-attention mechanism, the proposed model captures degradation features and achieves precise RUL prediction. Experiments conducted on the CMAPSS dataset demonstrate that the proposed method outperforms recent RUL prediction methods. The average root mean square error of the proposed method is reduced by 6.8%, and the average score is reduced by 8.5%, proving the effectiveness of the proposed approach.

关键词： remaining useful life transformer encoder gating mechanism turbofan engine

来源：评论

学校读者我要写书评

暂无评论

A Multi-Granularity Features Representation and Dimensionality Reduction Network for Website Fingerprinting

引用

IEEE ACCESS 2025年 13卷 574-587页

作者： Ding, Yaojun Hu, Bingxuan Gansu Univ Polit Sci & Law Sch Artificial Intelligence Lanzhou 730070 Peoples R China Gansu Univ Polit Sci & Law Sch Cyberspace Secur Lanzhou 730070 Peoples R China

Website fingerprinting (WF) refers to the identification of target websites accessed by users in anonymous communication scenarios, playing a critical role in cybercrime investigation and forensics. In recent years, deep learning-based WF has become a research focus in this field. Although these methods can achieve high accuracy, their performance tends to decline over time. This degradation results from the evolution of network protocol versions and the ongoing development of obfuscation techniques, a phenomenon known as concept drift. To address the problem of concept drift, this paper presents a multi-granularity features representation and dimensionality reduction network for Website Fingerprinting, referred to as LRCT. The LRCT network effectively leverages the temporal learning advantages of Local Recurrent Networks (Local RNN) and the spatial learning strengths of Convolutional Neural Network (CNN) by designing the local feature extraction block (denoted as LRC Block), which extracts fine-grained local features from 2000-dimensional original sequences and reduces the dimensionality to 125. The network then uses a transformer encoder to capture more robust global features from the low-dimensional data. Experimental results comparing LRCT with state-of-the-art WF methods demonstrate that the proposed LRCT achieves 99.34% accuracy in a closed-world scenario, outperforming other benchmark models. Compared to the benchmark WF-transformer method, the LRCT network reduces the parameter count by half and decreases the average training time per epoch to approximately 20% of the benchmark. Even under concept drift conditions, LRCT maintains an accuracy of 89.9%. In open-world scenarios, LRCT also has better precision and recall.

关键词： Feature extraction Fingerprint recognition Accuracy Computer architecture Monitoring Convolutional neural networks Concept drift Microprocessors transformers Long short term memory Website fingerprinting local recurrent neural network transformer encoder traffic analysis

来源：评论

学校读者我要写书评

暂无评论

Large language models based vulnerability detection: How does it enhance performance?

引用

INTERNATIONAL JOURNAL OF INFORMATION SECURITY 2025年第1期24卷 1-18页

作者： Xuan, Cho Do Quang, Dat Bui Quang, Vinh Dang Posts & Telecommun Inst Technol Dept Informat Secur Hanoi Vietnam Posts & Telecommun Inst Technol Dept Informat Technol Hanoi Vietnam

Detecting vulnerabilities in C/C + + source code has become a critical challenge in information security, especially as the growing number and severity of new vulnerabilities increasingly impact organizations. In this context, Large Language Models (LLMs) have emerged as a promising approach;however, building a model capable of effectively predicting and classifying various types of vulnerabilities from diverse datasets remains a complex problem, demanding innovative and comprehensive solutions. Our research proposes a breakthrough approach by developing the FG-CVD ensemble learning model, an advanced architecture that combines code embedding techniques and Knowledge Argument to enhance feature representation and the ability to learn complex relationships within source code. These improvements are specifically designed on the foundation of code embedding and the transformer architecture of LLMs to boost the detection and classification of sophisticated vulnerability patterns. To evaluate the model's effectiveness, we conducted extensive experiments on four representative datasets: Reveal, BigVul, RealVul, and FFMQ + QEmu. The experimental results demonstrated FG-CVD's superior performance with an average accuracy of 85%, a prediction precision of 43%, a recall of 65%, and an F1-score of 47%. Notably, the model exhibited flexible adaptability to datasets with different structures and efficiently addressed data imbalance between labels. Moreover, through rigorous cross-dataset testing, the model showcased strong generalization capabilities and high stability, underscoring not only the academic value of the approach but also its practical potential, outperforming traditional approaches across a range of metrics and experimental scenarios.

关键词： Software vulnerability detection Embedding code transformer encoder Large language models Ensemble learning

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：