检索结果-内蒙古大学图书馆

Incorporating BERT into parallel sequence decoding with adapters 20

学校读者我要写书评

暂无评论

Incorporating BERT into parallel sequence decoding with adap...

Proceedings of the 34th International Conference on Neural Information Processing Systems

作者： Junliang Guo Zhirui Zhang Linli Xu Hao-Ran Wei Boxing Chen Enhong Chen Anhui Province Key Laboratory of Big Data Analysis and Application School of Computer Science and Technology University of Science and Technology of China Alibaba DAMO Academy Anhui Province Key Laboratory of Big Data Analysis and Application School of Computer Science and Technology University of Science and Technology of China and IFLYTEK Co. Ltd.

ISBN: (纸本)9781713829546

While large scale pre-trained language models such as BERT [5] have achieved great success on various natural language understanding tasks, how to efficiently and effectively incorporate them into sequence-to-sequence models and the corresponding text generation tasks remains a non-trivial problem. In this paper, we propose to address this problem by taking two different BERT models as the encoder and decoder respectively, and fine-tuning them by introducing simple and lightweight adapter modules, which are inserted between BERT layers and tuned on the task-specific dataset. In this way, we obtain a flexible and efficient model which is able to jointly leverage the information contained in the source-side and target-side BERT models, while bypassing the catastrophic forgetting problem. Each component in the framework can be considered as a plug-in unit, making the framework flexible and task agnostic. Our framework is based on a parallel sequence decoding algorithm named Mask-Predict [8] considering the bi-directional and conditional independent nature of BERT, and can be adapted to traditional autoregressive decoding easily. We conduct extensive experiments on neural machine translation tasks where the proposed method consistently outperforms autoregressive baselines while reducing the inference latency by half, and achieves 36.49/33.57 BLEU scores on IWSLT14 German-English/WMT14 German-English translation. When adapted to autoregressive decoding, the proposed method achieves 30.60/43.56 BLEU scores on WMT14 English-German/English-French translation, on par with the state-of-the-art baseline models.

关键词：

application of a neural network model with multimodal fusion for fluorescence spectroscopy

学校读者我要写书评

暂无评论

Nuclear Science and Techniques 2024年第10期35卷 135-148页

作者： Lin Tang Shuang Zhou Kai-Bo Shi Hong-Tao Shen Lei You College of Electronic Information and Electrical Engineering Chengdu UniversityChengdu 610106China Guangxi Key Laboratory of Nuclear Physics and Nuclear Technology Guangxi Normal UniversityGuilin 541004China National Engineering Research Center for Agro-Ecological Big Data Analysis&Application Anhui UniversityHefei 230039China School of Electrical and Electronic Engineering Nanyang Technological UniversitySingapore 639798Singapore Vanadium and Titanium Resource Comprehensive Utilization Key Laboratory of Sichuan Province Panzhihua UniversityPanzhihua 617000China Key Laboratory of Interior Layout optimization and Security Institutions of Higher Education of Sichuan ProvinceChengdu Normal UniversityChengdu 611130China

In energy-dispersive X-ray fluorescence spectroscopy,the estimation of the pulse amplitude determines the accuracy of the spectrum *** error generated by the amplitude estimation of the pulse output distorted by the measurement system leads to false peaks in the measured *** eliminate these false peaks and achieve an accurate estimation of the distorted pulse amplitude,a composite neural network model is proposed,which embeds long and short-term memory(LSTM)into the UNet *** UNet network realizes the fusion of pulse sequence features and the LSTM model realizes pulse amplitude *** model is trained using simulated pulse datasets with different amplitudes and distortion *** the pulse height estimation,the average relative error of the trained model on the test set was approximately 0.64%,which is 27.37% lower than that of the traditional trapezoidal shaping *** processing of a standard iron source further validated the pulse height estimation performance of the UNet-LSTM *** estimating the amplitude of the distorted pulses using the model,the false peak area was reduced by approximately 91% over the full spectrum and was corrected to the characteristic peak region of interest(ROI).The corrected peak area accounted for approximately 1.32%of the characteristic peak ROI *** results indicate that the model can accurately estimate the height of distorted pulses and has substantial corrective effects on false peaks.

关键词： UNet Long-and short-term memory Pulse distortion Pulse height estimation Fluorescent spectroscopy

FairLISA: fair user modeling with limited sensitive attributes information 23

学校读者我要写书评

暂无评论

FairLISA: fair user modeling with limited sensitive attribut...

Proceedings of the 37th International Conference on Neural Information Processing Systems

作者： Zheng Zhang Qi Liu Hao Jiang Fei Wang Yan Zhuang Le Wu Weibo Gao Enhong Chen Anhui Province Key Laboratory of Big Data Analysis and Application University of Science and Technology of China and State Key Laboratory of Cognitive Intelligence Hefei University of Technology

User modeling techniques profile users' latent characteristics (e.g., preference) from their observed behaviors, and play a crucial role in decision-making. Unfortunately, traditional user models may unconsciously capture biases related to sensitive attributes (e.g., gender) from behavior data, even when this sensitive information is not explicitly provided. This can lead to unfair issues and discrimination against certain groups based on these sensitive attributes. Recent studies have been proposed to improve fairness by explicitly decorrelating user modeling results and sensitive attributes. However, most existing approaches assume that fully sensitive attribute labels are available in the training set, which is unrealistic due to collection limitations like privacy concerns, and hence bear the limitation of performance. In this paper, we focus on a practical situation with limited sensitive data and propose a novel FairLISA framework, which can efficiently utilize data with known and unknown sensitive attributes to facilitate fair model training. We first propose a novel theoretical perspective to build the relationship between data with both known and unknown sensitive attributes with the fairness objective. Then, based on this, we provide a general adversarial framework to effectively leverage the whole user data for fair user modeling. We conduct experiments on representative user modeling tasks including recommender system and cognitive diagnosis. The results demonstrate that our FairLISA can effectively improve fairness while retaining high accuracy in scenarios with different ratios of missing sensitive attributes.

关键词：

Drug package recommendation via interaction-aware graph induction

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Zheng, Zhi Wang, Chao Xu, Tong Shen, Dazhong Qin, Penggang Huai, Baoxing Liu, Tongzhu Chen, Enhong Anhui Province Key Lab of Big Data Analysis and Application University of Science and Technology of China China Huawei Technologies China First Affiliated Hospital of USTC China

Recent years have witnessed the rapid accumulation of massive electronic medical records (EMRs), which highly support the intelligent medical services such as drug recommendation. However, prior arts mainly follow the traditional recommendation strategies like collaborative filtering, which usually treat individual drugs as mutually independent, while the latent interactions among drugs, e.g., synergistic or antagonistic effect, have been largely ignored. To that end, in this paper, we target at developing a new paradigm for drug package recommendation with considering the interaction effect within drugs, in which the interaction effects could be affected by patient conditions. Specifically, we first design a pre-training method based on neural collaborative filtering to get the initial embedding of patients and drugs. Then, the drug interaction graph will be initialized based on medical records and domain knowledge. Along this line, we propose a new Drug Package Recommendation (DPR) framework with two variants, respectively DPR on Weighted Graph (DPR-WG) and DPR on Attributed Graph (DPR-AG) to solve the problem, in which each the interactions will be described as signed weights or attribute vectors. In detail, a mask layer is utilized to capture the impact of patient condition, and graph neural networks (GNNs) are leveraged for the final graph induction task to embed the package. Extensive experiments on a real-world data set from a first-rate hospital demonstrate the effectiveness of our DPR framework compared with several competitive baseline methods, and further support the heuristic study for the drug package generation task with adequate performance. Copyright © 2021, The Authors. All rights reserved.

关键词： Multilayer neural networks

Queue Length Prediction Method of Short-linked Intersections Based on Long Short-Term Memory 9

学校读者我要写书评

暂无评论

Queue Length Prediction Method of Short-linked Intersections...

9th International Conference on Systems and Informatics, ICSAI 2023

作者： Shi, Kang Shi, Kang Yang, Jun Liang, Zi Jun Jun Liang, Zi Li, Yu Qi Wang, Rui Han Tongji University Key Laboratory of Road and Traffic Engineering Ministry of Education Shanghai201804 China Anhui Chaoyuan Information Technology Co. Ltd HeFei230088 China Anhui Province Transportation Big Data Analysis and Application Engineering Laboratory Hefei230601 China Hefei University School of Urban Construction and Transportation Hefei230601 China

ISBN: (纸本)9798350383706

To address traffic congestion and vehicle queue overflow at short-link intersections, it is essential to conduct a predictive analysis of vehicle queue length between these intersections. This helps in proactively implementing appropriate traffic management measures to prevent traffic overflow and congestion. In this paper, we propose a short-link intersection vehicle queue length prediction method based on Long Short-Term Memory (LSTM). First, historical traffic data from the morning peak hours over the past month is collected. After comparing multiple machine learning models, LSTM is chosen for traffic flow prediction. Subsequently, mathematical models are used to predict queue length at future time intervals. Through practical validation, we found that the proposed method for predicting queue length between short-link intersections in this paper is effective, with an average prediction error rate of 11.9%. It meets the requirements for predicting vehicle queue length between intersections and proactively controlling them, holding significant practical significance for preventing traffic flow and congestion events. © 2023 IEEE.

关键词： Long short-term memory

Aftershock Detection with Multi-scale Description Based Neural Network

学校读者我要写书评

暂无评论

Aftershock Detection with Multi-scale Description Based Neur...

IEEE International Conference on data Mining (ICDM)

作者： Qi Zhang Tong Xu Hengshu Zhu Lifu Zhang Hui Xiong Enhong Chen Qi Liu Anhui Province Key Lab of Big Data Analysis and Application University of Science and Technology of China Baidu Talent Intelligence Center Baidu Inc Business Intelligence Lab Baidu Research

Aftershocks refer to the smaller earthquakes that occur following large earthquakes, in the same area of the main shock. The task of aftershocks detection, as a crucial and challenging issue in disaster monitoring, has attracted wide research attention in relevant fields. Compared with the traditional detection methods like STA/LTA algorithms or heuristic matching, neural network techniques are regarded as an advanced choice with better pattern recognition ability. However, current neural network-based solutions mainly formulate the seismic wave as ordinary time series, where existing techniques are directly deployed without adaption, and thus fail to obtain competitive performance on the intensive and highly-noise waveforms of aftershocks. To that end, in this paper, we propose a novel framework named Multi-Scale Description based Neural Network (MSDNN) for enhancing aftershock detection. Specifically, MSDNN contains a delicately-designed network structure for capturing both short-term scale and long-term scale seismic features. Therefore, the unique characteristics of seismic waveforms can be fully-exploited for aftershock detection. Furthermore, a multi-task learning strategy is introduced to model the seismic waveforms of multiple monitoring stations simultaneously, which can not only refine the detection performance but also provide additionally quantitative clues for discovering homologous earthquakes. Finally, comprehensive experiments on the data set from aftershocks of the Wenchuan M8.0 Earthquake have clearly validated the effectiveness of our framework compared with several state-of-the-art baselines.

关键词：

Preference Enhanced Social Influence Modeling for Network-Aware Cascade Prediction

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Wu, Likang Wang, Hao Chen, Enhong Li, Zhi Zhao, Hongke Ma, Jianhui Anhui Province Key Laboratory of Big Data Analysis and Application School of Computer Science and Technology University of Science and Technology of China State Key Laboratory of Cognitive Intelligence Anhui Hefei China Anhui Province Key Laboratory of Big Data Analysis and Application School of Data Science University of Science and Technology of China State Key Laboratory of Cognitive Intelligence Anhui Hefei China College of Management and Economics Tianjin University Tianjin China

Network-aware cascade size prediction aims to predict the final reposted number of user-generated information via modeling the propagation process in social networks. Estimating the user's reposting probability by social influence, namely state activation plays an important role in the information diffusion process. Therefore, Graph Neural Networks (GNN), which can simulate the information interaction between nodes, has been proved as an effective scheme to handle this prediction task. However, existing studies including GNN-based models usually neglect a vital factor of user's preference which influences the state activation deeply. To that end, we propose a novel framework to promote cascade size prediction by enhancing the user preference modeling according to three stages, i.e., preference topics generation, preference shift modeling, and social influence activation. Our end-to-end method makes the user activating process of information diffusion more adaptive and accurate. Extensive experiments on two large-scale real-world datasets have clearly demonstrated the effectiveness of our proposed model compared to state-of-the-art baselines. Copyright © 2022, The Authors. All rights reserved.

关键词： Forecasting

GraphMI: Extracting private graph data from graph neural networks

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Zhang, Zaixi Liu, Qi Huang, Zhenya Wang, Hao Lu, Chengqiang Liu, Chuanren Chen, Enhong Anhui Province Key Lab. of Big Data Analysis and Application School of Computer Science and Technology University of Science and Technology of China Alibaba Group University of Tennessee Knoxville

As machine learning becomes more widely used for critical applications, the need to study its implications in privacy turns to be urgent. Given access to the target model and auxiliary information, model inversion attack aims to infer sensitive features of the training dataset, which leads to great privacy concerns. Despite its success in grid-like domains, directly applying model inversion techniques on non-grid domains such as graph achieves poor attack performance due to the difficulty to fully exploit the intrinsic properties of graphs and attributes of nodes used in Graph Neural Networks (GNN). To bridge this gap, we present Graph Model Inversion attack (GraphMI), which aims to extract private graph data of the training graph by inverting GNN, one of the state-of-the-art graph analysis tools. Specifically, we firstly propose a projected gradient module to tackle the discreteness of graph edges while preserving sparsity and smoothness of graph features. Then we design a graph auto-encoder module to efficiently exploit graph topology, node attributes, and target model parameters for edge inference. With the proposed methods, we study the connection between model inversion risk and edge influence and show that edges with greater influence are more likely to be recovered. Extensive experiments over several public datasets demonstrate the effectiveness of our method. We also show that differential privacy in its canonical form can hardly defend our attack while preserving decent utility. Copyright © 2021, The Authors. All rights reserved.

关键词： Graph neural networks

ProtGNN: Towards self-explaining Graph Neural Networks

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Zhang, Zaixi Liu, Qi Wang, Hao Lu, Chengqiang Lee, Cheekong Anhui Province Key Lab. of Big Data Analysis and Application School of Computer Science and Technology University of Science and Technology of China China Tencent America United States

Despite the recent progress in Graph Neural Networks (GNNs), it remains challenging to explain the predictions made by GNNs. Existing explanation methods mainly focus on post-hoc explanations where another explanatory model is employed to provide explanations for a trained GNN. The fact that post-hoc methods fail to reveal the original reasoning process of GNNs raises the need of building GNNs with built-in interpretability. In this work, we propose Prototype Graph Neural Network (ProtGNN), which combines prototype learning with GNNs and provides a new perspective on the explanations of GNNs. In ProtGNN, the explanations are naturally derived from the case-based reasoning process and are actually used during classification. The prediction of ProtGNN is obtained by comparing the inputs to a few learned prototypes in the latent space. Furthermore, for better interpretability and higher efficiency, a novel conditional subgraph sampling module is incorporated to indicate which part of the input graph is most similar to each prototype in ProtGNN+. Finally, we evaluate our method on a wide range of datasets and perform concrete case studies. Extensive results show that ProtGNN and ProtGNN+ can provide inherent interpretability while achieving accuracy on par with the non-interpretable counterparts. © 2021, CC BY.

关键词： Graph neural networks