检索结果-内蒙古大学图书馆

Bio-K-Transformer: A pre-trained transformer-based sequence-to-sequence model for adverse drug reactions prediction

COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2025年 260卷 108524-108524页

作者： Qiu, Xihe Shao, Siyue Wang, Haoyu Tan, Xiaoyu Shanghai Univ Engn Sci Sch Elect & Elect Engn Shanghai Peoples R China INF Technol Shanghai Co Ltd Shanghai Peoples R China

Background and Objective: Adverse drug reactions (ADRs) pose a serious threat to patient health, potentially resulting in severe consequences, including mortality. Accurate prediction of ADRs before drug market release is crucial for early prevention. Traditional ADR detection, relying on clinical trials and voluntary reporting, has inherent limitations. Clinical trials face challenges in capturing rare and long-term reactions due to scale and time constraints, while voluntary reporting tends to neglect mild and common reactions. Consequently, drugs on the market may carry unknown risks, leading to an increasing demand for more accurate predictions of ADRs before their commercial release. This study aims to develop a more accurate prediction model for ADRs prior to drug market release. Methods: We frame the ADR prediction task as a sequence-to-sequence problem and propose the Bio-KTransformer, which integrates the transformer model with pre-trained models ( i.e. , Bio_ClinicalBERT and K-bert), to forecast potential ADRs. We enhance the attention mechanism of the Transformer encoder structure and adjust embedding layers to model diverse relationships between drug adverse reactions. Additionally, we employ a masking technique to handle target data. Experimental findings demonstrate a notable improvement in predicting potential adverse reactions, achieving a predictive accuracy of 90.08%. It significantly exceeds current state-of-the-art baseline models and even the fine-tuned Llama-3.1-8B and Llama3-Aloe-8B-Alpha model, while being cost-effective. The results highlight the model's efficacy in identifying potential adverse reactions with high precision, sensitivity, and specificity. Conclusion: The Bio-K-Transformer significantly enhances the prediction of ADRs, offering a cost-effective method with strong potential for improving pre-market safety evaluations of pharmaceuticals.

关键词： Adverse drug reactions Transformer Diagonal-masked sequence-to-sequence model

来源：评论

学校读者我要写书评

暂无评论

sequence-to-sequence model for Trajectory Planning of Nonprehensile Manipulation Including Contact model

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2018年第4期3卷 3606-3613页

作者： Kutsuzawa, Kyo Sakaino, Sho Tsuji, Toshiaki Saitama Univ Grad Sch Sci & Engn Saitama 3388570 Japan Saitama Univ JST PRESTO Saitama 3388570 Japan

Nonprehensile manipulation is necessary for robots to operate in humans' daily lives. As nonprehensile manipulation should satisfy both kinematics and dynamics requirements simultaneously, it is difficult to manipulate objects along given paths. Previous studies have considered the problems with sequence-to-sequence models, which are neural networks for time-series conversion. However, they did not consider nonlinear contact models, such as friction models. When we train the seq2seq models using end-to-end backpropagation, training losses vanish owing to static friction. In this letter, we realize sequence-to-sequence models for trajectory planning of nonprehensile manipulation including contact models between the robots and target objects. This letter proposes a training curriculum that commences training without contact models to bring the seq2seq models outside of the gradient-vanishing zone. This letter discusses sliding manipulation, which includes a friction model between objects and tools, such as frying pans fixed onto the robots. We validated the proposed curriculum through a simulation. In addition, we observed that the trained seq2seq models could handle parameter fluctuations that did not exist during training.

关键词： Deep learning in robotics and automation motion and path planning nonprehensile manipulation sequence-to-sequence model

来源：评论

学校读者我要写书评

暂无评论

sequence-to-sequence model with Attention for Time Series Classification 16

Sequence-to-Sequence Model with Attention for Time Series Cl...

引用

16th IEEE International Conference on Data Mining (ICDM)

作者： Tang, Yujin Xu, Jianfeng Matsumoto, Kazunori Ono, Chihiro KDDI R&D Labs Inc 2-1-15 Ohara Saitama 3568502 Japan

ISBN: (纸本)9781509059102

Encouraged by recent waves of successful applications of deep learning, some researchers have demonstrated the effectiveness of applying convolutional neural networks (CNN) to time series classification problems. However, CNN and other traditional methods require the input data to be of the same dimension which prevents its direct application on data of various lengths and multi-channel time series with different sampling rates across channels. Long short-term memory (LSTM), another tool in the deep learning arsenal and with its design nature, is more appropriate for problems involving time series such as speech recognition and language translation. In this paper, we propose a novel model incorporating a sequence-to-sequence model that consists two LSTMs, one encoder and one decoder. The encoder LSTM accepts input time series of arbitrary lengths, extracts information from the raw data and based on which the decoder LSTM constructs fixed length sequences that can be regarded as discriminatory features. For better utilization of the raw data, we also introduce the attention mechanism into our model so that the feature generation process can peek at the raw data and focus its attention on the part of the raw data that is most relevant to the feature under construction. We call our model S2SwA, as the short for sequence-to-sequence with Attention. We test S2SwA on both uni-channel and multi-channel time series datasets and show that our model is competitive with the state-of-the-art in real world tasks such as human activity recognition.

关键词： time series classification human activity recognition deep learning sequence-to-sequence model attention mechanism

来源：评论

学校读者我要写书评

暂无评论

Universal Lemmatizer: A sequence-to-sequence model for lemmatizing Universal Dependencies treebanks

引用

NATURAL LANGUAGE ENGINEERING 2021年第5期27卷 545-574页

作者： Kanerva, Jenna Ginter, Filip Salakoski, Tapio Univ Turku Dept Future Technol TurkuNLP Grp Turku Finland

In this paper, we present a novel lemmatization method based on a sequence-to-sequence neural network architecture and morphosyntactic context representation. In the proposed method, our context-sensitive lemmatizer generates the lemma one character at a time based on the surface form characters and its morphosyntactic features obtained from a morphological tagger. We argue that a sliding window context representation suffers from sparseness, while in majority of cases the morphosyntactic features of a word bring enough information to resolve lemma ambiguities while keeping the context representation dense and more practical for machine learning systems. Additionally, we study two different data augmentation methods utilizing autoencoder training and morphological transducers especially beneficial for low-resource languages. We evaluate our lemmatizer on 52 different languages and 76 different treebanks, showing that our system outperforms all latest baseline systems. Compared to the best overall baseline, UDPipe Future, our system outperforms it on 62 out of 76 treebanks reducing errors on average by 19% relative. The lemmatizer together with all trained models is made available as a part of the Turku-neural-parsing-pipeline under the Apache 2.0 license.

关键词： Lemmatization Universal Dependencies Parsing sequence-to-sequence model

来源：评论

学校读者我要写书评

暂无评论

A sequence-to-sequence model-based deep learning approach for recognizing activity of daily living for senior care

引用

JOURNAL OF BIOMEDICAL INFORMATICS 2018年 84卷 148-158页

作者： Zhu, Hongyi Chen, Hsinchun Brown, Randall Univ Arizona Dept Management Informat Syst Tucson AZ 85721 USA

Ensuring the health and safety of independent-living senior citizens is a growing societal concern. Researchers have developed sensor based systems to monitor senior citizens' Activity of Daily Living (ADL), a set of daily activities that can indicate their self-caring ability. However, most ADL monitoring systems are designed for one specific sensor modality, resulting in less generalizable models that is not flexible to account variations in real life monitoring settings. Current classic machine learning and deep learning methods do not provide a generalizable solution to recognize complex ADLs for different sensor settings. This study proposes a novel sequence-to-sequence model based deep-learning framework to recognize complex ADLs leveraging an activity state representation. The proposed activity state representation integrated motion and environment sensor data without labor-intense feature engineering. We evaluated our proposed framework against several state-of-the-art machine leaming and deep learning benchmarks. Overall, our approach outperformed baselines in most performance metrics, accurately recognized complex ADLs from different types of sensor input. This framework can generalize to different sensor settings and provide a viable approach to understand senior citizen's daily activity patterns with smart home health monitoring systems.

关键词： Activity of daily living ADL recognition Deep learning Activity state representation sequence-to-sequence model

来源：评论

学校读者我要写书评

暂无评论

Trajectory adjustment for nonprehensile manipulation using latent space of trained sequence-to-sequence model

引用

ADVANCED ROBOTICS 2019年第21期33卷 1144-1154页

作者： Kutsuzawa, K. Sakaino, S. Tsuji, T. Saitama Univ Grad Sch Sci & Engn Saitama Japan Saitama Univ Saitama Japan Univ Tsukuba Grad Sch Syst & Informat Engn Ibaraki Japan Univ Tsukuba JST PRESTO Ibaraki Japan

When robots are used to manipulate objects in various ways, they often have to consider the dynamic constraint. Machine learning is a good candidate for such complex trajectory planning problems. However, it sometimes does not satisfy the task objectives due to a change in the objective or a lack of guarantee that the objective functions will be satisfied. To overcome this issue, we applied a method of trajectory deformation by using sequence-to-sequence (seq2seq) models. We propose a method of adjusting the generated trajectories, by utilizing the architecture of seq2seq models. The proposed method optimizes the latent variables of the seq2seq models instead of the trajectories to minimize the given objective functions. The verification results show that the use of latent variables can obtain the desired trajectories faster than direct optimization of the trajectories.

关键词： Nonprehensile manipulation sequence-to-sequence model latent variable neural network motion optimization

来源：评论

学校读者我要写书评

暂无评论

SC-NER: A sequence-to-sequence model with Sentence Classification for Named Entity Recognition 23rd

SC-NER: A Sequence-to-Sequence Model with Sentence Classific...

引用

23rd Pacific-Asia Conference on Knowledge Discovery and Data Mining (PAKDD)

作者： Wang, Yu Li, Yun Zhu, Ziye Xia, Bin Liu, Zheng Nanjing Univ Posts & Telecommun Sch Comp Sci Jiangsu Key Lab Big Data Secur & Intelligent Proc Nanjing 210023 Jiangsu Peoples R China

ISBN: (纸本)9783030161484;9783030161477

Named Entity Recognition (NER) is a basic task in Natural Language Processing (NLP). Recently, the sequence-to-sequence (seq2seq) model has been widely used in NLP task. Different from the general NLP task, 60% sentences in the NER task do not contain entities. Traditional seq2seq method cannot address this issue effectively. To solve the aforementioned problem, we propose a novel seq2seq model, named SC-NER, for NER task. We construct a classifier between the encoder and decoder. In particular, the classifier's input is the last hidden state of the encoder. Moreover, we present the restricted beam search to improve the performance of the proposed SC-NER. To evaluate our proposed model, we construct the patent documents corpus in the communications field, and conduct experiments on it. Experimental results show that our SC-NER model achieves better performance than other baseline methods.

关键词： Named Entity Recognition sequence-to-sequence model Deep learning

来源：评论

学校读者我要写书评

暂无评论

Centralized Traffic Signal Control for Multiple Intersections based on sequence-to-sequence model and Attention Mechanism

Centralized Traffic Signal Control for Multiple Intersection...

引用

IEEE Intelligent Transportation Systems Conference (ITSC)

作者： Ma, Le Xue, Bo Wu, Jia Univ Elect Sci & Technol China Sch Informat & Software Engn Chengdu Peoples R China

ISBN: (纸本)9781728191423

Recently, the use of deep reinforcement learning techniques (DRL) has attracted increasing interest due to its ability of dynamical traffic signal control for multiple intersections. Only a few researches use the centralized control with single-agent to intelligently control all the signals, because the major problem, i.e., the curse of dimensionality, has not been successfully solved. We propose a novel centralized control method based on sequence-to-sequence model and attention mechanism to deal with this problem. The idea is similar to the Divide and Conquer paradigm. We mitigate the difficulty of searching in the huge space by dividing the state and action space into sub-spaces through sequence-to-sequence model. In addition, we greatly facilitate the communication and cooperation of traffic signals among intersections by introducing the attention mechanism. The DRL agent is trained by an efficient off-policy learning method - Proximal Policy Optimization. To the best of our knowledge, we are the first to use sequence-to-sequence method to deal with the huge search space problem in the traffic control. The comprehensive experiments demonstrate that our method can efficiently solve the curse of dimensionalitly problem and outperforms the traditional methods and other centralized control methods based on DRL.

关键词： Traffic signal control Deep reinforcement learning sequence-to-sequence model Attention mechanism

来源：评论

学校读者我要写书评

暂无评论

GENERATING SOUND WORDS FROM AUDIO SIGNALS OF ACOUSTIC EVENTS WITH sequence-to-sequence model

GENERATING SOUND WORDS FROM AUDIO SIGNALS OF ACOUSTIC EVENTS...

引用

IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

作者： Ikawa, Shota Kashino, Kunio Univ Tokyo Grad Sch Informat Sci & Technol Tokyo Japan NTT Corp NTT Commun Sci Labs Tokyo Japan

ISBN: (纸本)9781538646588

Representing various sounds in language, such as sound words, or onomatopoeias, is not only useful as an auxiliary means for automatic speech recognition, but also essential in emerging fields such as natural human-machine communication, searching audio archives for acoustic events, and abnormality detection based on sounds. This paper proposes a novel method for sound word generation from audio signals. The method is based on an end-to-end, sequence-to-sequence framework to solve the audio segmentation problem to find an appropriate segment of audio signals along time that corresponds to a sequence of phonemes, and the ambiguity problem, where multiple words may correspond to the same sound, depending on the situations or listeners. Our tests show that the method worked efficiently and achieved a 2.8 % mean phoneme error rate (MPER) and a 7.2 % word error rate (WER) in a sound word generation task.

关键词： Sound word onomatopoeia sequence-to-sequence model sound transcription

来源：评论

学校读者我要写书评

暂无评论

Direct speech-to-speech translation with a sequence-to-sequence model 20

Direct speech-to-speech translation with a sequence-to-seque...

引用

Interspeech Conference

作者： Jia, Ye Weiss, Ron J. Biadsy, Fadi Macherey, Wolfgang Johnson, Melvin Chen, Zhifeng Wu, Yonghui Google Mountain View CA 94043 USA

We present an attention-based sequence-to-sequence neural network which can directly translate speech from one language into speech in another language, without relying on an intermediate text representation. The network is trained end-to-end, learning to map speech spectrograms into target spectrograms in another language, corresponding to the translated content (in a different canonical voice). We further demonstrate the ability to synthesize translated speech using the voice of the source speaker. We conduct experiments on two Spanish-to-English speech translation datasets, and find that the proposed model slightly underperforms a baseline cascade of a direct speech-to-text translation model and a text-to-speech synthesis model, demonstrating the feasibility of the approach on this very challenging task.

关键词： speech-to-speech translation voice transfer attention sequence-to-sequence model end-to-end model

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：