检索结果-内蒙古大学图书馆

iPINNs: Incremental learning for Physics-informed neural networks

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Dekhovich, Aleksandr Sluiter, Marcel H.F. Tax, David M.J. Bessa, Miguel A. Department of Materials Science and Engineering Delft University of Technology Mekelweg 2 Delft2628 CD Netherlands Pattern Recognition and Bioinformatics Laboratory Delft University of Technology Van Mourik Broekmanweg 6 Delft2628 XE Netherlands School of Engineering Brown University 184 Hope St. ProvidenceRI02912 United States

Physics-informed neural networks (PINNs) have recently become a powerful tool for solving partial differential equations (PDEs). However, finding a set of neural network parameters that lead to fulfilling a PDE can be challenging and non-unique due to the complexity of the loss landscape that needs to be traversed. Although a variety of multi-task learning and transfer learning approaches have been proposed to overcome these issues, there is no incremental training procedure for PINNs that can effectively mitigate such training challenges. We propose incremental PINNs (iPINNs) that can learn multiple tasks (equations) sequentially without additional parameters for new tasks and improve performance for every equation in the sequence. Our approach learns multiple PDEs starting from the simplest one by creating its own subnetwork for each PDE and allowing each subnetwork to overlap with previously learned subnetworks. We demonstrate that previous subnetworks are a good initialization for a new equation if PDEs share similarities. We also show that iPINNs achieve lower prediction error than regular PINNs for two different scenarios: (1) learning a family of equations (e.g., 1-D convection PDE);and (2) learning PDEs resulting from a combination of processes (e.g., 1-D reaction-diffusion PDE). The ability to learn all problems with a single network together with learning more complex PDEs with better generalization than regular PINNs will open new avenues in this field. © 2023, CC BY.

关键词： Complex networks

Cooperative data-driven modeling

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Dekhovich, Aleksandr Turan, O. Taylan Yi, Jiaxiang Bessa, Miguel A. Department of Material Science and Engineering Delft University of Technology Mekelweg 2 Delft2628 CD Netherlands Pattern Recognition and Bioinformatics Laboratory Delft University of Technology Van Mourik Broekmanweg 6 Delft2628 XE Netherlands School of Engineering Brown University 184 Hope St. ProvidenceRI02912 United States

Data-driven modeling in mechanics is evolving rapidly based on recent machine learning advances, especially on artificial neural networks. As the field matures, new data and models created by different groups become available, opening possibilities for cooperative modeling. However, artificial neural networks suffer from catastrophic forgetting, i.e. they forget how to perform an old task when trained on a new one. This hinders cooperation because adapting an existing model for a new task affects the performance on a previous task trained by someone else. The authors developed a continual learning method1 that addresses this issue, applying it here for the first time to solid mechanics. In particular, the method is applied to recurrent neural networks to predict history-dependent plasticity behavior, although it can be used on any other architecture (feedforward, convolutional, etc.) and to predict other phenomena. This work intends to spawn future developments on continual learning that will foster cooperative strategies among the mechanics community to solve increasingly challenging problems. We show that the chosen continual learning strategy can sequentially learn several constitutive laws without forgetting them, using less data to achieve the same error as standard (non-cooperative) training of one law per model. © 2022, CC BY.

关键词： Recurrent neural networks

Neural network relief: a pruning algorithm based on neural activity

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Dekhovich, Aleksandr Tax, David M.J. Sluiter, Marcel H.F. Bessa, Miguel A. Department of Materials Science and Engineering Delft University of Technology Mekelweg 2 Delft2628 CD Netherlands Pattern Recognition and Bioinformatics Laboratory Delft University of Technology Van Mourik Broekmanweg 6 Delft2628 XE Netherlands School of Engineering Brown University 184 Hope St. ProvidenceRI02912 United States

Current deep neural networks (DNNs) are overparameterized and use most of their neuronal connections during inference for each task. The human brain, however, developed specialized regions for different tasks and performs inference with a small fraction of its neuronal connections. We propose an iterative pruning strategy introducing a simple importance-score metric that deactivates unimportant connections, tackling overparameterization in DNNs and modulating the firing patterns. The aim is to find the smallest number of connections that is still capable of solving a given task with comparable accuracy, i.e. a simpler subnetwork. We achieve comparable performance for LeNet architectures on MNIST, and significantly higher parameter compression than state-of-the-art algorithms for VGG and ResNet architectures on CIFAR-10/100 and Tiny-ImageNet.∗ Our approach also performs well for the two different optimizers considered – Adam and SGD. The algorithm is not designed to minimize FLOPs when considering current hardware and software implementations, although it performs reasonably when compared to the state of the art. © 2021, CC BY.

关键词： Deep neural networks

Predicting functional impact of single amino acid polymorphisms by integrating sequence and structural features

学校读者我要写书评

暂无评论

Predicting functional impact of single amino acid polymorphi...

IEEE International Conference on Systems Biology

作者： Mingjun Wang Hong-Bin Shen Tatsuya Akutsu Jiangning Song State Engineering Laboratory for Industrial Enzymes Tianjin Institute of Industrial Biotechnology Chinese Academy of Sciences (CAS) Tianjin China Institute of Image Processing and Pattern Recognition Shanghai Jiao Tong University Shanghai China Bioinformatics Center Institute for Chemical Research Kyoto University Uji Kyoto Japan Department of Biochemistry and Molecular Biology Faculty of Medicine Monash University Melbourne VIC Australia

Single amino acid polymorphisms (SAPs) are the most abundant form of known genetic variations associated with human diseases. It is of great interest to study the sequence-structure-function relationship underlying SAPs. In this work, we collected the human variant data from three databases and divided them into three categories, i.e. cancer somatic mutations (CSM), Mendelian disease-related variant (SVD) and neutral polymorphisms (SVP). We built support vector machine (SVM) classifiers to predict these three classes of SAPs, using the optimal features selected by a random forest algorithm. Consequently, 280 sequence-derived and structural features were initially extracted from the curated datasets from which 18 optimal candidate features were further selected by random forest. Furthermore, we performed a stepwise feature selection to select characteristic sequence and structural features that are important for predicting each SAPs class. As a result, our predictors achieved a prediction accuracy (ACC) of 84.97, 96.93, 86.98 and 88.24%, for the three classes, CSM, SVD and SVP, respectively. Performance comparison with other previously developed tools such as SIFT, SNAP and Polyphen2 indicates that our method provides a favorable performance with higher Sensitivity scores and Matthew's correlation coefficients (MCC). These results indicate that the prediction performance of SAPs classifiers can be effectively improved by feature selection. Moreover, division of SAPs into three respective categories and construction of accurate SVM-based classifiers for each class provides a practically useful way for investigating the difference between Mendelian disease-related variants and cancer somatic mutations.

关键词： Support vector machines Accuracy Proteins Feature extraction Humans Amino acids Databases

ROSE: Robust Selective Fine-tuning for Pre-trained Language Models

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Jiang, Lan Zhou, Hao Lin, Yankai Li, Peng Zhou, Jie Jiang, Rui Ministry of Education Key Laboratory of Bioinformatics Center for Synthetic and Systems Biology Department of Automation BNRist Tsinghua University China Pattern Recognition Center WeChat AI Tencent Inc. China Gaoling School of Artificial Intelligence Renmin University of China Beijing China Beijing Key Laboratory of Big Data Management and Analysis Methods Beijing China Tsinghua University China

Even though the large-scale language models have achieved excellent performances, they suffer from various adversarial attacks. A large body of defense methods has been proposed. However, they are still limited due to redundant attack search spaces and the inability to defend against various types of attacks. In this work, we present a novel fine-tuning approach called RObust SEletive fine-tuning (ROSE) to address this issue. ROSE conducts selective updates when adapting pretrained models to downstream tasks, filtering out invaluable and unrobust updates of parameters. Specifically, we propose two strategies: the first-order and second-order ROSE for selecting target robust parameters. The experimental results show that ROSE achieves significant improvements in adversarial robustness on various downstream NLP tasks, and the ensemble method even surpasses both variants above. Furthermore, ROSE can be easily incorporated into existing fine-tuning methods to improve their adversarial robustness further. The empirical analysis confirms that ROSE eliminates unrobust spurious updates during fine-tuning, leading to solutions corresponding to flatter and wider optima than the conventional method. Code is available at https://***/jiangllan/ROSE. © 2022, CC BY.

关键词： Computational linguistics

POS0750 WHO PARTICIPATES IN AN ONLINE SYMPTOM CHECKER? CHARACTERISTICS, ATTRITION AND REPRESENTATIVENESS IN THE RHEUMATIC? STUDY

学校读者我要写书评

暂无评论

Annals of the Rheumatic Diseases 2024年 83卷 1128-1129页

作者： F. Zegers L. Qin D. Selani G. Gomon T.D. Maarseveen H.K. Glas A. Van Tubergen Y. Goekoop-Ruiterman M.J.T. Reinders E.B. Van den Akker C. Baatenburg de Jong L. Klareskog B. Axnäs E. Steyerberg R. Bos S. Le Cessie R. Knevel Leiden University Medical Center Department of Biomedical Data Sciences Leiden Netherlands Leiden University Medical Center Department of Rheumatology Leiden Netherlands Delft University of Technology Department of Intelligent Systems Pattern Recognition and Bioinformatics Delft Netherlands Reumazorg Zuid West Neder；anmd Goes Netherlands Maastricht University Medical Center Department of Internal Medicine Division of Rheumatology Maastricht Netherlands Maastricht University Care and Public Health Research Institute (CAPHRI) Maastricht Netherlands Haga Hospital Department of Rheumatology Den Haag Netherlands Delft Technical University Delft Bioinformatics Lab Delft Netherlands Leiden University Medical Center Leiden Computational Biology Department of Biomedical Data Sciences Leiden Netherlands Delft University of Technology The Delft Bioinformatics Lab Pattern Recognition and Bioinformatics Delft Netherlands ReumaNederland Amsterdam Netherlands (Solna) Karolinska Institutet Division for Rheumatology Department of Medicine Stockholm Sweden Elsa Science Stockholm Sweden University Medical Center Rotterdam Department of Public Health Erasmus MC Rotterdam Netherlands Medical Center Leeuwarden Department of Rheumatology Leeuwarden Netherlands Nij Smellinghe Hospital Department of Rheumatology Drachten Netherlands Leiden University Medical Center Department of Clinical Epidemiology Leiden Netherlands Newcastle University Translational and Clinical Research Institute Rheumatology Newcastle upon Tyne United Kingdom

Background: Online symptom checkers (OSCs) could provide guidance in the diagnostic process. The number of OSCs are expanding yet often lack rigorous validation. Rheumatic? is an online questionnaire designed to provide a fast and reliable risk assessment of possible rheumatic diagnoses for individuals with musculoskeletal complaints (MSCs). We set out to validate this tool by following individuals with MSCs. A key question is representativity of the participants as attrition bias can complicate the construction of a precise prediction model. Objectives: This project aims to assess aspects of attrition and selection bias in Rheumatic? by comparing participants who did and did not respond to the first follow-up survey, by comparing participants versus patients from general practitioners (GP) and rheumatology clinics, and by exploring variations in characteristics across different recruitment policies. Methods: Rheumatic? includes a baseline questionnaire about current symptoms and past diagnoses. Participants were recruited online, at primary care or at rheumatology outpatient clinics. Four surveys followed after 0, 3, 6 and 12 months, about source of recruitment, medication, further diagnoses etc. We considered data of July 2021 to July 2023. We compared participants who solely answered the questionnaire to those who also answered the first follow up survey. Secondly, we compared characteristics of participants across different sources of recruitment (i.e. at primary or secondary care clinics or online (Instagram, Facebook, Google, the Dutch Arthritis Association)). For this, we calculated differences in percentages with confidence intervals and applied chi squared tests. We assessed representability of the cohort with the expected target population by comparing age, sex and BMI of the participants with information on new patients with MSC from the same period at primary care (GP), Rheumatology clinics in the South West of the Netherlands (RZWN), and the Rheumatolog

关键词：

Hybrid genetic-discretized algorithm to handle data uncertainty in diagnosing stenosis of coronary arteries

学校读者我要写书评

暂无评论

Hybrid genetic-discretized algorithm to handle data uncertai...

作者： Alizadehsani, Roohallah Roshanzamir, Mohamad Abdar, Moloud Beykikhoshk, Adham Khosravi, Abbas Nahavandi, Saeid Plawiak, Pawel Tan, Ru San Acharya, U Rajendra Deakin University GeelongVIC Australia Department of Engineering Fasa Branch Islamic Azad University Fars Fasa Iran Centre for Pattern Recognition and Data Analytics Deakin University GeelongVIC Australia Department of Information and Communications Technology Faculty of Computer Science and Telecommunications Cracow University of Technology Warszawska 24 st. F-3 31-155 Krakow Poland Institute of Theoretical and Applied Informatics Polish Academy of Sciences Baltycka 5 Gliwice44-100 Poland National Heart Centre Singapore Singapore Department of Electronics and Computer Engineering Ngee Ann Polytechnic Singapore Department of Biomedical Engineering School of Science and Technology Singapore University of Social Sciences Singapore Department of Bioinformatics and Medical Engineering Asia University Taiwan

Coronary artery disease (CAD) is the leading cause of morbidity and death worldwide. Invasive coronary angiography is the most accurate technique for diagnosing CAD, but is invasive and costly. Hence, analytical methods such as machine learning and data mining techniques are becoming increasingly more popular. Although physicians need to know which arteries are stenotic, most of the researchers focus only on CAD detection and few studies have investigated stenosis of the right coronary artery (RCA), left circumflex (LCX) artery and left anterior descending (LAD) artery separately. Meanwhile, most of the datasets in this field are noisy (data uncertainty). However, to the best of our knowledge, there is no study conducted to address this important problem. This study uses the extension of the Z-Alizadeh Sani dataset, containing 303 records with 54 features. A new feature selection algorithm is proposed in this work. Meanwhile, by discretization of data, we also handle the uncertainty in CAD prediction. To the best of our knowledge, this is the first study attempted to handle uncertainty in CAD prediction. Finally, the genetic algorithm (GA) is used to determine the hyper-parameters of the support vector machine (SVM) kernels. We have achieved high accuracy for the stenosis diagnosis of each main coronary artery. The results of this study can aid the clinicians to validate their manual stenosis diagnosis of RCA, LCX and LAD coronary arteries. © 2020 John Wiley & Sons, Ltd.

关键词： Feature Selection

MICAI 2010 Organization and Conference Committee

学校读者我要写书评

暂无评论

Proceedings of Special Session - 9th Mexican International C...

Proceedings of Special Session - 9th Mexican International Conference on Artificial Intelligence: Advances in Artificial Intelligence and Applications, MICAI 2010 2010年 viii页

作者： Reyes-García, Carlos Alberto Sidorov, Grigori Hernández-Aguirre, Arturo Arroyo, Gustavo Murrieta, Rafael Gonzalez, Jesus A. Gonzalez, Miguel Herrera, Oscar Peña, Alejandro Espinoza, Félix A. Castro Cansino, Joel Suárez Galicia-Haro, Sofia N. Koeppen, Mario Reyes-García, Carlos A. Monroy, Raul Gelbukh, Alexander Mezura-Montes, Efrén Leguizamón, Guillermo Ramírez-Manzanares, Alonso Castillo, Oscar Fuentes, Olac Sánchez, Gildardo Natural Language Processing Mexico Machine Learning and Pattern Recognition Mexico Hybrid Intelligent Systems and Neural Networks Mexico Logic Reasoning Ontologies Knowledge Mgmt. Knowledge-Based Syst. Multi-agent Syst. Mexico Data Mining Mexico Intelligent Tutoring Systems Mexico Evolutionary Algorithms and Other Naturally Inspired Algorithms Mexico Computer Vision and Image Processing Mexico Fuzzy Logic Uncertainty and Probabilistic Reasoning Mexico Bioinformatics and Medical Applications Mexico Robotics Planning and Scheduling Mexico

Dynamic vortical flow profile quantitatively characterizes mitral valvular-left ventricular hemodynamic coupling: In vivo analysis by dynamic 3D enstrophy mapping from 4D Flow MRI

学校读者我要写书评

暂无评论

arXiv 2018年

作者： Elbaz, Mohammed S.M. Le, Trung Bao Garg, Pankaj Roest, Arno A.W. Lelieveldt, Boudewijn P.F. Plein, Sven Westenberg, Jos J.M. Sotiropoulos, Fotis van der Geest, Rob J. Department of Radiology Northwestern University ChicagoIL United States Department of Radiology Leiden University Medical Center Leiden Netherlands Department of Civil Engineering Stony Brook University Stony BrookNY United States University of Leeds Leeds United Kingdom Department of Pediatrics Division of Pediatric Cardiology Leiden University Medical Center Leiden Netherlands Pattern Recognition and Bioinformatics Group Faculty of Electrical Engineering Mathematics and Computer Science Delft University of Technology Delft Netherlands Computational Fluids Laboratory North Dakota State University United States

Vortical blood flow in the human left ventricular (LV) inflow initiates from the mitral valve (MV) and evolves within the LV during diastolic E-filling. Hence, vortical flow links MV and LV hemodynamics. This study sought to elucidate and quantitatively characterize the in vivo 3D dynamics of LV vortical flow over E-filling and relation to MV-LV hemodynamic coupling using 4D Flow MRI flow field. 34 healthy volunteers and 5 example patients underwent 4D Flow MRI. Vortical blood flow evolution was mapped in the LV over E-wave using enstrophy density. A new dimensionless profile PMV-LV was derived as a function of both MV vortex formation time (VFT) and LV volumetric enstrophy density. Results reveal that 3D vortical flow evolution in the healthy LV follows a bi-phasic behavior with a vortical growth phase followed by a vortical decay phase. In healthy LVs studied, the PMV-LV profile showed that the vortical growth and decay phases are characterized by a vortical growth time Tgrowth = 1.23 ± 0.25, growth rate α = 0.80 ± 0.17, decay time Tdecay = 0.96 ± 0.39 and decay rate β = -1.02 ± 0.49. Distinctly altered parameters were found in the pilot patients studied. The derived PMV-LV profile quantitatively characterizes MV-LV hemodynamic coupling by vortical flow dynamics. Results herein unravel new insights into cardiac physiology and could enable a novel standardized methodology to study MV-LV hemodynamic coupling and association to cardiac function in health and disease. Copyright © 2018, The Authors. All rights reserved.

关键词： Heart