检索结果-内蒙古大学图书馆

iPINNs: Incremental learning for Physics-informed neural networks

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Dekhovich, Aleksandr Sluiter, Marcel H.F. Tax, David M.J. Bessa, Miguel A. Department of Materials Science and Engineering Delft University of Technology Mekelweg 2 Delft2628 CD Netherlands Pattern Recognition and Bioinformatics Laboratory Delft University of Technology Van Mourik Broekmanweg 6 Delft2628 XE Netherlands School of Engineering Brown University 184 Hope St. ProvidenceRI02912 United States

Physics-informed neural networks (PINNs) have recently become a powerful tool for solving partial differential equations (PDEs). However, finding a set of neural network parameters that lead to fulfilling a PDE can be challenging and non-unique due to the complexity of the loss landscape that needs to be traversed. Although a variety of multi-task learning and transfer learning approaches have been proposed to overcome these issues, there is no incremental training procedure for PINNs that can effectively mitigate such training challenges. We propose incremental PINNs (iPINNs) that can learn multiple tasks (equations) sequentially without additional parameters for new tasks and improve performance for every equation in the sequence. Our approach learns multiple PDEs starting from the simplest one by creating its own subnetwork for each PDE and allowing each subnetwork to overlap with previously learned subnetworks. We demonstrate that previous subnetworks are a good initialization for a new equation if PDEs share similarities. We also show that iPINNs achieve lower prediction error than regular PINNs for two different scenarios: (1) learning a family of equations (e.g., 1-D convection PDE);and (2) learning PDEs resulting from a combination of processes (e.g., 1-D reaction-diffusion PDE). The ability to learn all problems with a single network together with learning more complex PDEs with better generalization than regular PINNs will open new avenues in this field. © 2023, CC BY.

关键词： Complex networks

Cooperative data-driven modeling

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Dekhovich, Aleksandr Turan, O. Taylan Yi, Jiaxiang Bessa, Miguel A. Department of Material Science and Engineering Delft University of Technology Mekelweg 2 Delft2628 CD Netherlands Pattern Recognition and Bioinformatics Laboratory Delft University of Technology Van Mourik Broekmanweg 6 Delft2628 XE Netherlands School of Engineering Brown University 184 Hope St. ProvidenceRI02912 United States

Data-driven modeling in mechanics is evolving rapidly based on recent machine learning advances, especially on artificial neural networks. As the field matures, new data and models created by different groups become available, opening possibilities for cooperative modeling. However, artificial neural networks suffer from catastrophic forgetting, i.e. they forget how to perform an old task when trained on a new one. This hinders cooperation because adapting an existing model for a new task affects the performance on a previous task trained by someone else. The authors developed a continual learning method1 that addresses this issue, applying it here for the first time to solid mechanics. In particular, the method is applied to recurrent neural networks to predict history-dependent plasticity behavior, although it can be used on any other architecture (feedforward, convolutional, etc.) and to predict other phenomena. This work intends to spawn future developments on continual learning that will foster cooperative strategies among the mechanics community to solve increasingly challenging problems. We show that the chosen continual learning strategy can sequentially learn several constitutive laws without forgetting them, using less data to achieve the same error as standard (non-cooperative) training of one law per model. © 2022, CC BY.

关键词： Recurrent neural networks

ROSE: Robust Selective Fine-tuning for Pre-trained Language Models

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Jiang, Lan Zhou, Hao Lin, Yankai Li, Peng Zhou, Jie Jiang, Rui Ministry of Education Key Laboratory of Bioinformatics Center for Synthetic and Systems Biology Department of Automation BNRist Tsinghua University China Pattern Recognition Center WeChat AI Tencent Inc. China Gaoling School of Artificial Intelligence Renmin University of China Beijing China Beijing Key Laboratory of Big Data Management and Analysis Methods Beijing China Tsinghua University China

Even though the large-scale language models have achieved excellent performances, they suffer from various adversarial attacks. A large body of defense methods has been proposed. However, they are still limited due to redundant attack search spaces and the inability to defend against various types of attacks. In this work, we present a novel fine-tuning approach called RObust SEletive fine-tuning (ROSE) to address this issue. ROSE conducts selective updates when adapting pretrained models to downstream tasks, filtering out invaluable and unrobust updates of parameters. Specifically, we propose two strategies: the first-order and second-order ROSE for selecting target robust parameters. The experimental results show that ROSE achieves significant improvements in adversarial robustness on various downstream NLP tasks, and the ensemble method even surpasses both variants above. Furthermore, ROSE can be easily incorporated into existing fine-tuning methods to improve their adversarial robustness further. The empirical analysis confirms that ROSE eliminates unrobust spurious updates during fine-tuning, leading to solutions corresponding to flatter and wider optima than the conventional method. Code is available at https://***/jiangllan/ROSE. © 2022, CC BY.

关键词： Computational linguistics

Neural network relief: a pruning algorithm based on neural activity

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Dekhovich, Aleksandr Tax, David M.J. Sluiter, Marcel H.F. Bessa, Miguel A. Department of Materials Science and Engineering Delft University of Technology Mekelweg 2 Delft2628 CD Netherlands Pattern Recognition and Bioinformatics Laboratory Delft University of Technology Van Mourik Broekmanweg 6 Delft2628 XE Netherlands School of Engineering Brown University 184 Hope St. ProvidenceRI02912 United States

Current deep neural networks (DNNs) are overparameterized and use most of their neuronal connections during inference for each task. The human brain, however, developed specialized regions for different tasks and performs inference with a small fraction of its neuronal connections. We propose an iterative pruning strategy introducing a simple importance-score metric that deactivates unimportant connections, tackling overparameterization in DNNs and modulating the firing patterns. The aim is to find the smallest number of connections that is still capable of solving a given task with comparable accuracy, i.e. a simpler subnetwork. We achieve comparable performance for LeNet architectures on MNIST, and significantly higher parameter compression than state-of-the-art algorithms for VGG and ResNet architectures on CIFAR-10/100 and Tiny-ImageNet.∗ Our approach also performs well for the two different optimizers considered – Adam and SGD. The algorithm is not designed to minimize FLOPs when considering current hardware and software implementations, although it performs reasonably when compared to the state of the art. © 2021, CC BY.

关键词： Deep neural networks

Probabilistic Inference for Camera Calibration in Light Microscopy Under Circular Motion

学校读者我要写书评

暂无评论

Probabilistic Inference for Camera Calibration in Light Micr...

IEEE International Symposium on Biomedical Imaging

作者： Yuanhao Guo Fons J. Verbeek Ge Yang Computational Biology and Machine Intelligence National Laboratory of Pattern Recognition Institute of Automation Chinese Academy of Sciences LIACS Bioinformatics and Imaging Leiden University School of Artificial Intelligence University of Chinese Academy of Sciences

ISBN: (数字)9781538693308

ISBN: (纸本)9781538693315

Robust and accurate camera calibration is essential for 3D reconstruction in light microscopy under circular motion. Conventional methods require either accurate key point matching or precise segmentation of the axial-view images. Both remain challenging because specimens often exhibit transparency/translucency in a light microscope. To address these issues, we propose a probabilistic inference based method for the camera calibration that does not require sophisticated image pre-processing. Based on 3D projective geometry, our method assigns a probability on each of a range of voxels that cover the whole object. The probability indicates the likelihood of a voxel belonging to the object to be reconstructed. Our method maximizes a joint probability that distinguishes the object from the background. Experimental results show that the proposed method can accurately recover camera configurations in both light microscopy and natural scene imaging. Furthermore, the method can be used to produce high-fidelity 3D reconstructions and accurate 3D measurements.

关键词： Cameras Three-dimensional displays Calibration Probabilistic logic Microscopy Solid modeling

NiuEM: A Nested-iterative Unsupervised Learning Model for Single-particle Cryo-EM Image Processing

学校读者我要写书评

暂无评论

NiuEM: A Nested-iterative Unsupervised Learning Model for Si...

2020 IEEE International Conference on bioinformatics and Biomedicine, BIBM 2020

作者： Hu, Rui Cai, Jiaming Zheng, Wangjie Yang, Yang Shen, Hong-Bin Shanghai Jiao Tong University Key Lab. of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering Department of Computer Science and Engineering Shanghai200240 China Institute of Image Processing and Pattern Recognition Shanghai Jiao Tong University Key Laboratory of System Control and Information Processing Ministry of Education of China Shanghai200240 China Shanghai Jiao Tong University Department of Bioinformatics and Biostatistics Shanghai200240 China

ISBN: (纸本)9781728162157

Cryo-electron microscopy (cryo-EM) has become a mainstream technology for solving spatial structures of biomacromolecules, while the processing of cryo-EM images is a very challenging task. One of the great challenges is the high noise in the images. A common method is to cluster the images with close projecting angles to get mean images, which are used for 3D reconstruction. However, due to the extremely low signal-to-noise-ratio, common clustering methods often fail to obtain high-quality mean images, leading to poorly reconstructed structures. In this study, we present a new unsupervised learning framework, called NiuEM, to discriminate images captured from different angles and yield cluster-mean images. NiuEM first generates pseudo-labels and then exploits both contrastive loss and cross-entropy loss for training convolutional layers to learn feature representations. Moreover, the pseudo-labels are updated iteratively to enhance the reliability of labels. We assess the performance of NiuEM on four data sets via both visualized and quantitative experiments. Especially, two kinds of metrics are adopted to measure the performance, regarding the clustering quality and the resolution of reconstructed 3D models, respectively. The experimental results show that NiuEM achieves very competitive clustering accuracy in the comparison with the state-of-the-art image clustering methods. Moreover, the cluster mean images yielded by NiuEM lead to better initial 3D models compared with the mainstream reconstruction tools. © 2020 IEEE.

关键词： Image reconstruction

Probabilistic inference for camera calibration in light microscopy under circular motion

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Guo, Yuanhao Verbeek, Fons J. Yang, Ge Computational Biology and Machine Intelligence Group National Laboratory of Pattern Recognition Institute of Automation Chinese Academy of Sciences Beijing China School of Artificial Intelligence University of Chinese Academy of Sciences Beijing China Bioinformatics and Imaging LIACS Leiden University Leiden Netherlands

Robust and accurate camera calibration is essential for 3D reconstruction in light microscopy under circular motion. Conventional methods require either accurate key point matching or precise segmentation of the axial-view images. Both remain challenging because specimens often exhibit transparency/translucency in a light microscope. To address those issues, we propose a probabilistic inference based method for the camera calibration that does not require sophisticated image pre-processing. Based on 3D projective geometry, our method assigns a probability on each of a range of voxels that cover the whole object. The probability indicates the likelihood of a voxel belonging to the object to be reconstructed. Our method maximizes a joint probability that distinguishes the object from the background. Experimental results show that the proposed method can accurately recover camera configurations in both light microscopy and natural scene imaging. Furthermore, the method can be used to produce highfidelity 3D reconstructions and accurate 3D measurements. Copyright © 2019, The Authors. All rights reserved.

关键词： Probability

Comparative cellular analysis of motor cortex in human, marmoset and mouse (vol 598, pg 111, 2021)

学校读者我要写书评

暂无评论

NATURE 2022年第7904期604卷 E8-E8页

作者： Bakken, Trygve E. Jorstad, Nikolas L. Hu, Qiwen Lake, Blue B. Tian, Wei Kalmbach, Brian E. Crow, Megan Hodge, Rebecca D. Krienen, Fenna M. Sorensen, Staci A. Eggermont, Jeroen Yao, Zizhen Aevermann, Brian D. Aldridge, Andrew I. Bartlett, Anna Bertagnolli, Darren Casper, Tamara Castanon, Rosa G. Crichton, Kirsten Daigle, Tanya L. Dalley, Rachel Dee, Nick Dembrow, Nikolai Diep, Dinh Ding, Song-Lin Dong, Weixiu Fang, Rongxin Fischer, Stephan Goldman, Melissa Goldy, Jeff Graybuck, Lucas T. Herb, Brian R. Hou, Xiaomeng Kancherla, Jayaram Kroll, Matthew Lathia, Kanan van Lew, Baldur Li, Yang Eric Liu, Christine S. Liu, Hanqing Lucero, Jacinta D. Mahurkar, Anup McMillen, Delissa Miller, Jeremy A. Moussa, Marmar Nery, Joseph R. Nicovich, Philip R. Niu, Sheng-Yong Orvis, Joshua Osteen, Julia K. Owen, Scott Palmer, Carter R. Pham, Thanh Plongthongkum, Nongluk Poirion, Olivier Reed, Nora M. Rimorin, Christine Rivkin, Angeline Romanow, William J. Sedeno-Cortes, Adriana E. Siletti, Kimberly Somasundaram, Saroja Sulc, Josef Tieu, Michael Torkelson, Amy Tung, Herman Wang, Xinxin Xie, Fangming Yanny, Anna Marie Zhang, Renee Ament, Seth A. Behrens, M. Margarita Bravo, Hector Corrada Chun, Jerold Dobin, Alexander Gillis, Jesse Hertzano, Ronna Hof, Patrick R. Hollt, Thomas Horwitz, Gregory D. Keene, C. Dirk Kharchenko, Peter V. Ko, Andrew L. Lelieveldt, Boudewijn P. Luo, Chongyuan Mukamel, Eran A. Pinto-Duarte, Antonio Preiss, Sebastian Regev, Aviv Ren, Bing Scheuermann, Richard H. Smith, Kimberly Spain, William J. White, Owen R. Koch, Christof Hawrylycz, Michael Tasic, Bosiljka Macosko, Evan Z. McCarroll, Steven A. Ting, Jonathan T. Zeng, Hongkui Zhang, Kun Feng, Guoping Ecker, Joseph R. Linnarsson, Sten Lein, Ed S. Allen Institute for Brain Science Seattle WA USA Department of Physiology and Biophysics University of Washington Seattle WA USA Department of Biomedical Informatics Harvard Medical School Boston MA USA Department of Bioengineering University of California San Diego La Jolla CA USA The Salk Institute for Biological Studies La Jolla CA USA Epilepsy Center of Excellence Department of Veterans Affairs Medical Center Seattle WA USA Stanley Institute for Cognitive Genomics Cold Spring Harbor Laboratory Cold Spring Harbor NY USA Department of Genetics Harvard Medical School Boston MA USA Broad Institute of MIT and Harvard Cambridge MA USA LKEB Department of Radiology Leiden University Medical Center Leiden The Netherlands Pattern Recognition and Bioinformatics group Delft University of Technology Delft The Netherlands J. Craig Venter Institute La Jolla CA USA Department of Pathology University of California San Diego CA USA Division of Vaccine Discovery La Jolla Institute for Immunology La Jolla CA USA Genomic Analysis Laboratory The Salk Institute for Biological Studies La Jolla CA USA Computer Science and Engineering Program University of California San Diego La Jolla CA USA Howard Hughes Medical Institute The Salk Institute for Biological Studies La Jolla CA USA Bioinformatics and Systems Biology Graduate Program University of California San Diego La Jolla CA USA Institute for Genomes Sciences University of Maryland School of Medicine Baltimore MD USA Center for Epigenomics Department of Cellular and Molecular Medicine University of California San Diego La Jolla CA USA Ludwig Institute for Cancer Research La Jolla CA USA Department of Computer Science University of Maryland College Park College Park MD USA Sanford Burnham Prebys Medical Discovery Institute La Jolla CA USA Biomedical Sciences Program School of Medicine University of California San Diego La Jolla CA USA University of Connecticut Storrs CT USA Department

Markov Random Field for wind farm planning

学校读者我要写书评

暂无评论

Markov Random Field for wind farm planning

IEEE International Conference on Smart Energy Grid Engineering (SEGE)

作者： Hale Cetinay Taygun Kekeç Fernando A. Kuipers D. M. J. Tax Network Architectures and Services Delft University of Technology Delft The Netherlands Pattern Recognition and Bioinformatics Laboratory Delft University of Technology Delft The Netherlands

Many countries aim to integrate a substantial amount of wind energy in the near future. This requires meticulous planning, which is challenging due to the uncertainty in wind profiles. In this paper, we propose a novel framework to discover and investigate those geographic areas that are well suited for building wind farms. We combine the key indicators of wind farm investment using fuzzy sets, and employ multiple-criteria decision analysis to obtain a coarse wind farm suitability value. We further demonstrate how this suitability value can be refined by a Markov Random Field (MRF) that takes the dependencies between adjacent areas into account. As a proof of concept, we take wind farm planning in Turkey, and demonstrate that our MRF modeling can accurately find promising areas.

关键词： Wind farms Investment Markov random fields Wind power generation Decision analysis Planning