Training of Physical Neural Networks

作     者：Momeni, Ali Rahmani, Babak Scellier, Benjamin Wright, Logan G. McMahon, Peter L. Wanjura, Clara C. Li, Yuhang Skalli, Anas Berloff, Natalia G. Onodera, Tatsuhiro Oguz, Ilker Morichetti, Francesco del Hougne, Philipp Le Gallo, Manuel Sebastian, Abu Mirhoseini, Azalia Zhang, Cheng Marković, Danijela Brunner, Daniel Moser, Christophe Gigan, Sylvain Marquardt, Florian Ozcan, Aydogan Grollier, Julie Liu, Andrea J. Psaltis, Demetri Alù, Andrea Fleury, Romain 

作者机构： Lausanne Switzerland Microsoft Research 198 Cambridge Science Park CambridgeCB4 0AB United Kingdom Rain AI San Francisco United States Department of Applied Physics Yale University CT United States School of Applied and Engineering Physics Cornell University IthacaNY14853 United States Max Planck Institute for the Science of Light Staudtstraße 2 Erlangen91058 Germany Department of Electrical and Computer Engineering University of California Los AngelesCA90095 United States FEMTO-ST Institute Optics Department CNRS University Bourgogne Franche-Comté Besançon25030 Cedex France Department of Applied Mathematics and Theoretical Physics University of Cambridge Cambridge United Kingdom NTT Physics and Informatics Laboratories NTT Research Inc. Sunnyvale United States  Lausanne Switzerland Dipartimento di Elettronica Informazione e Bioingegneria Politecnico di Milano Milan Italy Univ Rennes CNRS IETR UMR 6164 RennesF-35000 France IBM Research Europe– Zurich Rüschlikon8803 Switzerland Department of Computer Science Stanford University United States Google DeepMind 1600 Amphitheatre Parkway Mountain ViewCA94043 United States Unité Mixte de Physique CNRS/Thales CNRS Thales Université Paris-Saclay Palaiseau France Laboratoire Kastler Brossel Sorbonne Université École Normale Supérieure Collège de France CNRS UMR 8552 Paris France Laboratoire Albert Fert CNRS Thales UniversitéParis-Saclay Palaiseau91767 France Department of Physics and Astronomy University of Pennsylvania PhiladelphiaPA19104 United States  Lausanne Switzerland Photonics Initiative Advanced Science Research Center City University of New York New YorkNY10031 United States Physics Program Graduate Center City University of New York New YorkNY10016 United States 

出 版 物：《arXiv》 (arXiv)

年 卷 期：2024年

核心收录：

主　　题：Deep learning 

摘      要：Physical neural networks (PNNs) are a class of neural-like networks that leverage the properties of physical systems to perform computation. While PNNs are so far a niche research area with small-scale laboratory demonstrations, they are arguably one of the most underappreciated important opportunities in modern artificial intelligence (AI). Could we train AI models 1000x larger than current ones? Could we do this and also have them perform inference locally and privately on edge devices, such as smartphones or sensors? Research over the past few years has shown that the answer to all these questions is likely textityes, with enough research: PNNs could one day radically change what is possible and practical for AI systems. To do this will however require rethinking both how AI models work, and how they are trained – primarily by considering the problems through the constraints of the underlying hardware physics. To train PNNs at large scale, many methods including backpropagation-based and backpropagation-free approaches are now being explored. These methods have various trade-offs, and so far no method has been shown to scale to the same scale and performance as the backpropagation algorithm widely used in deep learning today. However, this is rapidly changing, and a diverse ecosystem of training techniques provides clues for how PNNs may one day be utilized to create both more efficient realizations of current-scale AI models, and to enable unprecedented-scale models. Copyright © 2024, The Authors. All rights reserved.