Optoelectronic nonlinear Softmax operator based on diffractive neural networks

作     者：Zhan, Ziyu Wang, Hao Liu, Qiang Fu, Xing 

作者机构：Tsinghua Univ Dept Precis Instrument Beijing 100084 Peoples R China State Key Lab Precis Space time Informat Sensing T Beijing 100084 Peoples R China Tsinghua Univ Key Lab Photon Control Technol Minist Educ Beijing 100084 Peoples R China 

出 版 物：《OPTICS EXPRESS》 (Opt. Express)

年 卷 期：2024年第32卷第15期

页      面：26458-26469页

核心收录：

学科分类：070207[理学-光学] 07[理学] 08[工学] 0803[工学-光学工程] 0702[理学-物理学] 

基　　金：Beijing Natural Science Foundation [JQ23021] 

主　　题：Deep learning Machine vision Neural networks Optical computing Probability theory Signal processing 

摘      要：Softmax, , a pervasive nonlinear operation, plays a pivotal role in numerous statistics and deep learning (DL) models such as ChatGPT. To compute it is expensive especially for at-scale models. Several software and hardware speed-up strategies are proposed but still suffer from low efficiency, poor scalability. Here we propose a photonic-computing solution including massive programmable neurons that is capable to execute such operation in an accurate, computation-efficient, robust and scalable manner. Experimental results show our diffraction- based computing system exhibits salient generalization ability in diverse artificial and real-world tasks (mean square error 10(-5)). We further analyze its performances against several realistic restricted factors. Such flexible system not only contributes to optimizing Softmax operation mechanism but may provide an inspiration of manufacturing a plug-and-play module for general optoelectronic accelerators.