检索结果-内蒙古大学图书馆

Conference on Optoelectronic Devices and Integration X

作者： Li, Hengkang Dong, Jianji Huazhong Univ Sci & Technol Wuhan Natl Lab Optoelect Wuhan 430074 Peoples R China

ISBN: (数字)9781510646384

ISBN: (纸本)9781510646384;9781510646377

Optical neural networks (ONNs), a kind of neural network implemented by optical hardware, have attracted more and more attention due to their excellent performance. As a part of ONNs, nonlinear activation functions the key for ONN to achieve more various functions. The realization of nonlinear activation functions by optical hardware has become a research hotspot. Here, we propose and demonstrate an all-optical implementation of a nonlinear activation function, which utilizes the intrinsic absorption and the plasma dispersion effect of germanium at 1550nm. The one has a loss about 4db and a threshold power of 5.1mW, while the another has a loss around 11db and a threshold power of 56.2mW. This proves that our scheme has great potential for nonlinear activation functions.

关键词： nonlinear activation functions germanium silicon hybrid integration plasma dispersion effect

来源：评论

学校读者我要写书评

暂无评论

Photonic Neural activation Function Based on High-Speed Electro-Absorption Modulated Laser

引用

IEEE PHOTONICS TECHNOLOGY LETTERS 2024年第22期36卷 1317-1320页

作者： Tian, Qi Li, Yunlong Zhou, Qihui Han, Yu Zhang, Ruigang Wang, Kaiyuan Liu, Deming Zheng, Shuang Zhang, Minming Huazhong Univ Sci & Technol Sch Opt & Elect Informat Wuhan 430074 Peoples R China Natl Engn Res Ctr Next Generat Internet Access Sys Wuhan 430074 Hubei Peoples R China Hubei Opt Fundamental Res Ctr Wuhan 430073 Peoples R China Opt Valley Lab Wuhan 430074 Hubei Peoples R China Acad Mil Sci Peoples Liberat Army Beijing 100071 Peoples R China

Integrated optics hold great potential to accelerate deep learning tasks with high clock rates, parallelism and low-loss data transmission. Silicon photonic integrated circuits can perform large-scale and low-power-consuming optical linear operations by using weighting mechanism through linear optics. However, on-chip light attenuation and nonlinear activation functions are still huge challenges for large-scale optical neural networks. Here, we demonstrate a high-speed electro-absorption modulator (EAM) monolithically integrated with a distributed feedback (DFB) laser that can deliver high output lasing power for larger scale expansion while acting as a nonlinear activation function unit. With the use of the obtained nonlinear activation function, a convolutional neural network (CNN) is simulated to perform a handwritten digit classification benchmark task with high accuracy. Thanks to its compactness, high response speed and laser integration, the demonstrated nonlinear unit has the potential to be used in heterogeneously integrated large-scale photonic neural networks.

关键词： nonlinear optics Optical device fabrication Optical attenuators Optical network units Neurons High-speed optical techniques Waveguide lasers Photonic neural networks electro-absorption modulator nonlinear activation functions

来源：评论

学校读者我要写书评

暂无评论

Web-aided data set expansion in deep learning: evaluating trainable activation functions in ResNet for improved image classification

引用

INTERNATIONAL JOURNAL OF WEB INFORMATION SYSTEMS 2024年第4期20卷 452-469页

作者： Zhang, Zhiqiang Li, Xiaoming Xu, Xinyi Lu, Chengjie Yang, Yihe Shi, Zhiyong Zhejiang Univ Sci & Technol Coll Sci Hangzhou Peoples R China Zhejiang Yuexiu Univ Foreign Languages Sch Int Business Shaoxing Peoples R China

PurposeThe purpose of this study is to explore the potential of trainable activation functions to enhance the performance of deep neural networks, specifically ResNet architectures, in the task of image classification. By introducing activation functions that adapt during training, the authors aim to determine whether such flexibility can lead to improved learning outcomes and generalization capabilities compared to static activation functions like ReLU. This research seeks to provide insights into how dynamic nonlinearities might influence deep learning models' efficiency and accuracy in handling complex image data ***/methodology/approachThis research integrates three novel trainable activation functions - CosLU, DELU and ReLUN - into various ResNet-n architectures, where "n" denotes the number of convolutional layers. Using CIFAR-10 and CIFAR-100 data sets, the authors conducted a comparative study to assess the impact of these functions on image classification accuracy. The approach included modifying the traditional ResNet models by replacing their static activation functions with the trainable variants, allowing for dynamic adaptation during training. The performance was evaluated based on accuracy metrics and loss profiles across different network *** findings indicate that trainable activation functions, particularly CosLU, can significantly enhance the performance of deep learning models, outperforming the traditional ReLU in deeper network configurations on the CIFAR-10 data set. CosLU showed the highest improvement in accuracy, whereas DELU and ReLUN offered varying levels of performance enhancements. These functions also demonstrated potential in reducing overfitting and improving model generalization across more complex data sets like CIFAR-100, suggesting that the adaptability of activation functions plays a crucial role in the training dynamics of deep neural ***/valueThis study contributes to the field of deep

关键词： Image classification ResNet nonlinear activation functions Trainable activation functions

来源：评论

学校读者我要写书评

暂无评论

All-Optical nonlinear activation Function Based on Germanium Silicon Hybrid Asymmetric Coupler

引用

IEEE JOURNAL OF SELECTED TOPICS IN QUANTUM ELECTRONICS 2023年第2期29卷 1页

作者： Li, Hengkang Wu, Bo Tong, Weiyu Dong, Jianji Zhang, Xinliang Huazhong Univ Sci & Technol Wuhan Natl Lab Optoelect Wuhan 430074 Peoples R China Opt Valley Lab Wuhan 430074 Hubei Peoples R China

nonlinear activation functions are crucial for optical neural networks (ONNs) to achieve more various functions. However, the current nonlinear functions suffer from some dilemma, including high power consumption, high loss, and limited bandwidth. Here, we propose and demonstrate an all-optical implementation of a nonlinear activation function based on germanium silicon hybrid integration. The principle lies in the intrinsic absorption and the carrier-induced refractive index change of germanium in C -band. It has a large operating bandwidth and a response frequency of 70 MHz, with a loss of 4.28 dB and a threshold power of 5.1 mW. Adopting it to the MNIST handwriting data set classification, it shows an improvement in accuracy from 91.6% to 96.8%. This proves that our scheme has great potential for advanced ONN applications.

关键词： Germanium Silicon Optical losses Stimulated emission Optical waveguides Optical resonators Optical interferometry Optical neural network nonlinear activation functions germanium silicon hybrid integration

来源：评论

学校读者我要写书评

暂无评论

nonlinearly Activated IEZNN Model for Solving Time-Varying Sylvester Equation

引用

IEEE ACCESS 2022年 10卷 121520-121530页

作者： Lei, Yihui Luo, Jiamei Chen, Tengxiao Ding, Lei Liao, Bolin Xia, Guangping Dai, Zhengqi Jishou Univ Coll Math & Stat Jishou Peoples R China Jishou Univ Coll Comp Sci & Engn Jishou Peoples R China

Zeroing neural network (ZNN) is an effective method to calculate time-varying problems. However, the ZNN and its extensions separately addressed the robustness and the convergence. To simultaneously promote the robustness and finite-time convergence, a nonlinearly activated integration-enhanced ZNN (NIEZNN) model based on a coalescent activation function (C-AF) has been designed for solving the time-varying Sylvester equation in various noise situations. The C-AF with an optimized structure is convenient for simulations and calculations, which promotes NIEZNN accelerates convergence speed without remarkable efficiency loss. The robustness and the finite-time convergence of the NIEZNN model have been proved in theoretical analyses. Furthermore, the upper bounds of convergence time of the NIEZNN model and the noise-attached NIEZNN model have been deduced in theory. At last, numerical comparative results and the application to mobile manipulator have validated the efficiency and superiority of the NIEZNN model based on the designed coalescent activation function.

关键词： Zeroing neural network time-varying Sylvester equation nonlinear activation functions robustness Finite-time convergence

来源：评论

学校读者我要写书评

暂无评论

Heterogeneous III-V-on-Silicon photonic non-linear activation function for scalable photonic neural networks

Heterogeneous III-V-on-Silicon photonic non-linear activatio...

引用

International Conference on Photonics in Switching and Computing (PSC)

作者： Giamougiannis, George Tossoun, Bassem Kurczveil, Geza Beausoleil, Raymond G. Hewlett Packard Enterprise Hewlett Packard Labs Santa Barbara CA 93106 USA Aristotle Univ Thessaloniki Dept Informat Thessaloniki Greece Hewlett Packard Enterprise Hewlett Packard Labs Milpitas CA USA

ISBN: (纸本)9798350323702

Targeting a large scale deep neural network implementation in the photonic domain, we experimentally demonstrate an optoelectronic activation function generated on our heterogeneous III-V-on-Si platform and the combination of a quantum dot avalanche photodiode with a quantum dot distributed feedback laser. The power threshold for the light activation out of the ReLU function is 5.25 mW.

关键词： Neuromorphic photonics nonlinear activation functions III-V-on-Si Quantum Dot APD Quantum Dot DFB

来源：评论

学校读者我要写书评

暂无评论

A reconfigurable multi-precision quantization-aware nonlinear activation function hardware module for DNNs

引用

MICROELECTRONICS JOURNAL 2024年 151卷

作者： Hong, Qi Liu, Zhiming Long, Qiang Tong, Hao Zhang, Tianxu Zhu, Xiaowen Zhao, Yunong Ru, Hua Zha, Yuxing Zhou, Ziyuan Wu, Jiashun Tan, Hongtao Hong, Weiqiang Xu, Yaohua Guo, Xiaohui Anhui Univ Sch Integrated Circuits Key Lab Intelligent Comp & Signal Proc Minist Educ Hefei 230601 Peoples R China Huadong Photoelectron IC Inst Bengbu 233030 Anhui Peoples R China Dalian Univ Technol State Key Lab High Performance Precis Mfg Dalian 116024 Peoples R China Dalian Univ Technol Key Lab Micro Nano Technol & Syst Liaoning Prov Dalian 116024 Peoples R China

In recent years, the increasing variety of nonlinear activation functions (NAFs) in deep neural networks (DNNs) has led to higher computational demands. However, hardware implementation faces challenges such as lack of flexibility, high hardware cost, and limited accuracy. This paper proposes a highly flexible and low-cost hardware solution for implementing activation functions to overcome these issues. Based on the piecewise linear (PWL) approximation method, our method supports NAFs with different accuracy configurations through a customized implementation strategy to meet the requirements in different scenario applications. In this paper, the symmetry of the activation function is investigated, and incorporate curve translation preprocessing and data quantization to significantly reduce hardware storage costs. The modular hardware architecture proposed in this study supports NAFs of multiple accuracies, which is suitable for designing deep learning neural network accelerators in various scenarios, avoiding the need to design dedicated hardware circuits for the activation function layer and enhances circuit design efficiency. The proposed hardware architecture is validated on the Xilinx XC7Z010 development board. The experimental results show that the average absolute error (AAE) is reduced by about 35.6 % at a clock frequency of 312.5 MHz. Additionally, the accuracy loss of the model is maximized to-0.684 % after replacing the activation layer function of DNNs under the PyTorch framework.

关键词： nonlinear activation functions Deep neural networks Piecewise linear approximation Hardware architecture Customized floating-point quantization

来源：评论

学校读者我要写书评

暂无评论

Performance analysis of nonlinear activation function in convolution neural network for image classification

引用

INTERNATIONAL JOURNAL OF COMPUTATIONAL SCIENCE AND ENGINEERING 2020年第4期21卷 522-535页

作者： Too, Edna C. Li, Yujian Gadosey, Pius Kwao Njuki, Sam Essaf, Firdaous Beijing Univ Technol Dept Comp Sci & Technol Beijing 100124 Peoples R China Chuka Univ Dept Comp Sci POB 109-60400 Chuka Kenya

Deep learning architectures which are exceptionally deep have exhibited to be incredibly powerful models for image processing. As the architectures become deep, it introduces challenges and difficulties in the training process such as overfitting, computational cost, and exploding/vanishing gradients and degradation. A new state-of-the-art densely connected architecture, called DenseNets, has exhibited an exceptionally outstanding result for image classification. However, it still computationally costly to train DenseNets. The choice of the activation function is also an important aspect in training of deep learning networks because it has a considerable impact on the training and performance of a network model. Therefore, an empirical analysis of some of the nonlinear activation functions used in deep learning is done for image classification. The activation functions evaluated include ReLU, Leaky ReLU, ELU, SELU and an ensemble of SELU and ELU. Publicly available datasets Cifar-10, SVHN, and PlantVillage are used for evaluation.

关键词： deep learning convolution neural networks activation functions nonlinear activation functions image classification rectified linear unit exponential linear unit scaled exponential linear unit leaky rectified linear unit DenseNet

来源：评论

学校读者我要写书评

暂无评论

Performance analysis of nonlinear activated zeroing neural networks for time-varying matrix pseudoinversion with application

引用

APPLIED SOFT COMPUTING 2021年 98卷 106735-106735页

作者： Hu, Zeshan Xiao, Lin Li, Kenli Li, Keqin Li, Jichun Hunan Univ Coll Informat Sci & Engn Changsha 410082 Hunan Peoples R China Hunan Normal Univ Hunan Prov Key Lab Intelligent Comp & Language In Changsha 410081 Peoples R China Univ Essex Sch Comp Sci & Elect Engn Colchester CO4 3SQ Essex England

By exploiting two simplified nonlinear activation functions, two zeroing neural network (ZNN) models are designed and studied to efficiently tackle the time-varying matrix pseudoinversion problem. Compared with ZNN activated by previously presented activation functions, these two simplified finite-time ZNN (SFTZNN) models (called SFTZNN1 and SFTZNN2) not only achieve faster finite-time convergence, but also possess better robustness. In addition, the SFTZNN1 and SFTZNN2 models have simpler structure compared with the widely used sign-bi-power activated ZNN model. Theoretical analysis is presented to obtain the maximum convergence time for the SFTZNN models in ideal conditions. Besides, when external perturbations are injected into the proposed SFTZNN models, upper bounds of the steady-state residual error are theoretically calculated. Comparative simulations and one engineering application case validate the feasibility and superiority of the two new SFTZNN models when solving time-varying matrix pseudoinversion. (C) 2020 Elsevier B.V. All rights reserved.

关键词： Time-varying matrix pseudoinverse Finite-time nonlinear activation functions Zeroing neural network Robot manipulator

来源：评论

学校读者我要写书评

暂无评论

Solving Time-Varying System of nonlinear Equations by Finite-Time Recurrent Neural Networks With Application to Motion Tracking of Robot Manipulators

引用

IEEE TRANSACTIONS ON SYSTEMS MAN CYBERNETICS-SYSTEMS 2019年第11期49卷 2210-2220页

作者： Xiao, Lin Zhang, Zhijun Li, Shuai Jishou Univ Coll Informat Sci & Engn Jishou 416000 Peoples R China South China Univ Technol Sch Automat Sci & Engn Guangzhou 510640 Guangdong Peoples R China Hong Kong Polytech Univ Dept Comp Hong Kong Peoples R China

Two novel nonlinearly activated recurrent neural networks (RNNs) with finite-time convergence [called finite-time RNNs (FTRNNs)] are proposed and analyzed to solve efficiently time-varying systems of nonlinear equations (SoNEs). Compared with previously presented neural networks for solving such a SoNE, the FTRNNs are activated by new nonlinear activation functions and thus possess a better finite-time convergence property. In addition, theoretical analyses about FTRNNs are presented to determine the upper bounds of convergence time under the context of using such two novel nonlinear activation functions. Computer simulations based on a numerical example validate the preponderance of the proposed FTRNNs for time-varying SoNE, as compared to the recently proposed Zhang neural network and its improved version. Finally, an engineering practical example to motion tracking of a robot manipulator demonstrates the feasibility and applicability of the FTRNNs.

关键词： Finite-time neural networks nonlinear activation functions robot manipulators time-varying systems of nonlinear equations

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：