Multi-objective evolutionary optimization for hardware-aware neural network pruning

作     者：Wenjing Hong Guiying Li Shengcai Liu Peng Yang Ke Tang Wenjing Hong;Guiying Li;Shengcai Liu;Peng Yang;Ke Tang

作者机构：Guangdong Provincial Key Laboratory of Brain-Inspired Inteligent ComputationDepartment of Computer Science and EngineeringSouthern University of Science and TechnologyShenzhen 518055China Research Institute of Trustworthy Autonomous SystemsSouthern University of Science and TechnologyShenzhen 518055China Department of Statistics and Data ScienceSouthern University of Science and TechnologyShenzhen 518055China 

出 版 物：《Fundamental Research》 (自然科学基础研究（英文版）)

年 卷 期：2024年第4卷第4期

页      面：941-950页

核心收录：

学科分类：12[管理学] 1201[管理学-管理科学与工程(可授管理学、工学学位)] 081104[工学-模式识别与智能系统] 08[工学] 0835[工学-软件工程] 0811[工学-控制科学与工程] 0812[工学-计算机科学与技术（可授工学、理学学位）] 

基　　金：the National Natural Science Foundation of China(62106098) the Stable Support Plan Program of Shenzhen Natural Science Fund(20200925154942002) the M0E University Scientific-Technological Innovation Plan Program 

主　　题：Multi-objective optimization Evolutionary algorithm Neural network pruning Hardware-awaremachine learning Hardware efficiency 

摘      要：Neural network pruning is a popular approach to reducing the computational complexity of deep neural *** recent years,as growing evidence shows that conventional network pruning methods employ inappropriate proxy metrics,and as new types of hardware become increasingly available,hardware-aware network pruning that incorporates hardware characteristics in the loop of network pruning has gained growing attention,Both network accuracy and hardware efficiency(latency,memory consumption,etc.)are critical objectives to the success of network pruning,but the conflict between the multiple objectives makes it impossible to find a single optimal *** studies mostly convert the hardware-aware network pruning to optimization problems with a single *** this paper,we propose to solve the hardware-aware network pruning problem with Multi-Objective Evolutionary Algorithms(MOEAs).Specifically,we formulate the problem as a multi-objective optimization problem,and propose a novel memetic MOEA,namely HAMP,that combines an efficient portfoliobased selection and a surrogate-assisted local search,to solve *** studies demonstrate the potential of MOEAs in providing simultaneously a set of alternative solutions and the superiority of HAMP compared to the state-of-the-art hardware-aware network pruning method.