Mathematical programming for piecewise linear regression analysis

作     者：Yang, Lingjian Liu, Songsong Tsoka, Sophia Papageorgiou, Lazaros G. 

作者机构：UCL Dept Chem Engn Ctr Proc Syst Engn London WC1E 7JE England Kings Coll London Sch Nat & Mahtemat Sci Dept Informat London WC2R 2LS England 

出 版 物：《EXPERT SYSTEMS WITH APPLICATIONS》 (专家系统及其应用)

年 卷 期：2016年第44卷

页      面：156-167页

核心收录：

学科分类：1201[管理学-管理科学与工程(可授管理学、工学学位)] 0808[工学-电气工程] 08[工学] 0812[工学-计算机科学与技术（可授工学、理学学位）] 

基　　金：UK Engineering and Physical Sciences Research Council (through the EPSRC Centre for Innovative Manufacturing in Emergent Macromolecular Therapies) UK Leverhulme Trust [RPG-2012-686] European Union [HEALTH-F2-2011-261366] Centre for Process Systems Engineering (CPSE) at Imperial and University College London 

主　　题：Regression analysis Surrogate model Piecewise linear function Mathematical programming Optimisation 

摘      要：In data mining, regression analysis is a computational tool that predicts continuous output variables from a number of independent input variables, by approximating their complex inner relationship. A large number of methods have been successfully proposed, based on various methodologies, including linear regression, support vector regression, neural network, piece-wise regression, etc. In terms of piece-wise regression, the existing methods in literature are usually restricted to problems of very small scale, due to their inherent non-linear nature. In this work, a more efficient piece-wise linear regression method is introduced based on a novel integer linear programming formulation. The proposed method partitions one input variable into multiple mutually exclusive segments, and fits one multivariate linear regression function per segment to minimise the total absolute error. Assuming both the single partition feature and the number of regions are known, the mixed integer linear model is proposed to simultaneously determine the locations of multiple break-points and regression coefficients for each segment. Furthermore, an efficient heuristic procedure is presented to identify the key partition feature and final number of break-points. 7 real world problems covering several application domains have been used to demonstrate the efficiency of our proposed method. It is shown that our propbsed piece-wise regression method can be solved to global optimality for datasets of thousands samples, which also consistently achieves higher prediction accuracy than a number of state-of-the-art regression methods. Another advantage of the proposed method is that the learned model can be conveniently expressed as a small number of if-then rules that are easily interpretable. Overall, this work proposes an efficient rule-based multivariate regression method based on piece-wise functions and achieves better prediction performance than state-of-the-arts approaches. This novel method