检索结果-内蒙古大学图书馆

Nyström regularization for time series forecasting

学校读者我要写书评

暂无评论

The Journal of Machine Learning Research

The Journal of machine learning Research 2022年第1期23卷 14082-14123页

作者： Zirui Sun Mingwei Dai Yao Wang Shao-Bo Lin Center for Intelligent Decision-Making and Machine Learning School of Management Xi 'an Jiaotong University Xi 'an China Center of Statistical Research and School of Statistics Southwestern University of Finance and Economics Chengdu China

This paper focuses on learning rate analysis of Nyström regularization with sequential subsampling for τ-mixing time series. Using a recently developed Banach-valued Bernstein inequality for τ-mixing sequences and an integral operator approach based on secondorder decomposition, we succeed in deriving almost optimal learning rates of Nyström regularization with sequential sub-sampling for τ-mixing time series. A series of numerical experiments are carried out to verify our theoretical results, showing the excellent learning performance of Nyström regularization with sequential sub-sampling in learning massive time series data. All these results extend the applicable range of Nyström regularization from i.i.d. samples to non-i.i.d. sequences.

关键词： time series forecasting sub-sampling Nyström regularization τ-mixing process

Efficient Generalized Low-Rank Tensor Contextual Bandits

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Yi, Qianxin Yang, Yiyang Tang, Shaojie Liu, Jiapeng Wang, Yao Center for Intelligent Decision-Making and Machine Learning School of Management Xi’an Jiaotong University Xi’an China Naveen Jindal School of Management The University of Texas at Dallas RichardsonTX United States

In this paper, we aim to build a novel bandits algorithm that is capable of fully harnessing the power of multi-dimensional data and the inherent non-linearity of reward functions to provide high-usable and accountable decision-making services. To this end, we introduce a generalized low-rank tensor contextual bandits model in which an action is formed from three feature vectors, and thus can be represented by a tensor. In this formulation, the reward is determined through a generalized linear function applied to the inner product of the action’s feature tensor and a fixed but unknown parameter tensor with a low tubal rank. To effectively achieve the trade-off between exploration and exploitation, we introduce a novel algorithm called "Generalized Low-Rank Tensor Exploration Subspace then Refine" (G-LowTESTR). This algorithm first collects raw data to explore the intrinsic low-rank tensor subspace information embedded in the decision-making scenario, and then converts the original problem into an almost lower-dimensional generalized linear contextual bandits problem. Rigorous theoretical analysis shows that the regret bound of G-LowTESTR is superior to those in vectorization and matricization cases. We conduct a series of simulations and real data experiments to further highlight the effectiveness of G-LowTESTR, leveraging its ability to capitalize on the low-rank tensor structure for enhanced learning. Copyright © 2023, The Authors. All rights reserved.

关键词： Tensors

Randomized Spectral Co-Clustering for Large-Scale Directed Networks

学校读者我要写书评

暂无评论

Journal of machine learning Research 2023年 24卷

作者： Guo, Xiao Qiu, Yixuan Zhang, Hai Chang, Xiangyu Center for Modern Statistics School of Mathematics Northwest University Xi'an China School of Statistics and Management Shanghai University of Finance and Economics Shanghai China Center for Intelligent Decision-Making and Machine Learning School of Management Xi'an Jiaotong University Xi'an China

Directed networks are broadly used to represent asymmetric relationships among units. Co-clustering aims to cluster the senders and receivers of directed networks simultaneously. In particular, the well-known spectral clustering algorithm could be modified as the spectral co-clustering to co-cluster directed networks. However, large-scale networks pose great computational challenges to it. In this paper, we leverage sketching techniques and derive two randomized spectral co-clustering algorithms, one random-projection-based and the other random-sampling-based, to accelerate the co-clustering of large-scale directed networks. We theoretically analyze the resulting algorithms under two generative models – the stochastic co-block model and the degree-corrected stochastic co-block model, and establish their approximation error rates and misclustering error rates, indicating better bounds than the state-of-the-art results of co-clustering literature. Numerically, we design and conduct simulations to support our theoretical results and test the efficiency of the algorithms on real networks with up to millions of nodes. A publicly available R package RandClust is developed for better usability and reproducibility of the proposed methods. ©2023 Xiao Guo, Yixuan Qiu, Hai Zhang and Xiangyu Chang.

关键词： Clustering algorithms

Reliable and Autonomy-Enabled Collaborative Medical Prediction System Using Distributed learning

学校读者我要写书评

暂无评论

SSRN

SSRN 2023年

作者： Liu, Xiaotong Wang, Yao Tang, Shaojie Lin, Shao-Bo Center for Intelligent Decision-Making and Machine Learning School of Management Xi’an Jiaotong University Xi’an China Naveen Jindal School of Management University of Texas at Dallas RichardsonTX United States

Multi-institutional collaboration is a promising way to improve small-scale institutions’ competitiveness in prediction. However, rising privacy concerns among these institutions hinder such collaboration. Considering the proliferated research and development of medical prediction, we focus on the medical field and propose a collaborative medical prediction system that makes this cooperation feasible, efficient, and effective through a novel distributed learning scheme. This system gives institutions the autonomy to determine their learning algorithms, algorithmic parameters, and attendance and develops a corresponding qualification mechanism to ensure high-quality synthesized results. Our theoretical results demonstrate that the proposed collaborative medical prediction system can achieve almost optimal generalization error bounds in the framework of statistical learning theory. Numerical experiments, including toy simulations and real-world data analyses, verify our theoretical assertions and show the feasibility and power of this system. © 2023, The Authors. All rights reserved.

关键词： Forecasting

Distributed Uncertainty Quantification of Kernel Interpolation on Spheres

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Lin, Shao-Bo Sun, Xingping Wang, Di Center for Intelligent Decision-Making and Machine Learning School of Management Xi’an Jiaotong University Xi’an710049 China Department of Mathematics Missouri State University SpringfieldMO65897 United States

For radial basis function (RBF) kernel interpolation of scattered data, Schaback [30] in 1995 proved that the attainable approximation error and the condition number of the underlying interpolation matrix cannot be made small simultaneously. He referred to this finding as an "uncertainty relation", an undesirable consequence of which is that RBF kernel interpolation is susceptible to noisy data. In this paper, we propose and study a distributed interpolation method to manage and quantify the uncertainty brought on by interpolating noisy spherical data of non-negligible magnitude. We also present numerical simulation results showing that our method is practical and robust in terms of handling noisy data from challenging computing environments. © 2023, CC0.

关键词： Spheres

Early stopping for iterative regularization with general loss functions

学校读者我要写书评

暂无评论

The Journal of Machine Learning Research

The Journal of machine learning Research 2022年第1期23卷 15320-15355页

作者： Ting Hu Yunwen Lei Center for Intelligent Decision-Making and Machine Learning School of Management Xi 'an Jiaotong University Xi 'an China Department of Mathematics Hong Kong Baptist University Kowloon Hong Kong China

In this paper, we investigate the early stopping strategy for the iterative regularization technique, which is based on gradient descent of convex loss functions in reproducing kernel Hilbert spaces without an explicit regularization term. This work shows that projecting the last iterate of the stopping time produces an estimator that can improve the generalization ability. Using the upper bound of the generalization errors, we establish a close link between the iterative regularization and Tikhonov regularization scheme and explain theoretically why the two schemes have similar regularization paths in the existing numerical simulations. We introduce a data-dependent way based on cross-validation to select the stopping time. We prove that the a-posteriori selection way can retain the comparable generalization errors to those obtained by our stopping rules with a-prior parameters.

关键词： iterative regularization early stopping reproducing kernel Hilbert spaces stopping rule cross-validation

Feature Qualification by Deep Nets: A Constructive Approach

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Cao, Feilong Lin, Shao-Bo School of Mathematics Zhejiang Normal University Jinhua321014 China Institute of Mathematics and Cross-disciplinary Science Zhejiang Normal University Hangzhou310012 China Center for Intelligent Decision-Making and Machine Learning School of Management Xi’an Jiaotong University Xi’an710049 China

The great success of deep learning has stimulated avid research activities in verifying the power of depth in theory, a common consensus of which is that deep net are versatile in approximating and learning numerous functions. Such a versatility certainly enhances the understanding of the power of depth, but makes it difficult to judge which data features are crucial in a specific learning task. This paper proposes a constructive approach to equip deep nets for the feature qualification purpose. Using the product-gate nature and localized approximation property of deep nets with sigmoid activation (deep sigmoid nets), we succeed in constructing a linear deep net operator that possesses optimal approximation performance in approximating smooth and radial functions. Furthermore, we provide theoretical evidences that the constructed deep net operator is capable of qualifying multiple features such as the smoothness and radialness of the target functions. © 2025, CC0.

关键词： Deep neural networks

PPFL: A Personalized Federated learning Framework for Heterogeneous Population

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Di, Hao Yang, Yi Ye, Haishan Chang, Xiangyu School of Management Xi'an Jiaotong University China International Business School Suzhou Xi'an Jiaotong-Liverpool University China Center for Intelligent Decision-Making and Machine Learning School of Management Xi'an Jiaotong University China

Personalization aims to characterize individual preferences and is widely applied across many fields. However, conventional personalized methods operate in a centralized manner and potentially expose the raw data when pooling individual information. In this paper, with privacy considerations, we develop a flexible and interpretable personalized framework within the paradigm of federated learning, called PPFL (Population Personalized Federated learning). By leveraging "canonical models" to capture fundamental characteristics among the heterogeneous population and employing "membership vectors" to reveal clients' preferences, it models the heterogeneity as clients' varying preferences for these characteristics and provides substantial insights into client characteristics, which is lacking in existing Personalized Federated learning (PFL) methods. Furthermore, we explore the relationship between our method and three main branches of PFL methods: clustered FL, multi-task PFL, and decoupling PFL, and demonstrate the advantages of PPFL. To solve PPFL (a non-convex constrained optimization problem), we propose a novel random block coordinate descent algorithm and present the convergence property. We conduct experiments on both pathological and practical datasets, and the results validate the effectiveness of PPFL. Copyright © 2023, The Authors. All rights reserved.

关键词： Constrained optimization

Toward Fairness-Aware Gradient Boosting decision Trees for Ranking

学校读者我要写书评

暂无评论

SSRN

SSRN 2024年

作者： Xu, Biao Wang, Yao Gao, Shanxing Shu, Chengli Tang, Shaojie Center for Intelligent Decision-Making and Machine Learning School of Management Xi’an Jiaotong University Xi’an710049 China School of Management Xi’an Jiaotong University Xi’an710049 China Naveen Jindal School of Management The University of Texas DallasTX75080 United States

Ranking involves training models to prioritize items based on their relevance to a given query, and Gradient Boosting decision Trees (GBDT)-based ranking methods stand out as a robust choice for addressing the intricacies of ranking tasks. Nonetheless, the inherent risk associated with fairness in ranking tasks requires the incorporation of fairness-aware considerations into GBDT-based models. As GBDT-based ranking methods are indifferentiable, widely used fairness-aware models do not embed GBDT-based ranking methods effectively. In response to this research gap, we propose a fairness-aware GBDT-based ranking method, namely individual fairness gradient boosting decision trees method for ranking (IFGBDTRank), to deal with these issues. To facilitate the differentiable GBDT-based model, we introduce the concept of Soft decision Trees (SDT). We then integrate SDTs using gradient boosting techniques, leading to the development of Soft Gradient Boosting decision Trees. Then we formulate soft gradient boosting decision trees for ranking problems. This ensemble referred to as GBDT-Rank, is designed for ranking problems and is formulated to create a differentiable model that maintains high-ranking performance. To address fairness-aware consideration in GBDT-Rank, we opt for fairness-aware ranking based on individual fairness, and incorporate an optimal transport-based regularize into GBDT-Rank to deal with individual fairness at the algorithm level, thus introducing IFGBDT-Rank. Comprehensive experiments on synthetic and real-world datasets confirm that IFGBDT-Rank maintains robust ranking performance while improving individual fairness simultaneously. Our code is available at https://***/dfhjdskfds/IFGBDT-Rank. © 2024, The Authors. All rights reserved.

关键词： decision trees