检索结果-内蒙古大学图书馆

2D-Shapley: A Framework for Fragmented Data Valuation

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Liu, Zhihong Just, Hoang Anh Chang, Xiangyu Chen, Xi Jia, Ruoxi Center for Intelligent Decision-Making and Machine Learning Department of Information Systems and Intelligent Business School of Management Xi'an Jiaotong University Xi’an710049 China Bradley Department of Electrical and Computer Engineering Virginia Tech VA United States Department of Technology Operations and Statistics Stern School of Business New York University New York10012 United States

Data valuation-quantifying the contribution of individual data sources to certain predictive behaviors of a model-is of great importance to enhancing the transparency of machine learning and designing incentive systems for data sharing. Existing work has focused on evaluating data sources with the shared feature or sample space. How to valuate fragmented data sources of which each only contains partial features and samples remains an open question. We start by presenting a method to calculate the counterfactual of removing a fragment from the aggregated data matrix. Based on the counterfactual calculation, we further propose 2D-Shapley, a theoretical framework for fragmented data valuation that uniquely satisfies some appealing axioms in the fragmented data context. 2D-Shapley empowers a range of new use cases, such as selecting useful data fragments, providing interpretation for sample-wise data values, and fine-grained data issue diagnosis. Copyright © 2023, The Authors. All rights reserved.

关键词： machine learning

A Fast and Accurate Frequent Directions Algorithm for Low Rank Approximation via Block Krylov Iteration

学校读者我要写书评

暂无评论

A Fast and Accurate Frequent Directions Algorithm for Low Ra...

IEEE International Conference on Acoustics, Speech and Signal Processing

作者： Qianxin Yi Chenhao Wang Xiuwu Liao Yao Wang Center for Intelligent Decision-making and Machine Learning Xi’an Jiaotong University China

ISBN: (数字)9781509066315

ISBN: (纸本)9781509066322

It is known that frequent directions (FD) is a popular deterministic matrix sketching technique for low rank approximation. However, FD and its randomized variants usually meet high computational cost or computational instability in dealing with large-scale datasets, which limits their use in practice. To remedy such issues, this paper aims at improving the efficiency and effectiveness of FD. Specifically, by utilizing the power of Block Krylov Iteration and count sketch techniques, we propose a fast and accurate FD algorithm dubbed as BKICS-FD. We derive the error bound of the proposed BKICS-FD and then carry out extensive numerical experiments to illustrate its superiority over several popular FD algorithms, both in terms of computational speed and accuracy.

关键词：

Nyström regularization for time series forecasting

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Sun, Zirui Dai, Mingwei Wang, Yao Lin, Shao-Bo Center of Intelligent Decision-Making and Machine Learning School of Management Xi'an Jiaotong University Xi'an China Center of Statistical Research School of Statistics Southwestern University of Finance and Economics Chengdu China

This paper focuses on learning rate analysis of Nyström regularization with sequential subsampling for τ-mixing time series. Using a recently developed Banach-valued Bernstein inequality for τ-mixing sequences and an integral operator approach based on second-order decomposition, we succeed in deriving almost optimal learning rates of Nyström regularization with sequential sub-sampling for τ-mixing time series. A series of numerical experiments are carried out to verify our theoretical results, showing the excellent learning performance of Nyström regularization with sequential sub-sampling in learning massive time series data. All these results extend the applicable range of Nyström regularization from i.i.d. samples to non-i.i.d. sequences. © 2021, CC0.

关键词： Time series

Effective streaming low-tubal-rank tensor approximation via frequent directions

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Yi, Qianxin Wang, Chenhao Wang, Kaidong Wang, Yao The Center for Intelligent Decision-making and Machine Learning School of Management Xi’an Jiaotong University Xi’an710049 China The School of Mathematics and Statistics Xi’an Jiaotong University Xi’an710049 China

—Low-tubal-rank tensor approximation has been proposed to analyze large-scale and multi-dimensional data. However, finding such an accurate approximation is challenging in the streaming setting, due to the limited computational resources. To alleviate this issue, this paper extends a popular matrix sketching technique, namely Frequent Directions, for constructing an efficient and accurate low-tubal-rank tensor approximation from streaming data based on the tensor Singular Value Decomposition (t-SVD). Specifically, the new algorithm allows the tensor data to be observed slice by slice, but only needs to maintain and incrementally update a much smaller sketch which could capture the principal information of the original tensor. The rigorous theoretical analysis shows that the approximation error of the new algorithm can be arbitrarily small when the sketch size grows linearly. Extensive experimental results on both synthetic and real multi-dimensional data further reveal the superiority of the proposed algorithm compared with other sketching algorithms for getting low-tubal-rank approximation, in terms of both efficiency and accuracy. Copyright © 2021, The Authors. All rights reserved.

关键词： Tensors

Approximate Newton methods

学校读者我要写书评

暂无评论

The Journal of Machine Learning Research

The Journal of machine learning Research 2021年第1期22卷 3067-3107页

作者： Haishan Ye Luo Luo Zhihua Zhang Center for Intelligent Decision-Making and Machine Learning School of Management Xi 'an Jiaotong University Xi 'an China Department of Mathematics Hong Kong University of Science and Technology Kowloon Hong Kong School of Mathematical Sciences Peking University Beijing China

Many machine learning models involve solving optimization problems. Thus, it is important to address a large-scale optimization problem in big data applications. Recently, subsampled Newton methods have emerged to attract much attention due to their efficiency at each iteration, rectified a weakness in the ordinary Newton method of suffering a high cost in each iteration while commanding a high convergence rate. Other efficient stochastic second order methods have been also proposed. However, the convergence properties of these methods are still not well understood. There are also several important gaps between the current convergence theory and the empirical performance in real applications. In this paper, we aim to fill these gaps. We propose a unifying framework to analyze both local and global convergence properties of second order methods. Accordingly, we present our theoretical results which match the empirical performance in real applications well.

关键词： approximate newton stochastic second-order hessian approximation

Generalization Performance of Empirical Risk Minimization on Over-parameterized Deep ReLU Nets

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Lin, Shao-Bo Wang, Yao Zhou, Ding-Xuan Center for Intelligent Decision-Making and Machine Learning School of Management Xi’an Jiaotong University Xi’an710049 China School of Mathematics and Statistics University of Sydney SydneyNSW2006 Australia

关键词： Deep learning

Distributed learning with dependent samples

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Sun, Zirui Lin, Shao-Bo The Center of Intelligent Decision-Making and Machine Learning School of Management Xi'an Jiaotong University China

This paper focuses on learning rate ansalysis of distributed kernel ridge regression (DKRR) for strong mixing sequences. Using a recently developed integral operator approach and a classical covariance inequality for Banach-valued strong mixing sequences, we succeed in deriving optimal learning rates of DKRR. As a byproduct, we deduce a sufficient condition for the mixing property to guarantee the optimal learning rates for kernel ridge regression, which fills the gap of learning rates between i.i.d. samples and strong mixing sequences. A series of numerical experiments are conducted to verify our theoretical assertions via showing excellent learning performance of DKRR in learning both toy and real world time series data. All these results extend the applicable range of distributed learning from i.i.d. samples to non-i.i.d. sequences. Copyright © 2020, The Authors. All rights reserved.

关键词： learning algorithms

learning Personalized Brain Functional Connectivity of MDD Patients from Multiple Sites via Federated Bayesian Networks

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Liu, Shuai Guo, Xiao Qi, Shun Wang, Huaning Chang, Xiangyu Department of Information Systems and Intelligent Business School of Management Xi’an Jiaotong University China Key Laboratory of Biomedical Information Engineering Ministry of Education Institute of Health and Rehabilitation Science School of Life Science and Technology Xi’an Jiaotong University China Department of Psychiatry Xijing Hospital Air Force Medical University China Center for Intelligent Decision-Making and Machine Learning School of Management Xi’an Jiaotong University China

Identifying functional connectivity biomarkers of major depressive disorder (MDD) patients is essential to advance understanding of the disorder mechanisms and early intervention. However, due to the small sample size and the high dimension of available neuroimaging data, the performance of existing methods is often limited. Multi-site data could enhance the statistical power and sample size, while they are often subject to inter-site heterogeneity and data-sharing policies. In this paper, we propose a federated joint estimator, NOTEARS-PFL, for simultaneous learning of multiple Bayesian networks (BNs) with continuous optimization, to identify disease-induced alterations in MDD patients. We incorporate information shared between sites and site-specific information into the proposed federated learning framework to learn personalized BN structures by introducing the group fused lasso penalty. We develop the alternating direction method of multipliers, where in the local update step, the neuroimaging data is processed at each local site. Then the learned network structures are transmitted to the center for the global update. In particular, we derive a closed-form expression for the local update step and use the iterative proximal projection method to deal with the group fused lasso penalty in the global update step. We evaluate the performance of the proposed method on both synthetic and real-world multi-site rs-fMRI datasets. The results suggest that the proposed NOTEARS-PFL yields superior effectiveness and accuracy than the comparable methods. © 2023, CC BY.

关键词： Bayesian networks

Effective Tensor Completion via Element-wise Weighted Low-rank Tensor Train with Overlapping Ket Augmentation

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Zhang, Yang Wang, Yao Han, Zhi Chen, Xi'ai Tang, Yandong The State Key Laboratory of Robotics Shenyang Institute of Automation Chinese Academy of Sciences Shenyang110016 China The Center for Intelligent Decision-Making and Machine Learning School of Mangement Xi'an Jiaotong University Xi'An710049 China

In recent years, there have been an increasing number of applications of tensor completion based on the tensor train (TT) format because of its efficiency and effectiveness in dealing with higher-order tensor data. However, existing tensor completion methods using TT decomposition have two obvious drawbacks. One is that they only consider mode weights according to the degree of mode balance, even though some elements are recovered better in an unbalanced mode. The other is that serious blocking artifacts appear when the missing element rate is relatively large. To remedy such two issues, in this work, we propose a novel tensor completion approach via the element-wise weighted technique. Accordingly, a novel formulation for tensor completion and an effective optimization algorithm, called as tensor completion by parallel weighted matrix factorization via tensor train (TWMac-TT), is proposed. In addition, we specifically consider the recovery quality of edge elements from adjacent blocks. Different from traditional reshaping and ket augmentation, we utilize a new tensor augmentation technique called overlapping ket augmentation, which can further avoid blocking artifacts. We then conduct extensive performance evaluations on synthetic data and several real image data sets. Our experimental results demonstrate that the proposed algorithm TWMac-TT outperforms several other competing tensor completion methods. The code is available at https://***/yzcv/TWMac-TT-OKA. Copyright © 2021, The Authors. All rights reserved.

关键词： Tensors