检索结果-内蒙古大学图书馆

Uncertainty Quantification of Data Shapley via Statistical Inference

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Wu, Mengmeng Liu, Zhihong Li, Xiang Jia, Ruoxi Chang, Xiangyu Center for Intelligent Decision-Making and Machine Learning School of Management Xi’an Jiaotong University China University of Pennsylvania United States Virginia Tech Blacksburg United States

As data plays an increasingly pivotal role in decision-making, the emergence of data markets underscores the growing importance of data valuation. Within the machine learning landscape, Data Shapley stands out as a widely embraced method for data valuation. However, a limitation of Data Shapley is its assumption of a fixed dataset, contrasting with the dynamic nature of real-world applications where data constantly evolves and expands. This paper establishes the relationship between Data Shapley and infinite-order U-statistics and addresses this limitation by quantifying the uncertainty of Data Shapley with changes in data distribution from the perspective of U-statistics. We make statistical inferences on data valuation to obtain confidence intervals for the estimations. We construct two different algorithms to estimate this uncertainty and provide recommendations for their applicable situations. We also conduct a series of experiments on various datasets to verify asymptotic normality and propose a practical trading scenario enabled by this method. Copyright © 2024, The Authors. All rights reserved.

关键词： Data assimilation

Snap-Shot Decentralized Stochastic Gradient Tracking Methods

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Ye, Haishan Chang, Xiangyu Center for Intelligent Decision-Making and Machine Learning School of Management Xi'an Jiaotong University China

In decentralized optimization, m agents form a network and only communicate with their neighbors, which gives advantages in data ownership, privacy, and scalability. At the same time, decentralized stochastic gradient descent (SGD) methods, as popular decentralized algorithms for training large-scale machine learning models, have shown their superiority over centralized counterparts. Distributed stochastic gradient tracking (DSGT) [Pu and Nedić, 2021] has been recognized as the popular and state-of-the-art decentralized SGD method due to its proper theoretical guarantees. However, the theoretical analysis of DSGT [Koloskova et al., 2021] shows that its iteration complexity is (Equation presented), where W is a double stochastic mixing matrix that presents the network topology and CW is a parameter that depends on W. Thus, it indicates that the convergence property of DSGT is heavily affected by the topology of the communication network. To overcome the weakness of DSGT, we resort to the snap-shot gradient tracking skill and propose two novel algorithms. We further justify that the proposed two algorithms are more robust to the topology of communication networks under similar algorithmic structures and the same communication strategy to DSGT. Compared with DSGT, their iteration complexity are (Equation presented) and (Equation presented) which reduce the impact on network topology (no CW ). © 2022, CC BY-NC-ND.

关键词： Network topology

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Han, Yishan Xu, Biao Wang, Yao Gao, Shanxing Center for Intelligent Decision-Making and Machine Learning School of Management Xi’an Jiaotong University Shaanxi Xi’an China Department of Marketing School of Management Xi’an Jiaotong University Shaanxi Xi’an China

Top-K recommendation involves inferring latent user preferences and generating personalized recommendations accordingly, which is now ubiquitous in various decision systems. Nonetheless, recommender systems usually suffer from severe popularity bias, leading to the over-recommendation of popular items. Such a bias deviates from the central aim of reflecting user preference faithfully, compromising both customer satisfaction and retailer profits. Despite the prevalence, existing methods tackling popularity bias still have limitations due to the considerable accuracy-debias tradeoff and the sensitivity to extensive parameter selection, further exacerbated by the extreme sparsity in positive user-item interactions. In this paper, we present a Popularity-aware top-K recommendation algorithm integrating multi-behavior Side Information (PopSI), aiming to enhance recommendation accuracy and debias performance simultaneously. Specifically, by leveraging multiple user feedback that mirrors similar user preferences and formulating it as a three-dimensional tensor, PopSI can utilize all slices to capture the desiring user preferences effectively. Subsequently, we introduced a novel orthogonality constraint to refine the estimated item feature space, enforcing it to be invariant to item popularity features thereby addressing our model’s sensitivity to popularity bias. Comprehensive experiments on real-world e-commerce datasets demonstrate the general improvements of PopSI over state-of-the-art debias methods with a marginal accuracy-debias tradeoff and scalability to practical applications. The source code for our algorithm and experiments is available at https://***/Eason-sys/PopSI. Copyright © 2024, The Authors. All rights reserved.

关键词： Customer satisfaction

Adaptive Distributed Kernel Ridge Regression: A Feasible Distributed learning Scheme for Data Silos

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Wang, Di Liu, Xiaotong Lin, Shao-Bo Zhou, Ding-Xuan Center for Intelligent Decision-Making and Machine Learning School of Management Xi'an Jiaotong University Xi'An China School of Mathematics and Statistics University of Sydney Sydney Australia

Data silos, mainly caused by privacy and interoperability, significantly constrain collaborations among different organizations with similar data for the same purpose. Distributed learning based on divide-and-conquer provides a promising way to settle the data silos, but it suffers from several challenges, including autonomy, privacy guarantees, and the necessity of collaborations. This paper focuses on developing an adaptive distributed kernel ridge regression (AdaDKRR) by taking autonomy in parameter selection, privacy in communicating non-sensitive information, and the necessity of collaborations in performance improvement into account. We provide both solid theoretical verification and comprehensive experiments for AdaDKRR to demonstrate its feasibility and effectiveness. Theoretically, we prove that under some mild conditions, AdaDKRR performs similarly to running the optimal learning algorithms on the whole data, verifying the necessity of collaborations and showing that no other distributed learning scheme can essentially beat AdaDKRR under the same conditions. Numerically, we test AdaDKRR on both toy simulations and two real-world applications to show that AdaDKRR is superior to other existing distributed learning schemes. All these results show that AdaDKRR is a feasible scheme to defend against data silos, which are highly desired in numerous application regions such as intelligent decision-making, pricing forecasting, and performance prediction for products. Copyright © 2023, The Authors. All rights reserved.

关键词： learning algorithms

Component-based Sketching for Deep ReLU Nets

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Wang, Di Lin, Shao-Bo Meng, Deyu Cao, Feilong Center for Intelligent Decision-Making and Machine Learning School of Management Xi'an Jiaotong University China School of Mathematics and Statistics Xi'an Jiaotong University China School of Science China Jiliang University China

Deep learning has made profound impacts in the domains of data mining and AI, distinguished by the groundbreaking achievements in numerous real-world applications and the innovative algorithm design philosophy. However, it suffers from the inconsistency issue between optimization and generalization, as achieving good generalization, guided by the bias-variance trade-off principle, favors under-parameterized networks, whereas ensuring effective convergence of gradientbased algorithms demands over-parameterized networks. To address this issue, we develop a novel sketching scheme based on deep net components for various tasks. Specifically, we use deep net components with specific efficacy to build a sketching basis that embodies the advantages of deep networks. Subsequently, we transform deep net training into a linear empirical risk minimization problem based on the constructed basis, successfully avoiding the complicated convergence analysis of iterative algorithms. The efficacy of the proposed componentbased sketching is validated through both theoretical analysis and numerical experiments. Theoretically, we show that the proposed component-based sketching provides almost optimal rates in approximating saturated functions for shallow nets and also achieves almost optimal generalization error bounds. Numerically, we demonstrate that, compared with the existing gradientbased training methods, component-based sketching possesses superior generalization performance with reduced training costs. © 2024, CC0.

关键词： Deep learning

Preference Construction: A Bayesian Interactive Preference Elicitation Framework Based on Monte Carlo Tree Search

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Wang, Yan Liu, Jiapeng Kadziński, Milosz Liao, Xiuwu Center for Intelligent Decision-making and Machine Learning School of Management Xi’an Jiaotong University Shaanxi Xi’an710049 China Faculty of Computing and Telecommunications Poznan University of Technology Piotrowo 2 Poznań60-965 Poland

We present a novel preference learning framework to capture participant preferences efficiently within limited interaction rounds. It involves three main contributions. First, we develop a variational Bayesian approach to infer the participant’s preference model by estimating posterior distributions and managing uncertainty from limited information. Second, we propose an adaptive questioning policy that maximizes cumulative uncertainty reduction, formulating questioning as a finite Markov decision process and using Monte Carlo Tree Search to prioritize promising question trajectories. By considering long-term effects and leveraging the efficiency of the Bayesian approach, the policy avoids shortsightedness. Third, we apply the framework to Multiple Criteria decision Aiding, with pairwise comparison as the preference information and an additive value function as the preference model. We integrate the reparameterization trick to address high-variance issues, enhancing robustness and efficiency. Computational studies on real-world and synthetic datasets demonstrate the framework’s practical usability, outperforming baselines in capturing preferences and achieving superior uncertainty reduction within limited interactions. Copyright © 2025, The Authors. All rights reserved.

关键词： Markov processes

Robust Tensor Completion With Side Information

学校读者我要写书评

暂无评论

IEEE Transactions on Knowledge and Data Engineering 2025年

作者： Wang, Yao Yi, Qianxin Yang, Yiyang Gao, Shanxing Tang, Shaojie Wang, Di Xi'an Jiaotong University Center for Intelligent Decision-making and Machine Learning School of Management Xi'an China Xi'an Jiaotong University Department of Marketing School of Management Xi'an China University at Buffalo Department of Management Science and Systems BuffaloNY United States

Although robust tensor completion has been extensively studied, the effect of incorporating side information has not been explored. In this article, we fill this gap by developing a novel high-order robust tensor completion model that incorporates both latent and explicit side information. We base our model on the transformed t-product because the corresponding tensor tubal rank can characterize the inherent low-rank structure of a tensor. We study the effect of side information on sample complexity and prove that our model needs fewer observations than other tensor recovery methods when side information is perfect. This theoretically shows that informative side information is beneficial for learning. Extensive experimental results on synthetic and real data further demonstrate the superiority of the proposed method over several popular alternatives. In particular, we evaluate the performance of our solution based on two important applications, namely, link prediction in signed networks and rating prediction in recommender systems. We show that the proposed model, which manages to exploit side information in learning, outperforms other methods in the learning of such low-rank tensor data. Furthermore, when dealing with varying dimensions, we also design an online robust tensor completion with side information algorithm and validate its effectiveness using a real-world traffic dataset in the supplementary material. © 1989-2012 IEEE.

关键词： Recommender systems

Randomized spectral co-clustering for large-scale directed networks

学校读者我要写书评

暂无评论

The Journal of Machine Learning Research

The Journal of machine learning Research 2023年第1期24卷 18206-18273页

作者： Xiao Guo Yixuan Qiu Hai Zhang Xiangyu Chang Center for Modern Statistics School of Mathematics Northwest University Xi'an China School of Statistics and Management Shanghai University of Finance and Economics Shanghai China Center for Intelligent Decision-Making and Machine Learning School of Management Xi'an Jiaotong University Xi'an China

Directed networks are broadly used to represent asymmetric relationships among units. Co-clustering aims to cluster the senders and receivers of directed networks simultaneously. In particular, the well-known spectral clustering algorithm could be modified as the spectral co-clustering to co-cluster directed networks. However, large-scale networks pose great computational challenges to it. In this paper, we leverage sketching techniques and derive two randomized spectral co-clustering algorithms, one random-projection-basedand the other random-sampling-based, to accelerate the co-clustering of large-scale directed networks. We theoretically analyze the resulting algorithms under two generative models - the stochastic co-block model and the degree-corrected stochastic co-block model, and establish their approximation error rates and misclustering error rates, indicating better bounds than the state-of-the-art results of co-clustering literature. Numerically, we design and conduct simulations to support our theoretical results and test the efficiency of the algorithms on real networks with up to millions of nodes. A publicly available R package RandClust is developed for better usability and reproducibility of the proposed methods.

关键词： co-clustering directed network random projection random sampling stochastic co-block model

SKETCHING WITH SPHERICAL DESIGNS FOR NOISY DATA FITTING ON SPHERES∗

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Lin, Shao-Bo Wang, Di Zhou, Ding-Xuan Center for Intelligent Decision-Making and Machine Learning School of Management Xi’an Jiaotong University Xi’an710049 China School of Mathematics and Statistics University of Sydney SydneyNSW2006 Australia

This paper proposes a sketching strategy based on spherical designs, which is applied to the classical spherical basis function approach for massive spherical data fitting. We conduct theoretical analysis and numerical verifications to demonstrate the feasibility of the proposed sketching strategy. From the theoretical side, we prove that sketching based on spherical designs can reduce the computational burden of the spherical basis function approach without sacrificing its approximation capability. In particular, we provide upper and lower bounds for the proposed sketching strategy to fit noisy data on spheres. From the experimental side, we numerically illustrate the feasibility of the sketching strategy by showing its comparable fitting performance with the spherical basis function approach. These interesting findings show that the proposed sketching strategy is capable of fitting massive and noisy data on spheres. Copyright © 2023, The Authors. All rights reserved.

关键词： Spheres