检索结果-内蒙古大学图书馆

Revisiting Stochastic Multi-Level Compositional Optimization

IEEE Transactions on Pattern Analysis and Machine Intelligence 2025年第7期47卷 5613-5624页

作者： Wei Jiang Sifan Yang Yibo Wang Tianbao Yang Lijun Zhang National Key Laboratory for Novel Software Technology Nanjing University Nanjing China National Key Laboratory for Novel Software Technology School of Artiicial Intelligence Nanjing University Nanjing China Department of Computer Science and Engineering Texas A&M University College Station TX USA

This paper explores stochastic multi-level compositional optimization, where the objective function is a composition of multiple smooth functions. Traditional methods for solving this problem suffer from either sub-optimal sample complexities or require huge batch sizes. To address these limitations, we introduce the Stochastic Multi-level Variance Reduction (SMVR) method. In the expectation case, our SMVR method attains the optimal sample complexity of $\mathcal {O}(1/\epsilon ^{3})$ to find an $\epsilon$ -stationary point for non-convex objectives. When the function satisfies convexity or the Polyak-Łojasiewicz (PL) condition, we propose a stage-wise SMVR variant. This variant improves the sample complexity to $\mathcal {O}(1/\epsilon ^{2})$ for convex functions and $\mathcal {O}(1/(\mu \epsilon ))$ for functions meeting the $\mu$ -PL condition or $\mu$ -strong convexity. These complexities match the lower bounds not only in terms of $\epsilon$ but also in terms of $\mu$ (for PL or strongly convex functions), without relying on large batch sizes in each iteration. Furthermore, in the finite-sum case, we develop the SMVR-FS algorithm, which can achieve a complexity of $\mathcal {O}(\sqrt{n}/\epsilon ^{2})$ for non-convex objectives, $\mathcal {O}(\sqrt{n}/\epsilon \log (1/\epsilon ))$ for convex functions and $\mathcal {O}(\sqrt{n}/\mu \log (1/\epsilon ))$ for objectives satisfying the $\mu$ -PL condition, where $n$ denotes the number of functions in each level. To make use of adaptive learning rates, we propose the Adaptive SMVR method, which maintains the same complexities while demonstrating faster convergence in practice.

关键词： Complexity theory Optimization Convex functions Convergence Training Stochastic processes Lower bound Jacobian matrices Adaptive learning Storms

来源：评论

学校读者我要写书评

暂无评论

Near-Linear Time Approximation Algorithms for k-means with Outliers 41

Near-Linear Time Approximation Algorithms for k-means with O...

引用

41st International Conference on Machine Learning, ICML 2024

作者： Huang, Junyu Feng, Qilong Huang, Ziyun Xu, Jinhui Wang, Jianxin School of Computer Science and Engineering Central South University Changsha410083 China Xiangjiang Laboratory Changsha410205 China Department of Computer Science and Software Engineering Penn State Erie The Behrend College United States Department of Computer Science and Engineering State University of New York BuffaloNY United States The Hunan Provincial Key Lab of Bioinformatics Central South University Changsha410083 China

The k-means with outliers problem is one of the most extensively studied clustering problems in the field of machine learning, where the goal is to discard up to z outliers and identify a minimum k-means clustering on the remaining data points. Most previous results for this problem have running time dependent on the aspect ratio ∆ (the ratio between the maximum and the minimum pairwise distances) to achieve fast approximations. To address the issue of aspect ratio dependency on the running time, we propose sampling-based algorithms with almost linear running time in the data size, where a crucial component of our approach is an algorithm called Fast-Sampling. Fast-Sampling algorithm can find inliers that well approximate the optimal clustering centers without relying on a guess for the optimal clustering costs, where a 4-approximate solution can be obtained in time O(ndk log log n/∊2) with O(k/∊) centers opened and (1 + ∊)z outliers discarded. To reduce the number of centers opened, we propose a center reduction algorithm, where an O(1/∊)-approximate solution can be obtained in time O(ndk log log n/∊2 + dpoly(k, 1/∊) log(n∆)) with (1 + ∊)z outliers discarded and exactly k centers opened. Empirical experiments suggest that our proposed sampling-based algorithms outperform state-of-the-art algorithms for the k-means with outliers problem. Copyright 2024 by the author(s)

关键词： K-means clustering

来源：评论

学校读者我要写书评

暂无评论

Linear Time Approximation Algorithm for Column Subset Selection with Local Search 38

Linear Time Approximation Algorithm for Column Subset Select...

引用

38th Conference on Neural Information Processing Systems, NeurIPS 2024

作者： Zou, Yuanbin Huang, Ziyun Xu, Jinhui Wang, Jianxin Feng, Qilong School of Computer Science and Engineering Central South University Changsha410083 China Xiangjiang Laboratory Changsha410205 China Department of Computer Science and Software Engineering Penn State Erie The Behrend College United States Department of Computer Science and Engineering State University of New York BuffaloNY United States The Hunan Provincial Key Lab of Bioinformatics Central South University Changsha410083 China

The Column Subset Selection (CSS) problem has been widely studied in dimensionality reduction and feature selection. The goal of the CSS problem is to output a submatrix S, consisting of k columns from an n × d input matrix A that minimizes the residual error A − SS†A2F, where S† is the Moore-Penrose inverse matrix of S. Many previous approximation algorithms have non-linear running times in both n and d, while the existing linear-time algorithms have a relatively larger approximation ratios. Additionally, the local search algorithms in existing results for solving the CSS problem are heuristic. To achieve linear running time while maintaining better approximation using a local search strategy, we propose a local search-based approximation algorithm for the CSS problem with exactly k columns selected. A key challenge in achieving linear running time with the local search strategy is how to avoid exhaustive enumerations of candidate columns for constructing swap pairs in each local search step. To address this issue, we propose a two-step mixed sampling method that reduces the number of enumerations for swap pair construction from O(dk) to k in linear time. Although the two-step mixed sampling method reduces the search space of local search strategy, bounding the residual error after swaps is a non-trivial task. To estimate the changes in residual error after swaps, we propose a matched swap pair construction method to bound the approximation loss, ensuring a constant probability of loss reduction in each local search step. In expectation, these techniques enable us to obtain the local search algorithm for the CSS problem with theoretical guarantees, where a 53(k + 1)-approximate solution can be obtained in linear running time O(ndk4 log k). Empirical experiments show that our proposed algorithm achieves better quality and time compared to previous algorithms on both small and large datasets. Moreover, it is at least 10 times faster than state-of-the-art algorithms a

关键词：

来源：评论

学校读者我要写书评

暂无评论

Learning Order Forest for Qualitative-Attribute Data Clustering 27

Learning Order Forest for Qualitative-Attribute Data Cluster...

引用

27th European Conference on Artificial Intelligence, ECAI 2024

作者： Zhao, Mingjie Feng, Sen Zhang, Yiqun Li, Mengke Lu, Yang Cheung, Yiu-Ming School of Computer Science and Technology Guangdong University of Technology Guangzhou China Shenzhen China School of Computer Science and Software Engineering Shenzhen University Shenzhen China Fujian Key Laboratory of Sensing and Computing for Smart City School of Informatics Xiamen University China Key Laboratory of Multimedia Trusted Perception and Efficient Computing Ministry of Education of China Xiamen University China Department of Computer Science Hong Kong Baptist University Hong Kong

ISBN: (纸本)9781643685489

Clustering is a fundamental approach to understanding data patterns, wherein the intuitive Euclidean distance space is commonly adopted. However, this is not the case for implicit cluster distributions reflected by qualitative attribute values, e.g., the nominal values of attributes like symptoms, marital status, etc. This paper, therefore, discovered a tree-like distance structure to flexibly represent the local order relationship among intra-attribute qualitative values. That is, treating a value as the vertex of the tree allows to capture rich order relationships among the vertex value and the others. To obtain the trees in a clustering-friendly form, a joint learning mechanism is proposed to iteratively obtain more appropriate tree structures and clusters. It turns out that the latent distance space of the whole dataset can be well-represented by a forest consisting of the learned trees. Extensive experiments demonstrate that the joint learning adapts the forest to the clustering task to yield accurate results. Comparisons of 10 counterparts on 12 real benchmark datasets with significance tests verify the superiority of the proposed method. Source code of the proposed method is available at [39]. © 2024 The Authors.

关键词： Adversarial machine learning

来源：评论

学校读者我要写书评

暂无评论

A Comprehensive Survey on Communication-Efficient Federated Learning in Mobile Edge Environments

引用

IEEE Communications Surveys and Tutorials 2025年

作者： Jia, Ninghui Qu, Zhihao Ye, Baoliu Wang, Yanyan Hu, Shihong Guo, Song Hohai University Key Laboratory of Water Big Data Technology of Ministry of Water Resources College of Computer Science and Software Engineering Nanjing211100 China Nanjing University State Key Laboratory for Novel Software Technology Department of Computer Science and Technology Nanjing210023 China The Hong Kong University of Science and Technology Department of Computer Science and Engineering Kowloon Hong Kong

In traditional centralized machine learning, transmitting raw data to a cloud center incurs high communication costs and raises privacy concerns. This is particularly challenging in mobile edge environments, where devices are dynamic and resource-constrained. Federated Learning (FL) addresses these issues by allowing devices to train models locally and upload parameters to a central server without sharing local data. However, limited wireless channel resources and dynamic transmission performance make communication overhead a major bottleneck of FL in mobile edge environments. Concerning this challenge, this survey provides a comprehensive summary of methods to improve communication efficiency in FL, focusing on: 1) minimizing communication complexity to reduce total transmission volume, 2) scheduling resources appropriately to improve training efficiency, 3) utilizing over-the-air computation (OTA) to integrate computation into communication for accommodating the computation/communication characteristics of FL in mobile edge environments. Thus, this work analyzes research from the perspective of convergence and data heterogeneity to reduce communication rounds by optimizing algorithm performance. We hope that this survey could offer insights into communication-efficient FL for future research. © 2025 IEEE.

关键词： Federated learning

来源：评论

学校读者我要写书评

暂无评论

Class-Aware Prompting for Federated Few-Shot Class-Incremental Learning

引用

IEEE Transactions on Circuits and Systems for Video Technology 2025年

作者： Liang, Fang-Yi Zhan, Yu-Wei Liu, Jiale Zhang, Chong-Yu Chen, Zhen-Duo Luo, Xin Xu, Xin-Shun Shandong University School of Software Jinan250101 China Tsinghua University Department of Computer Science and Technology Beijing100084 China Yunnan Key Laboratory of Software Engineering Kunming650504 China

Few-Shot Class-Incremental Learning (FSCIL) aims to continuously learn new classes from limited samples while preventing catastrophic forgetting. With the increasing distribution of learning data across different clients and privacy concerns, FSCIL faces a more realistic scenario where few learning samples are distributed across different clients, thereby necessitating a Federated Few-Shot Class-Incremental Learning (FedFSCIL) scenario. However, this integration faces challenges from non-IID problem, which affects model generalization and training efficiency. The communication overhead in federated settings also presents a significant challenge. To address these issues, we propose Class-Aware Prompting for Federated Few-Shot Class-Incremental Learning (FedCAP). Our framework leverages pre-trained models enhanced by a class-wise prompt pool, where shared class-wise keys enable clients to utilize global class information during training. This unifies the understanding of base class features across clients and enhances model consistency. We further incorporate a class-level information fusion module to improve class representation and model generalization. Our approach requires very few parameter transmission during model aggregation, ensuring communication efficiency. To our knowledge, this is the first study to explore the scenario of FedFSCIL. Consequently, we designed comprehensive experimental setups and made the code publicly available. © 2025 IEEE. All rights reserved.

关键词： Federated learning

来源：评论

学校读者我要写书评

暂无评论

SFPL: Improving CRS via Prompt Learning based Semantic Fusion Module

SFPL: Improving CRS via Prompt Learning based Semantic Fusio...

引用

Information Technology,Big Data and Artificial Intelligence (ICIBA), International Conference on IEEE International Conference on

作者： Ao Song Zhou Lei Shengbo Chen College of Computer Engineering and Science Shanghai University Shanghai China Shanghai Key Laboratory of Computer Software Testing and Evaluating Shanghai China

ISBN: (数字)9798350363609

ISBN: (纸本)9798350363616

With the popularity of the Internet and the massive increase of content production, users are faced with massive information and content, and it is often difficult to accurately and efficiently find the content that meets their needs. This information overload motivates the emergence of recommendation systems that help users filter and recommend relevant content. Dialogue recommendation systems (CRS) aim to dynamically recommend high-quality items to users while eliciting user preferences by means of dialogue. CRS mainly consists of two parts, namely, the dialogue module and the recommendation module. An efficient CRS is to solve the problems existing in the two sub-tasks of dialogue and recommendation, namely, generating appropriate responses in dialogue and predicting user favorite items in recommendation. In addition, it is necessary to adopt a suitable method to integrate these two modules to ensure the accuracy of CRS recommendation. This paper proposes an efficient way of semantic fusion module to solve the integration problem of two modules, and proves the effectiveness of our method through experiments

关键词： Semantics Training data Production Learning (artificial intelligence) Knowledge representation Information filters Data models Internet Information technology Recommender systems

来源：评论

学校读者我要写书评

暂无评论

CODESWIFT: Accelerating LLM Inference for Efficient Code Generation

arXiv

引用

arXiv 2025年

作者： Zhao, Qianhui Zhang, Li Liu, Fang Lian, Xiaoli Meng, Qiaoyuanhe Jiao, Ziqian Zhou, Zetong Zhang, Borui Guo, Runlin Li, Jia School of Computer Science and Engineering State Key Laboratory of Complex & Critical Software Environment Beihang University China Key Lab of High Confidence Software Technology Peking University China

Code generation is a latency-sensitive task that demands high timeliness, but the autoregressive decoding mechanism of Large Language Models (LLMs) leads to poor inference efficiency. Existing LLM inference acceleration methods mainly focus on standalone functions using only built-in components. Moreover, they treat code like natural language sequences, ignoring its unique syntax and semantic characteristics. As a result, the effectiveness of these approaches in code generation tasks remains limited and fails to align with real-world programming scenarios. To alleviate this issue, we propose CODESWIFT, a simple yet highly efficient inference acceleration approach specifically designed for code generation, without comprising the quality of the output. CODESWIFT constructs a multi-source datastore, providing access to both general and project-specific knowledge, facilitating the retrieval of high-quality draft sequences. Moreover, CODESWIFT reduces retrieval cost by controlling retrieval timing, and enhances efficiency through parallel retrieval and a context- and LLM preference-aware cache. Experimental results show that CODESWIFT can reach up to 2.53× and 2.54× speedup compared to autoregressive decoding in repository-level and standalone code generation tasks, respectively, outperforming state-of-the-art inference acceleration approaches by up to 88%. Our code and data are available at https://***/r/CodeSwift. Copyright © 2025, The Authors. All rights reserved.

关键词： Decoding

来源：评论

学校读者我要写书评

暂无评论

A Neural Architecture Search Method with Multi-Dimensional Correlation Representation for Fault Diagnosis 9

A Neural Architecture Search Method with Multi-Dimensional C...

引用

9th IEEE Smart World Congress, SWC 2023

作者： Liu, Yumeng Wang, Tingqi Zheng, Xu Tian, Ling Wang, Hongan Institute of Software Chinese Academy of Sciences Beijing Key Laboratory of Human-Computer Interaction Beijing China University of Chinese Academy of Sciences School of Computer Science and Technology Beijing China University of Electronic Science and Technology of China School of Computer Science and Engineering Chengdu China

ISBN: (纸本)9798350319804

As deep learning has become more widely used for fault diagnosis, the shortcomings of model transferability and human model design costs are growing increasingly evident. The current work has tackled each of these two issues. Domain adaptation methods can provide transportability by mapping data to a common domain. For the consumption of manually constructed networks, Neural Architecture Search (NAS) is an intelligent solution that automatically select the ideal network structure. Unfortunately, there is little work to connect these two methodologies in fault diagnosis. Furthermore, these approaches are incapable in extracting internal correlations from instrumental signal data. In this paper, we propose a transferrable NAS fault diagnosis method, which maximizes savings in manual costs. We also integrate graph neural networks (GNN) in a stepwise manner into the NAS and domain adaptation to provide a multi-dimension correlation representation. Firstly, we structure the signal data into graph based on temporal relationships. Then, NAS designs automatically for selecting the optimal network structure and useful features. In NAS, graph encoder of GNN is used to establish internal relationships. Finally, domain adaptation provides transferability. At the same time, GNN's weights in domain adaptation help to further mine data internal associations. We tested our method on two publicly available datasets and performed comparisons. The experimental results show the proposed method has superior performance. © 2023 IEEE.

关键词： Domain Adaptation Fault Diagnosis Graph Neural Network Neural Architecture Search

来源：评论

学校读者我要写书评

暂无评论

dgQuEST: Accelerating Large Scale Quantum Circuit Simulation through Hybrid CPU-GPU Memory Hierarchies 18th

dgQuEST: Accelerating Large Scale Quantum Circuit Simulation...

引用

18th IFIP WG 10.3 International Conference on Network and Parallel Computing, NPC 2021

作者： Feng, Tianyu Chen, Siyan You, Xin Zhong, Shuzhang Yang, Hailong Luan, Zhongzhi Qian, Depei State Key Laboratory of Software Development Environment Beihang University Beijing China School of Computer Science and Engineering Beihang University Beijing China

ISBN: (纸本)9783030935702

With the advancement of quantum computing, verifying the correctness of the quantum circuits becomes critical while developing new quantum algorithms. Constrained by the obstacles of building practical quantum computers, quantum circuit simulation has become a feasible approach to develop and verify quantum algorithms. Although there are many quantum simulators available, they either achieve low performance on CPUs, or limited simulation scale (e.g., number of qubits) on GPUs due to limited memory capacity. Therefore, we propose dgQuEST, a novel acceleration method that utilizes hybrid CPU-GPU memory hierarchies for large-scale quantum circuit simulation across multiple nodes. dgQuEST adopts efficient memory management and communication schemes to leverage the distributed CPU and GPU memories for accelerating large-scale quantum simulation. Our evaluation demonstrates that dgQuEST achieves an average speedup of 403 × compared to QuEST on quantum circuit simulation with 32 qubits, and scales to quantum circuit simulation with 35 qubits on two GPU nodes, far beyond the state-of-the-art implementation HyQuas can support. © 2022, IFIP International Federation for Information Processing.

关键词： Circuit simulation

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：