检索结果-内蒙古大学图书馆

Proceedings of the 38th International Conference on Neural Information Processing Systems

作者： Weixin An Yuanyuan Liu Fanhua Shang Hongying Liu Key Laboratory of Intelligent Perception and Image Understanding of Ministry of Education School of Artificial Intelligence Xidian University China College of Intelligence and Computing Tianjin University China Medical School Tianjin University China and Peng Cheng Lab Shenzhen China

ISBN: (纸本)9798331314385

Many zeroth-order (ZO) optimization algorithms have been developed to solve nonconvex minimax problems in machine learning and computer vision areas. However, existing ZO minimax algorithms have high complexity and rely on some strict restrictive conditions for ZO estimations. To address these issues, we design a new unified ZO gradient descent extragradient ascent (ZO-GDEGA) algorithm, which reduces the overall complexity to O(dε-6) to find an ε-stationary point of the function ψ for nonconvex-concave (NC-C) problems, where d is the variable dimension. To the best of our knowledge, ZO-GDEGA is the first ZO algorithm with complexity guarantees to solve stochastic NC-C problems. Moreover, ZO-GDEGA requires weaker conditions on the ZO estimations and achieves more robust theoretical results. As a by-product, ZO-GDEGA has advantages on the condition number for the NC-strongly concave case. Experimentally, ZO-GDEGA can generate more effective poisoning attack data with an average accuracy reduction of 5%. The improved AUC performance also verifies the robustness of gradient estimations.

关键词：

来源：评论

学校读者我要写书评

暂无评论

AttentionPredictor: Temporal Pattern Matters for Efficient LLM Inference

arXiv

引用

arXiv 2025年

作者： Yang, Qingyue Wang, Jie Li, Xing Wang, Zhihai Chen, Chen Chen, Lei Yu, Xianzhi Liu, Wulong Hao, Jianye Yuan, Mingxuan Li, Bin MoE Key Laboratory of Brain-inspired Intelligent Perception and Cognition University of Science and Technology of China China Noah’s Ark Lab Huawei Technologies China College of Intelligence and Computing Tianjin University China

With the development of large language models (LLMs), efficient inference through key-Value (KV) cache compression has attracted considerable attention, especially for long-context generation. To compress the KV cache, recent methods identify critical KV tokens through heuristic ranking with attention scores. However, these methods often struggle to accurately determine critical tokens as they neglect the temporal patterns in attention scores, resulting in a noticeable degradation in LLM performance. To address this challenge, we propose AttentionPredictor, which is the first learning-based critical token identification approach. Specifically, AttentionPredictor learns a lightweight convolution model to capture spatiotemporal patterns and predict the next-token attention score. An appealing feature of AttentionPredictor is that it accurately predicts the attention score while consuming negligible memory. Moreover, we propose a cross-token critical cache prefetching framework that hides the token estimation time overhead to accelerate the decoding stage. By retaining most of the attention information, AttentionPredictor achieves 16× KV cache compression with comparable LLM performance, significantly outperforming the state-of-the-art. Copyright © 2025, The Authors. All rights reserved.

关键词： Cache memory

来源：评论

学校读者我要写书评

暂无评论

Towards next-generation logic synthesis: a scalable neural circuit generation framework 24

Towards next-generation logic synthesis: a scalable neural c...

引用

Proceedings of the 38th International Conference on Neural Information Processing Systems

作者： Zhihai Wang Jie Wang Qingyue Yang Yinqi Bai Xing Li Lei Chen Jianye Hao Mingxuan Yuan Bin Li Yongdong Zhang Feng Wu MoE Key Laboratory of Brain-inspired Intelligent Perception and Cognition University of Science and Technology of China Noah's Ark Lab Huawei Technologies Noah's Ark Lab Huawei Technologies and College of Intelligence and Computing Tianjin University

ISBN: (纸本)9798331314385

Logic Synthesis (LS) aims to generate an optimized logic circuit satisfying a given functionality, which generally consists of circuit translation and optimization. It is a challenging and fundamental combinatorial optimization problem in integrated circuit design. Traditional LS approaches rely on manually designed heuristics to tackle the LS task, while machine learning recently offers a promising approach towards next-generation logic synthesis by neural circuit generation and optimization. In this paper, we first revisit the application of differentiable neural architecture search (DNAS) methods to circuit generation and found from extensive experiments that existing DNAS methods struggle to exactly generate circuits, scale poorly to large circuits, and exhibit high sensitivity to hyper-parameters. Then we provide three major insights for these challenges from extensive empirical analysis: 1) DNAS tends to overfit to too many skip-connections, consequently wasting a significant portion of the network's expressive capabilities; 2) DNAS suffers from the structure bias between the network architecture and the circuit inherent structure, leading to inefficient search; 3) the learning difficulty of different input-output examples varies significantly, leading to severely imbalanced learning. To address these challenges in a systematic way, we propose a novel regularized triangle-shaped circuit network generation framework, which leverages our key insights for completely accurate and scalable circuit generation. Furthermore, we propose an evolutionary algorithm assisted by reinforcement learning agent restarting technique for efficient and effective neural circuit optimization. Extensive experiments on four different circuit benchmarks demonstrate that our method can precisely generate circuits with up to 1200 nodes. Moreover, our synthesized circuits significantly outperform the state-of-the-art results from several competitive winners in IWLS 2022 and 2023 competiti

关键词：

来源：评论

学校读者我要写书评

暂无评论

Towards general algorithm discovery for combinatorial optimization: learning symbolic branching policy from bipartite graph 24

Towards general algorithm discovery for combinatorial optimi...

引用

Proceedings of the 41st International Conference on Machine Learning

作者： Yufei Kuang Jie Wang Yuyan Zhou Xijun Li Fangzhou Zhu Jianye Hao Feng Wu CAS Key Laboratory of Technology in GIPAS and MoE Key Laboratory of Brain-inspired Intelligent Perception and Cognition University of Science and Technology of China CAS Key Laboratory of Technology in GIPAS and MoE Key Laboratory of Brain-inspired Intelligent Perception and Cognition University of Science and Technology of China and Noah's Ark Lab Huawei Technologies Noah's Ark Lab Huawei Technologies Noah's Ark Lab Huawei Technologies and College of Intelligence and Computing Tianjin University

Machine learning (ML) approaches have been successfully applied to accelerating exact combinatorial optimization (CO) solvers. However, many of them fail to explain what patterns they have learned that accelerate the CO algorithms due to the black-box nature of ML models like neural networks, and thus they prevent researchers from further understanding the tasks they are interested in. To tackle this problem, we propose the first graph-based algorithm discovery framework--namely, graph symbolic discovery for exact combinatorial optimization solver (GS4CO)--that learns interpretable branching policies directly from the general bipartite graph representation of CO problems. Specifically, we mainly focus on the variable selection part of the branching policy. We design a unified representation for symbolic variable selection policies with graph inputs, and then we employ a Transformer with multiple treestructural encodings to generate symbolic trees end-to-end, which effectively reduces the cumulative error from iteratively distilling graph neural networks. Experiments show that GS4CO learned interpretable and lightweight policies outperform all the baselines on CPU machines, including both the human-designed and the learning-based. GS4CO shows an encouraging step towards general algorithm discovery on modern CO solvers. Codes are available at https://***/MIRALab-USTC/L2O-GS4CO.

关键词：

来源：评论

学校读者我要写书评

暂无评论

A circuit domain generalization framework for efficient logic synthesis in chip design 24

A circuit domain generalization framework for efficient logi...

引用

Proceedings of the 41st International Conference on Machine Learning

作者： Zhihai Wang Lei Chen Jie Wang Yinqi Bai Xing Li Xijun Li Mingxuan Yuan Jianye Hao Yongdong Zhang Feng Wu CAS Key Laboratory of Technology in GIPAS & MoE Key Laboratory of Brain-inspired Intelligent Perception and Cognition University of Science and Technology of China Noah's Ark Lab Huawei Technologies Noah's Ark Lab Huawei Technologies and College of Intelligence and Computing Tianjin University

Logic Synthesis (LS) plays a vital role in chip design. A key task in LS is to simplify circuits--modeled by directed acyclic graphs (DAGs)--with functionality-equivalent transformations. To tackle this task, many LS heuristics apply transformations to subgraphs--rooted at each node on an input DAG--sequentially. However, we found that a large number of transformations are ineffective, which makes applying these heuristics highly time-consuming. In particular, we notice that the runtime of the Resub and Mfs2 heuristics often dominates the overall runtime of LS optimization processes. To address this challenge, we propose a novel data-driven LS heuristic paradigm, namely PruneX, to reduce ineffective transformations. The major challenge of developing PruneX is to learn models that well generalize to unseen circuits, i.e., the out-of-distribution (OOD) generalization problem. Thus, the major technical contribution of PruneX is the novel circuit domain generalization framework, which learns domain-invariant representations based on the transformation-invariant domain-knowledge. To the best of our knowledge, PruneX is the first approach to tackle the OOD problem in LS heuristics. We integrate PruneX with the aforementioned Resub and Mfs2 heuristics. Experiments demonstrate that PruneX significantly improves their efficiency while keeping comparable optimization performance on industrial and very largescale circuits, achieving up to 3.1× faster runtime.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Learning to Learn Transferable Generative Attack for Person Re-Identification

引用

IEEE Transactions on Image Processing 2025年 PP卷 PP页

作者： Bian, Yuan Liu, Min Wang, Xueping Ma, Yunfeng Wang, Yaonan Hunan University National Engineering Research Center of Robot Visual Perception and Control Technology College of Electrical and Information Engineering Hunan Changsha China Hunan Normal University Hunan Provincial Key Laboratory of Intelligent Computing and Language Information Processing College of Information Science and Engineering Hunan Changsha China

Deep learning-based person re-identification (reid) models are widely employed in surveillance systems and inevitably inherit the vulnerability of deep networks to adversarial attacks. Existing attacks merely consider cross-dataset and cross-model transferability, ignoring the cross-test capability to perturb models trained in different domains. To powerfully examine the robustness of real-world re-id models, the Meta Transferable Generative Attack (MTGA) method is proposed, which adopts meta-learning optimization to promote the generative attacker producing highly transferable adversarial examples by learning comprehensively simulated transfer-based crossmodel&dataset&test black-box meta attack tasks. Specifically, cross-model&dataset black-box attack tasks are first mimicked by selecting different re-id models and datasets for meta-train and meta-test attack processes. As different models may focus on different feature regions, the Perturbation Random Erasing module is further devised to prevent the attacker from learning to only corrupt model-specific features. To boost the attacker learning to possess cross-test transferability, the Normalization Mix strategy is introduced to imitate diverse feature embedding spaces by mixing multi-domain statistics of target models. Extensive experiments show the superiority of MTGA, especially in cross-model&dataset and cross-model&dataset&test attacks, our MTGA outperforms the SOTA methods by 20.0% and 11.3% on mean mAP drop rate, respectively. © 1992-2012 IEEE.

关键词： Generative adversarial networks

来源：评论

学校读者我要写书评

暂无评论

Predicting Gradient is Better: Exploring Self-Supervised Learning for SAR ATR with a Joint-Embedding Predictive Architecture

arXiv

引用

arXiv 2023年

作者： Li, Weijie Wei, Yang Liu, Tianpeng Hou, Yuenan Li, Yuxuan Liu, Zhen Liu, Yongxiang Liu, Li The College of Electronic Science and Technology National University of Defense Technology Changsha410073 China The Shanghai AI Laboratory Shanghai200000 China The Visual Computing and Intelligent Perception Lab Nankai University Tianjin300071 China

The growing Synthetic Aperture Radar (SAR) data can build a foundation model using self-supervised learning (SSL) methods, which can achieve various SAR automatic target recognition (ATR) tasks with pretraining in large-scale unlabeled data and fine-tuning in small-labeled samples. SSL aims to construct supervision signals directly from the data, minimizing the need for expensive expert annotation and maximizing the use of the expanding data pool for a foundational model. This study investigates an effective SSL method for SAR ATR, which can pave the way for a foundation model in SAR ATR. The primary obstacles faced in SSL for SAR ATR are small targets in remote sensing and speckle noise in SAR images, corresponding to the SSL approach and signals. To overcome these challenges, we present a novel joint-embedding predictive architecture for SAR ATR (SAR-JEPA) that leverages local masked patches to predict the multi-scale SAR gradient representations of an unseen context. The key aspect of SAR-JEPA is integrating SAR domain features to ensure high-quality self-supervised signals as target features. In addition, we employ local masks and multi-scale features to accommodate various small targets in remote sensing. By fine-tuning and evaluating our framework on three target recognition datasets (vehicle, ship, and aircraft) with four other datasets as pretraining, we demonstrate its outperformance over other SSL methods and its effectiveness as the SAR data increases. This study demonstrates the potential of SSL for the recognition of SAR targets across diverse targets, scenes, and sensors. Our codes and weights are available in https://***/waterdisappear/SAR-JEPA. Copyright © 2023, The Authors. All rights reserved.

关键词： Self-supervised learning

来源：评论

学校读者我要写书评

暂无评论

A hierarchical adaptive multi-task reinforcement learning framework for multiplier circuit design 24

A hierarchical adaptive multi-task reinforcement learning fr...

引用

Proceedings of the 41st International Conference on Machine Learning

作者： Zhihai Wang Jie Wang Dongsheng Zuo Yunjie Ji Xilin Xia Yuzhe Ma Jianye Hao Mingxuan Yuan Yongdong Zhang Feng Wu CAS Key Laboratory of Technology in GIPAS & MoE Key Laboratory of Brain-inspired Intelligent Perception and Cognition University of Science and Technology of China Microelectronics Thrust Hong Kong University of Science and Technology (Guangzhou) Noah's Ark Lab Huawei Technologies and College of Intelligence and Computing Tianjin University Noah's Ark Lab Huawei Technologies

Multiplier design--which aims to explore a large combinatorial design space to simultaneously optimize multiple conflicting objectives--is a fundamental problem in the integrated circuits industry. Although traditional approaches tackle the multi-objective multiplier optimization problem by manually designed heuristics, reinforcement learning (RL) offers a promising approach to discover high-speed and area-efficient multipliers. However, the existing RL-based methods struggle to find Pareto-optimal circuit designs for all possible preferences, i.e., weights over objectives, in a sample-efficient manner. To address this challenge, we propose a novel hierarchical adaptive (HAVE) multi-task reinforcement learning framework. The hierarchical framework consists of a meta-agent to generate diverse multiplier preferences, and an adaptive multi-task agent to collaboratively optimize multipliers conditioned on the dynamic preferences given by the meta-agent. To the best of our knowledge, HAVE is the first to well approximate Pareto-optimal circuit designs for the entire preference space with high sample efficiency. Experiments on multipliers across a wide range of input widths demonstrate that HAVE significantly Pareto-dominates state-of-the-art approaches, achieving up to 28% larger hypervolume. Moreover, experiments demonstrate that multipliers designed by HAVE can well generalize to large-scale computation-intensive circuits.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Learning to Learn Transferable Generative Attack for Person Re-Identification

arXiv

引用

arXiv 2024年

作者： Bian, Yuan Liu, Min Wang, Xueping Ma, Yunfeng Wang, Yaonan The College of Electrical and Information Engineering Hunan University National Engineering Research Center of Robot Visual Perception and Control Technology Hunan Changsha China The College of Information Science and Engineering Hunan Normal University Hunan Provincial Key Laboratory of Intelligent Computing and Language Information Processing Hunan Changsha China

Deep learning-based person re-identification (re-id) models are widely employed in surveillance systems and inevitably inherit the vulnerability of deep networks to adversarial attacks. Existing attacks merely consider cross-dataset and cross-model transferability, ignoring the cross-test capability to perturb models trained in different domains. To powerfully examine the robustness of real-world re-id models, the Meta Transferable Generative Attack (MTGA) method is proposed, which adopts meta-learning optimization to promote the generative attacker producing highly transferable adversarial examples by learning comprehensively simulated transfer-based cross-model&dataset&test black-box meta attack tasks. Specifically, cross-model&dataset black-box attack tasks are first mimicked by selecting different re-id models and datasets for meta-train and meta-test attack processes. As different models may focus on different feature regions, the Perturbation Random Erasing module is further devised to prevent the attacker from learning to only corrupt model-specific features. To boost the attacker learning to possess cross-test transferability, the Normalization Mix strategy is introduced to imitate diverse feature embedding spaces by mixing multi-domain statistics of target models. Extensive experiments show the superiority of MTGA, especially in cross-model&dataset and cross-model&dataset&test attacks, our MTGA outperforms the SOTA methods by 21.5% and 11.3% on mean mAP drop rate, respectively. The code of MTGA will be released after the paper is accepted. Copyright © 2024, The Authors. All rights reserved.

关键词： Generative adversarial networks

来源：评论

学校读者我要写书评

暂无评论

High-Speed Trajectory Tracking Control for Quadrotors via Deep Reinforcement Learning

High-Speed Trajectory Tracking Control for Quadrotors via De...

引用

Youth Academic Annual Conference of Chinese Association of Automation (YAC)

作者： Hean Hua Hang Zhong Hui Zhang Yongchun Fang Yaonan Wang College of Electrical and Information Engineering Hunan University Changsha China National Engineering Research Center for Robot Visual Perception and Control Technology Changsha China School of Robotics Hunan University Changsha China Institute of Robotics and Automatic Information System College of Artificial Intelligence Nankai University Tianjin Key Laboratory of Intelligent Robotics Nankai University Tianjin China

ISBN: (数字)9798350379228

ISBN: (纸本)9798350390780

This paper presents a learning-based high-speed trajectory tracking control strategy for quadrotors, which achieves efficient learning and strong reliability by the collaboration of deep reinforcement learning (RL) and self-tuning mechanism. Different from existing methods, the proposed strategy is designed to explore optimal control performance by taking advantage of model-based self-tuning mechanism and deep reinforcement learning. Specifically, the self-tuning guided deep RL scheme is put forward for quadrotors, with superior learning efficiency and strong adaptability. Firstly, a novel self-tuning mechanism is constructed and some auxiliary variables are introduced to enhance the tracking performance. Then, based on the model-driven self-tuning design, the deep RL is proposed to achieve model-guided learning, where the tuning actions are adopted in the evaluation process during training, aiming at removing the bad explorations by the carefully designed parallel evaluation. Finally, the convergence is analyzed based on the proposed learning framework, which indicates the efficient cooperation of exploration and self-tuning mechanism. To verify the effectiveness of the proposed controller, the guided training and hardware experiments are implemented to show efficient cooperation and satisfactory high-speed trajectory tracking control of the proposed method.

关键词： Training Adaptation models Trajectory tracking Heuristic algorithms Optimal control Deep reinforcement learning Hardware

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：