检索结果-内蒙古大学图书馆

36th Conference on Neural Information Processing Systems, NeurIPS 2022

作者： Luo, Shengjie Li, Shanda Zheng, Shuxin Liu, Tie-Yan Wang, Liwei He, Di National Key Laboratory of General Artificial Intelligence School of Intelligence Science and Technology Peking University China Machine Learning Department School of Computer Science Carnegie Mellon University United States Microsoft Research United States Center for Data Science Peking University China Zhejiang Lab China

ISBN: (纸本)9781713871088

Relative Positional Encoding (RPE), which encodes the relative distance between any pair of tokens, is one of the most successful modifications to the original Transformer. As far as we know, theoretical understanding of the RPE-based Transformers is largely unexplored. In this work, we mathematically analyze the power of RPE-based Transformers regarding whether the model is capable of approximating any continuous sequence-to-sequence functions. One may naturally assume the answer is in the affirmative-RPE-based Transformers are universal function approximators. However, we present a negative result by showing there exist continuous sequence-to-sequence functions that RPE-based Transformers cannot approximate no matter how deep and wide the neural network is. One key reason lies in that most RPEs are placed in the softmax attention that always generates a right stochastic matrix. This restricts the network from capturing positional information in the RPEs and limits its capacity. To overcome the problem and make the model more powerful, we first present sufficient conditions for RPE-based Transformers to achieve universal function approximation. With the theoretical guidance, we develop a novel attention module, called Universal RPE-based (URPE) Attention, which satisfies the conditions. Therefore, the corresponding URPE-based Transformers become universal function approximators. Extensive experiments covering typical architectures and tasks demonstrate that our model is parameter-efficient and can achieve superior performance to strong baselines in a wide range of applications. The code will be made publicly available at https://***/lsj2408/URPE. © 2022 Neural information processing systems foundation. All rights reserved.

关键词： Encoding (symbols)

来源：评论

学校读者我要写书评

暂无评论

GraphMorph: Tubular Structure Extraction by Morphing Predicted Graphs

arXiv

引用

arXiv 2025年

作者： Zhang, Zhao Zhao, Ziwei Wang, Dong Wang, Liwei Center for Data Science Peking University China Yizhun Medical AI Co. Ltd China State Key Laboratory of General Artificial Intelligence School of Intelligence Science and Technology Peking University China Center for Machine Learning Research Peking University China China

Accurately restoring topology is both challenging and crucial in tubular structure extraction tasks, such as blood vessel segmentation and road network extraction. Diverging from traditional approaches based on pixel-level classification, our proposed method, named GraphMorph, focuses on branch-level features of tubular structures to achieve more topologically accurate predictions. GraphMorph comprises two main components: a Graph Decoder and a Morph Module. Utilizing multi-scale features extracted from an image patch by the segmentation network, the Graph Decoder facilitates the learning of branch-level features and generates a graph that accurately represents the tubular structure in this patch. The Morph Module processes two primary inputs: the graph and the centerline probability map, provided by the Graph Decoder and the segmentation network, respectively. Employing a novel SkeletonDijkstra algorithm, the Morph Module produces a centerline mask that aligns with the predicted graph. Furthermore, we observe that employing centerline masks predicted by GraphMorph significantly reduces false positives in the segmentation task, which is achieved by a simple yet effective post-processing strategy. The efficacy of our method in the centerline extraction and segmentation tasks has been substantiated through experimental evaluations across various datasets. Source code will be released soon. Copyright © 2025, The Authors. All rights reserved.

关键词： Image segmentation

来源：评论

学校读者我要写书评

暂无评论

Spatially-Consistent Implicit Volumetric Function for Uni- and Bi-Planar X-Ray-Based Computed Tomography Reconstruction

Spatially-Consistent Implicit Volumetric Function for Uni- a...

引用

IEEE International Symposium on Biomedical Imaging

作者： Yikun Jiang Xiaoru Yuan Yuru Pei Center for Data Science Peking University Beijing China Key Laboratory of Machine Perception School of Intelligence Science and Technology Peking University Beijing China

We introduce a spatially-consistent implicit f unction representation (sci-f) for high-resolution volumetric computed tomography (CT) image recovery from uni- and bi-planar X-ray images. We devise a deep end-to-end learning scheme to parameterize the unified implicit function conditioned on the input 2D X-rays and predict the 3D anatomies. Instead of the discretized voxel representation in the existing deep learning-based CT reconstruction from sparse 2D X-rays, the lightweight and memory-efficient sci-f enables a volume recovery with continuous resolutions. The sci-f bridges the 2D pixels with their counterpart voxel through the 3D ray for generalized and spatially consistent 3D reconstruction of anatomical structures, instead of existing per-subject learning of neural representation. Experiments demonstrate the efficacy of the proposed approach in X-ray-based volume recovery from clinically obtained X-rays, with performance gains over state-of-the-art deep X-ray-based CT reconstruction.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Towards Highly Efficient Anomaly Detection for Predictive Maintenance

Towards Highly Efficient Anomaly Detection for Predictive Ma...

引用

International Conference on machine Learning and Applications (ICMLA)

作者： Simon Klüttermann Vanlal Peka Philipp Doebler Emmanuel Müller TU Dortmund University Dortmund Germany Lamarr Institute for Machine Learning and Artificial Intelligence Dortmund Germany Research Center Trustworthy Data Science and Security Dortmund Germany

ISBN: (数字)9798350374889

ISBN: (纸本)9798350374896

This paper introduces SEAN, a novel anomaly detection algorithm designed for real-time applications in predictive maintenance. SEAN leverages an ensemble-based approach to deliver competitive performance while drastically reducing computational costs. In our comprehensive evaluation across 121 datasets, SEAN consistently outperforms comparable shallow anomaly detection algorithms. Our comparisons reveal that SEAN operates over 20,000 times faster than a similar state-of-the-art deep learning alternative, with negligible sacrifice in detection accuracy. We further demonstrate SEAN's versatility through an ablation study, highlighting how its hyperparameters can be tuned to balance runtime and performance effectively. Additionally, we present a practical C++ export tool that enables the deployment of SEAN on resource-constrained devices, meeting the stringent requirements of on-device predictive maintenance tasks. Our findings underscore SEAN as a powerful and efficient solution for anomaly detection in real-world engineering applications.

关键词： Performance evaluation Accuracy Runtime C++ languages Prediction algorithms Real-time systems Computational efficiency Time factors Anomaly detection Predictive maintenance

来源：评论

学校读者我要写书评

暂无评论

NC-ALG: Graph-Based Active Learning under Noisy Crowd 40

NC-ALG: Graph-Based Active Learning under Noisy Crowd

引用

40th IEEE International Conference on data Engineering, ICDE 2024

作者： Zhang, Wentao Wang, Yexin You, Zhenbang Li, Yang Cao, Gang Yang, Zhi Cui, Bin Center for Machine Learning Research Peking University China Key Lab of High Confidence Software Technologies Peking University China Institute of Advanced Algorithms Research Shanghai China Institute of Computational Social Science Peking University Qingdao China National Engineering Labratory for Big Data Analytics and Applications China TEG Tencent Inc. Department of Data Platform China Beijing Academy of Artificial Intelligence China

ISBN: (纸本)9798350317152

Graph Neural Networks (GNNs) have achieved great success in various data mining tasks but they heavily rely on a large number of annotated nodes, requiring considerable human efforts. Despite the effectiveness of existing GNN-based Active Learning (AL) methods, they assume that the annotated labels are always correct, which is contradictory to the error-prone labeling process in a practical crowdsourcing environment. Besides, due to this impractical assumption, existing works only focus on optimizing the node selection in AL but neglect optimizing the labeling process. Therefore, we present NC-ALG, the first GNN-based AL framework that optimizes both the node selection and node labeling process under a noisy crowd. For node selection, NC-ALG introduces a new measurement to model influence reliability and an effective influence maximization objective to select nodes. For node labeling, NC-ALG significantly reduces the labeling cost by considering the model-predicted labels and the labels of mirror nodes. To the best of our knowledge, this is the first attempt to consider GNN-based AL under the practical noisy crowd. Empirical studies on public datasets demonstrate that NC-ALG significantly outperforms existing methods in terms labeling efficiency. Notably, it only takes NC-ALG one-third of the labeling budget that the competitive baseline GRAIN needs to achieve an accuracy of 70.7 % on PubMed. © 2024 IEEE.

关键词： Crowdsourcing

来源：评论

学校读者我要写书评

暂无评论

A REDUCTION-BASED FRAMEWORK FOR CONSERVATIVE BANDITS AND REINFORCEMENT LEARNING 10

A REDUCTION-BASED FRAMEWORK FOR CONSERVATIVE BANDITS AND REI...

引用

10th International Conference on Learning Representations, ICLR 2022

作者： Yang, Yunchang Wu, Tianhao Zhong, Han Garcelon, Evrard Pirotta, Matteo Lazaric, Alessandro Wang, Liwei Du, Simon S. Center for Data Science Peking University China University of California Berkeley United States Facebook AI Research Key Laboratory of Machine Perception MOE School of Artificial Intelligence Peking University China International Center for Machine Learning Research Peking University China University of Washington United States

We study bandits and reinforcement learning (RL) subject to a conservative constraint where the agent is asked to perform at least as well as a given baseline policy. This setting is particular relevant in real-world domains including digital marketing, healthcare, production, finance, etc. In this paper, we present a reduction-based framework for conservative bandits and RL, in which our core technique is to calculate the necessary and sufficient budget obtained from running the baseline policy. For lower bounds, we improve the existing lower bound for conservative multi-armed bandits and obtain new lower bounds for conservative linear bandits, tabular RL and low-rank MDP, through a black-box reduction that turns a certain lower bound in the nonconservative setting into a new lower bound in the conservative setting. For upper bounds, in multi-armed bandits, linear bandits and tabular RL, our new upper bounds tighten or match existing ones with significantly simpler analyses. We also obtain a new upper bound for conservative low-rank MDP. © 2022 ICLR 2022 - 10th International Conference on Learning Representationss. All rights reserved.

关键词： Reinforcement learning

来源：评论

学校读者我要写书评

暂无评论

Reinforcement Learning as an Improvement Heuristic for Real-World Production Scheduling

arXiv

引用

arXiv 2024年

作者： Müller, Arthur Vollenkemper, Lukas Department of Machine Intelligence Fraunhofer IOSB-INA Lemgo32657 Germany Center for Applied Data Science Bielefeld University of Applied Sciences and Arts Gütersloh33330 Germany

The integration of Reinforcement Learning (RL) with heuristic methods is an emerging trend for solving optimization problems, which leverages RL’s ability to learn from the data generated during the search process. One promising approach is to train an RL agent as an improvement heuristic, starting with a suboptimal solution that is iteratively improved by applying small changes. We apply this approach to a real-world multi-objective production scheduling problem. Our approach utilizes a network architecture that includes Transformer encoding to learn the relationships between jobs. Afterwards, a probability matrix is generated from which pairs of jobs are sampled and then swapped to improve the solution. We benchmarked our approach against other heuristics using real data from our industry partner, demonstrating its superior performance. Copyright © 2024, The Authors. All rights reserved.

关键词： Reinforcement learning

来源：评论

学校读者我要写书评

暂无评论

Securing Consumer IoT Swarms Using Graph Transformers and SRAM for Firmware Attestation

Securing Consumer IoT Swarms Using Graph Transformers and SR...

引用

Consumer Communications and Networking Conference, CCNC IEEE

作者： Varun Kohli Bhavya Kohli Muhammad Naveed Aman Biplab Sikdar Department of Electrical and Computer Engineering National University of Singapore Singapore Center for Machine Intelligence and Data Science Indian Institute of Technology Bombay India School of Computing University of Nebraska-Lincoln USA

ISBN: (数字)9798331508050

ISBN: (纸本)9798331508067

Consumer Internet of Things (IoT) networks have gained widespread popularity due to their convenience, automation, and security provisions in personal and home environments. Ubiquitous resource-constrained devices, however, are plagued with security issues that often arise from firmware-related issues and their propagated effects. While various studies on firmware attestation are available, they require firmware copies, specific hardware, and complex computation on the IoT device. This paper presents a study on the application of Graph Transformer Networks (GTN) in verifying the firmware integrity of consumer IoT swarms using SRAM as an attestation feature. The proposed method achieves an overall 0.99 accuracy on authentic samples from development and physical twin networks, 0.99 on malware, and 0.97 on propagated misbehavior at a $\sim 10^{-4}$ second inference latency on a laptop CPU.

关键词： Portable computers Accuracy Random access memory Transformers Malware Graph neural networks Hardware Internet of Things Security Microprogramming

来源：评论

学校读者我要写书评

暂无评论

Smaller Batches, Bigger Gains? Investigating the Impact of Batch Sizes on Reinforcement Learning Based Real-World Production Scheduling

arXiv

引用

arXiv 2024年

作者： Müller, Arthur Grumbach, Felix Sabatelli, Matthia Department of Machine Intelligence Fraunhofer IOSB-INA Lemgo32657 Germany Center for Applied Data Science Hochschule Bielefeld Gütersloh33330 Germany Department of Artificial Intelligence University of Groningen Groningen9712 CP Netherlands

Production scheduling is an essential task in manufacturing, with Reinforcement Learning (RL) emerging as a key solution. In a previous work, RL was utilized to solve an extended permutation flow shop scheduling problem (PFSSP) for a real-world production line with two stages, linked by a central buffer. The RL agent was trained to sequence equallysized product batches to minimize setup efforts and idle times. However, the substantial impact caused by varying the size of these product batches has not yet been explored. In this follow-up study, we investigate the effects of varying batch sizes, exploring both the quality of solutions and the training dynamics of the RL agent. The results demonstrate that it is possible to methodically identify reasonable boundaries for the batch size. These boundaries are determined on one side by the increasing sample complexity associated with smaller batch sizes, and on the other side by the decreasing flexibility of the agent when dealing with larger batch sizes. This provides the practitioner the ability to make an informed decision regarding the selection of an appropriate batch size. Moreover, we introduce and investigate two new curriculum learning strategies to enable the training with small batch sizes. The findings of this work offer the potential for application in several industrial use cases with comparable scheduling problems. Copyright © 2024, The Authors. All rights reserved.

关键词： Reinforcement learning

来源：评论

学校读者我要写书评

暂无评论

Optimal bounds for p sensitivity sampling via 2 augmentation

arXiv

引用

arXiv 2024年

作者： Munteanu, Alexander Omlor, Simon Dortmund Data Science Center Faculties of Statistics and Computer Science TU Dortmund University Dortmund Germany Faculty of Statistics and Lamarr Institute for Machine Learning and Artificial Intelligence TU Dortmund University Dortmund Germany

data subsampling is one of the most natural methods to approximate a massively large data set by a small representative proxy. In particular, sensitivity sampling received a lot of attention, which samples points proportional to an individual importance measure called sensitivity. This framework reduces in very general settings the size of data to roughly the VC dimension d times the total sensitivity S while providing strong (1 ± Ε) guarantees on the quality of approximation. The recent work of Woodruff and Yasuda (2023c) improved substantially over the general Õ(Ε−2Sd) bound for the important problem of p subspace embeddings to Õ(Ε−2S2/p) for p ∈ [1, 2]. Their result was subsumed by an earlier Õ(Ε−2Sd1−p/2) bound which was implicitly given in the work of Chen and Derezinski (2021). We show that their result is tight when sampling according to plain p sensitivities. We observe that by augmenting the p sensitivities by 2 sensitivities, we obtain better bounds improving over the aforementioned results to optimal linear Õ(Ε−2(S + d)) = Õ(Ε−2d) sampling complexity for all p ∈ [1, 2]. In particular, this resolves an open question of Woodruff and Yasuda (2023c) in the affirmative for p ∈ [1, 2] and brings sensitivity subsampling into the regime that was previously only known to be possible using Lewis weights (Cohen and Peng, 2015). As an application of our main result, we also obtain an Õ(Ε−2µd) sensitivity sampling bound for logistic regression, where µ is a natural complexity measure for this problem. This improves over the previous Õ(Ε−2µ2d) bound of Mai et al. (2021) which was based on Lewis weights subsampling. Copyright © 2024, The Authors. All rights reserved.

关键词： Importance sampling

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：