检索结果-内蒙古大学图书馆

arXiv 2022年

作者： Luo, Shengjie Li, Shanda Zheng, Shuxin Liu, Tie-Yan Wang, Liwei He, Di Key Laboratory of Machine Perception MOE School of Intelligence Science and Technology Peking University China Machine Learning Department School of Computer Science Carnegie Mellon University United States Microsoft Research United States Center for Data Science Peking University China Zhejiang Lab China

Relative Positional Encoding (RPE), which encodes the relative distance between any pair of tokens, is one of the most successful modifications to the original Transformer. As far as we know, theoretical understanding of the RPE-based Transformers is largely unexplored. In this work, we mathematically analyze the power of RPE-based Transformers regarding whether the model is capable of approximating any continuous sequence-to-sequence functions. One may naturally assume the answer is in the affirmative-RPE-based Transformers are universal function approximators. However, we present a negative result by showing there exist continuous sequence-to-sequence functions that RPE-based Transformers cannot approximate no matter how deep and wide the neural network is. One key reason lies in that most RPEs are placed in the softmax attention that always generates a right stochastic matrix. This restricts the network from capturing positional information in the RPEs and limits its capacity. To overcome the problem and make the model more powerful, we first present sufficient conditions for RPE-based Transformers to achieve universal function approximation. With the theoretical guidance, we develop a novel attention module, called Universal RPE-based (URPE) Attention, which satisfies the conditions. Therefore, the corresponding URPE-based Transformers become universal function approximators. Extensive experiments covering typical architectures and tasks demonstrate that our model is parameter-efficient and can achieve superior performance to strong baselines in a wide range of applications. The code will be made publicly available at https://***/lsj2408/URPE. Copyright © 2022, The Authors. All rights reserved.

关键词： Stochastic systems

来源：评论

学校读者我要写书评

暂无评论

RBGNet: Ray-based Grouping for 3D Object Detection

arXiv

引用

arXiv 2022年

作者： Wang, Haiyang Shi, Shaoshuai Yang, Ze Fang, Rongyao Qian, Qi Li, Hongsheng Schiele, Bernt Wang, Liwei Center for Data Science Peking University China Max Planck Institute for Informatics Germany University of Toronto Canada The Chinese University of Hong Kong Hong Kong Alibaba Group China Key Laboratory of Machine Perception MOE School of Artificial Intelligence Peking University China International Center for Machine Learning Research Peking University China

As a fundamental problem in computer vision, 3D object detection is experiencing rapid growth. To extract the point-wise features from the irregularly and sparsely distributed points, previous methods usually take a feature grouping module to aggregate the point features to an object candidate. However, these methods have not yet leveraged the surface geometry of foreground objects to enhance grouping and 3D box generation. In this paper, we propose the RBGNet framework, a voting-based 3D detector for accurate 3D object detection from point clouds. In order to learn better representations of object shape to enhance cluster features for predicting 3D boxes, we propose a ray-based feature grouping module, which aggregates the point-wise features on object surfaces using a group of determined rays uniformly emitted from cluster centers. Considering the fact that foreground points are more meaningful for box estimation, we design a novel foreground biased sampling strategy in downsample process to sample more points on object surfaces and further boost the detection performance. Our model achieves state-of-the-art 3D detection performance on ScanNet V2 and SUN RGB-D with remarkable performance gains. Code will be available at https://***/Haiyang-W/RBGNet. Copyright © 2022, The Authors. All rights reserved.

关键词： Object recognition

来源：评论

学校读者我要写书评

暂无评论

Robustness in Fatigue Strength Estimation

arXiv

引用

arXiv 2022年

作者： Weichert, Dorina Kister, Alexander Houben, Sebastian Ernis, Gunar Wrobel, Stefan Fraunhofer Institute for Intelligent Analysis and Information Systems IAIS Germany German Federal Institute for Materials Research and Testing Section S.3 eScience Germany University of Applied Sciences Bonn-Rhein-Sieg Germany Fraunhofer Center for Machine Learning Germany

Fatigue strength estimation is a costly manual material characterization process in which state-of-the-art approaches follow a standardized experiment and analysis procedure. In this paper, we examine a modular, machine learning-based approach for fatigue strength estimation that is likely to reduce the number of experiments and, thus, the overall experimental costs. Despite its high potential, deployment of a new approach in a real-life lab requires more than the theoretical definition and simulation. Therefore, we study the robustness of the approach against misspecification of the prior and discretization of the specified loads. We identify its applicability and its advantageous behavior over the state-of-the-art methods, potentially reducing the number of costly experiments. Copyright © 2022, The Authors. All rights reserved.

关键词： machine learning

来源：评论

学校读者我要写书评

暂无评论

Active learning meets fractal decision boundaries: a cautionary tale from the Sitnikov three-body problem

arXiv

引用

arXiv 2023年

作者： Payot, Nicolas Pasquato, Mario Trani, Alessandro Alberto Hezaveh, Yashar Perreault-Levasseur, Laurence Département de Physique Université de Montréal Mila Quebec Artificial Intelligence Institute Ciela - Montreal Institute for Astrophysical Data Analysis and Machine Learning Canada Dipartimento di Fisica e Astronomia Università di Padova Vicolo dell’Osservatorio 5 PadovaI-35122 Italy Niels Bohr Institute Copenhagen Denmark Research Center for the Early Universe School of Science The University of Tokyo Tokyo Japan Okinawa Institute of Science and Technology Okinawa Japan

Chaotic systems such as the gravitational N-body problem are ubiquitous in astronomy. machine learning (ML) is increasingly deployed to predict the evolution of such systems, e.g. with the goal of speeding up simulations. Strategies such as active learning (AL) are a natural choice to optimize ML training. Here we showcase an AL failure when predicting the stability of the Sitnikov three-body problem, the simplest case of N-body problem displaying chaotic behavior. We link this failure to the fractal nature of our classification problem’s decision boundary. This is a potential pitfall in optimizing large sets of N-body simulations via AL in the context of star cluster physics, galactic dynamics, or cosmology. © 2023, CC BY.

关键词： Fractals

来源：评论

学校读者我要写书评

暂无评论

Topology-Preserving Automatic Labeling of Coronary Arteries via Anatomy-aware Connection Classifier

arXiv

引用

arXiv 2023年

作者： Zhang, Zhixing Zhao, Ziwei Wang, Dong Zhao, Shishuang Liu, Yuhang Liu, Jia Wang, Liwei Center for Data Science Peking University Beijing China National Key Laboratory of General Artificial Intelligence School of Intelligence Science and Technology Peking University Beijing China Yizhun Medical AI Co. Ltd Beijing China Peking University First Hospital Beijing China Center for Machine Learning Research Peking University Beijing China Pazhou Lab Guangzhou China

Automatic labeling of coronary arteries is an essential task in the practical diagnosis process of cardiovascular diseases. For experienced radiologists, the anatomically predetermined connections are important for labeling the artery segments accurately, while this prior knowledge is barely explored in previous studies. In this paper, we present a new framework called TopoLab which incorporates the anatomical connections into the network design explicitly. Specifically, the strategies of intra-segment feature aggregation and inter-segment feature interaction are introduced for hierarchical segment feature extraction. Moreover, we propose the anatomy-aware connection classifier to enable classification for each connected segment pair, which effectively exploits the prior topology among the arteries with different categories. To validate the effectiveness of our method, we contribute high-quality annotations of artery labeling to the public orCaScore dataset. The experimental results on both the orCaScore dataset and an in-house dataset show that our TopoLab has achieved state-of-the-art performance. © 2023, CC BY.

关键词： Topology

来源：评论

学校读者我要写书评

暂无评论

xMIL: Insightful Explanations for Multiple Instance learning in Histopathology

arXiv

引用

arXiv 2024年

作者： Hense, Julius Idaji, Mina Jamshidi Eberle, Oliver Schnake, Thomas Dippel, Jonas Ciernik, Laure Buchstab, Oliver Mock, Andreas Klauschen, Frederick Müller, Klaus-Robert Berlin Institute for the Foundations of Learning and Data Berlin Germany Machine Learning Group Technische Universität Berlin Berlin Germany Aignostics GmbH Berlin Germany Institute of Pathology Ludwig Maximilian University Munich Germany German Cancer Research Center Heidelberg Germany German Cancer Consortium Munich Germany Institute of Pathology Charité Universitätsmedizin Berlin Germany Department of Artificial Intelligence Korea University Seoul Korea Republic of Max-Planck Institute for Informatics Saarbrücken Germany

Multiple instance learning (MIL) is an effective and widely used approach for weakly supervised machine learning. In histopathology, MIL models have achieved remarkable success in tasks like tumor detection, biomarker prediction, and outcome prognostication. However, MIL explanation methods are still lagging behind, as they are limited to small bag sizes or disregard instance interactions. We revisit MIL through the lens of explainable AI (XAI) and introduce xMIL, a refined framework with more general assumptions. We demonstrate how to obtain improved MIL explanations using layer-wise relevance propagation (LRP) and conduct extensive evaluation experiments on three toy settings and four real-world histopathology datasets. Our approach consistently outperforms previous explanation attempts with particularly improved faithfulness scores on challenging biomarker prediction tasks. Finally, we showcase how xMIL explanations enable pathologists to extract insights from MIL models, representing a significant advance for knowledge discovery and model debugging in digital histopathology. Codes are available at: https://***/bifold-pathomics/xMIL. © 2024, CC BY-NC-ND.

关键词： Self-supervised learning

来源：评论

学校读者我要写书评

暂无评论

Preface

Cognitive Technologies

引用

Cognitive Technologies 2025年 Part F287卷 v-vi页

作者： Schulz, Daniel Bauckhage, Christian Research Center Machine Learning Fraunhofer Institute for Intelligent Analysis and Information Systems IAIS Sankt Nordrhein-Westfalen Augustin Germany Fraunhofer Institute for Intelligent Analysis and Information Systems Sankt Nordrhein-Westfalen Augustin Germany

来源：评论

学校读者我要写书评

暂无评论

Caustics: A Python Package for Accelerated Strong Gravitational Lensing Simulations

arXiv

引用

arXiv 2024年

作者： Stone, Connor Adam, Alexandre Coogan, Adam Yantovski-Barth, M.J. Filipp, Andreas Setiawan, Landung Core, Cordero Legin, Ronan Wilson, Charles Barco, Gabriel Missael Hezaveh, Yashar Perreault-Levasseur, Laurence Ciela Institute Montréal Institute for Astrophysical Data Analysis and Machine Learning MontréalQC Canada Department of Physics Université de Montréal MontréalQC Canada Mila Québec Artificial Intelligence Institute MontréalQC Canada Center for Computational Astrophysics Flatiron Institute 162 5th Avenue New YorkNY10010 United States eScience Institute Scientific Software Engineering Center 1410 NE Campus Pkwy SeattleWA98195 United States

Gravitational lensing is the deflection of light rays due to the gravity of intervening masses. This phenomenon is observed in a variety of scales and configurations, involving any non-uniform mass such as planets, stars, galaxies, clusters of galaxies, and even the large scale structure of the universe. Strong lensing occurs when the distortions are significant and multiple images of the background source are observed. The lens objects must align on the sky of order ∼ 1 arcsecond for galaxy-galaxy lensing, or 10’s of arcseonds for cluster-galaxy lensing. As the discovery of lens systems has grown to the low thousands, these systems have become pivotal for precision measurements and addressing critical questions in astrophysics. Notably, they facilitate the measurement of the Universe’s expansion rate (e.g. K. C. Wong et al., 2020), dark matter (e.g. Hezaveh et al., 2016;Vegetti & Vogelsberger, 2014), supernovae (e.g. Rodney et al., 2021), quasars (e.g. Peng et al., 2006), and the first stars (e.g. Welch et al., 2022) among other topics. With future surveys expected to discover hundreds of thousands of lensing systems, the modelling and simulation of such systems must occur at orders of magnitude larger scale then ever before. Here we present caustics, a Python package designed to handle the extensive computational demands of modeling such a vast number of lensing systems. © 2024, CC BY.

关键词： Python

来源：评论

学校读者我要写书评

暂无评论

Achieving Linear Speedup with Network-Independent learning Rates in Decentralized Stochastic Optimization

Achieving Linear Speedup with Network-Independent Learning R...

引用

IEEE Conference on Decision and Control

作者： Hao Yuan Sulaiman A. Alghunaim Kun Yuan Center for Machine Learning Research Peking University Beijing P. R. China Dept. Electrical Engr. Kuwait University Safat Kuwait AI for Science Institute Beijing P. R. China National Engineering Labratory for Big Data Analytics and Applications Beijing P. R. China

Decentralized stochastic optimization has become a crucial tool for addressing large-scale machine learning and control problems. In decentralized algorithms, all computing nodes are connected through a network topology, and each node communicates only with its direct neighbors. Decentralized algorithms can significantly reduce communication overhead by eliminating the need for global communication. However, existing research on the linear speedup analysis of decentralized stochastic algorithms is limited to the condition of network-dependent learning rates, which rarely holds in practice since the network connectivity is typically unknown to each node. As a result, it remains an open question whether a linear speedup bound can be achieved using network-independent learning rates. This paper provides an affirmative answer. By utilizing a new analysis framework, we prove that D-SGD and Exact-Diffusion, two representative decentralized stochastic algorithms, can achieve linear speedup with network-independent learning rates. Simulations are provided to validate our theories.

关键词：

来源：评论

学校读者我要写书评

暂无评论

learning an Effective Evolution Equation for Particle-Mesh Simulations Across Cosmologies

arXiv

引用

arXiv 2023年

作者： Payot, Nicolas Lemos, Pablo Perreault-Levasseur, Laurence Cuesta-Lazaro, Carolina Modi, Chirag Hezaveh, Yashar Département de Physique Université de Montréal Mila Quebec Artificial Intelligence Institute Ciela - Montreal Institute for Astrophysical Data Analysis and Machine Learning Canada Center for Computational Astrophysics Flatiron Institute NY United States The NSF AI Institute for Artificial Intelligence and Fundamental Interactions Massachusetts Institute of Technology CambridgeMA United States Center for Astrophysics Harvard & Smithsonian CambridgeMA United States Center for Computational Mathematics Flatiron Institute NY United States

Particle-mesh simulations trade small-scale accuracy for speed compared to traditional, computationally expensive N-body codes in cosmological simulations. In this work, we show how a data-driven model could be used to learn an effective evolution equation for the particles, by correcting the errors of the particle-mesh potential incurred on small scales during simulations. We find that our learnt correction yields evolution equations that generalize well to new, unseen initial conditions and cosmologies. We further demonstrate that the resulting corrected maps can be used in a simulation-based inference framework to yield an unbiased inference of cosmological parameters. The model, a network implemented in Fourier space, is exclusively trained on the particle positions and velocities. © 2023, CC BY.

关键词： Cosmology

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：