检索结果-内蒙古大学图书馆

Towards Expressive Spectral-Temporal Graph Neural Networks for Time Series Forecasting

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Jin, Ming Shi, Guangsi Li, Yuan-Fang Xiong, Bo Zhou, Tian Salim, Flora D. Zhao, Liang Wu, Lingfei Wen, Qingsong Pan, Shirui School of Information and Communication Technology Griffith University Gold Coast Australia Department of Data Science and AI Monash University Melbourne Australia International Max Plank Research School for Intelligent Systems The University of Stuttgart Stuttgart Germany Alibaba Group Hangzhou China School of Computer Science and Engineering University of New South Wales Sydney Australia Department of Computer Science Emory University Atlanta United States Anytime.AI New York United States Squirrel Ai Learning Bellevue United States

Time series forecasting has remained a focal point due to its vital applications in sectors such as energy management and transportation planning. Spectral-temporal graph neural network is a promising abstraction underlying most time series forecasting models that are based on graph neural networks (GNNs). However, more is needed to know about the underpinnings of this branch of methods. In this paper, we establish a theoretical framework that unravels the expressive power of spectral-temporal GNNs. Our results show that linear spectral-temporal GNNs are universal under mild assumptions, and their expressive power is bounded by our extended first-order Weisfeiler-Leman algorithm on discrete-time dynamic graphs. To make our findings useful in practice on valid instantiations, we discuss related constraints in detail and outline a theoretical blueprint for designing spatial and temporal modules in spectral domains. Building on these insights and to demonstrate how powerful spectral-temporal GNNs are based on our framework, we propose a simple instantiation named Temporal Graph Gegenbauer Convolution (TGGC), which significantly outperforms most existing models with only linear components and shows better model efficiency. Our findings pave the way for devising a broader array of provably expressive GNN-based models for time series. © 2023, CC BY.

关键词： Time series

Your transformer may not be as powerful as you expect 22

学校读者我要写书评

暂无评论

Your transformer may not be as powerful as you expect

Proceedings of the 36th International Conference on Neural Information Processing Systems

作者： Shengjie Luo Shanda Li Shuxin Zheng Tie-Yan Liu Liwei Wang Di He National Key Laboratory of General Artificial Intelligence School of Intelligence Science and Technology Peking University and Zhejiang Lab Machine Learning Department School of Computer Science Carnegie Mellon University Microsoft Research National Key Laboratory of General Artificial Intelligence School of Intelligence Science and Technology Peking University and Center for Data Science Peking University National Key Laboratory of General Artificial Intelligence School of Intelligence Science and Technology Peking University

ISBN: (纸本)9781713871088

Relative Positional Encoding (RPE), which encodes the relative distance between any pair of tokens, is one of the most successful modifications to the original Transformer. As far as we know, theoretical understanding of the RPE-based Transformers is largely unexplored. In this work, we mathematically analyze the power of RPE-based Transformers regarding whether the model is capable of approximating any continuous sequence-to-sequence functions. One may naturally assume the answer is in the affirmative—RPE-based Transformers are universal function approximators. However, we present a negative result by showing there exist continuous sequence-to-sequence functions that RPE-based Transformers cannot approximate no matter how deep and wide the neural network is. One key reason lies in that most RPEs are placed in the softmax attention that always generates a right stochastic matrix. This restricts the network from capturing positional information in the RPEs and limits its capacity. To overcome the problem and make the model more powerful, we first present sufficient conditions for RPE-based Transformers to achieve universal function approximation. With the theoretical guidance, we develop a novel attention module, called Universal RPE-based (URPE) Attention, which satisfies the conditions. Therefore, the corresponding URPE-based Transformers become universal function approximators. Extensive experiments covering typical architectures and tasks demonstrate that our model is parameter-efficient and can achieve superior performance to strong baselines in a wide range of applications.

关键词：

A Generic and Robust System for Automated Detection of Different Classes of Arrhythmia

学校读者我要写书评

暂无评论

Procedia Computer Science 2020年 167卷 1801-1810页

作者： Vandana Singh U. Srinivasulu Reddy G. Manju Bhargavia M.Tech (Data Analytics) National Institute of Technology Tiruchirappalli 620015 India Assistant Professor Machine Learning and Data Analytics Lab Department of Computer Applications National Institute of Technology Tiruchirappalli 620015 India Post-Doc Department of Physics National Institute of Technology Tiruchirappalli 620015 India

Cardiovascular Arrhythmias (irregular beat) are related to the sudden death, can be characterized into two kinds, life-threatening (dangerous) and non-life-threatening. In clinical research and diagnosis analysis electrocardiography (ECG) plays an important role. The Advancement of Medical Instrumentation (AAMI) suggests, all the heartbeat of MIT-BIH dataset into four classes, namely, normal or bundle branch block (N), supraventricular ectopic (S), ventricular ectopic (V) and fusion of ventricular and normal (F). The objective of this paper is the classification of arrhythmias on the ECG as per AAMI standards. First beat detection and then on a given window size beat segmentation is performed. Then, features computed from ECG signals are RR (separation between two successive heartbeat pulses), local binary pattern(LBP), morphological changes, wavelet, high order statistics, and several amplitude values; Pre-RR, Post-RR, Local RR and Global RR. The main reason is to computing many morphological features to comparison analysis with the model when trained and tested on single features vs combining all features together. Model gives highest performance which is 90.8% accuracy using LBP as single feature compared with other features. Combined all important features together and optimal features (total 104 ) are given as an input to Single SVM classifier as well as ensembles of SVM which maps the component feature vectors to the particular class name. Principal Component Analysis (PCA) technique is also applied to the model which reduces the total number of features from 104 to 30 features. The Single SVM system can accurately classify four beat types and has an overall classification rate of 94.05% without PCA and using PCA 92.84% while the Ensemble SVM classifies and with an accuracy of 92.96%

关键词： Electrocardiogram (ECG) signal MIT-BIH dataset Wavelet Transform Support Vector Machines OVO

Risk-based decision making: estimands for sequential prediction under interventions

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Luijken, Kim Morzywolek, Pawel van Amsterdam, Wouter Cinà, Giovanni Hoogland, Jeroen Keogh, Ruth Krijthe, Jesse Magliacane, Sara van Ommen, Thijs Peek, Niels Putter, Hein van Smeden, Maarten Sperrin, Matthew Wang, Junfeng Weir, Daniala Didelez, Vanessa van Geloven, Nan Department of Epidemiology Julius Center for Health Sciences and Primary Care University Medical Center Utrecht Utrecht Netherlands Department of Applied Mathematics Computer Science and Statistics Ghent University Ghent Belgium Department of Statistics University of Washington Seattle United States Department of Medical Informatics Amsterdam University Medical Centers Amsterdam Netherlands Institute for Logic Language and Computation University of Amsterdam Amsterdam Netherlands Pacmed Amsterdam Netherlands Department of Epidemiology and Data Science Amsterdam University Medical Centers Amsterdam Netherlands Department of Medical Statistics London School of Hygiene & Tropical Medicine Keppel Street London United Kingdom Pattern Recognition and Bio-Informatics Group EEMCS Delft University of Technology Delft Netherlands Amsterdam Machine Learning Lab University of Amsterdam Amsterdam Netherlands Department of Information and Computing Sciences Utrecht University Utrecht Netherlands Division of Informatics Imaging and Data Science Faculty of Biology Medicine and Health University of Manchester Manchester Academic Health Science Centre Manchester United Kingdom Department of Biomedical Data Sciences Leiden University Medical Center Leiden Netherlands Division of Pharmacoepidemiology and Clinical Pharmacology Department of Pharmaceutical Sciences Utrecht University Utrecht Netherlands Department of Biometry and Data Management Leibniz Institute for Prevention Research Epidemiology - BIPS Bremen Germany Faculty of Mathematics/Computer Science University of Bremen Bremen Germany

Prediction models are used amongst others to inform medical decisions on interventions. Typically, individuals with high risks of adverse outcomes are advised to undergo an intervention while those at low risk are advised to refrain from it. Standard prediction models do not always provide risks that are relevant to inform such decisions: e.g., an individual may be estimated to be at low risk because similar individuals in the past received an intervention which lowered their risk. Therefore, prediction models supporting decisions should target risks belonging to defined intervention strategies. Previous works on prediction under interventions assumed that the prediction model was used only at one time point to make an intervention decision. In clinical practice, intervention decisions are rarely made only once: they might be repeated, deferred and re-evaluated. This requires estimated risks under interventions that can be reconsidered at several potential decision moments. In the current work, we highlight key considerations for formulating estimands in sequential prediction under interventions that can inform such intervention decisions. We illustrate these considerations by giving examples of estimands for a case study about choosing between vaginal delivery and cesarean section for women giving birth. Our formalization of prediction tasks in a sequential, causal, and estimand context provides guidance for future studies to ensure that the right question is answered and appropriate causal estimation approaches are chosen to develop sequential prediction models that can inform intervention decisions. © 2023, CC BY.

关键词： Forecasting

Unintended consequences: Factors influencing oil palm plantation expansion 3

学校读者我要写书评

暂无评论

Unintended consequences: Factors influencing oil palm planta...

3rd Life and Environmental Sciences Academics Forum, LEAF 2019

作者： Wongsai, S. Keson, J. Wongsai, N. Thammasat University Research Unit in Data Learning Division of Mathematics and Statistics Faculty of Science and Technology Thammasat University Pathumthani12120 Thailand Faculty of Technology and Environment Prince of Songkla University Phuket Campus Phuket83120 Thailand Department of Mathematics and Computer Science Faculty of Science and Technology Prince of Songkla University Pattani Campus Pattani94000 Thailand

We investigated the landscape variables affecting the current dramatic expansion of oil palm plantations in Lam Thap district Krabi Province Thailand. THEOS satellite data was used to map land use classifications using the support vector machine method. Seven land use classes were consolidated into oil palm plantations and other rural uses. A logistic regression model was then applied to search for the relationship between two land use classes and the landscape variables of slope (three classes) and soil drainage (four classes). Overall, slope and drainage were statistically significant in explaining oil palm plantation expansion. Flat areas with poor drainage were the strongest factors. This contradicts the known optimal agronomic requirements for oil palms of well drainage. We suggest that what is observed is an unintended consequence of government oil palm support policies. © Published under licence by IOP Publishing Ltd.

关键词： Land use

Optimal sampling density for nonparametric regression

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Panknin, Danny Müller, Klaus-Robert Nakajima, Shinichi Machine Learning Department Berlin Institute of Technology Berlin10587 Germany BIFOLD-Berlin Institute for the Foundations of Learning and Data Germany Department of Artificial Intelligence Korea University Seoul136-713 Korea Republic of Max Planck Institute for Informatics Saarbrücken66123 Germany RIKEN AIP 1-4-1 Nihonbashi Chuo-ku Tokyo Japan

We propose a novel active learning strategy for regression, which is model-agnostic, robust against model mismatch, and interpretable. Assuming that a small number of initial samples are available, we derive the optimal training density that minimizes the generalization error of local polynomial smoothing (LPS) with its kernel bandwidth tuned locally: We adopt the mean integrated squared error (MISE) as a generalization criterion, and use the asymptotic behavior of the MISE as well as the locally optimal bandwidths (LOB) – the bandwidth function that minimizes MISE in the asymptotic limit. The asymptotic expression of our objective then reveals the dependence of the MISE on the training density, enabling analytic minimization. As a result, we obtain the optimal training density in a closed-form. The almost model-free nature of our approach thus helps to encode the essential properties of the target problem, providing a robust and model-agnostic active learning strategy. Furthermore, the obtained training density factorizes the influence of local function complexity, noise level and test density in a transparent and interpretable way. We validate our theory in numerical simulations, and show that the proposed active learning method outperforms the existing state-of-the-art model-agnostic approaches. Copyright © 2021, The Authors. All rights reserved.

关键词： Bandwidth

Parameter estimation for cellular automata

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Kazarnikov, Alexey Ray, Nadja Haario, Heikki Lappalainen, Joona Rupp, Andreas Heidelberg University Mathematikon Im Neuenheimer Feld 205 Heidelberg69120 Germany Mathematical Institute for Machine Learning and Data Science Catholic University of Eichstätt-Ingolstadt Hohe-Schul-Str. 5 Ingolstadt85049 Germany School of Engineering Science Lappeenranta–Lahti University of Technology P.O. Box 20 Lappeenranta53851 Finland Department of Mathematics Faculty of Mathematics and Computer Science Saarland University SaarbrückenDE-66123 Germany

Self-organizing complex systems can be modeled using cellular automaton models. However, the parametrization of these models is crucial and significantly determines the resulting structural pattern. In this research, we introduce and successfully apply a sound statistical method to estimate these parameters. The decisive difference to earlier applications of such approaches is that, in our case, both the CA rules and the resulting patterns are discrete. The method is based on constructing Gaussian likelihoods using characteristics of the structures, such as the mean particle size. We show that our approach is robust for the method parameters, domain size of patterns, or CA iterations. Copyright © 2023, The Authors. All rights reserved.

关键词： Statistical methods

SDCOR: Scalable density-based clustering for local outlier detection in massive-scale datasets

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Naghavi Nozad, Sayyed Ahmad Haeri, Maryam Amir Folino, Gianluigi Department of Computer Engineering Amirkabir University of Technology Tehran Iran Learning Data-Analytics Technology Department University of Twente Enschede Netherlands ICAR-CNR Rende Italy

This paper presents a batch-wise density-based clustering approach for local outlier detection in massive-scale datasets. Unlike the well-known traditional algorithms, which assume that all the data is memory-resident, our proposed method is scalable and processes the input data chunk-by-chunk within the confines of a limited memory buffer. A temporary clustering model is built at the first phase;then, it is gradually updated by analyzing consecutive memory loads of points. Subsequently, at the end of scalable clustering, the approximate structure of the original clusters is obtained. Finally, by another scan of the entire dataset and using a suitable criterion, an outlying score is assigned to each object called SDCOR (Scalable Density-based Clustering Outlierness Ratio). Evaluations on real-life and synthetic datasets demonstrate that the proposed method has a low linear time complexity and is more effective and efficient compared to best-known conventional density-based methods, which need to load all data into the memory;and also, to some fast distance-based methods, which can perform on data resident in the disk. Copyright © 2020, The Authors. All rights reserved.

关键词： Anomaly detection

Connectome Computation System:2015–2021 updates

学校读者我要写书评

暂无评论

Science Bulletin 2022年第5期67卷 448-451页

作者： Xiu-Xia Xing Ting Xu Chao Jiang Yin-Shan Wang Xi-Nian Zuo Department of Applied Mathematics College of MathematicsFaculty of ScienceBeijing University of TechnologyBeijing 100124China Center for the Developing Brain Child Mind InstituteNY 10022USA School of Psychology Capital Normal UniversityBeijing 100048China State Key Laboratory of Cognitive Neuroscience and Learning Beijing Normal UniversityBeijing 100875China National Basic Science Data Center Beijing 100190China Developmental Population Neuroscience Research Center IDG/McGovern Institute for Brain ResearchBeijing Normal UniversityBeijing 100875China

The Connectome Computation System(CCS)was previously reported in the Science Bulletin[1].Here,we describe a summary of the 6-year CCS updates(2015–2021),which are accessible at https://***/zuoxinian/*** updates contain many utilities of mining developmental neuroimaging ***,H1 module covers the data cleaning and preprocessing and refines the implementations of face-masking,brain extraction,denoising,and frequency banding.

关键词： Bulletin cleaning mining