检索结果-内蒙古大学图书馆

A COMPREHENSIVE SURVEY ON APPLICATIONS OF TRANSFORMERS FOR DEEP LEARNING TASKS

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Islam, Saidul Elmekki, Hanae Elsebai, Ahmed Bentahar, Jamal Drawel, Najat Rjoub, Gaith Pedrycz, Witold Concordia Institute for Information Systems Engineering Concordia University Montreal Canada Department of Electrical Engineering and Computer Science Khalifa University Abu Dhabi United Arab Emirates King Hussein School of Computing Sciences Princess Sumaya University for Technology Jordan Department of Electrical and Computer Engineering University of Alberta Edmonton Canada Systems Research Institute Polish Academy of Sciences Warsaw Poland Department of Computer Engineering Istinye University Sariyer Istanbul Turkey Department of Electrical and Computer Engineering King Abdulaziz University Jeddah Saudi Arabia

Transformer is a deep neural network that employs a self-attention mechanism to comprehend the contextual relationships within sequential data. Unlike conventional neural networks or updated versions of Recurrent Neural Networks (RNNs) such as Long Short-Term Memory (LSTM), transformer models excel in handling long dependencies between input sequence elements and enable parallel processing. As a result, transformer-based models have attracted substantial interest among researchers in the field of artificial intelligence. This can be attributed to their immense potential and remarkable achievements, not only in Natural Language Processing (NLP) tasks but also in a wide range of domains, including computer vision, audio and speech processing, healthcare, and the Internet of Things (IoT). Although several survey papers have been published highlighting the transformer's contributions in specific fields, architectural differences, or performance evaluations, there is still a significant absence of a comprehensive survey paper encompassing its major applications across various domains. Therefore, we undertook the task of filling this gap by conducting an extensive survey of proposed transformer models from 2017 to 2022. Our survey encompasses the identification of the top five application domains for transformer-based models, namely: NLP, computer Vision, Multi-Modality, Audio and Speech Processing, and Signal Processing. We analyze the impact of highly influential transformer-based models in these domains and subsequently classify them based on their respective tasks using a proposed taxonomy. Our aim is to shed light on the existing potential and future possibilities of transformers for enthusiastic researchers, thus contributing to the broader understanding of this groundbreaking technology. © 2023, CC BY-NC-ND.

关键词： Long short-term memory

How Can We Reward You? An Ontology for Eliciting Quantitative Reward Rules for Engagement in Mhealth App and Healthy Behaviors

学校读者我要写书评

暂无评论

SSRN

SSRN 2024年

作者： Peleg, Mor Veggiotti, Nicole Sacchi, Lucia Wilk, Szymon Department of Information Systems University of Haifa Haifa Israel Department of Electrical Computer and Biomedical Engineering University of Pavia Pavia Italy Institute of Computing Science Poznan University of Technology Poznan Poland

Objective. When developing mHealth apps with point reward systems, knowledge engineers and domain experts should define app requirements capturing quantitative reward patterns that reflect patient compliance to health behaviors. This is a difficult task, and they could be aided by an ontology that defines systematically quantitative behavior goals that address more than merely the recommended behavior but also rewards for partial compliance or practicing the behavior more than recommended. No ontology and algorithm exist for defining point rewards systematically. Methods. We developed an OWL ontology for point rewards that leverages the Basic Formal Ontology, The Behaviour Change Intervention Ontology and the Gamification Domain Ontology. The ontology allows defining temporal elementary reward patterns for single and multiple sessions of practicing a behavior. These could be assembled to define more complex temporal patterns for persistent behavior over longer time intervals as well as logical combinations of simpler reward patterns. We also developed an algorithm for calculating the points that should be awarded to users, given data regarding their actual performance. A natural language generation algorithm generates from ontology instances app requirements in the form of user stories. To assess the usefulness of the ontology and algorithms we evaluated whether the ontology and algorithms can improve the requirement elicitation of point rewards for compliance patterns more completely and precisely, by information system students who are trained to be system analysts/knowledge engineers. Results. For single-session rewards, the ontology improved formulation of 2 of the 6 requirements as well as the total time for specifying them. For multi-session rewards, the ontology improved formulation of 5 of the 11 requirements. Conclusion. The ontology is a first attempt of its kind, and it covers all of the cases of compliance and award pattern definitions that were needed f

关键词： Ontology

Risk-Averse Learning with Non-Stationary Distributions

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Wang, Siyi Wang, Zifan Yi, Xinlei Zavlanos, Michael M. Johansson, Karl H. Hirche, Sandra Department of Electrical and Computer Engineering Technical University of Munich Munich80333 Germany Division of Decision and Control Systems School of Electrical Engineering and Computer Science KTH Royal Institute of Technology Stockholm10044 Sweden Lab for Information & Decision Systems Massachusetts Institute of Technology CambridgeMA02139 United States Mechanical Engineering and Material Science Duke University DurhamNC27708 United States

Considering non-stationary environments in online optimization enables decision-maker to effectively adapt to changes and improve its performance over time. In such cases, it is favorable to adopt a strategy that minimizes the negative impact of change to avoid potentially risky situations. In this paper, we investigate risk-averse online optimization where the distribution of the random cost changes over time. We minimize risk-averse objective function using the Conditional Value at Risk (CVaR) as risk measure. Due to the difficulty in obtaining the exact CVaR gradient, we employ a zeroth-order optimization approach that queries the cost function values multiple times at each iteration and estimates the CVaR gradient using the sampled values. To facilitate the regret analysis, we use a variation metric based on Wasserstein distance to capture time-varying distributions. Given that the distribution variation is sub-linear in the total number of episodes, we show that our designed learning algorithm achieves sub-linear dynamic regret with high probability for both convex and strongly convex functions. Moreover, theoretical results suggest that increasing the number of samples leads to a reduction in the dynamic regret bounds until the sampling number reaches a specific limit. Finally, we provide numerical experiments of dynamic pricing in a parking lot to illustrate the efficacy of the designed algorithm. Copyright © 2024, The Authors. All rights reserved.

关键词： Learning algorithms

Learning-Based Detection of Malicious Volt-VAr Control Parameters in Smart Inverters

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Saber, Ahmad Mohammad Youssef, Amr Svetinovic, Davor Zeineldin, Hatem El-Saadany, Ehab Electrical Engineering and Computer Science Department Khalifa University Abu Dhabi United Arab Emirates Concordia Institute for Information Systems Engineering MontrealQC Canada

Distributed Volt-Var Control (VVC) is a widely used control mode of smart inverters. However, necessary VVC curve parameters are remotely communicated to the smart inverter, which opens doors for cyberattacks. If VVC curves of an inverter are maliciously manipulated, the attacked inverter's reactive power injection will oscillate, causing undesirable voltage oscillations to manifest in the distribution system, which, in turn, threatens the system's stability. In contrast with previous works which proposed methods to mitigate the oscillations after they are already present in the system, this paper presents an intrusion detection method to detect malicious VVC curves once they are communicated to the inverter. The proposed method utilizes a Multi-Layer Perceptron (MLP) that is trained on features extracted from only the local measurements of the inverter. After a smart inverter is equipped with the proposed method, any communicated VVC curve will be verified by the MLP once received. If the curve is found to be malicious, it will be rejected, thus preventing unwanted oscillations beforehand. Otherwise, legitimate curves will be permitted. The performance of the proposed scheme is verified using the 9-bus Canadian urban benchmark distribution system simulated in PSCAD/EMTDC environment. Our results show that the proposed solution can accurately detect malicious VVC curves. © 2023, CC BY-NC-SA.

关键词： Distributed power generation

Non-Convex Potential Games for Finding Global Solutions to Sensor Network Localization

学校读者我要写书评

暂无评论

Non-Convex Potential Games for Finding Global Solutions to S...

European Control Conference (ECC)

作者： Gehui Xu Guanpu Chen Yiguang Hong Baris Fidan Thomas Parisini Karl H. Johansson Key Laboratory of Systems and Control Academy of Mathematics and Systems Science Beijing China Division of Decision and Control Systems School of Electrical Engineering and Computer Science KTH Royal Institute of Technology Stockholm Sweden Department of Control Science and Engineering Tongji University Shanghai China Shanghai Research Institute for Intelligent Autonomous Systems Shanghai China Department of Mechanical and Mechatronics Engineering University of Waterloo Waterloo ON Canada Department of Electrical and Electronic Engineering Imperial College London London UK Department of Engineering and Architecture University of Trieste Trieste Italy

ISBN: (数字)9783907144107

ISBN: (纸本)9798331540920

Sensor network localization (SNL) problems require determining the physical coordinates of all sensors in a network. This process relies on the global coordinates of anchors and the available measurements between non-anchor and anchor nodes. Attributed to the intrinsic non-convexity, obtaining a globally optimal solution to SNL is challenging, as well as implementing corresponding algorithms. In this paper, we formulate a non-convex multi-player potential game for a generic SNL problem to investigate the identification condition of the global Nash equilibrium (NE) therein, where the global NE represents the global solution of SNL. We employ canonical duality theory to transform the non-convex game into a complementary dual problem. Then we develop a conjugation-based algorithm to compute the stationary points of the complementary dual problem. On this basis, we show an identification condition of the global NE: the stationary point of the proposed algorithm satisfies a duality relation. Finally, simulation results are provided to validate the effectiveness of the theoretical results.

关键词： Location awareness Coordinate measuring machines Simulation Current measurement Noise Europe Games

Participation Factors for Nonlinear Autonomous Dynamical systems in the Koopman Operator Framework

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Takamichi, Kenji Susuki, Yoshihiko Netto, Marcos The Department of Electrical and Information Systems Osaka Prefecture University Sakai599-8531 Japan The Department of Electrical Engineering Kyoto University Kyoto615-8510 Japan The Helen and John C. Hartmann Department of Electrical and Computer Engineering New Jersey Institute of Technology NewarkNJ07102 United States

We devise a novel formulation and propose the concept of modal participation factors to nonlinear dynamical systems. The original definition of modal participation factors (or simply participation factors) provides a simple yet effective metric. It finds use in theory and practice, quantifying the interplay between states and modes of oscillation in a linear time-invariant (LTI) system. In this paper, with the Koopman operator framework, we present the results of participation factors for nonlinear dynamical systems with an asymptotically stable equilibrium point or limit cycle. We show that participation factors are defined for the entire domain of attraction, beyond the vicinity of an attractor, where the original definition of participation factors for LTI systems is a special case. Finally, we develop a numerical method to estimate participation factors using time series data from the underlying nonlinear dynamical system. The numerical method can be implemented by leveraging a well-established numerical scheme in the Koopman operator framework called dynamic mode decomposition. © 2024, CC BY.

关键词： Dynamical systems

Variational Covariance Smoothing for Dynamic Functional Connectivity Analysis

学校读者我要写书评

暂无评论

Variational Covariance Smoothing for Dynamic Functional Conn...

Asilomar Conference on Signals, systems & computers

作者： Ruwanthi Abeysekara Jonathan Z. Simon Behtash Babadi Department of Electrical & Computer Engineering University of Maryland College Park MD Institute for Systems Research University of Maryland College Park MD Department of Biology University of Maryland College Park MD

ISBN: (数字)9798350354058

ISBN: (纸本)9798350354065

Functional connectivity is among the widely used metrics to assess the network-level attributes of brain function. While most existing analysis frameworks assume static functional connectivity during the course of an experiment, to capture neural dynamics over short time scales, a time-varying notion of functional connectivity is required. By revealing how neural networks reconfigure in response to changing external stimuli, internal states, and task demands, time-varying functional connectivity can be leveraged to study flexible cognition, such as working memory, attention, and decision-making. A major challenge in estimating time-varying functional connectivity from high-dimensional neural is the associated computational complexity. Existing methods trade off accuracy for computational efficiency, especially in applications that require real-time or near real-time processing. Here, we build on existing work using covariance-domain state-space models and introduce a framework based on variational inference that allows low-complexity estimation of time-varying functional connectivity and construction of confidence intervals. We validate the performance of the proposed method using simulation studies. Our results reveal significant gains in computational complexity compared to existing methods, while maintaining high accuracy.

关键词： Measurement Accuracy Smoothing methods Filtering Real-time systems Computational efficiency State-space methods Recording Computational complexity Biological neural networks

CosSIF: Cosine similarity-based image filtering to overcome low inter-class variation in synthetic medical image datasets

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Islam, Mominul Zunair, Hasib Mohammed, Nabeel Department of Electrical and Computer Engineering North South University Bashundhara Dhaka Bangladesh Concordia Institute for Information Systems Engineering Concordia University MontrealQC Canada

Crafting effective deep learning models for medical image analysis is a complex task, particularly in cases where the medical image dataset lacks significant inter-class variation. This challenge is further aggravated when employing such datasets to generate synthetic images using generative adversarial networks (GANs), as the output of GANs heavily relies on the input data. In this research, we propose a novel filtering algorithm called Cosine Similarity-based Image Filtering (CosSIF). We leverage CosSIF to develop two distinct filtering methods: Filtering Before GAN Training (FBGT) and Filtering After GAN Training (FAGT). FBGT involves the removal of real images that exhibit similarities to images of other classes before utilizing them as the training dataset for a GAN. On the other hand, FAGT focuses on eliminating synthetic images with less discriminative features compared to real images used for training the GAN. Experimental results reveal that employing either the FAGT or FBGT method with modern transformer and convolutional-based networks leads to substantial performance gains in various evaluation metrics. FAGT implementation on the ISIC-2016 dataset surpasses the baseline method in terms of sensitivity by 1.59% and AUC by 1.88%. Furthermore, for the HAM10000 dataset, applying FABT outperforms the baseline approach in terms of recall by 13.75%, and with the sole implementation of FAGT, achieves a maximum accuracy of 94.44%. Code and implementation details are available at: https://***/mominul-ssv/cossif. © 2023, CC BY.

关键词： Generative adversarial networks

Universal Energy Access: Enhancing Networked Rural Electrification Using Modified Fuzzy Inference Method

学校读者我要写书评

暂无评论

Universal Energy Access: Enhancing Networked Rural Electrifi...

2022 IEEE Region 10 International Conference, TENCON 2022

作者： Li, Jerry C.F. Zimmerle, Daniel Young, Peter Colorado State University Department of Systems Engineering Fort CollinsCO United States Energy Institute Colorado State University Fort CollinsCO United States Colorado State University Department of Electrical and Computer Engineering Fort CollinsCO United States

ISBN: (数字)9781665450959

ISBN: (纸本)9781665450959

To achieve UN Sustainable Development Goal 7 (SDG7) by 2030, better rural electrification is required. Most current projects provide electricity inefficiently to rural users rather using stand-alone designs. As most remaining unserved users are located in difficult-to-reach areas, IEA has pointed out that the marginal cost of rural electrification is increasing. Earlier work indicates that networking rural villages would be more cost-effective than prior models. This work uses modified fuzzy inference to improve power distribution costs by improving the optimality of distribution network design. © 2022 IEEE.

关键词： Fuzzy inference