检索结果-内蒙古大学图书馆

Hardware acceleration of sparse and irregular tensor computations of ML models: A survey and insights

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Dave, Shail Baghdadi, Riyadh Nowatzki, Tony Avancha, Sasikanth Shrivastava, Aviral Li, Baoxin School of Computing Informatics and Decision Systems Engineering Arizona State University TempeAZ85281 United States Computer Science and Artificial Intelligence Laboratory Massachusetts Institute of Technology CambridgeMA02139 United States School of Computer Science University of California Los AngelesCA90095 United States Parallel Computing Lab at Intel Labs Bangalore India

Machine learning (ML) models are widely used in many important domains. For efficiently processing these computational- and memory-intensive applications, tensors of these over-parameterized models are compressed by leveraging sparsity, size reduction, and quantization of tensors. Unstructured sparsity and tensors with varying dimensions yield irregular computation, communication, and memory access patterns;processing them on hardware accelerators in a conventional manner does not inherently leverage acceleration opportunities. This paper provides a comprehensive survey on the efficient execution of sparse and irregular tensor computations of ML models on hardware accelerators. In particular, it discusses enhancement modules in the architecture design and the software support;categorizes different hardware designs and acceleration techniques and analyzes them in terms of hardware and execution costs;analyzes achievable accelerations for recent DNNs;highlights further opportunities in terms of hardware/software/model co-design optimizations (inter/intra-module). The takeaways from this paper include: understanding the key challenges in accelerating sparse, irregular-shaped, and quantized tensors;understanding enhancements in accelerator systems for supporting their efficient computations;analyzing trade-offs in opting for a specific design choice for encoding, storing, extracting, communicating, computing, and load-balancing the non-zeros;understanding how structured sparsity can improve storage efficiency and balance computations;understanding how to compile and map models with sparse tensors on the accelerators;understanding recent design trends for efficient accelerations and further opportunities. Copyright © 2020, The Authors. All rights reserved.

关键词： Energy efficiency

A Prompting-based Approach for Adversarial Example Generation and Robustness Enhancement

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Yang, Yuting Huang, Pei Cao, Juan Li, Jintao Lin, Yun Dong, Jin Song Ma, Feifei Zhang, Jian Key Lab of Intelligent Information Processing Institute of Computing Technology Chinese Academy of Sciences Beijing China University of Chinese Academy of Sciences Beijing China Beijing China National University of Singapore Singapore Laboratory of Parallel Software and Computational Science ISCAS Beijing China

Recent years have seen the wide application of NLP models in crucial areas such as finance, medical treatment, and news media, raising concerns of the model robustness and vulnerabilities. In this paper, we propose a novel prompt-based adversarial attack to compromise NLP models and robustness enhancement technique. We first construct malicious prompts for each instance and generate adversarial examples via mask-and-filling under the effect of a malicious purpose. Our attack technique targets the inherent vulnerabilities of NLP models, allowing us to generate samples even without interacting with the victim NLP model, as long as it is based on pre-trained language models (PLMs). Furthermore, we design a prompt-based adversarial training method to improve the robustness of PLMs. As our training method does not actually generate adversarial samples, it can be applied to large-scale training sets efficiently. The experimental results show that our attack method can achieve a high attack success rate with more diverse, fluent and natural adversarial examples. In addition, our robustness enhancement method can significantly improve the robustness of models to resist adversarial attacks. Our work indicates that prompting paradigm has great potential in probing some fundamental flaws of PLMs and fine-tuning them for downstream tasks. Copyright © 2022, The Authors. All rights reserved.

关键词： Natural language processing systems

Resource-Aware Multi-Criteria Vehicle Participation for Federated Learning in Internet of Vehicles

学校读者我要写书评

暂无评论

SSRN

SSRN 2023年

作者： Wen, Jie Zhang, Jingbo Zhang, Zhixia Cui, Zhihua Cai, Xingjuan Chen, Jinjun The Shanxi Key Laboratory of Advanced Control and Equipment intelligence Taiyuan University of Science and Technology Shanxi Taiyuan China The Shanxi Key Laboratory of Big Data Analysis and Parallel Computing Taiyuan University of Science and Technology Shanxi Taiyuan China The State Key Lab for Novel Software Technology Nanjing University China Department of Computing Technologies Swinburne University of Technology Melbourne Australia

Federated learning (FL), as a safe distributed training mode, provides strong support for the edge intelligence of the Internet of Vehicles (IoV) to realize efficient collaborative control and safe data sharing. However, due to the resource limitation and the instability of training environment in the complex IoV, ideal performance of FL cannot be achieved. Since considering the actual resource constraints and federated task requirements, the diversified device selection criteria make the resource-aware vehicle selection problem become a multi-criteria selection problem. To effectively support FL for IoV, the resource-aware multi-criteria vehicle selection problem was described as a many-objective optimization problem, and proposed a resource-aware many-objective vehicle selection model (RA-MaOVSM) to optimize resource efficiency. The RA-MaOVSM considering heterogeneous resources (like computation resources, communication resources, energy resources and data resources) of on-board devices in IoV, and realizes the joint optimization of learning efficiency, energy cost and global performance. Additionally, a novel probability distribution combination game strategy is applied to many-objective evolutionary algorithm (MaOEA) for improving the model solving performance. Simulation results demonstrate that RA-MaOVSM can effectively optimize the IoV resources and FL model performance, and the designed algorithm exhibits good convergence and distribution, achieving a good balance among multiple device selection criteria. © 2023, The Authors. All rights reserved.

关键词： Evolutionary algorithms

Quantum Neuron: An elementary building block for machine learning on quantum computers

学校读者我要写书评

暂无评论

arXiv 2017年

作者： Cao, Yudong Guerreschi, Gian Giacomo Aspuru-Guzik, Alán Department of Chemistry and Chemical Biology Harvard University CambridgeMA02138 United States Parallel Computing Lab Intel Corporation Santa ClaraCA95054 Canadian Institute for Advanced Research TorontoONM5G 1Z8 Canada

Even the most sophisticated artificial neural networks are built by aggregating substantially identical units called neurons. A neuron receives multiple signals, internally combines them, and applies a non-linear function to the resulting weighted sum. Several attempts to generalize neurons to the quantum regime have been proposed, but all proposals collided with the difficulty of implementing non-linear activation functions, which is essential for classical neurons, due to the linear nature of quantum mechanics. Here we propose a solution to this roadblock in the form of a small quantum circuit that naturally simulates neurons with threshold activation. Our quantum circuit defines a building block, the "quantum neuron", that can reproduce a variety of classical neural network constructions while maintaining the ability to process superpositions of inputs and preserve quantum coherence and entanglement. In the construction of feedforward networks of quantum neurons, we provide numerical evidence that the network not only can learn a function when trained with superposition of inputs and the corresponding output, but that this training suffices to learn the function on all individual inputs separately. When arranged to mimic Hopfield networks, quantum neural networks exhibit properties of associative memory. Patterns are encoded using the simple Hebbian rule for the weights and we demonstrate attractor dynamics from corrupted inputs. Finally, the fact that our quantum model closely captures (traditional) neural network dynamics implies that the vast body of literature and results on neural networks becomes directly relevant in the context of quantum machine learning. Copyright © 2017, The Authors. All rights reserved.

关键词： Neurons

parallel Bayesian Network Structure Learning for Genome-Scale Gene Networks

学校读者我要写书评

暂无评论

Parallel Bayesian Network Structure Learning for Genome-Scal...

Supercomputing Conference

作者： Sanchit Misra Md. Vasimuddin Kiran Pamnany Sriram P. Chockalingam Yong Dong Min Xie Maneesha R. Aluru Srinivas Aluru Parallel Computing Lab Intel Corporation Bangalore India Dept. of Computer Science and Engineering Indian Institute of Technology Bombay Mumbai India State Key Laboratory of High Performance Computing National University of Defense Technology Changsha China School of Biology Georgia Institute of Technology Atlanta USA School of Computational Science and Engineering Georgia Institute of Technology Atlanta USA

Learning Bayesian networks is NP-hard. Even with recent progress in heuristic and parallel algorithms, modeling capabilities still fall short of the scale of the problems encountered. In this paper, we present a massively parallel method for Bayesian network structure learning, and demonstrate its capability by constructing genome-scale gene networks of the model plant Arabidopsis thaliana from over 168.5 million gene expression values. We report strong scaling efficiency of 75% and demonstrate scaling to 1.57 million cores of the Tianhe-2 supercomputer. Our results constitute three and five orders of magnitude increase over previously published results in the scale of data analyzed and computations performed, respectively. We achieve this through algorithmic innovations, using efficient techniques to distribute work across all compute nodes, all available processors and coprocessors on each node, all available threads on each processor and coprocessor, and vectorization techniques to maximize single thread performance.

关键词： Bayes methods Hypercubes Instruction sets Genomics Bioinformatics Coprocessors Vectors

Quantifying robustness to adversarial word substitutions

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Yang, Yuting Huang, Pei Ma, Fei Fei Cao, Juan Zhang, Meishan Zhang, Jian Li, Jintao Key Lab Of Intelligent Information Processing Institute Of Computing Technology Chinese Academy Of Sciences Beijing100190 China University Of Chinese Academy Of Sciences Beijing100049 China Beijing100190 China Laboratory Of Parallel Software And Computational Science ISCAS Beijing100190 China China

Deep-learning-based NLP models are found to be vulnerable to word substitution perturbations. Before they are widely adopted, the fundamental issues of robustness need to be addressed. Along this line, we propose a formal framework to evaluate word-level robustness. First, to study safe regions for a model, we introduce robustness radius which is the boundary where the model can resist any perturbation. As calculating the maximum robustness radius is computationally hard, we estimate its upper and lower bound. We repurpose attack methods as ways of seeking upper bound and design a pseudo-dynamic programming algorithm for a tighter upper bound. Then verification method is utilized for a lower bound. Further, for evaluating the robustness of regions outside a safe radius, we reexamine robustness from another view: quantification. A robustness metric with a rigorous statistical guarantee is introduced to measure the quantification of adversarial examples, which indicates the model's susceptibility to perturbations outside the safe radius. The metric helps us figure out why state-of-the-art models like BERT can be easily fooled by a few word substitutions, but generalize well in the presence of real-world noises. Copyright © 2022, The Authors. All rights reserved.

关键词： Dynamic programming

GREYONE: data flow sensitive fuzzing 20

学校读者我要写书评

暂无评论

GREYONE: data flow sensitive fuzzing

Proceedings of the 29th USENIX Conference on Security Symposium

作者： Shuitao Gan Chao Zhang Peng Chen Bodong Zhao Xiaojun Qin Dong Wu Zuoning Chen State Key Laboratory of Mathematical Engineering and Advanced Computing Institute for Network Science and Cyberspace Tsinghua University and Beijing National Research Center for Information Science and Technology ByteDance AI lab Institute for Network Science and Cyberspace Tsinghua University National Research Center of Parallel Computer Engineering and Technology

ISBN: (纸本)9781939133175

Data flow analysis (e.g., dynamic taint analysis) has proven to be useful for guiding fuzzers to explore hard-to-reach code and find vulnerabilities. However, traditional taint analysis is labor-intensive, inaccurate and slow, affecting the fuzzing efficiency. Apart from taint, few data flow features are *** this paper, we proposed a data flow sensitive fuzzing solution GREYONE. We first utilize the classic feature taint to guide fuzzing. A lightweight and sound fuzzing-driven taint inference (FTI) is adopted to infer taint of variables, by monitoring their value changes while mutating input bytes during fuzzing. With the taint, we propose a novel input prioritization model to determine which branch to explore, which bytes to mutate and how to mutate. Further, we use another data flow feature constraint conformance, i.e., distance of tainted variables to values expected in untouched branches, to tune the evolution direction of *** implemented a prototype of GREYONE and evaluated it on the LAVA data set and 19 real world programs. The results showed that it outperforms various state-of-the-art fuzzers in terms of both code coverage and vulnerability discovery. In the LAVA data set, GREYONE found all listed bugs and 336 more unlisted. In real world programs, GREYONE on average found 2.12X unique program paths and 3.09X unique bugs than state-of-the-art evolutionary fuzzers, including AFL, VUzzer, CollAFL, Angora and Honggfuzz, Moreover, GREYONE on average found 1.2X unique program paths and 1.52X unique bugs than a state-of-the-art symbolic exeuction assisted fuzzer QSYM. In total, it found 105 new security bugs, of which 41 are confirmed by CVE.

关键词：

Situation and location awareness in harsh environment

学校读者我要写书评

暂无评论

Situation and location awareness in harsh environment

Proceedings of the International Convention MIPRO

作者： Miklos Kozlovszky Daniela Zavec Pavlinić Andreja Oder Gábor Fehér Pál Bogdanov MTA SZTAKI Laboratory of Parallel and Distributed Computing Budapest Hungary John von Neumann Faculty of Informatics Óbuda University Budapest Hungary Titera Technical Innovative Technologies Ltd. Slovenia Prevent&Deloza Ltd. Celje Slovenia OMT-LAB Budapest University Hungary

Enhanced environment awareness is a key element of life protection in dangerous situations. Thus the Personal Protection Equipment (PPE) clothing industry is forced in the direction to develop highly innovative “intelligent” products, and equip their products with sensors and actuators combined with smart technologies. Besides basic functions of protection, thermal and ergonomic comfort, these new generation protective garments have additional functions such as monitoring and/or alarming for prevention in case of emergency situation. Such systems are costly because even if the clothing and engineering parts of the development are feasible, their combination is far from trivial. The different system parts have different requirements, and these requirements are sometimes not compatible with each other. Parameters such as cost, bulkiness, accuracy, independency, scalability, robustness are key parameters, which can cause high acceptance or usage prevention rapidly by the end-users. Our research is focusing on the seamless integration of the various subsystems to develop effective and user friendly, intelligent situation aware personal protection systems. Our solution helps to receive accurate information about user performance and activity conditions. The main goal of our work is to adapt the already existing monitoring solution and provide it to people working in harsh environment. The paper describes the results of our development with the description of the main functionalities.

关键词： Data acquisition Sensor fusion Monitoring Accuracy Data integration Data visualization

Efficient architecture-aware acceleration of BWA-MEM for multicore systems

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Md, Vasimuddin Misra, Sanchit Li, Heng Aluru, Srinivas Parallel Computing Lab Intel Corporation Bangalore India Department of Biostatistics and Computational Biology Dana-Farber Cancer Institute Boston United States Department of Biomedical Informatics Harvard Medical School Boston United States School of Computational Science and Engineering Georgia Institute of Technology Atlanta United States

Innovations in Next-Generation Sequencing are enabling generation of DNA sequence data at ever faster rates and at very low cost. For example, the Illumina NovaSeq 6000 sequencer can generate 6 Terabases of data in less than two days, sequencing nearly 20 Billion short DNA fragments called reads at the low cost of $1000 per human genome. Large sequencing centers typically employ hundreds of such systems. Such high-throughput and low-cost generation of data underscores the need for commensurate acceleration in downstream computational analysis of the sequencing data. A fundamental step in downstream analysis is mapping of the reads to a long reference DNA sequence, such as a reference human genome. Sequence mapping is a compute-intensive step that accounts for more than 30% of the overall time of the GATK (Genome Analysis ToolKit) best practices workflow. BWA-MEM is one of the most widely used tools for sequence mapping and has tens of thousands of users. Copyright © 2019, The Authors. All rights reserved.

关键词： Costs