检索结果-内蒙古大学图书馆

International Symposium on parallel and Distributed Processing (IPDPS)

作者： Dhiraj D. Kalamkar Joshua D. Trzaskoz Srinivas Sridharan Mikhail Smelyanskiy Daehyun Kim Armando Manduca Yunhong Shu Matt A. Bernstein Bharat Kaul Pradeep Dubey Parallel Computing Lab Intel Labs Bangalore KA India Mayo Clinic Minnesota Rochester MN US Parallel Computing Laboratory INTEL Research Laboratory Santa Clara CA USA Department of Physiology and Biomedical Engineering Mayo Clinic Rochester MN USA Parallel Computing Lab Intel Labs Santa Clara CA USA Department of Radiology Mayo Clinic Rochester MN USA

The Non-Uniform Fast Fourier Transform (NUFFT) is a generalization of FFT to non-equidistant samples. It has many applications which vary from medical imaging to radio astronomy to the numerical solution of partial differential equations. Despite recent advances in speeding up NUFFT on various platforms, its practical applications are still limited, due to its high computational cost, which is significantly dominated by the convolution of a signal between a non-uniform and uniform grids. The computational cost of the NUFFT is particularly detrimental in cases which require fast reconstruction times, such as iterative 3D non-Cartesian MRI reconstruction. We propose novel and highly scalable parallel algorithm for performing NUFFT on x86-based multi-core CPUs. The high performance of our algorithm relies on good SIMD utilization and high parallel efficiency. On convolution, we demonstrate on average 90% SIMD efficiency using SSE, as well up to linear scalability using a quad-socket 40-core intel(R) Xeon(R) E7-4870 Processors based system. As a result, on dual socket intel(R) Xeon(R) X5670 based server, our NUFFT implementation is more than 4x faster compared to the best available NUFFT3D implementation, when run on the same hardware. On intel(R) Xeon(R) E5-2670 processor based server, our NUFFT implementation is 1.5X faster than any published NUFFT implementation today. Such speed improvement opens new usages for NUFFT. For example, iterative multi channel reconstruction of a 240×240×240 image could execute in just over 3 minutes, which is on the same order as contemporary non-iterative (and thus less-accurate) 3D NUFFT-based MRI reconstructions.

关键词： Convolution Kernel Interpolation Magnetic resonance imaging Image reconstruction Hardware Transforms

来源：评论

学校读者我要写书评

暂无评论

Practical massively parallel monte-carlo tree search applied to molecular design

arXiv

引用

arXiv 2020年

作者： Yang, Xiufeng Aasawat, Tanuj Kr Yoshizoe, Kazuki Chugai Pharmaceutical Co. Ltd Parallel Computing Lab - India Intel Labs RIKEN Center for Advanced Intelligence Project

It is common practice to use large computational resources to train neural networks, known from many examples, such as reinforcement learning applications. However, while massively parallel computing is often used for training models, it is rarely used to search solutions for combinatorial optimization problems. This paper proposes a novel massively parallel Monte-Carlo Tree Search (MP-MCTS) algorithm that works efficiently for a 1,000 worker scale on a distributed memory environment using multiple compute nodes and applies it to molecular design. This paper is the first work that applies distributed MCTS to a real-world and non-game problem. Existing works on large-scale parallel MCTS show efficient scalability in terms of the number of rollouts up to 100 workers. Still, they suffer from the degradation in the quality of the solutions. MP-MCTS maintains the search quality at a larger scale. By running MP-MCTS on 256 CPU cores for only 10 minutes, we obtained candidate molecules with similar scores to non-parallel MCTS running for 42 hours. Moreover, our results based on parallel MCTS (combined with a simple RNN model) significantly outperform existing state-of-the-art work. Our method is generic and is expected to speed up other applications of MCTS1 Copyright © 2020, The Authors. All rights reserved.

关键词： Trees (mathematics)

来源：评论

学校读者我要写书评

暂无评论

A semi-supervised method for multi-subject FMRI functional alignment

A semi-supervised method for multi-subject FMRI functional a...

引用

IEEE International Conference on Acoustics, Speech and Signal Processing

作者： Javier S. Turek Theodore L. Willke Po-Hsuan Chen Peter J. Ramadge Parallel Computing Lab Intel Labs Hillsboro Oregon USA Department of Electrical Engineering Princeton University New Jersey USA

ISBN: (纸本)9781509041183

Practical limitations on the duration of individual fMRI scans have led neuroscientist to consider the aggregation of data from multiple subjects. Differences in anatomical structures and functional topographies of brains require aligning data across subjects. Existing functional alignment methods serve as a preprocessing step that allows subsequent statistical methods to learn from the aggregated multi-subject data. Despite their success, current alignment methods do not leverage the labeled data used in the subsequent methods. In this work we propose a semi-supervised scheme that simultaneously learns the alignment and performs the analysis. We derive a specific instance of the scheme using the Shared Response Model for alignment and Multinomial Logistic Regression for classification. In our experiments this method improves the average classification accuracy from 65.5% to 68.5%, and from 5.3% to 6.1% over the independently-trained methods. Furthermore, our method achieves similar prediction with almost half the samples used for alignment.

关键词： fMRI functional alignment shared response model semi-supervised method

来源：评论

学校读者我要写书评

暂无评论

Scalable Bayesian optimization using deep neural networks 32

Scalable Bayesian optimization using deep neural networks

引用

32nd International Conference on Machine Learning, ICML 2015

作者： Snoek, Jasper Ripped, Oren Swersky, Kevin Kiros, Ryan Satish, Nadathur Sundaram, Narayanan Patwary, Md. Mostofa Ali Prabhat Adams, Ryan P. Harvard University School of Engineering and Applied Sciences United States Massachusetts Institute of Technology Department of Mathematics United States University of Toronto Department of Computer Science Canada Intel Labs Parallel Computing Lab Switzerland NERSC Lawrence Berkeley National Laboratory United States

ISBN: (纸本)9781510810587

Bayesian optimization is an effective methodology for the global optimization of functions with expensive evaluations. It relies on querying a distribution over functions defined by a relatively cheap surrogate model. An accurate model for this distribution over functions is critical to the effectiveness of the approach, and is typically fit using Gaussian processes (GPs). However, since GPs scale cubically with the number of observations, it has been challenging to handle objectives whose optimization requires many evaluations, and as such, massively parallelizing the optimization. In this work, we explore the use of neural networks as an alternative to GPs to model distributions over functions. We show that performing adaptive basis function regression with a neural network as the parametric form performs competitively with state-of-the-art GP-based approaches, but scales linearly with the number of data rather than cubically. This allows us to achieve a previously intractable degree of parallelism, which we apply to large scale hyperparameter optimization, rapidly finding competitive models on benchmark object recognition tasks using convolutional networks, and image caption generation using neural language models. © Copyright 2015 by International Machine Learning Society (IMLS). All rights reserved.

关键词： Global optimization

来源：评论

学校读者我要写书评

暂无评论

Miss-Correlation Folding: Encoding Per-Block Miss Correlations in Compressed DRAM for Data Prefetching

Miss-Correlation Folding: Encoding Per-Block Miss Correlatio...

引用

IEEE International parallel & Distributed Processing Symposium

作者： Gang Liu Jih-Kwon Peir Victor Lee Department of Computer & Information Science & Eng University of Florida Gainesville FL USA Parallel Computing Lab Intel Labs Intel Corporation Santa Clara CA USA

ISBN: (纸本)9781467309752

Cache misses frequently exhibit repeated streaming behavior, i.e. a sequence of cache misses has a high tendency of being repeated. Correlation-based prefetchers record the missing streams in a history table for accurate prefetching. Saving a large miss history in off-chip DRAM is a practical implementation, but incurs access latency and consumes memory bandwidth which leads to performance degradation. In this paper, we investigate a new data prefetching mechanism based on per-block miss correlation where a miss is correlated with an earlier miss when the two misses are closely encountered both in time and space. The miss correlations are captured dynamically and saved along with the content of the data block using a simple data compression technique. As a result of this novel combination, our scheme provides unbounded correlation history and its prefetch metadata can be fetched together with demand data without incurring additional latency nor consuming any memory bandwidth. Performance evaluations using data-parallel applications demonstrate that prefetchers based on per-block miss correlations can improve IPC by 42-139% with an average of 88% compared to the IPC without prefetching. In comparison with regular stream prefetcher, sampled temporal streaming prefetcher and spatial-temporal memory streaming prefetcher, up to 115%, 99% and 98% IPC improvement can be obtained with an average about 36%, 26% and 27% respectively.

关键词： prefetch data parallel miss correlation compress cache temporal spatial

来源：评论

学校读者我要写书评

暂无评论

Hierarchical block sparse neural networks

arXiv

引用

arXiv 2018年

作者： Vooturi, Dharma Teja Mudigere, Dheevatsa Avancha, Sashikanth Center for Security Theory and Algorithmic Research International Institute of Information Technology - Hyderabad Hyderabad India Parallel Computing Lab Intel Labs India

Sparse deep neural networks(DNNs) are efficient in both memory and compute when compared to dense DNNs. But due to irregularity in computation of sparse DNNs, their efficiencies are much lower than that of dense DNNs on regular parallel hardware such as TPU. This inefficiency leads to poor/no performance benefits for sparse DNNs. Performance issue for sparse DNNs can be alleviated by bringing structure to the sparsity and leveraging it for improving runtime efficiency. But such structural constraints often lead to suboptimal accuracies. In this work, we jointly address both accuracy and performance of sparse DNNs using our proposed class of sparse neural networks called HBsNN (Hierarchical Block sparse Neural Networks). For a given sparsity, HBsNN models achieve better runtime performance than unstructured sparse models and better accuracy than highly structured sparse models. Copyright © 2018, The Authors. All rights reserved.

关键词： Deep neural networks

来源：评论

学校读者我要写书评

暂无评论

A study of BFLOAT16 for deep learning training

arXiv

引用

arXiv 2019年

作者： Kalamkar, Dhiraj Mudigere, Dheevatsa Mellempudi, Naveen Das, Dipankar Banerjee, Kunal Avancha, Sasikanth Vooturi, Dharma Teja Jammalamadaka, Nataraj Huang, Jianyu Yuen, Hector Yang, Jiyan Park, Jongsoo Heinecke, Alexander Georganas, Evangelos Srinivasan, Sudarshan Kundu, Abhisek Smelyanskiy, Misha Kaul, Bharat Dubey, Pradeep Parallel Computing Lab Intel Labs Facebook 1 Hacker Way Menlo ParkCA United States IIIT Hyderabad Lab126 Amazon

This paper presents the first comprehensive empirical study demonstrating the efficacy of the Brain Floating Point (BFLOAT16) half-precision format for Deep Learning training across image classification, speech recognition, language modeling, generative networks and industrial recommendation systems. BFLOAT16 is attractive for Deep Learning training for two reasons: the range of values it can represent is the same as that of IEEE 754 floating-point format (FP32) and conversion to/from FP32 is simple. Maintaining the same range as FP32 is important to ensure that no hyper-parameter tuning is required for convergence;e.g., IEEE 754 compliant half-precision floating point (FP16) requires hyper-parameter tuning. In this paper, we discuss the flow of tensors and various key operations in mixed precision training, and delve into details of operations, such as the rounding modes for converting FP32 tensors to BFLOAT16. We have implemented a method to emulate BFLOAT16 operations in Tensorflow, Caffe2, intelCaffe, and Neon for our experiments. Our results show that deep learning training using BFLOAT16 tensors achieves the same state-of-the-art (SOTA) results across domains as FP32 tensors in the same number of iterations and with no changes to hyper-parameters. Copyright © 2019, The Authors. All rights reserved.

关键词： Tensors

来源：评论

学校读者我要写书评

暂无评论

MILC staggered conjugate gradient performance on intel KNL 34

MILC staggered conjugate gradient performance on Intel KNL

引用

34th Annual International Symposium on Lattice Field Theory, LATTICE 2016

作者： Li, Ruizi DeTar, Carleton Doerfler, Douglas Gottlieb, Steven Jha, Ashish Kalamkar, Dhiraj Toussaint, Doug Department of Physics Indiana University BloomingtonIN47405 United States Department of Physics and Astronomy University of Utah Salt Lake CityUT84112 United States National Energy Research Scientific Computing Center Lawrence Berkeley National Laboratory BerkeleyCA94720 United States Software and Services Group Intel Corporation HillsboroOR97124 United States Parallel Computing Lab Intel Labs Bangalore560103 India Physics Department University of Arizona TucsonAZ85721 United States

We review our work done to optimize the staggered conjugate gradient (CG) algorithm in the MILC code for use with the intel Knights Landing (KNL) architecture. KNL is the second generation intel Xeon Phi processor. It is capable of massive thread parallelism, data parallelism, and high on-board memory bandwidth and is being adopted in supercomputing centers for scientific research. The CG solver consumes the majority of time in production running, so we have spent most of our effort on it. We compare performance of an MPI+OpenMP baseline version of the MILC code with a version incorporating the QPhiX staggered CG solver, for both one-node and multi-node runs. © Copyright owned by the author(s).

关键词： Application programming interfaces (API)

来源：评论

学校读者我要写书评

暂无评论

Z-Inspection®: A Process to Assess Trustworthy AI

IEEE Transactions on Technology and Society

引用

IEEE Transactions on Technology and Society 2021年第2期2卷 83-97页

作者： Zicari, Roberto V. Brodersen, John Brusseau, James Dudder, Boris Eichhorn, Timo Ivanov, Todor Kararigas, Georgios Kringen, Pedro McCullough, Melissa Moslein, Florian Mushtaq, Naveed Roig, Gemma Sturtz, Norman Tolle, Karsten Tithi, Jesmin Jahan Van Halem, Irmhild Westerlund, Magnus Frankfurt Big Data Lab Goethe University Frankfurt Frankfurt Germany Department of Public Health Section of General Practice and Research Unit for General Practice Faculty of Health and Medical Sciences University of Copenhagen Copenhagen Denmark Philosophy Department Pace University New YorkNY United States Department of Computer Science University of Copenhagen Copenhagen Denmark Department of Physiology Faculty of Medicine University of Iceland Iceland Institute of the Law and Regulation of Digitalization Philipps-University Marburg Marburg Germany Parallel Computing Labs Intel Labs Santa ClaraCA United States Department of Computer Science Arcada University of Applied Sciences Helsinki Finland Primary Health Care Research Unit Copenhagen1014 Denmark

The ethical and societal implications of artificial intelligence systems raise concerns. In this article, we outline a novel process based on applied ethics, namely, Z-Inspection®, to assess if an AI system is trustworthy. We use the definition of trustworthy AI given by the high-level European Commission's expert group on AI. Z-Inspection® is a general inspection process that can be applied to a variety of domains where AI systems are used, such as business, healthcare, and public sector, among many others. To the best of our knowledge, Z-Inspection® is the first process to assess trustworthy AI in practice. © 2020 IEEE.

关键词： Deep learning

来源：评论

学校读者我要写书评

暂无评论

Hardware acceleration of sparse and irregular tensor computations of ML models: A survey and insights

arXiv

引用

arXiv 2020年

作者： Dave, Shail Baghdadi, Riyadh Nowatzki, Tony Avancha, Sasikanth Shrivastava, Aviral Li, Baoxin School of Computing Informatics and Decision Systems Engineering Arizona State University TempeAZ85281 United States Computer Science and Artificial Intelligence Laboratory Massachusetts Institute of Technology CambridgeMA02139 United States School of Computer Science University of California Los AngelesCA90095 United States Parallel Computing Lab at Intel Labs Bangalore India

Machine learning (ML) models are widely used in many important domains. For efficiently processing these computational- and memory-intensive applications, tensors of these over-parameterized models are compressed by leveraging sparsity, size reduction, and quantization of tensors. Unstructured sparsity and tensors with varying dimensions yield irregular computation, communication, and memory access patterns;processing them on hardware accelerators in a conventional manner does not inherently leverage acceleration opportunities. This paper provides a comprehensive survey on the efficient execution of sparse and irregular tensor computations of ML models on hardware accelerators. In particular, it discusses enhancement modules in the architecture design and the software support;categorizes different hardware designs and acceleration techniques and analyzes them in terms of hardware and execution costs;analyzes achievable accelerations for recent DNNs;highlights further opportunities in terms of hardware/software/model co-design optimizations (inter/intra-module). The takeaways from this paper include: understanding the key challenges in accelerating sparse, irregular-shaped, and quantized tensors;understanding enhancements in accelerator systems for supporting their efficient computations;analyzing trade-offs in opting for a specific design choice for encoding, storing, extracting, communicating, computing, and load-balancing the non-zeros;understanding how structured sparsity can improve storage efficiency and balance computations;understanding how to compile and map models with sparse tensors on the accelerators;understanding recent design trends for efficient accelerations and further opportunities. Copyright © 2020, The Authors. All rights reserved.

关键词： Energy efficiency

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：