检索结果-内蒙古大学图书馆

Depth selection for deep ReLU nets in feature extraction and generalization

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Han, Zhi Yu, Siquan Lin, Shao-Bo Zhou, Ding-Xuan State Key Laboratory of Robotics Shenyang Institute of Automation Chinese Academy of Sciences Shenyang China Institutes for Robotics and Intelligent Manufacturing Chinese Academy of Sciences Shenyang China School of Information Science and Engineering Northeastern University Shenyang China Center of Intelligent Decision-Making and Machine Learning School of Management Xi'an Jiaotong University Xi'an China School of Data Science Department of Mathematics City University of Hong Kong Hong Kong

Deep learning is recognized to be capable of discovering deep features for representation learning and pattern recognition without requiring elegant feature engineering techniques by taking advantage of human ingenuity and prior knowledge. Thus it has triggered enormous research activities in machine learning and pattern recognition. One of the most important challenge of deep learning is to figure out relations between a feature and the depth of deep neural networks (deep nets for short) to reflect the necessity of depth. Our purpose is to quantify this feature-depth correspondence in feature extraction and generalization. We present the adaptivity of features to depths and vice-verse via showing a depth-parameter trade-off in extracting both single feature and composite features. Based on these results, we prove that implementing the classical empirical risk minimization on deep nets can achieve the optimal generalization performance for numerous learning tasks. Our theoretical results are verified by a series of numerical experiments including toy simulations and a real application of earthquake seismic intensity prediction. Copyright © 2020, The Authors. All rights reserved.

关键词： Feature extraction

Feature extraction for hyperspectral imagery: The evolution from shallow to deep

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Rasti, Behnood Hong, Danfeng Hang, Renlong Ghamisi, Pedram Kang, Xudong Chanussot, Jocelyn Benediktsson, Jon Atli Machine Learning Group Exploration Division Helmholtz Institute Freiberg for Resource Technology Helmholtz-Zentrum Dresden-Rossendorf Freiberg09599 Germany Univ. Grenoble Alpes CNRS Grenoble INP GIPSAlab Grenoble38000 France Jiangsu Key Laboratory of Big Data Analysis Technology School of Automation Nanjing University of Information Science and Technology Nanjing210044 China Machine Learning Group Exploration Division Helmholtz Institute Freiberg for Resource Technology Helmholtz-Zentrum Dresden-Rossendorf Freiberg09599 Germany College of Electrical and Information Engineering Hunan University Changsha410082 China Key Laboratory of Visual Perception and Artificial Intelligence of Hunan Province Changsha410082 China Univ. Grenoble Alpes Inria CNRS Grenoble INP LJK GrenobleF-38000 France Faculty of Electrical and Computer Engineering University of Iceland Reykjavik101 Iceland Faculty of Electrical and Computer Engineering University of Iceland Reykjavik107 Iceland

The final version of the paper can be found in IEEE Geoscience and Remote Sensing Magazine. Hyperspectral images provide detailed spectral information through hundreds of (narrow) spectral channels (also known as dimensionality or bands) with continuous spectral information that can accurately classify diverse materials of interest. The increased dimensionality of such data makes it possible to significantly improve data information content but provides a challenge to the conventional techniques (the so-called curse of dimensionality) for accurate analysis of hyperspectral images. Feature extraction, as a vibrant field of research in the hyperspectral community, evolved through decades of research to address this issue and extract informative features suitable for data representation and classification. The advances in feature extraction have been inspired by two fields of research, including the popularization of image and signal processing as well as machine (deep) learning, leading to two types of feature extraction approaches named shallow and deep techniques. This article outlines the advances in feature extraction approaches for hyperspectral imagery by providing a technical overview of the state-of-the-art techniques, providing useful entry points for researchers at different levels, including students, researchers, and senior researchers, willing to explore novel investigations on this challenging topic. In more detail, this paper provides a bird's eye view over shallow (both supervised and unsupervised) and deep feature extraction approaches specifically dedicated to the topic of hyperspectral feature extraction and its application on hyperspectral image classification. Additionally, this paper compares 15 advanced techniques with an emphasis on their methodological foundations in terms of classification accuracies. Furthermore, to push this vibrant field of research forward an impressive amount of codes and libraries is shared at: https://***/Behnoo

关键词： Feature extraction

Uplink-downlink duality between multiple-access and broadcast channels with compressing relays

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Liu, Liang Liu, Ya-Feng Patil, Pratik Yu, Wei the Department of Electronic and Information Engineering The Hong Kong Polytechnic University Hong Kong the State Key Laboratory of Scientific and Engineering Computing Institute of Computational Mathematics and Scientific/Engineering Computing Academy of Mathematics and Systems Science Chinese Academy of Sciences Beijing100190 China The Edward S. Rogers Sr. Department of Electrical and Computer Engineering the University of Toronto the Department of Statistics and Data Science and the Machine Learning Department Carnegie Mellon University PittsburghPA15213 United States The Edward S. Rogers Sr. Department of Electrical and Computer Engineering University of Toronto 10 King’s College Road TorontoONM5S3G4 Canada

—Uplink-downlink duality refers to the fact that under a sum-power constraint, the capacity regions of a Gaussian multiple-access channel and a Gaussian broadcast channel with Hermitian transposed channel matrices are identical. This paper generalizes this result to a cooperative cellular network, in which remote access-points are deployed as relays in serving the users under the coordination of a central processor (CP). In this model, the users and the relays are connected over noisy wireless links, while the relays and the CP are connected over noiseless but rate-limited fronthaul links. Based on a Lagrangian technique, this paper establishes a duality relationship between such a multiple-access relay channel and broadcast relay channel, under the assumption that the relays use compression-based strategies. Specifically, we show that under the same total transmit power constraint and individual fronthaul rate constraints, the achievable rate regions of the Gaussian multiple-access and broadcast relay channels are identical, when either independent compression or Wyner-Ziv and multivariate compression strategies are used. The key observations are that if the beamforming vectors at the relays are fixed, the sum-power minimization problems under the achievable rate and fronthaul constraints in both the uplink and the downlink can be transformed into either a linear programming or a semidefinite programming problem depending on the compression technique, and that the uplink and downlink problems are Lagrangian duals of each other. Moreover, the dual variables corresponding to the downlink rate constraints become the uplink powers;the dual variables corresponding to the downlink fronthaul constraints become the uplink quantization noises. This duality relationship enables an efficient algorithm for optimizing the downlink transmission and relaying strategies based on the uplink. Copyright © 2020, The Authors. All rights reserved.

关键词： Linear programming

The state-of-the-art in Cardiac MRI Reconstruction: Results of the CMRxRecon Challenge in MICCAI 2023

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Lyu, Jun Qin, Chen Wang, Shuo Wang, Fanwen Li, Yan Wang, Zi Guo, Kunyuan Ouyang, Cheng Tänzer, Michael Liu, Meng Sun, Longyu Sun, Mengting Li, Qin Shi, Zhang Hua, Sha Li, Hao Chen, Zhensen Zhang, Zhenlin Xin, Bingyu Metaxas, Dimitris N. Yiasemis, George Teuwen, Jonas Zhang, Liping Chen, Weitian Pang, Yanwei Liu, Xiaohan Razumov, Artem Dylov, Dmitry V. Dou, Quan Yan, Kang Xue, Yuyang Du, Yuning Dietlmeier, Julia Garcia-Cabrera, Carles Hemidi, Ziad Al-Haj Vogt, Nora Xu, Ziqiang Zhang, Yajing Chu, Ying-Hua Chen, Weibo Bai, Wenjia Zhuang, Xiahai Qin, Jing Wu, Lianmin Yang, Guang Qu, Xiaobo Wang, He Wang, Chengyan Psychiatry Neuroimaging Laboratory Brigham and Women’s Hospital Harvard Medical School 399 Revolution Drive BostonMA02215 United States Department of Electrical and Electronic Engineering & I-X Imperial College London United Kingdom Digital Medical Research Center School of Basic Medical Sciences Fudan University Shanghai China Department of Bioengineering & I-X Imperial College London LondonW12 7SL United Kingdom Cardiovascular Magnetic Resonance Unit Royal Brompton Hospital Guy’s and St Thomas’ NHS Foundation Trust LondonSW3 6NP United Kingdom Department of Radiology Ruijin Hospital Shanghai Jiao Tong University School of Medicine Shanghai China Department of Electronic Science Fujian Provincial Key Laboratory of Plasma and Magnetic Resonance National Institute for Data Science in Health and Medicine Institute of Artificial Intelligence Xiamen University Xiamen361102 China Department of Computing Department of Brain Sciences Imperial College London LondonSW7 2AZ United Kingdom Human Phenome Institute Fudan University 825 Zhangheng Road Pudong New District Shanghai201203 China Department of Radiology Zhongshan Hospital Fudan University Shanghai China Department of Cardiovascular Medicine Ruijin Hospital Lu Wan Branch Shanghai Jiao Tong University School of Medicine Shanghai China Institute of Science and Technology for Brain-Inspired Intelligence Fudan University Shanghai200433 China Department of Computer Science Rutgers University PiscatawayNJ08854 United States AI for Oncology Netherlands Cancer Institute Plesmanlaan 121 Amsterdam1066 CX Netherlands Department of Imaging and Interventional Radiology The Chinese University of Hong Kong Hong Kong TJK-BIIT Lab School of Electrical and Information Engineering Tianjin University Tianjin300072 China Skolkovo Institute Of Science And Technology Center for Artificial Intelligence Technology 30/1 Bolshoy blvd. Moscow121205 Russia Department of Biomedical Engineering University of Virginia

Cardiac magnetic resonance imaging (MRI) provides detailed and quantitative evaluation of the heart’s structure, function, and tissue characteristics with high-resolution spatial-temporal imaging. However, its slow imaging speed and motion artifacts are notable limitations. Undersampling reconstruction, especially data-driven algorithms, has emerged as a promising solution to accelerate scans and enhance imaging performance using highly under-sampled data. Nevertheless, the scarcity of publicly available cardiac k-space datasets and evaluation platform hinder the development of data-driven reconstruction algorithms. To address this issue, we organized the Cardiac MRI Reconstruction Challenge (CMRxRecon) in 2023, in collaboration with the 26th International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI). CMRxRecon presented an extensive k-space dataset comprising cine and mapping raw data, accompanied by detailed annotations of cardiac anatomical structures. With overwhelming participation, the challenge attracted more than 285 teams and over 600 participants. Among them, 22 teams successfully submitted Docker containers for the testing phase, with 7 teams submitted for both cine and mapping tasks. All teams use deep learning based approaches, indicating that deep learning has predominately become a promising solution for the problem. The first-place winner of both tasks utilizes the E2E-VarNet architecture as backbones. In contrast, U-Net is still the most popular backbone for both multi-coil and single-coil reconstructions. This paper provides a comprehensive overview of the challenge design, presents a summary of the submitted results, reviews the employed methods, and offers an in-depth discussion that aims to inspire future advancements in cardiac MRI reconstruction models. The summary emphasizes the effective strategies observed in Cardiac MRI reconstruction, including backbone architecture, loss function, pre-processing techn

关键词： Mapping

Lessons Learned from Assessing Trustworthy AI in Practice

学校读者我要写书评

暂无评论

Digital Society 2023年第3期2卷 1-25页

作者： Vetter, Dennis Amann, Julia Bruneault, Frédérick Coffee, Megan Düdder, Boris Gallucci, Alessio Gilbert, Thomas Krendl Hagendorff, Thilo van Halem, Irmhild Hickman, Eleanore Hildt, Elisabeth Holm, Sune Kararigas, Georgios Kringen, Pedro Madai, Vince I. Wiinblad Mathez, Emilie Tithi, Jesmin Jahan Westerlund, Magnus Wurth, Renee Zicari, Roberto V. Computational Vision and Artificial Intelligence Lab Goethe University Frankfurt Frankfurt Am Main Germany Z-Inspection® Initiative Venice Italy Health Ethics and Policy Lab ETH Zurich Zurich Switzerland Strategy and Innovation Careum Foundation Zurich Switzerland Philosophie Departement Collège André-Laurendeau Montréal Canada École Des Médias Université du Québec À Montréal Montréal Canada Department of Medicine Division of Infectious Diseases and Immunology New York University Grossman School of Medicine New York City USA Department of Computer Science University of Copenhagen Copenhagen Denmark Digital Life Initiative Cornell Tech New York City USA Cluster of Excellence “Machine Learning: New Perspectives for Science” University of Tuebingen Tuebingen Germany School of Law University of Bristol Bristol UK Center for the Study of Ethics in the Professions Illinois Institute of Technology Chicago USA Department of Business Management and Analytics Arcada University of Applied Sciences Helsinki Finland Department of Food & Resource Economics University of Copenhagen Copenhagen Denmark Department of Physiology Faculty of Medicine University of Iceland Reykjavik Iceland QUEST Centre for Responsible Research Berlin Institute of Health Charité Universitätsmedizin Berlin Berlin Germany Faculty of Computing Engineering and the Built Environment School of Computing and Digital Technology Birmingham City University Birmingham UK Parallel Computing Labs Intel Santa Clara USA School of Economics Innovation and Technology Kristiania University College Oslo Norway Data Science Graduate School Seoul National University Seoul South Korea

Building artificial intelligence (AI) systems that adhere to ethical standards is a complex problem. Even though a multitude of guidelines for the design and development of such trustworthy AI systems exist, these guidelines focus on high-level and abstract requirements for AI systems, and it is often very difficult to assess if a specific system fulfills these requirements. The Z-Inspection® process provides a holistic and dynamic framework to evaluate the trustworthiness of specific AI systems at different stages of the AI lifecycle, including intended use, design, and development. It focuses, in particular, on the discussion and identification of ethical issues and tensions through the analysis of socio-technical scenarios and a requirement-based framework for ethical and trustworthy AI. This article is a methodological reflection on the Z-Inspection® process. We illustrate how high-level guidelines for ethical and trustworthy AI can be applied in practice and provide insights for both AI researchers and AI practitioners. We share the lessons learned from conducting a series of independent assessments to evaluate the trustworthiness of real-world AI systems, as well as key recommendations and practical suggestions on how to ensure a rigorous trustworthiness assessment throughout the lifecycle of an AI system. The results presented in this article are based on our assessments of AI systems in the healthcare sector and environmental monitoring, where we used the framework for trustworthy AI proposed in the Ethics Guidelines for Trustworthy AI by the European Commission’s High-Level Expert Group on AI. However, the assessment process and the lessons learned can be adapted to other domains and include additional frameworks.

关键词：

Analyzing the Structure of Attention in a Transformer Language Model

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Vig, Jesse Belinkov, Yonatan Palo Alto Research Center Machine Learning and Data Science Group Interaction and Analytics Lab Palo AltoCA United States Harvard John A. Paulson School of Engineering and Applied Sciences MIT Computer Science and Artificial Intelligence Laboratory CambridgeMA United States

The Transformer is a fully attention-based alternative to recurrent networks that has achieved state-of-the-art results across a range of NLP tasks. In this paper, we analyze the structure of attention in a Transformer language model, the GPT-2 small pretrained model. We visualize attention for individual instances and analyze the interaction between attention and syntax over a large corpus. We find that attention targets different parts of speech at different layer depths within the model, and that attention aligns with dependency relations most strongly in the middle layers. We also find that the deepest layers of the model capture the most distant relationships. Finally, we extract exemplar sentences that reveal highly specific patterns targeted by particular attention heads. Copyright © 2019, The Authors. All rights reserved.

关键词： Recurrent neural networks

Structuring the Problem Space for Model-Based Systems engineering

学校读者我要写书评

暂无评论

Insight 2019年第1期22卷 45-50页

作者： Wrigley, J. Craig Wide-ranging experience of systems engineering for over 35 years through innovative work on systems ranging from new demand access protocols for military data links through to large scale intelligence processing systems applying artificial intelligence and machine learning and assessments of quantum technologies.

Systems engineering is evolving from a largely text-based endeavour towards a more graphical and model-based approach. While modelling of the structural aspects of a system is well developed, the modelling of requirements is still evolving. This paper proposes a traceable method for transforming textual functional requirements or business activities into required behaviours and proposes a multiscale approach to refining the behavioural descriptions into "atomic" behavioural requirements which can be satisfied either by existing systems or which can form the basis for development of new subsystems. © 2019 by J. Craig Wrigley. Published and used by INCOSE with permission.

关键词： Requirements engineering

Automatic identification of types of alterations in historical manuscripts

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Lassner, David Baillot, Anne Dogadov, Sergej Müller, Klaus-Robert Nakajima, Shinichi Machine Learning Group Technische Universität Berlin Berlin10587 Germany Le Mans Université Le Mans72085 France Berlin Big Data Center Berlin10587 Germany Department of Brain and Cognitive Engineering Korea University Anam-dong Seongbuk-gu Seoul136-713 Korea Republic of Max-Planck-Institut für Informatik Saarbrücken Germany Berliner Zentrum für Maschinelles Lernen Berlin10587 Germany RIKEN Center for AIP Tokyo103-0027 Japan

Alterations in historical manuscripts such as letters represent a promising field of research. On the one hand, they help understand the construction of text. On the other hand, topics that are being considered sensitive at the time of the manuscript gain coherence and contextuality when taking alterations into account, especially in the case of deletions. The analysis of alterations in manuscripts, though, is a traditionally very tedious work. In this paper, we present a machine learning-based approach to help categorize alterations in documents. In particular, we present a new probabilistic model (Alteration Latent Dirichlet Allocation, alterLDA in the following) that categorizes content-related alterations. The method proposed here is developed based on experiments carried out on the digital scholarly edition Berlin Intellectuals, for which alterLDA achieves high performance in the recognition of alterations on labelled data. On unlabelled data, applying alterLDA leads to interesting new insights into the alteration behavior of authors, editors and other manuscript contributors, as well as insights into sensitive topics in the correspondence of Berlin intellectuals around 1800. In addition to the findings based on the digital scholarly edition Berlin Intellectuals, we present a general framework for the analysis of text genesis that can be used in the context of other digital resources representing document variants. To that end, we present in detail the methodological steps that are to be followed in order to achieve such results, giving thereby a prime example of an machine learning application the Digital Humanities. Copyright © 2020, The Authors. All rights reserved.

关键词： Statistics

VulnerVAN: A Vulnerable Network Generation Tool

学校读者我要写书评

暂无评论

VulnerVAN: A Vulnerable Network Generation Tool

MILCOM, Military Communications Conference

作者： Sridhar Venkatesan Jason A. Youzwak Shridatt Sugrim Cho-Yu J. Chiang Alexander Poylisher Matthew Witkowski Gary Walther Michelle Wolberg Ritu Chadha E. Allison Newcomb Blaine Hoffman Norbou Buchler Machine Learning and Data Analytics Research Perspecta Labs Inc. Basking Ridge NJ USA Computational and Information Science Directorate U.S. CCDC Army Research Laboratory Aberdeen MD USA Human Research and Engineering Directorate U.S. CCDC Army Research Laboratory

ISBN: (数字)9781728142807

ISBN: (纸本)9781728142814

Cyber training, security testing, and research and development activities are vital to improve the security posture of a network. Currently, many institutions use cyber security testbeds to conduct these activities in an isolated virtual environment. One of the important requirements for such an environment is to provide organizers (or experimenters) with a library of vulnerable network scenarios and capabilities to mount attacks against them. However, the task of preparing a vulnerable network scenario in current testbed environments is costly in time and labor, requires significant support from the testbed staff. To this end, we present a toolset called VulnerVAN that creates a vulnerable network scenario to realize an attack sequence. In this paper, we discuss the design of VulnerVAn - our proof-of-concept implementation on CyberVAN - and present a new high-level attack specification language that enables users to chain attack steps into an attack sequence. For a given attack sequence and network scenario, VulnerVAN identifies all possible attack paths through the network that can realize the attack sequence, and provides instructions to configure machines on an attack path selected by the user. VulnerVAn also provides an attack blueprint that can guide a Red team or an automated attacker to execute the attack sequence. To demonstrate VulnerVAN's capability, we consider the use-case of a typical data exfiltration attack sequence conducted by APTs and study the performance of VulnerVAn in mapping the attack sequence to different networks.

关键词： data security Training Specification languages Virtual environments