检索结果-内蒙古大学图书馆

Meta-ticket: Finding optimal subnetworks for few-shot learning within randomly initialized neural networks

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Chijiwa, Daiki Yamaguchi, Shin'ya Kumagai, Atsutoshi Ida, Yasutoshi NTT Computer and Data Science Laboratories NTT Corporation Japan Kyoto University Japan

Few-shot learning for neural networks (NNs) is an important problem that aims to train NNs with a few data. The main challenge is how to avoid overfitting since over-parameterized NNs can easily overfit to such small dataset. Previous work (e.g. MAML by Finn et al. 2017) tackles this challenge by meta-learning, which learns how to learn from a few data by using various tasks. On the other hand, one conventional approach to avoid overfitting is restricting hypothesis spaces by endowing sparse NN structures like convolution layers in computer vision. However, although such manually-designed sparse structures are sample-efficient for sufficiently large datasets, they are still insufficient for few-shot learning. Then the following questions naturally arise: (1) Can we find sparse structures effective for few-shot learning by meta-learning? (2) What benefits will it bring in terms of meta-generalization? In this work, we propose a novel meta-learning approach, called Meta-ticket, to find optimal sparse subnetworks for few-shot learning within randomly initialized NNs. We empirically validated that Meta-ticket successfully discover sparse subnetworks that can learn specialized features for each given task. Due to this task-wise adaptation ability, Meta-ticket achieves superior meta-generalization compared to MAML-based methods especially with large NNs. Copyright © 2022, The Authors. All rights reserved.

关键词： Large dataset

q-Neurons: Neuron activations based on stochastic Jackson's derivative operators

学校读者我要写书评

暂无评论

arXiv 2018年

作者： Nielsen, Frank Sun, Ke Sony Computer Science Laboratories Inc. Japan Data61 Australia

We propose a new generic type of stochastic neurons, called q-neurons, that considers activation functions based on Jackson's q-derivatives, with stochastic parameters q. Our generalization of neural network architectures with q-neurons is shown to be both scalable and very easy to implement. We demonstrate experimentally consistently improved performances over state-of-the-art standard activation functions, both on training and testing loss functions. Copyright © 2018, The Authors. All rights reserved.

关键词： Neurons

Effects of local minima and bifurcation delay on combinatorial optimization with continuous variables

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Sato, Shintaro NTT Computer and Data Science Laboratories NTT Corporation Musashino180-8585 Japan

Combinatorial optimization problems can be mapped onto Ising models, and their ground state is generally difficult to find. A lot of heuristics for these problems have been proposed, and one promising approach is to use continuous variables. In recent years, one such algorithm has been implemented by using parametric oscillators known as coherent Ising machines. Although these algorithms have been confirmed to have high performance through many experiments, unlike other familiar algorithms such as simulated annealing, their computational ability has not been fully investigated. In this paper, we propose a simple heuristic based on continuous variables whose static and dynamical properties are easy to investigate. Through the analyses of the proposed algorithm, we find that many local minima in the early stage of the optimization and bifurcation delay reduce its performance in a certain class of Ising models. Copyright © 2022, The Authors. All rights reserved.

关键词： Combinatorial optimization

Modeling, Capacity Studies, Antenna and System Designs for 6G/B6G 3D Continuous-Space Radio Channels Enabled by Electromagnetic Information Theory

学校读者我要写书评

暂无评论

IEEE Communications Surveys and Tutorials 2025年

作者： Wang, Cheng-Xiang Li, Junling Huang, Jie Huang, Chen Zhang, Zhaoyang Liu, Ying Zhou, Shidong Chen, Yunfei You, Xiaohu Gao, Xiqi Cui, Tie Jun Renzo, Marco Di Thompson, John Haas, Harald Schober, Robert Zhang, Rui Southeast University National Mobile Communications Research Laboratory School of Information Science and Engineering Nanjing211189 China Purple Mountain Laboratories Nanjing211111 China Zhejiang University Zhejiang Provincial Key Laboratory of Information Processing Communication and Networking College of Information Science and Electronic Engineering Hangzhou310007 China Xidian University National Key Laboratory of Radar Detection and Sensing Xi'an710071 China Tsinghua University State Key Laboratory on Microwave and Digital Communications Beijing100084 China University of Durham Department of Engineering DurhamDH1 3LE United Kingdom Southeast University State Key Laboratory of Millimeter Waves School of Information Science and Engineering Nanjing211189 China UniversitèParis-Saclay Laboratoire des Signaux et Systèmes CNRS CentraleSupélec Gif-sur-Yvette91192 France Centre for Telecommunications Research - Department of Engineering King's College London LondonWC2R 2LS United Kingdom School of Engineering EdinburghEH9 3BF United Kingdom Cambridge University Department of Engineering CambridgeCB2 1PZ United Kingdom Institute for Digital Communications Erlangen91054 Germany The Chinese University of HongKong School of Science and Engineering Shenzhen Research Institute of Big Data Shenzhen518172 China National University of Singapore Department of Electrical and Computer Engineering 117583 Singapore

Channel theory is a fundamental theory of wireless communications. The sixth generation (6G) and beyond 6G (B6G) wireless communication networks are expected to provide space-air-ground-sea global coverage. Base stations and users tend to move in three dimensional (3D) continuous spaces, while antennas and propagation environments can be tightly coupled. The underlying channels show an evolutionary trend to 3D continuous-space radio channels that combine antennas and wireless propagation channels, in comparison to discrete local-space wireless propagation channels in previous generations. This introduces new challenges for channel modeling, channel capacity analysis, antenna design, system design, etc. To address these challenges, this paper performs a comprehensive study on 3D continuous-space radio channels in 6G/B6G with the aid of electromagnetic information theory (EIT) that integrates electromagnetic theory, information theory, wireless propagation channel modeling theory, and antenna theory. We start by revealing the connections and gaps between these four fundamental theories. Then, an in-depth investigation on the four major research thrusts of 3D continuous-space radio channels is provided: 1) channel measurements and modeling, 2) channel capacity analysis, 3) general antenna design, and 4) wireless system design. We aim to explore the intrinsic relationships between antenna parameters, channel parameters, channel characteristics, channel capacity, and communication system performance. Finally, future research directions and challenges for 3D continuous-space radio channels are outlined. Our study endeavors to establish a fundamental framework for 3D continuous-space radio channels, with the potential to catalyze breakthroughs in 6G/B6G theories. © 1998-2012 IEEE.

关键词： Antenna grounds

Pruning Randomly Initialized Neural Networks with Iterative Randomization

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Chijiwa, Daiki Yamaguchi, Shinya Ida, Yasutoshi Umakoshi, Kenji Inoue, Tomohiro NTT Computer and Data Science Laboratories NTT Corporation Japan NTT Social Informatics Laboratories NTT Corporation Japan

Pruning the weights of randomly initialized neural networks plays an important role in the context of lottery ticket hypothesis. Ramanujan et al. [26] empirically showed that only pruning the weights can achieve remarkable performance instead of optimizing the weight values. However, to achieve the same level of performance as the weight optimization, the pruning approach requires more parameters in the networks before pruning and thus more memory space. To overcome this parameter inefficiency, we introduce a novel framework to prune randomly initialized neural networks with iteratively randomizing weight values (IteRand). Theoretically, we prove an approximation theorem in our framework, which indicates that the randomizing operations are provably effective to reduce the required number of the parameters. We also empirically demonstrate the parameter efficiency in multiple experiments on CIFAR-10 and ImageNet. The code is available at https://***/dchiji-ntt/iterand. Copyright © 2021, The Authors. All rights reserved.

关键词： Machine learning

Attentive statistics pooling for deep speaker embedding

学校读者我要写书评

暂无评论

arXiv 2018年

作者： Okabe, Koji Koshinaka, Takafumi Shinoda, Koichi Data Science Research Laboratories NEC Corporation Japan Department of Computer Science Tokyo Institute of Technology Japan

This paper proposes attentive statistics pooling for deep speaker embedding in text-independent speaker verification. In conventional speaker embedding, frame-level features are averaged over all the frames of a single utterance to form an utterance-level feature. Our method utilizes an attention mechanism to give different weights to different frames and generates not only weighted means but also weighted standard deviations. In this way, it can capture long-term variations in speaker characteristics more effectively. An evaluation on the NIST SRE 2012 and the VoxCeleb data sets shows that it reduces equal error rates (EERs) from the conventional method by 7.5% and 8.1%, respectively. Copyright © 2018, The Authors. All rights reserved.

关键词： Deep neural networks

MagicPool: Dealing with Magic State Distillation Failures on Large-Scale Fault-Tolerant Quantum computer

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Hirano, Yutaka Suzuki, Yasunari Fujii, Keisuke Osaka University Japan NTT Computer and Data Science Laboratories NTT Corporation Japan Japan

Magic state distillation, which is a probabilistic process used to generate magic states, plays an important role in universal fault-tolerant quantum computers. On the other hand, to solve interesting problems, we need to run complex programs on fault-tolerant quantum computers, and hence, the system needs to use hardware resources efficiently. Taking advantage of parallelism is a major optimization strategy and compilers are responsible for performing optimizations to allow parallel processing. However, the probabilistic nature of magic state distillation is not compatible with compile-time optimizations and results in an additional run-time delay. To reduce the additional run-time delay, we propose introducing a pool of magic states. We run simulations of quantum circuits to verify the magnitude of the run-time delay and the usefulness of the mitigation approach. The experimental results show that the run-time delay is amplified by parallel processing, and pooling effectively reduces the run-time delay with a small spatial cost. © 2024, CC BY.

关键词： Time delay

GUARANTEED DETERMINISTIC BOUNDS ON THE TOTAL VARIATION DISTANCE BETWEEN UNIVARIATE MIXTURES

学校读者我要写书评

暂无评论

GUARANTEED DETERMINISTIC BOUNDS ON THE TOTAL VARIATION DISTA...

IEEE Workshop on Machine Learning for Signal Processing

作者： Frank Nielsen Ke Sun Sony Computer Science Laboratories Inc. Japan Data61 CSIRO Australia

The total variation distance is a core statistical distance between probability measures that satisfies the metric axioms, with value always falling in [0,1]. Since the total variation distance does not admit closed-form expressions for statistical mixtures, one often has to rely in practice on costly numerical integrations or on fast Monte Carlo approximations that however do not guarantee deterministic bounds. In this work, we consider two methods for bounding the total variation of univariate mixture models: The first is based on the information monotonicity property of the total variation to design guaranteed nested deterministic lower bounds. The second method relies on computing the geometric lower and upper envelopes of weighted mixture components to derive deterministic bounds based on density ratio. We demonstrate the tightness of our bounds through simulating Gaussian, Gamma and Rayleigh mixture models.

关键词： TV Upper bound Mixture models Density measurement Probability Monte Carlo methods

Modular Extremely Large-Scale Array Communication:Near-Field Modelling and Performance Analysis

学校读者我要写书评

暂无评论

China Communications 2023年第4期20卷 132-152页

作者： Xinrui Li Haiquan Lu Yong Zeng Shi Jin Rui Zhang National Mobile Communications Research Laboratory Frontiers Science Center for Mobile Information Communication and SecuritySoutheast UniversityNanjing 210096China Purple Mountain Laboratories Nanjing 211111China The Chinese University of Hong Kong Shenzhen Research Institute of Big DataShenzhen 518172China Department of Electrical and Computer Engineering National University of SingaporeSingapore 117583Singapore

This paper investigates the wireless communication with a novel architecture of antenna arrays,termed modular extremely large-scale array(XLarray),where array elements of an extremely large number/size are regularly mounted on a shared platform with both horizontally and vertically interlaced *** module consists of a moderate/flexible number of array elements with the inter-element distance typically in the order of the signal wavelength,while different modules are separated by the relatively large inter-module distance for convenience of practical *** accurately modelling the signal amplitudes and phases,as well as projected apertures across all modular elements,we analyse the near-field signal-to-noise ratio(SNR)performance for modular XL-array *** on the non-uniform spherical wave(NUSW)modelling,the closed-form SNR expression is derived in terms of key system parameters,such as the overall modular array size,distances of adjacent modules along all dimensions,and the user's three-dimensional(3D)*** addition,with the number of modules in different dimensions increasing infinitely,the asymptotic SNR scaling laws are ***,we show that our proposed near-field modelling and performance analysis include the results for existing array architectures/modelling as special cases,e.g.,the collocated XL-array architecture,the uniform plane wave(UPW)based far-field modelling,and the modular extremely large-scale uniform linear array(XL-ULA)of *** simulation results are presented to validate our findings.

关键词： modular extremely large-scale array practical deployment projected apertures non-uniform spherical wave near-field modelling