检索结果-内蒙古大学图书馆

CROSS-INFERENTIAL NETWORKS FOR SOURCE-FREE UNSUPERVISED DOMAIN ADAPTATION

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Tang, Yushun Guo, Qinghai He, Zhihai Department of Electronic and Electrical Engineering Southern University of Science and Technology Shenzhen China Advanced Computing and Storage Laboratory Huawei Technologies Co. Ltd. Shenzhen China Pengcheng Laboratory Shenzhen China

关键词： Forecasting

Neuro-Modulated Hebbian Learning for Fully Test-Time Adaptation

学校读者我要写书评

暂无评论

Neuro-Modulated Hebbian Learning for Fully Test-Time Adaptat...

Conference on Computer Vision and Pattern Recognition (CVPR)

作者： Yushun Tang Ce Zhang Heng Xu Shuoshuo Chen Jie Cheng Luziwei Leng Qinghai Guo Zhihai He Department of Electronic and Electrical Engineering Southern University of Science and Technology Shenzhen China Advanced Computing and Storage Laboratory Huawei Technologies Co. Ltd. Shenzhen China Pengcheng Laboratory Shenzhen China

Fully test-time adaptation aims to adapt the network model based on sequential analysis of input samples during the inference stage to address the cross-domain performance degradation problem of deep neural networks. We take inspiration from the biological plausibility learning where the neuron responses are tuned based on a local synapse-change procedure and activated by competitive lateral inhibition rules. Based on these feed-forward learning rules, we design a soft Hebbian learning process which provides an unsupervised and effective mechanism for online adaptation. We observe that the performance of this feed-forward Hebbian learning for fully test-time adaptation can be significantly improved by incorporating a feedback neuromodulation layer. It is able to fine-tune the neuron responses based on the external feedback generated by the error backpropagation from the top inference layers. This leads to our proposed neuro-modulated Hebbian learning (NHL) method for fully test-time adaptation. With the unsupervised feed-forward soft Hebbian learning being combined with a learned neuromodulator to capture feedback from external responses, the source model can be effectively adapted during the testing process. Experimental results on benchmark datasets demonstrate that our proposed method can significantly improve the adaptation performance of network models and outperforms existing state-of-the-art methods.

关键词：

Explicit Mutual Information Maximization for Self-Supervised Learning

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Chang, Lele Liu, Peilin Guo, Qinghai Wen, Fei School of Electronic Information and Electrical Engineering Shanghai Jiao Tong University Shanghai China Advanced Computing and Storage Laboratory Huawei Technologies Co. Ltd. Shenzhen China

Recently, self-supervised learning (SSL) has been extensively studied. Theoretically, mutual information maximization (MIM) is an optimal criterion for SSL, with a strong theoretical foundation in information theory. However, it is difficult to directly apply MIM in SSL since the data distribution is not analytically available in applications. In practice, many existing methods can be viewed as approximate implementations of the MIM criterion. This work shows that, based on the invariance property of MI, explicit MI maximization can be applied to SSL under a generic distribution assumption, i.e., a relaxed condition of the data distribution. We further illustrate this by analyzing the generalized Gaussian distribution. Based on this result, we derive a loss function based on the MIM criterion using only second-order statistics. We implement the new loss for SSL and demonstrate its effectiveness via extensive experiments. Copyright © 2024, The Authors. All rights reserved.

关键词： Self-supervised learning

Tsdat: An Open-Source Data Standardization Framework for Marine Energy and Beyond

学校读者我要写书评

暂无评论

Tsdat: An Open-Source Data Standardization Framework for Mar...

OCEANS

作者： Carina Lansing Maxwell Levin Chitra Sivaraman Rebecca Fao Frederick Driscoll Advanced Computing Mathematics and Data Division Pacific Northwest National Laboratory Richland U.S.A. Water Power Energy Conversion & Storage Systems National Renewable Energy Laboratory Golden U.S.A.

ISBN: (纸本)9781665427883

Many organizations are tasked with the collection and processing of large quantities of data from various measurement devices. Data reported from these sources are often not interoperable with datasets and software used by analysts and other organizations in the same domain, introducing barriers for collaboration on large-scale projects. This poses a particular problem for cross-device comparisons and machine learning applications, which rely on large quantities of data from multiple sources. To address these challenges, the open-source Time-Series Data Pipelines (Tsdat) Python framework was developed by Pacific Northwest National laboratory, with strategic guidance and direction provided by the National Renewable Energy laboratory and Sandia National Laboratories to facilitate collaboration and accelerate advancements in the marine energy domain through the development of an open-source ecosystem of tools. This paper will describe the Tsdat framework and the data standards within which it operates. A beta version of Tsdat has been released and is being used by several projects in marine energy, wind energy, and building energy systems.

关键词： Wind energy Instruments Standards organizations Buildings Collaboration Organizations Task analysis

Learning Visual Conditioning Tokens to Correct Domain Shift for Fully Test-time Adaptation

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Tang, Yushun Chen, Shuoshuo Kan, Zhehan Zhang, Yi Guo, Qinghai He, Zhihai The Department of Electrical and Electronic Engineering Southern University of Science and Technology Shenzhen China The Pengcheng Lab Shenzhen China The Advanced Computing and Storage Laboratory Huawei Technologies Co. LTD China

Fully test-time adaptation aims to adapt the network model based on sequential analysis of input samples during the inference stage to address the cross-domain performance degradation problem of deep neural networks. This work is based on the following interesting finding: in transformer-based image classification, the class token at the first transformer encoder layer can be learned to capture the domain-specific characteristics of target samples during test-time adaptation. This learned token, when combined with input image patch embeddings, is able to gradually remove the domain-specific information from the feature representations of input samples during the transformer encoding process, thereby significantly improving the test-time adaptation performance of the source model across different domains. We refer to this class token as visual conditioning token (VCT). To successfully learn the VCT, we propose a bi-level learning approach to capture the long-term variations of domain-specific characteristics while accommodating local variations of instance-specific characteristics. Experimental results on the benchmark datasets demonstrate that our proposed bi-level visual conditioning token learning method is able to achieve significantly improved test-time adaptation performance by up to 1.9%. Copyright © 2024, The Authors. All rights reserved.

关键词： Deep neural networks

Statistical physics of deep neural networks: Initialization toward optimal channels

学校读者我要写书评

暂无评论

Physical Review Research 2023年第2期5卷 023023-023023页

作者： Kangyu Weng Aohua Cheng Ziyang Zhang Pei Sun Yang Tian Tsien Excellence in Engineering Program Tsinghua University Beijing 100084 China Laboratory of Advanced Computing and Storage Central Research Institute 2012 Laboratories Huawei Technologies Co. Ltd. Beijing 100084 China Department of Psychology & Tsinghua Laboratory of Brain and Intelligence Tsinghua University Beijing 100084 China

In deep learning, neural networks serve as noisy channels between input data and its latent representation. This perspective naturally relates deep learning with the pursuit of constructing channels with optimal performance in information transmission and representation. While considerable efforts are concentrated on realizing optimal channel properties during network optimization, we study a frequently overlooked possibility that neural networks can be initialized toward optimal channels. Our theory, consistent with experimental validation, identifies primary mechanics underlying this unknown possibility and suggests intrinsic connections between statistical physics and deep learning. Unlike the conventional theories that characterize neural networks applying the classic mean-field approximation, we offer analytic proof that this extensively applied simplification scheme is not appropriate in studying neural networks as information channels. To fill this gap, we develop a restricted mean-field framework applicable for characterizing the limiting behaviors of information propagation in neural networks without strong assumptions on inputs. Based on it, we propose an analytic theory to prove that mutual information maximization is realized between inputs and propagated signals when neural networks are initialized at dynamic isometry, a case where information transmits via norm-preserving mappings. These theoretical predictions are validated by experiments on real neural networks, suggesting the robustness of our theory against finite-size effects. Finally, we analyze our findings with information bottleneck theory to confirm the precise relations among dynamic isometry, mutual information maximization, and optimal channel properties in deep learning. Our work may lay a cornerstone for promoting deep learning in terms of network initialization and suggest general statistical physics mechanisms underlying diverse deep learning techniques.

关键词： Neuronal dynamics Artificial neural networks Deep learning Information theory Random matrix theory

Abnormal Thermal Conductivity Increase in β-Ga2O3 by an Unconventional Bonding Mechanism Using Machine-Learning Potential

学校读者我要写书评

暂无评论

SSRN

SSRN 2024年

作者： Zhou, Wu-Xing Wu, Cheng-Wei Cao, Hao-Ran Zeng, Yu-Jia Xie, Guofeng Zhang, Gang School of Materials Science and Engineering Hunan Provincial Key Laboratory of Advanced Materials for New Energy Storage and Conversion Hunan University of Science and Technology Xiangtan411201 China Institute of High Performance Computing A*STAR Singapore138632 Singapore

β-Ga2O3, with its ultrawide band gap (~4.9 eV) and high critical electric field, holds potential in power electronics but is limited by low thermal conductivity, which is critical to the performance and reliability because the high level of heat flux density induced by the extremely high levels of power density. Combining first-principles calculations, machine-learning potentials, and solving the phonon Boltzmann transport equation, we found that substituting octahedral-coordinated Ga with Al significantly enhances thermal conductivity from 100K to 800K. At room temperature, Al-substituted β-Ga2O3 achieves 38.91 W/mK, more than 2-fold that of pristine β-Ga2O3 (17.10 W/mK) and even higher than β-Al2O3 (34.77 W/mK). This enhancement, unusual due to the heavier atomic mass and mixed mass distribution, is rooted in suppressed anharmonic characteristics caused by reduced bonding strength inhomogeneity. Our results may inspire the rational design of materials with tailored thermal properties through chemical bonding mechanisms. © 2024, The Authors. All rights reserved.

关键词： Thermal conductivity

MDNet: Learning Monaural Speech Enhancement from Deep Prior Gradient

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Li, Andong Zheng, Chengshi Zhang, Ziyang Li, Xiaodong Key Laboratory Of Noise And Vibration Research Institute Of Acoustics Chinese Academy Of Sciences Beijing China University Of Chinese Academy Of Sciences Beijing China Advanced Computing And Storage Lab Huawei Technologies Co. Ltd. Beijing China

While traditional statistical signal processing model-based methods can derive the optimal estimators relying on specific statistical assumptions, current learning-based methods further promote the performance upper bound via deep neural networks but at the expense of high encapsulation and lack adequate interpretability. Standing upon the intersection between traditional model-based methods and learning-based methods, we propose a model-driven approach based on the maximum a posteriori (MAP) framework, termed as MDNet, for single-channel speech enhancement. Specifically, the original problem is formulated into the joint posterior estimation w.r.t. speech and noise components. Different from the manual assumption toward the prior terms, we propose to model the prior distribution via networks and thus can learn from training data. The framework takes the unfolding structure and in each step, the target parameters can be progressively estimated through explicit gradient descent operations. Besides, another network serves as the fusion module to further refine the previous speech estimation. The experiments are conducted on the WSJ0-SI84 and Interspeech2020 DNS-Challenge datasets, and quantitative results show that the proposed approach outshines previous state-of-the-art baselines. Copyright © 2022, The Authors. All rights reserved.

关键词： Deep neural networks

Enhanced High-Temperature Thermoelectric Performance by Strain Engineering in BiOCl

学校读者我要写书评

暂无评论

Physical Review Applied 2022年第1期18卷 014053-014053页

作者： Cheng-Wei Wu Xue Ren Guofeng Xie Wu-Xing Zhou Gang Zhang Ke-Qiu Chen School of Materials Science and Engineering & Hunan Provincial Key Laboratory of Advanced Materials for New Energy Storage and Conversion Hunan University of Science and Technology Xiangtan 411201 China Institute of High Performance Computing A*STAR Singapore 138632 Singapore Department of Applied Physics School of Physics and Electronics Hunan University Changsha 410082 China

Semiconductor BiOCl has a layered structure with ultralow lattice thermal conductivity [Q.D. Gibson et al., Science 373, 1017–1022 (2021)] and has potential applications in the field of thermoelectric materials. In the present study, the thermoelectric properties of BiOCl crystals are accurately predicted using the first-principles calculation combined with Boltzmann transport theory. The dimensionless figure of merit (ZT) of p-type BiOCl is found to be 0.2 at room temperature, reaching 1.1 at 800 K. In addition, applying in-plane biaxial tensile strain ϵxy can lead to a further increase in the value of ZT to 1.9 at 800 K. This means that p BiOCl is an excellent high-temperature thermoelectric material. And this is due to the fact that biaxial strain drastically reduces the lattice thermal conductivity and the shift of the valence band toward the Fermi level can optimize the carrier concentration. Thus, the present work paves a way for the design of adjustable high-temperature thermoelectric materials.

关键词： First-principles calculations Heat transfer Quantum transport Thermoelectric effects Thermoelectrics 2-dimensional systems