检索结果-内蒙古大学图书馆

Multi Strategy Improvement of Newton-Raphson-Based Optimizer and Engineering Application

学校读者我要写书评

暂无评论

Multi Strategy Improvement of Newton-Raphson-Based Optimizer...

International Conference on Machine Learning, Big Data and Business Intelligence (MLBDBI)

作者： Shaogang Cheng Jun Yin Tingting Liu College of Computing Qinghai Normal University Xining China The State Key Laboratory of Tibetan Intelligent Information Processing and Application Qinghai Normal University Xining China

ISBN: (数字)9798331541798

ISBN: (纸本)9798331505615

A multi strategy improved Newton- Raphson-Based Optimizer (NRBO) is proposed to address the problem of slow convergence speed in the later stage of algorithm iteration and difficulty in finding the global optimal solution when facing complex functions. Firstly, using a set of best points instead of randomly generated initial populations to evenly distribute individuals in the optimization space improves the efficiency of the algorithms global and local search. Secondly, based on the idea of differential evolution, the initial population is mutated and crossed to accelerate the iteration speed. Then, in the optimal update stage, the Levi flight strategy is introduced to enhance the algorithms ability to jump out of local optima. The effectiveness of the proposed algorithm was tested on the CEC2005 test set, and compared with other algorithms on two test sets. The results showed that the convergence speed and accuracy of the algorithm were significantly improved. Finally, the proposed algorithm was applied to two practical engineering optimization problems, and simulation results showed that the algorithm proposed in this paper has better practicality and robustness compared to other algorithms.

关键词： Machine learning algorithms Accuracy Simulation Machine learning Robustness Business intelligence Optimization Faces Convergence Testing

Linear Precoding for Faster-Than-Nyquist Signaling with Imperfect Channel State information

学校读者我要写书评

暂无评论

Linear Precoding for Faster-Than-Nyquist Signaling with Impe...

International Conference on Wireless Communications and Signal processing (WCSP)

作者： Yan Wang Qiang Li Yingsong Li Liping Li Xingwang Li Key Laboratory of Intelligent Computing and Signal Processing Ministry of Education Anhui University China School of Physics and Electronic Information Engineering Henan Polytechnic University China

ISBN: (数字)9798350390643

ISBN: (纸本)9798350390650

In faster-than-Nyquist (FTN) transmission over frequency-selective fading channels, it is essential to eliminate both FTN-induced intersymbol-interference (ISI) (FTN-ISI) and channel-induced ISI (channel-ISI). Although frequency domain equalization (FDE) techniques can mitigate channel-ISI at the receiver, their performance is often inadequate. In contrast, cyclic prefix/suffix (CPS)-G-to-minus-half (GTMH) precoding shows superior effectiveness in mitigating channel-ISI. However, existing CPS-GTMH precoding techniques assume the transmitter obtains perfect channel state information (CSI) estimated at the receiver. This is impractical due to the high latency in satellite communication links. Therefore, this paper explores precoding for FTN signaling under imperfect CSI (ICSI) and proposes CPS-minimum mean squared error (MMSE) and linear pre-equalization (LPE)-MMSE precoding schemes based on the MMSE criterion. Both schemes employ a cyclic prefix to transform the channel-ISI matrix into the circular matrix, thereby avoiding inter-block interference. The primary distinction is that CPS-MMSE precoding handles FTN-ISI and channel-ISI together using MMSE precoding. In contrast, LPE-MMSE precoding addresses FTN-ISI with LPE and channel-ISI with MMSE precoding separately. Simulation results indicate that CPS-MMSE and LPE-MMSE precodings outperform CPS-FDE and LPE-FDE in mitigating channel-ISI when the estimation error is 0.005 and 0.01. For instance, with a packing ratio of 0.9, a roll-off factor of 0.25, and an estimation error of 0.01, CPS-MMSE and LPE-MMSE precodings achieve bit error rate (BER) gains of approximately 0.8 dB and 1 dB at a BER of 1e4, respectively. However, when the estimation error increases to 0.1, the BER performance of CPS-FDE and LPE-FDE surpasses that of the CPS-MMSE and LPE-MMSE precodings, indicating the proposed precoding schemes' unsuitability for larger channel estimation errors.

关键词： Estimation error Precoding Simulation Bit error rate Channel estimation Interference Receivers Matrix converters Satellite communications Channel state information

Improving Bert Fine-Tuning via Stabilizing Cross-Layer Mutual information 48

学校读者我要写书评

暂无评论

Improving Bert Fine-Tuning via Stabilizing Cross-Layer Mutua...

48th IEEE International Conference on Acoustics, Speech and Signal processing, ICASSP 2023

作者： Li, Jicun Li, Xingjian Wang, Tianyang Wang, Shi Cao, Yanan Xu, Chengzhong Dou, Dejing Key Laboratory of Intelligent Information Processing Institute of Computing Technology Chinese Academy of Sciences China University of Chinese Academy of Sciences China Big Data Lab Baidu Research China University of Alabama Birmingham United States Institute of Information Engineering Chinese Academy of Sciences China State Key Lab of Iotsc University of Macau China

ISBN: (纸本)9781728163277

Fine-tuning pre-trained language models, such as BERT, has shown enormous success among various NLP tasks. Though simple and effective, the process of fine-tuning has been found unstable, which often leads to unexpected poor performance. To increase stability and generalizability, most existing works resort to maintaining the parameters or representations of pre-trained models during fine-tuning. Nevertheless, very little work explores mining the reliable part of pre-learned information that can help to stabilize fine-tuning. To address this challenge, we introduce a novel solution in which we fine-tune BERT with stabilized cross-layer mutual information. Our method aims to preserve the reliable behaviors of cross-layer information propagation, instead of preserving the information itself, of the pre-trained model. Therefore, our method circumvents the domain conflicts between pre-trained and target tasks. We conduct extensive experiments with popular pre-trained BERT variants on NLP datasets, demonstrating the universal effectiveness and robustness of our method. © 2023 IEEE.

关键词： Fine-tuning Stability Mutual information Pre-trained Language Model

A Hybrid Iterative Neural Solver Based on Spectral Analysis for Parametric PDEs

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Cui, Chen Jiang, Kai Liu, Yun Shu, Shi Hunan Key Laboratory for Computation and Simulation in Science and Engineering Key Laboratory of Intelligent Computing and Information Processing of Ministry of Education School of Mathematics and Computational Science Xiangtan University Hunan Xiangtan411105 China

Recently, deep learning-based hybrid iterative methods (DL-HIM) have emerged as a promising approach for designing fast neural solvers to tackle large-scale sparse linear systems. DL-HIM combine the smoothing effect of simple iterative methods with the spectral bias of neural networks, which allows them to effectively eliminate both high-frequency and low-frequency error components. However, their efficiency may decrease if simple iterative methods can not provide effective smoothing, making it difficult for the neural network to learn mid-frequency and high-frequency components. This paper first conducts a convergence analysis for general DL-HIM from a spectral viewpoint, concluding that under reasonable assumptions, DL-HIM exhibit a convergence rate independent of grid size h and physical parameters µ. To meet these assumptions, we design a neural network from an eigen perspective, focusing on learning the eigenvalues and eigenvectors corresponding to error components that simple iterative methods struggle to eliminate. Specifically, the eigenvalues are learned by a meta subnet, while the eigenvectors are approximated using Fourier modes with a transition matrix provided by another meta subnet. The resulting DL-HIM, termed the Fourier Neural Solver (FNS), can be trained to achieve a convergence rate independent of grid size and PDE parameters by designing a loss function that ensures the neural network complements the smoothing effect of Jacobi iterative methods. We verify the performance of FNS on six types of linear parametric *** Codes 65F10, 65F08, 68T07, 65N22, 90C06 Copyright © 2024, The Authors. All rights reserved.

关键词： Eigenvalues and eigenfunctions

Enhancing Neural Machine Translation with Semantic Units

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Huang, Langlin Gu, Shuhao Zhang, Zhuocheng Feng, Yang Key Laboratory of Intelligent Information Processing Institute of Computing Technology Chinese Academy of Sciences University of Chinese Academy of Sciences China

Conventional neural machine translation (NMT) models typically use subwords and words as the basic units for model input and comprehension. However, complete words and phrases composed of several tokens are often the fundamental units for expressing semantics, referred to as semantic units. To address this issue, we propose a method Semantic Units for Machine Translation (SU4MT) which models the integral meanings of semantic units within a sentence, and then leverages them to provide a new perspective for understanding the sentence. Specifically, we first propose Word Pair Encoding (WPE), a phrase extraction method to help identify the boundaries of semantic units. Next, we design an Attentive Semantic Fusion (ASF) layer to integrate the semantics of multiple subwords into a single vector: the semantic unit representation. Lastly, the semantic-unit-level sentence representation is concatenated to the token-level one, and they are combined as the input of encoder. Experimental results demonstrate that our method effectively models and leverages semantic-unit-level information and outperforms the strong baselines. The code is available at https://***/ictnlp/SU4MT. © 2023, CC BY-NC-ND.

关键词： Neural machine translation

Momentum-Accelerated Richardson(m) and Their Multilevel Neural Solvers

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Wang, Zhen Liu, Yun Cui, Chen Shu, Shi Hunan Key Laboratory for Computation and Simulation in Science and Engineering Key Laboratory of Intelligent Computing and Information Processing of Ministry of Education School of Mathematics and Computational Science Xiangtan University Hunan Xiangtan411105 China

Recently, designing neural solvers for large-scale linear systems of equations has emerged as a promising approach in scientific and engineering computing. This paper first introduce the Richardson(m) neural solver by employing a meta network to predict the weights of the long-step Richardson iterative method. Next, by incorporating momentum and preconditioning techniques, we further enhance convergence. Numerical experiments on anisotropic second-order elliptic equations demonstrate that these new solvers achieve faster convergence and lower computational complexity compared to both the Chebyshev iterative method with optimal weights and the Chebyshev semi-iteration method. To address the strong dependence of the aforementioned single-level neural solvers on PDE parameters and grid size, we integrate them with two multilevel neural solvers developed in recent years. Using alternating optimization techniques, we construct Richardson(m)FNS for anisotropic equations and NAG-Richardson(m)-WANS for the Helmholtz equation. Numerical experiments show that these two multilevel neural solvers effectively overcome the drawback of single-level methods, providing better robustness and computational efficiency. Copyright © 2024, The Authors. All rights reserved.

关键词： Iterative methods

Improved Coded Caching Scheme for Multi-User information Retrieval System

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Wang, Junyi Zang, Quan Wang, Jinyu Cheng, Minquan The Key Laboratory of Cognitive Radio and Information Processing Ministry of Education Guilin University of Electronic Technology Guilin541004 China The Key Laboratory of Education Blockchain and Intelligent Technology Ministry of Education The Guangxi Key Laboratory of Multi-Source Information Mining and Security Guangxi Normal University Guilin541004 China

In this paper, we study the coded caching scheme for the (L, K, M, N) multi-user information retrieval (MIR) system, which consists of a content library containing N files, a base station (BS) with L antennas that cannot access the library, and K single-antenna users, each of which can cache at most M files from the library. The users communicate with the others assisted by the BS to decode their required files. In this paper, we focus on designing a coded caching scheme with low communication latency measured by normalized delivery time (NDT), computational complexity, and subpacketizations. When KM/N ≥ L we first simply the precoding matrix in the downlink step to an identity matrix and use the multiple-antenna placement delivery array (MAPDA), which was originally proposed for the multiple-input single-output networks, to generate several new schemes for MIR system. Compared to the existing schemes, both the theoretical and numerical analyses show that our new schemes achieve much lower computational complexity and smaller subpacketizations with the same NDT. Copyright © 2025, The Authors. All rights reserved.

关键词： Antenna arrays

Conservative nonconforming virtual element method for stationary incompressible magnetohydrodynamics

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Dong, Xiaojing Huang, Yunqing Wang, Tianwen Hunan Key Laboratory for Computation and Simulation in Science and Engineering Key Laboratory of Intelligent Computing & Information Processing of Ministry of Education School of Mathematics and Computational Science Xiangtan University Hunan Xiangtan411105 China

In this paper, we propose a conservative nonconforming virtual element method for the full stationary incompressible magnetohydrodynamics model. We leverage the virtual element satisfactory divergence-free property to ensure mass conservation for the velocity field. The condition of the well-posedness of the proposed method, as well as the stability are derived. We establish optimal error estimates in the discrete energy norm for both the velocity and magnetic field. Furthermore, by employing a new technique, we obtain the optimal error estimates in L2-norm without any additional conditions. Finally, numerical experiments are presented to validate the theoretical analysis. In the implementation process, we adopt the effective Oseen iteration to handle the nonlinear system. © 2024, CC BY-NC-ND.

关键词： Magnetohydrodynamics

A virtual element method with IMEX-SAV scheme for the incompressible magnetohydrodynamics equations

学校读者我要写书评

暂无评论

arXiv 2024年

This paper proposes a virtual element method (VEM) combined with a second-order implicit-explicit scheme based on the scalar auxiliary variable (SAV) method for the incompressible magnetohydrodynamics (MHD) equations. We employ the BDF2 scheme for time discretization and a conservative VEM for spatial discretization, in which the mass conservation in the velocity field is kept by taking advantage of the virtual element method’s adaptability and its divergence-free characteristics. In our scheme, the nonlinear terms are handled explicitly using the SAV method, and the magnetic field is decoupled from the velocity and pressure. This decoupling only requires solving a sequence of linear systems with constant coefficient at each time step. The stability estimate of the fully discrete scheme is developed, demonstrating the scheme is unconditionally stable. Moreover, rigorous error estimates for the velocity and magnetic field are provided. Finally, numerical experiments are presented to verify the valid of theoretical analysis. © 2024, CC BY-NC-ND.

关键词： Discrete element methods