检索结果-内蒙古大学图书馆

arXiv 2021年

作者： Xiao, Fei Pang, Liang Lan, Yanyan Wang, Yan Shen, Huawei Cheng, Xueqi Data Intelligence System Research Center Cas Key Lab of Network Data Science and Technology Institute of Computing Technology Chinese Academy of Sciences Institute for Ai Industry Research Tsinghua University University of Chinese Academy of Sciences Tencent Ai Lab

Unsupervised style transfer models are mainly based on an inductive learning approach, which represents the style as embeddings, decoder parameters, or discriminator parameters and directly applies these general rules to the test cases. However, the lacking of parallel corpus hinders the ability of these inductive learning methods on this task. As a result, it is likely to cause severe inconsistent style expressions, like the salad is rude. To tackle this problem, we propose a novel transductive learning approach in this paper, based on a retrieval-based context-aware style representation. Specifically, an attentional encoderdecoder with a retriever framework is utilized. It involves top-K relevant sentences in the target style in the transfer process. In this way, we can learn a context-aware style embedding to alleviate the above inconsistency problem. In this paper, both sparse (BM25) and dense retrieval functions (MIPS) are used, and two objective functions are designed to facilitate joint learning. Experimental results show that our method outperforms several strong baselines. The proposed transductive learning approach is general and effective to the task of unsupervised style transfer, and we will apply it to the other two typical methods in the future. © 2021, CC BY.

关键词： Embeddings

来源：评论

学校读者我要写书评

暂无评论

On layer normalization in the transformer architecture 37

On layer normalization in the transformer architecture

引用

37th International Conference on Machine Learning, ICML 2020

作者： Xiong, Ruibin Yang, Yunchang He, Di Zheng, Kai Zheng, Shuxin Xing, Chen Zhang, Huishuai Lan, Yanyan Wang, Liwei Liu, Tie-Yan CAS Key Laboratory of Network Data Science and Technology Institute of Computing Technolog Chinese Academy of Sciences China University of Chinese Academy of Sciences China Center for Data Science Peking University Beijing Institute of Big Data Research China Key Laboratory of Machine Perception MOE School of EECS Peking University China Microsoft Research College of Computer Science Nankai University China

ISBN: (纸本)9781713821120

The Transformer is widely used in natural language processing tasks. To train a Transformer however, one usually needs a carefully designed learning rate warm-up stage, which is shown to be crucial to the final performance but will slow down the optimization and bring more hyperparameter tunings. In this paper, we first study theoretically why the learning rate warm-up stage is essential and show that the location of layer normalization matters. Specifically, we prove with mean field theory that at initialization, for the original-designed Post-LN Transformer, which places the layer normalization between the residual blocks, the expected gradients of the parameters near the output layer are large. Therefore, using a large learning rate on those gradients makes the training unstable. The warm-up stage is practically helpful for avoiding this problem. On the other hand, our theory also shows that if the layer normalization is put inside the residual blocks (recently proposed as Pre-LN Transformer), the gradients are well-behaved at initialization. This motivates us to remove the warm-up stage for the training of Pre-LN Transformers. We show in our experiments that Pre-LN Transformers without the warm-up stage can reach comparable results with baselines while requiring significantly less training time and hyper-parameter tuning on a wide range of applications. Copyright 2020 by the author(s).

关键词： Mean field theory

来源：评论

学校读者我要写书评

暂无评论

Research on the Design of Government data Governance Evaluation System under the Perspective of Digital Government

Research on the Design of Government Data Governance Evaluat...

引用

IEEE International Conference on data science in Cyberspace (DSC)

作者： Zirui Xu Jinyang Sun Zhe Lu Delong Han Information Research Institute of Shandong Academy of Sciences Qilu University of Technology(Shandong Academy of Sciences) Jinan China SHANDONG SCICOM Information and Economy Research Institute Co. Ltd Jinan China Key Laboratory of Computing Power Network and Information Security Ministry of Education Shandong Computer Science Center (National Supercomputer Center in Jinan) Qilu University of Technology (Shandong Academy of Sciences) Jinan China

ISBN: (数字)9798350391367

ISBN: (纸本)9798350391374

In the context of the digital era, government data governance is a key factor in promoting the modernization of government governance, but there is a lack of a complete set of evaluation index system to standardize the content and scope of government data governance. This paper adopts the policy tool theory to code and analyze the 12 policies from the three policy tool dimensions of supply layer, demand layer, and environment layer, and proposes three first-level evaluation indicators of basic environment, actual effectiveness, and safeguard mechanism, on the basis of which the evaluation content is selected in a focused manner based on the integration of policy documents related to data governance. Meanwhile, a comprehensive, scientific and objective evaluation system of government data governance is constructed by adopting assessment means combining subjectivity and objectivity, and quantitative and qualitative. The system is designed to assess and guide the data governance practices of the domestic government, point out the direction for government departments at all levels to carry out data governance, and play an important role in promoting the transparency and efficiency of data management.

关键词： Codes Government Cyberspace data science data governance Indexes

来源：评论

学校读者我要写书评

暂无评论

O-PRESS: Boosting OCT axial resolution with Prior guidance, Recurrence, and Equivariant Self-Supervision

arXiv

引用

arXiv 2024年

作者： Li, Kaiyan Yang, Jingyuan Liang, Wenxuan Li, Xingde Zhang, Chenxi Chen, Lulu Wu, Chan Zhang, Xiao Xu, Zhiyan Wang, Yuelin Meng, Lihui Zhang, Yue Chen, Youxin Kevin Zhou, S. School of Biomedical Engineering University of Science and Technology of China Hefei230026 China Suzhou Institute for Advanced Research University of Science and Technology of China Suzhou215123 China Department of Ophthalmology Peking Union Medical College Hospital Chinese Academy of Medical Sciences Beijing100730 China Key Laboratory of Ocular Fundus Diseases Chinese Academy of Medical Sciences Beijing100730 China Department of Biomedical Engineering Johns Hopkins Univesity Baltimore21287 United States Institute of Computing Technology CAS Beijing100190 China School of Physical Sciences University of Science and Technology of China Anhui Hefei230026 China

Optical coherence tomography (OCT) is a noninvasive technology that enables real-time imaging of tissue microanatomies. The axial resolution of OCT is intrinsically constrained by the spectral bandwidth of the employed light source while maintaining a fixed center wavelength for a specific application. Physically extending this bandwidth faces strong limitations and requires a substantial cost. We present a novel computational approach, called as O-PRESS, for boosting the axial resolution of OCT with Prior Guidance, a Recurrent mechanism, and Equivariant Self-Supervision. Diverging from conventional superresolution methods that rely on physical models or data-driven techniques, our method seamlessly integrates OCT modeling and deep learning, enabling us to achieve real-time axial-resolution enhancement exclusively from measurements without a need for paired images. Our approach solves two primary tasks of resolution enhancement and noise reduction with one treatment. Both tasks are executed in a self-supervised manner, with equivariance imaging and free space priors guiding their respective processes. Experimental evaluations, encompassing both quantitative metrics and visual assessments, consistently verify the efficacy and superiority of our approach, which exhibits performance on par with fully supervised methods. Importantly, the robustness of our model is affirmed, showcasing its dual capability to enhance axial resolution while concurrently improving the signal-to-noise ratio. Copyright © 2024, The Authors. All rights reserved.

关键词： Optical tomography

来源：评论

学校读者我要写书评

暂无评论

A Fault Detection Method for Tree Contact Single-Phase-To-Ground Fault

SSRN

引用

SSRN 2023年

作者： Wu, Juping Xv, Huikai Jin, Min Gong, Guoliang Chen, Gang Mao, Wenyu Lu, Huaxiang Chen, Tianxiang Institute of Semiconductors CAS Beijing100083 China University of Chinese Academy of Sciences Beijing China Semiconductor Neural Network Intelligent Perception and Computing Technology Beijing Key Laboratory Beijing China Collage of Microelectronics University of Chinese Academy of Sciences Beijing China Materials and Optoelectronics Research Center University of Chinese Academy of Sciences Beijing China Tianjin University of Science and Technology Tianjin300457 China College of Nuclear Technology and Automation Engineering Chengdu University of Technology Chengdu610059 China

High impedance fault(HIF), especially tree contact single-phase-to-ground fault(TSF), often occurs in resonant grounding systems and are difficult to detect. Therefore, there is an urgent need for an effective TSF detection technology. In this paper, by analyzing the equivalent circuits of different impedance faults in resonant grounding system, and analyzing the transient zero-sequence components of underdamped state and overdamped state under different impedance, the conclusion is drawn: When a TSF occurs, the transient zero-sequence voltage will have a unique " Beat frequency " characteristic, and the problem of covering this phenomenon due to the imbalance of distribution network is effectively solved by fast independent component analysis(FastICA). Based on this conclusion, A fault detection method for TSF based on FastICA is proposed. Finally, the study conducted real experiments using two different impedance trees, bamboo and pine, to verify the proposed detection method. The results of the experiment found clear support for that the detection method is accurate and effective. © 2023, The Authors. All rights reserved.

关键词： Fault detection

来源：评论

学校读者我要写书评

暂无评论

Toward the Understanding of Deep Text Matching Models for Information Retrieval

arXiv

引用

arXiv 2021年

作者： Chen, Lijuan Lan, Yanyan Pang, Liang Guo, Jiafeng Cheng, Xueqi Sogou Inc. Beijing China Institute for AI Industry Research Tsinghua University Beijing China Data Intelligence System Research Center Institute of Computing Technology Chinese Academy of Sciences Beijing China CAS Key Lab of Network Data Science and Technology Institute of Computing Technology Chinese Academy of Sciences Beijing China

Semantic text matching is a critical problem in information retrieval. Recently, deep learning techniques have been widely used in this area and obtained significant performance improvements. However, most models are black boxes and it is hard to understand what happened in the matching process, due to the poor interpretability of deep learning. This paper aims at tackling this problem. The key idea is to test whether existing deep text matching methods satisfy some fundamental heuristics in information retrieval. Specifically, four heuristics are used in our study, i.e., term frequency constraint, term discrimination constraint, length normalization constraints, and TF-length constraint. Since deep matching models usually contain many parameters, it is difficult to conduct a theoretical study for these complicated functions. In this paper, We propose an empirical testing method. Specifically, We first construct some queries and documents to make them satisfy the assumption in a constraint, and then test to which extend a deep text matching model trained on the original dataset satisfy the corresponding constraint. Besides, a famous attribution based interpretation method, namely integrated gradient, is adopted to conduct detailed analysis and guide for feasible improvement. Experimental results on LETOR 4.0 and MS Marco show that all the investigated deep text matching methods, both representation and interaction based methods, satisfy the above constraints with high probabilities in statistics. We further extend these constraints to the semantic settings, which are shown to be better satisfied for all the deep text matching models. These empirical findings give clear understandings on why deep text matching models usually perform well in information retrieval. We believe the proposed evaluation methodology will be useful for testing future deep text matching models. © 2021, CC BY.

关键词： Deep learning

来源：评论

学校读者我要写书评

暂无评论

Energy Efficient Collaborative Federated Learning Design: A Graph Neural network based Approach

Energy Efficient Collaborative Federated Learning Design: A ...

引用

IEEE Conference on Global Communications (GLOBECOM)

作者： Nuocheng Yang Sihua Wang Mingzhe Chen Christopher G. Brinton Changchuan Yin Beijing Laboratory of Advanced Information Network Beijing University of Posts and Telecommunications Beijing China State Key Laboratory of Networking and Switching Technology Beijing University of Posts and Telecommunications Beijing China Department of Electrical and Computer Engineering Institute for Data Science and Computing University of Miami Coral Gables FL USA School of Electrical and Computer Engineering Purdue University West Lafayette IN USA

In this paper, we consider the design of an energy efficient collaborative federated learning (CFL) methodology where devices exchange their local FL parameters with a subset of their neighbors without reliance on a parameter server. In the considered model, mobile devices implement the designed CFL to train their local FL models using their own datasets over a realistic wireless network. Due to the limited wireless resources and user movements, each device may not be able to transmit its FL parameters with all neighboring devices. Therefore, each device must select a subset of devices to share its FL parameters and optimize the transmit power. This problem is formulated as an optimization problem, whose goal is to minimize CFL training energy consumption while satisfying the delay and CFL training loss requirements. To solve this problem, a two-stage solution is proposed. At the first stage, a graph neural network (GNN) based algorithm is proposed, which enables each device to individually determine the subset of devices to transmit FL parameters using its neighboring devices' location and connection information. Compared to standard iterative algorithms that need to iteratively optimize device connections and transmit power, the proposed GNN based method can directly obtain the optimal device connections without iterative optimization. Given the optimal device connections, at the second stage, each device can directly obtain the optimal transmit power. Simulation results show that the proposed algorithm can decrease energy consumption by up to 46% compared to the algorithm where each device will directly connect to its first and second nearest neighbors.

关键词：

来源：评论

学校读者我要写书评

暂无评论

One Comment from One Perspective: An Effective Strategy for Enhancing Automatic Music Comment 28

One Comment from One Perspective: An Effective Strategy for ...

引用

28th International Conference on Computational Linguistics, COLING 2020

作者： Huo, Tengfei Liu, Zhiqiang Zhang, Jinchao Zhou, Jie CAS Key Lab of Network Data Science and Technology China Institute of Computing Technology Chinese Academy of Sciences China University of Chinese Academy of Sciences Beijing China Pattern Recognition Center WeChat AI Tencent Inc. China

ISBN: (纸本)9781952148279

The automatic generation of music comments is of great significance for increasing the popularity of music and the music platform’s activity. In human music comments, there exists high distinction and diverse perspectives for the same song. In other words, for a song, different comments stem from different musical perspectives. However, to date, this characteristic has not been considered well in research on automatic comment generation. The existing methods tend to generate common and meaningless comments. In this paper, we propose an effective multi-perspective strategy to enhance the diversity of the generated comments. The experiment results on two music comment datasets show that our proposed model can effectively generate a series of diverse music comments based on different perspectives, which outperforms state-of-the-art baselines by a substantial margin. © 2020 COLING 2020 - 28th International Conference on Computational Linguistics, Proceedings of the Conference. All rights reserved.

关键词： Music

来源：评论

学校读者我要写书评

暂无评论

RLEPSO:Reinforcement learning based Ensemble particle swarm optimizer

RLEPSO:Reinforcement learning based Ensemble particle swarm ...

引用

2021年第四届算法、计算和人工智能国际会议

作者： Shiyuan Yin Yi Liu GuoLiang Gong Huaxiang Lu Wenchang Li Institute of Semiconductors CAS Institute of Semiconductors CAS University of Chinese Academy of Sciences China Academy of Sciences brain science and intelligent technology excellence innovation center Beijing Key Laboratory of semiconductor neural network intelligent sensing and computing technology

Evolution is the driving force behind the evolution of biological intelligence. Learning is the driving force behind human civilization. The combination of evolution and learning can form an entire natural world. Now, reinforcement learning has shown significant effects in many places. However, Currently, researchers in the field of optimization algorithms mainly focus on evolution *** there is very little research on learning. Inspired by these ideas,this paper proposes a new particle swarm optimization algorithm Reinforcement learning based Ensemble particle swarm optimizer(RLEPSO) that combines reinforcement learning. The algorithm uses reinforcement learning for pre-training in the design phase to automatically find a more effective combination of parameters for the algorithm to run better and Complete optimization tasks faster. Besides, this algorithm integrates two robust particle swarm variants. And it sets the weight parameters for different algorithms to better adapt to the solution requirements of a variety of different optimization problems, which significantly improves the robustness of the algorithm. RLEPSO makes a certain number of sub-swarms to increase the probability of finding the global optimum and increasing the diversity of particle swarms. This proposed RLEPSO is evaluated on an optimization test functions benchmark set(CEC2013) with 28 functions and compared with other eight particle swarm optimization variants, including three state-of-theart optimization algorithms. The results show that RLEPSO has better performance and outperforms all compared algorithms.

关键词： Particle Swarm Optimization Reinforcement Learning Policy Gradient

来源：评论

学校读者我要写书评

暂无评论

RobustPFL: Robust Personalized Federated Learning

引用

IEEE Transactions on Dependable and Secure computing 2025年

作者： Chen, Guorong Wang, Wei Wu, Yufang Li, Chao Xu, Guangquan Ji, Shouling Li, Tao Shen, Meng Han, Yufei Xi'an Jiaotong University Ministry of Education Key Lab for Intelligent Networks and Network Security Xi'an China Beijing Jiaotong University 3 Shangyuancun Beijing Key Laboratory of Security and Privacy in Intelligent Transportation Beijing100044 China Tianjin University Tianjin Key Laboratory of Advanced Networking College of Intelligence and Computing Tianjin300350 China Zhejiang University College of Computer Science and Technology Hangzhou310027 China ITAI Haihe Lab Tianjin China Nankai University College of Computer Science Tianjin300350 China Beijing Institute of Technology School of Cyberspace Security Beijing100081 China Peng Cheng Laboratory Cyberspace Security Research Center Shenzhen518066 China INRIA Bretagne Rennes35042 France

Conventional federated learning (FL) coordinated by a central server focuses on training a global model and protecting the privacy of clients' training data by storing it locally. However, the statistical heterogeneity hinders the global model from adapting to the non-IID distributions among clients. Moreover, untrusted and unreliable central servers and malicious clients may compromise model integrity and availability, thus degrading the robustness of FL. To address these challenges, we present RobustPFL, a decentralized personalized federated learning (PFL) approach that combines α-based Layer-position Normalized Similarity (α-LNS) and local collaborative training to improve personalized performance while utilizing a blockchain-based committee mechanism to coordinate the aggregation process, thereby achieving high personalized accuracy and robustness. Extensive experiments show that our RobustPFL approach outperforms multiple algorithms, including Local training, FedAvg, FedReptile, Per-FedAvg, FedBN, and SPFL, on MNIST, CIFAR10, EMNIST, and N-BaIoT datasets in four non-IID settings. We also evaluate RobustPFL's effectiveness against attacks - poisoning attacks and free-riding attacks. Particularly, for three prevalent poisoning attacks (backdoor, label flipping, and model poisoning attacks), we compare non-defensive (FedAvg) and defensive (Krum, trimmed mean, Bulyan, FedBN, FLAME, and FangTrmean) methods with our proposed RobustPFL. The results show that our approach achieves significant defensive effects. © 2004-2012 IEEE.

关键词： Federated learning

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：