检索结果-内蒙古大学图书馆

Sparse optimization guided pruning for neural networks

NEUROCOMPUTING 2024年 574卷

作者： Shi, Yong Tang, Anda Niu, Lingfeng Zhou, Ruizhi Univ Chinese Acad Sci Sch Math Sci Beijing 100190 Peoples R China Univ Chinese Acad Sci Sch Econ & Management Beijing 100190 Peoples R China Chinese Acad Sci Res Ctr Fictitious Econ & Data Sci Beijing 100190 Peoples R China Chinese Acad Sci Key Lab Big Data Min & Knowledge Management Beijing 100190 Peoples R China Beijing Univ Technol Inst Operat Res & Informat Engn Beijing 100124 Peoples R China

Neural network pruning is a critical field aimed at reducing the infrastructure costs of neural networks by removing parameters. Traditional methods follow a fixed paradigm including pretraining, pruning, and fine-tuning. Despite the close relationship among these three stages, most pruning methods treat them as independent processes. In this paper, we propose a novel two-stage pruning method, which includes pretraining a network that is instructive for subsequent pruning, and a unified optimization model that integrates pruning and fine-tuning. Specifically, in the first stage, we design a group sparse regularized model for pretraining. This model not only safeguards the network from irreversible damage but also offers valuable insights for the pruning process. In the second stage, we introduce an element-wise sparse regularization into pruning model. This model enables us to pinpoint sparse weights more precisely than pretrained network. It automatically derives effective pruning criteria, and omits the step of fine-tuning. To implement the two-stage process in practice, we utilize stochastic gradient algorithm for the pretraining and design a threshold algorithm for pruning stage. Extensive experiments confirm the competitive performance of our proposed method in terms of both accuracy and memory cost when compared to various benchmarks. Furthermore, ablation experiments validate the effectiveness of the proposed pretraining model's guidance for the pruning process.

关键词： Deep neural networks Model compression Sparse optimization

来源：评论

学校读者我要写书评

暂无评论

The neural network methods for solving Traveling Salesman Problem 8

The neural network methods for solving Traveling Salesman Pr...

引用

8th International Conference on Information Technology and Quantitative management (ITQM) - Developing Global Digital Economy after COVID-19

作者： Shi, Yong Zhang, Yuanying Univ Chinese Acad Sci Sch Econ & Management Beijing 100190 Peoples R China Chinese Acad Sci Res Ctr Fictitious Econ & Data Sci Beijing 100190 Peoples R China Chinese Acad Sci Key Lab Big Data Min & Knowledge Management Beijing 100190 Peoples R China Univ Nebraska Omaha Coll Informat Sci & Technol Omaha NE 68182 USA Univ Chinese Acad Sci Sch Math Sci Beijing 100190 Peoples R China

Traveling Salesman Problem(TSP) is a main attention issue at present. Neural network can be used to solve combinatorial optimization problems. In recent years, there have existed many neural network methods for solving TSP, which has made a big step forward for solving combinatorial optimization problems. This paper reviews the neural network methods for solving TSP in recent years, including Hopfield neural network, graph neural network and neural network with reinforcement learning. Using neural network to solve TSP can effectively improve the accuracy of the approximate solution. Finally, we put forward the prospect of solving TSP in the future. (C) 2021 The Authors. Published by Elsevier B.V.

关键词： Traveling Salesman Problem Neural network Hopfield neural network Graph neural network Reinforcement learning

来源：评论

学校读者我要写书评

暂无评论

The Development of Green Olympic Games: A Comparison of CO2Emission Reduction Strategies between PyeongChang 2018 and Beijing 2022 9th

The Development of Green Olympic Games: A Comparison of CO2E...

引用

9th International Conference on Information Technology and Quantitative management, ITQM 2022

作者： Ding, Xuesong Jiang, Yilin Yang, Banban Su, Jingchun Chen, Liming Li, Na School of Economics and Management University of Chinese Academy of Sciences Beijing100190 China Chinese Academy of Fiscal Sciences Ministry of Finance Beijing100142 China Division of Sports Science and Physical Education Tsinghua University Beijing100084 China Division of Sports Science and Physical Education Tangshan Polytechnic College Tangshan063299 China Research Center on Fictitious Economy and Data Science Chinese Academy of Sciences Beijing100190 China Key Laboratory of Big Data Mining and Knowledge Management Chinese Academy Sciences Beijing100190 China

The Olympic Games are striking a balance between economic and ecological benefits. Taking the 5 ice event venues of the 2018 PyeongChang Winter Olympic Games as the research object, the CO2 emissions under the 2018 PyeongChang Low-carbon scenario and 2022 Beijing Low-carbon scenario are compared. The empirical results show that, in terms of venue operation, the Beijing strategies can reduce CO2 emissions by 715.7 t compared with the PyeongChang strategies. In terms of the refrigerant escape, Beijing's adoption of CO2 refrigerant can reduce the emission of CO2 by 1202.2 t compared with the R404 refrigerant used in PyeongChang. This paper confirms the progress of the Green Winter Olympic Games, and may provide implications for the future host countries as well as the cities which in the green transformation. © 2022 The Author(s).

关键词： Refrigerants

来源：评论

学校读者我要写书评

暂无评论

Optimization Strategies for Client Drift in Federated Learning: A review 9th

Optimization Strategies for Client Drift in Federated Learni...

引用

9th International Conference on Information Technology and Quantitative management, ITQM 2022

作者： Shi, Yong Zhang, Yuanying Xiao, Yang Niu, Lingfeng School of Economics and Management University of Chinese Academy of Sciences Beijing100190 China Research Center on Fictitious Economy and Data Science Chinese Academy of Sciences Beijing100190 China Key Laboratory of Big Data Mining and Knowledge Management Chinese Academy of Sciences Beijing100190 China College of Information Sciences and Technology University of Nebraska at Omaha NE68182 United States School of Mathematical Sciences University of Chinese Academy of Sciences Beijing100190 China Faculty of Information Technology Beijing University of Technology Beijing100124 China

With the development of AI technology, there is an increasing awareness and concern about data privacy, and training data is becoming more and more fragmented. To make better use of this distributed data while ensuring privacy, federated learning is proposed, which can federate multiple distributed devices to train a machine learning model and does not exchange the raw training data. A key challenge in federated learning is that the distribution of data across clients is heterogeneous, which leads to client drift, that is, local model updates toward local optimal solutions, resulting in performance degradation of the global model. To address this problem, many federated learning algorithms have been proposed. In this paper, we provide a review of existing federated learning optimization strategies. In our opinion, the existing optimization strategies for client drift can be roughly classified into the following two categories: model aggregation and model training. We introduce these two categories separately. Finally, we present our perspectives on the future development. © 2022 The Author(s).

关键词： data privacy

来源：评论

学校读者我要写书评

暂无评论

Learning With Multiclass AUC: Theory and Algorithms

引用

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 2022年第11期44卷 7747-7763页

作者： Yang, Zhiyong Xu, Qianqian Bao, Shilong Cao, Xiaochun Huang, Qingming Univ Chinese Acad Sci Sch Comp Sci & Technol Beijing 101408 Peoples R China Chinese Acad Sci Inst Comp Technol Key Lab Intelligent Informat Proc Beijing 100190 Peoples R China Chinese Acad Sci Inst Informat Engn State Key Lab Informat Secur Beijing 100093 Peoples R China Univ Chinese Acad Sci Sch Cyber Secur Beijing 100049 Peoples R China Sun Yat Sen Univ Sch Cyber Sci & Technol Shenzhen 518100 Peoples R China Univ Chinese Acad Sci Sch Comp Sci & Technol Key Lab Big Data Min & Knowledge Management BDKM Beijing 101408 Peoples R China Peng Cheng Lab Shenzhen 518055 Peoples R China

The Area under the ROC curve (AUC) is a well-known ranking metric for problems such as imbalanced learning and recommender systems. The vast majority of existing AUC-optimization-based machine learning methods only focus on binary-class cases, while leaving the multiclass cases unconsidered. In this paper, we start an early trial to consider the problem of learning multiclass scoring functions via optimizing multiclass AUC metrics. Our foundation is based on the M metric, which is a well-known multiclass extension of AUC. We first pay a revisit to this metric, showing that it could eliminate the imbalance issue from the minority class pairs. Motivated by this, we propose an empirical surrogate risk minimization framework to approximately optimize the M metric. Theoretically, we show that: (i) optimizing most of the popular differentiable surrogate losses suffices to reach the Bayes optimal scoring function asymptotically;(ii) the training framework enjoys an imbalance-aware generalization error bound, which pays more attention to the bottleneck samples of minority classes compared with the traditional O(root 1/N) THORN result. Practically, to deal with the low scalability of the computational operations, we propose acceleration methods for three popular surrogate loss functions, including the exponential loss, squared loss, and hinge loss, to speed up loss and gradient evaluations. Finally, experimental results on 11 real-world datasets demonstrate the effectiveness of our proposed framework. The code is now available at https://***/joshuaas/ Learning-with-Multiclass-AUC-Theory-and-Algorithms.

关键词： AUC optimization machine learning

来源：评论

学校读者我要写书评

暂无评论

Research on the Rules of Scientists’ knowledge Sharing based on Interdisciplinary Measurement

引用

Procedia Computer Science 2022年 199卷 657-664页

作者： Yong Shi Yunong Wang School of Economics and Management University of Chinese Academy of Sciences Beijing 100190 China Research Center on Fictitious Economy and Data Science Chinese Academy of Sciences Beijing 100190 China Key Laboratory of Big Data Mining and Knowledge Management Chinese Academy of Sciences Beijing 100190 China College of Information Science and Technology University of Nebraska at Omaha NE 68182 USA

With the development of modern science, the adjustment of knowledge structure and the deepening of knowledge, the mode of single discipline research has been gradually broken, and interdisciplinary research has become a necessary way. All the information science and library science articles collected in the core collection of web of science from 1982 to 2020 are used for research and analysis. Through the research of interdisciplinary measurement index, this paper improves the existing interdisciplinary measurement index, uses Excel, Python and other tools to carry out data mining, cleaning, processing, analysis, and determines the interdisciplinary literature data defined in this paper. Finally, with vosviewer and other visual tools, this paper uses the citation network of bibliometrics, keywords co-occurrence and other ways to summarize interdisciplinary science’s law of knowledge sharing and provides suggestions and directions for future research.

关键词： Interdisciplinarity Disciplinary Similarity knowledge Sharing

来源：评论

学校读者我要写书评

暂无评论

Price change prediction of Ultra high frequency financial data based on temporal convolutional network

引用

Procedia Computer Science 2022年 199卷 1177-1183页

作者： Wei Dai Yuan An Wen Long School of Economics and Management University of Chinese Academy of Sciences Beijing 100190 P.R. China Research Center on Fictitious Economy and Data Science Chinese Academy of Sciences Beijing 100190 P.R. China Key Laboratory of Big Data Mining and Knowledge Management Chinese Academy of Sciences Beijing 100190 P.R. China

Through in-depth analysis of Ultra high frequency (UHF) stock price change data, more reasonable discrete dynamic distribution models are proposed in this paper. Firstly, we classify the price changes into several categories. Then, temporal convolutional network (TCN) is utilized to predict the conditional probability for each category. Furthermore, attention mechanism is added into the TCN architecture to model the time-varying distribution for stock price change data. Empirical research on constituent stocks of Chinese Shenzhen Stock Exchange 100 Index (SZSE 100) found that the TCN framework model and the TCN (attention) framework have a better overall performance than GARCH family models and the long short-term memory (LSTM) framework model for the description of the dynamic process of the UHF stock price change sequence. In addition, the scale of the dataset reached nearly 10 million, to the best of our knowledge, there has been no previous attempt to apply TCN to such a large-scale UHF transaction price dataset in Chinese stock market.

关键词： Temporal convolutional network Ultra high frequency Financial prediction

来源：评论

学校读者我要写书评

暂无评论

Multi-view Feature Augmentation with Adaptive Class Activation Mapping 30

Multi-view Feature Augmentation with Adaptive Class Activati...

引用

30th International Joint Conference on Artificial Intelligence (IJCAI)

作者： Gao, Xiang Tian, Yingjie Qi, Zhiquan Univ Chinese Acad Sci Sch Comp Sci & Technol Beijing Peoples R China Chinese Acad Sci Res Ctr Fictitious Econ & Data Sci Beijing Peoples R China Chinese Acad Sci Key Lab Big Data Min & Knowledge Management Beijing Peoples R China

ISBN: (纸本)9780999241196

We propose an end-to-end-trainable feature augmentation module built for image classification that extracts and exploits multi-view local features to boost model performance. Different from using global average pooling (GAP) to extract vectorized features from only the global view, we propose to sample and ensemble diverse multi-view local features to improve model robustness. To sample class-representative local features, we incorporate a simple auxiliary classifier head (comprising only one 1x1 convolutional layer) which efficiently and adaptively attends to class-discriminative local regions of feature maps via our proposed AdaCAM (Adaptive Class Activation Mapping). Extensive experiments demonstrate consistent and noticeable performance gains achieved by our multi-view feature augmentation module.

关键词： Mapping

来源：评论

学校读者我要写书评

暂无评论

Real estate rental market: a 10-year bibliometric-based review

引用

ECONOMIC RESEARCH-EKONOMSKA ISTRAZIVANJA 2021年第1期34卷 1752-1788页

作者： Liu, Yixi Wang, Peilong He, Zhou Dong, Jichang Univ Chinese Acad Sci Sch Econ & Management Beijing Peoples R China CPC Cent Comm Cadre Bur Publ Dept Expert Off Beijing Peoples R China Chinese Acad Sci Key Lab Big Data Min & Knowledge Management Beijing Peoples R China

The real estate rental market (RERM) is considered to have an important role in the entire real estate market. It refers to a property composed of land and its buildings, including the natural resources that can be rented or leased. Previous researches show that most developed countries have experienced the historical process of passively renting, actively buying, and actively renting. Moreover, academic interest in the impact of different sectors of the RERM has been reviewed increasingly over the past decade. However, previous studies provide limited insights into a comprehensive review of the RERM. Based on a 10-year database of 790 articles collected from the Web of Science, a comprehensive literature review is presented to discover the knowledge structure of RERM using CiteSpace software. First, this study recognizes the cluster of the articles, and discusses six major clusters in detail. Next, this study has identified four research trends that emerged during the past decade. To reveal the differences between the studies in the United States (US), China and the United Kingdom (UK), this study compares their publication scales and co-word networks. Finally, this study suggests six meaningful future research directions.

关键词： Real estate rental market bibliometric knowledge structure CiteSpace

来源：评论

学校读者我要写书评

暂无评论

Size-Invariance Matters: Rethinking Metrics and Losses for Imbalanced Multi-object Salient Object Detection 41

Size-Invariance Matters: Rethinking Metrics and Losses for I...

引用

41st International Conference on Machine Learning, ICML 2024

作者： Li, Feiran Xu, Qianqian Bao, Shilong Yang, Zhiyong Cong, Runmin Cao, Xiaochun Huang, Qingming Institute of Information Engineering Chinese Academy of Sciences Beijing China School of Cyber Security University of Chinese Academy of Sciences Beijing China Key Laboratory of Intelligent Information Processing Institute of Computing Technology Chinese Academy of Sciences Beijing China School of Computer Science and Technology University of Chinese Academy of Sciences Beijing China Institute of Information Science Beijing Jiaotong University Beijing China School of Control Science and Engineering Shandong University Jinan China Key Laboratory of Machine Intelligence and System Control Ministry of Education Jinan China School of Cyber Science and Tech. Sun Yat-sen University Shenzhen Campus China Key Laboratory of Big Data Mining and Knowledge Management Chinese Academy of Sciences Beijing China

This paper explores the size-invariance of evaluation metrics in Salient Object Detection (SOD), especially when multiple targets of diverse sizes co-exist in the same image. We observe that current metrics are size-sensitive, where larger objects are focused, and smaller ones tend to be ignored. We argue that the evaluation should be size-invariant because bias based on size is unjustified without additional semantic information. In pursuit of this, we propose a generic approach that evaluates each salient object separately and then combines the results, effectively alleviating the imbalance. We further develop an optimization framework tailored to this goal, achieving considerable improvements in detecting objects of different sizes. Theoretically, we provide evidence supporting the validity of our new metrics and present the generalization analysis of SOD. Extensive experiments demonstrate the effectiveness of our method. The code is available at https://***/Ferry-Li/SI-SOD. Copyright 2024 by the author(s)

关键词： Object detection

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：