检索结果-内蒙古大学图书馆

Large Scale Foundation Models for Intelligent Manufacturing Applications: A Survey

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Zhang, Haotian Dereck, Semujju Stuart Wang, Zhicheng Lv, Xianwei Xu, Kang Wu, Liang Jia, Ye Wu, Jing Long, Zhuo Liang, Wensheng Ma, X.G. Zhuang, Ruiyan The College of Information Science and Engineering Northeastern University Shenyang110819 China The Foshan Graduate School Northeastern University Foshan528311 China The School of mechanical engineering and automation South China University of Technology Shenyang110819 China The School of Software Engineering South China University of Technology Guangzhou510006 China The Departement of Computer and Information Science Qinghai University of Science and Technology Xining China The Enterprise AI Midea Group Foshan528311 China

'Although the applications of artificial intelligence especially deep learning had greatly improved various aspects of intelligent manufacturing, they still face challenges for wide employment due to the poor generalization ability, difficulties to establish high-quality training datasets, and unsatisfactory performance of deep learning methods. The emergence of large scale foundational models(LSFMs) had triggered a wave in the field of artificial intelligence, shifting deep learning models from single-task, single-modal, limited data patterns to a paradigm encompassing diverse tasks, multimodal, and pre-training on massive datasets. Although LSFMs had demonstrated powerful generalization capabilities, automatic high-quality training dataset generation and superior performance across various domains, applications of LSFMs on intelligent manufacturing were still in their nascent stage. A systematic overview of this topic was lacking, especially regarding which challenges of deep learning can be addressed by LSFMs and how these challenges can be systematically tackled. To fill this gap, this paper systematically expounded current statue of LSFMs and their advantages in the context of intelligent manufacturing. and compared comprehensively with the challenges faced by current deep learning models in various intelligent manufacturing applications. We also outlined the roadmaps for utilizing LSFMs to address these challenges. Finally, case studies of applications of LSFMs in real-world intelligent manufacturing scenarios were presented to illustrate how LSFMs could help industries, improve their efficiency. Copyright © 2023, The Authors. All rights reserved.

关键词： Deep learning

Understanding Self-Supervised Pretraining with Part-Aware Representation Learning

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Zhu, Jie Qi, Jiyang Ding, Mingyu Chen, Xiaokang Luo, Ping Wang, Xinggang Liu, Wenyu Wang, Leye Wang, Jingdong Key Lab of High Confidence Software Technologies [Peking University Ministry of Education China School of Computer Science Peking University China School of AI Peking University China UC Berkeley United States School of EIC Huazhong University of Science & Technology China University of Hong Kong Hong Kong Baidu China

In this paper, we are interested in understanding self-supervised pretraining through studying the capability that self-supervised methods learn part-aware representations. The study is mainly motivated by that random views, used in contrastive learning, and random masked (visible) patches, used in masked image modeling, are often about object parts. We explain that contrastive learning is a part-to-whole task: the projection layer hallucinates the whole object representation from the object part representation learned from the encoder, and that masked image modeling is a part-to-part task: the masked patches of the object are hallucinated from the visible patches. The explanation suggests that the self-supervised pretrained encoder leans toward understanding the object part. We empirically compare the off-the-shelf encoders pretrained with several representative methods on object-level recognition and part-level recognition. The results show that the fully-supervised model outperforms self-supervised models for object-level recognition, and most self-supervised contrastive learning and masked image modeling methods outperform the fully-supervised method for part-level recognition. It is observed that the combination of contrastive learning and masked image modeling further improves the performance. Copyright © 2023, The Authors. All rights reserved.

关键词： Image enhancement

UNIFIED POLICY OPTIMIZATION FOR CONTINUOUS-ACTION REINFORCEMENT LEARNING IN NON-STATIONARY TASKS AND GAMES

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Qin, Rong-Jun Luo, Fan-Ming Qian, Hong Yu, Yang National Key Laboratory for Novel Software Technology Nanjing University China Polixir.ai United States School of Computer Science and Technology East China Normal University China

This paper addresses policy learning in non-stationary environments and games with continuous actions. Rather than the classical reward maximization mechanism, inspired by the ideas of follow-the-regularized-leader (FTRL) and mirror descent (MD) update, we propose a no-regret style reinforcement learning algorithm PORL for continuous action tasks. We prove that PORL has a last-iterate convergence guarantee, which is important for adversarial and cooperative games. Empirical studies show that, in stationary environments such as MuJoCo locomotion controlling tasks, PORL performs equally well as, if not better than, the soft actor-critic (SAC) algorithm;in non-stationary environments including dynamical environments, adversarial training, and competitive games, PORL is superior to SAC in both a better final policy performance and a more stable training process. © 2022, CC BY.

关键词： Reinforcement learning

TJCCT: A Two-timescale Approach for UAV-assisted Mobile Edge Computing

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Sun, Zemin Sun, Geng Wu, Qingqing He, Long Liang, Shuang Pan, Hongyang Niyato, Dusit Yuen, Chau Leung, Victor C.M. The College of Computer Science and Technology Jilin University Changchun130012 China Key Laboratory of Symbolic Computation and Knowledge Engineering Ministry of Education Jilin University Changchun130012 China Department of Electronic Engineering Shanghai Jiao Tong University Shanghai China The School of Information Science and Technology Northeast Normal University Changchun130024 China The School of Computer Science and Engineering Nanyang Technological University Singapore639798 Singapore The School of Electrical and Electronics Engineering Nanyang Technological University Singapore639798 Singapore The College of Computer Science and Software Engineering Shenzhen University Shenzhen518060 China Department of Electrical and Computer Engineering University of British Columbia VancouverBCV6T 1Z4 Canada

Unmanned aerial vehicle (UAV)-assisted mobile edge computing (MEC) is emerging as a promising paradigm to provide aerial-terrestrial computing services in close proximity to mobile devices (MDs). However, meeting the demands of computation-intensive and delay-sensitive tasks for MDs poses several challenges, including the demand-supply contradiction between MDs and MEC servers, the demand-supply heterogeneity between MDs and MEC servers, the trajectory control requirements on energy efficiency and timeliness, and the different time-scale dynamics of the network. To address these issues, we first present a hierarchical architecture by incorporating terrestrial-aerial computing capabilities and leveraging UAV flexibility. Furthermore, we formulate a joint computing resource allocation, computation offloading, and trajectory control problem to maximize the system utility. Since the problem is a non-convex and NP-hard mixed integer nonlinear programming (MINLP), we propose a two-timescale joint computing resource allocation, computation offloading, and trajectory control (TJCCT) approach for solving the problem. In the short timescale, we propose a price-incentive model for on-demand computing resource allocation and a matching mechanism-based method for computation offloading. In the long timescale, we propose a convex optimization-based method for UAV trajectory control. Besides, we theoretically prove the stability, optimality, and polynomial complexity of TJCCT. Extended simulation results demonstrate that the proposed TJCCT outperforms the comparative algorithms in terms of the system utility, average processing rate, average completion delay, and average completion ratio. Copyright © 2024, The Authors. All rights reserved.

关键词： Computation offloading

PMFSNet: Polarized Multi-scale Feature Self-attention Network For Lightweight Medical Image Segmentation

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Zhong, Jiahui Tian, Wenhong Xie, Yuanlun Liu, Zhijia Ou, Jie Tian, Taoran Zhang, Lei School of Information and Software Engineering University of Electronic Science and Technology of China Chengdu610054 China State Key Laboratory of Oral Diseases National Clinical Research Center for Oral Diseases West China Hospital of Stomatology Sichuan University Chengdu610041 China School of Computer Science University of Lincoln LN6 7TS United Kingdom

Current state-of-the-art medical image segmentation methods prioritize accuracy but often at the expense of increased computational demands and larger model sizes. Applying these large-scale models to the relatively limited scale of medical image datasets tends to induce redundant computation, complicating the process without the necessary benefits. This approach not only adds complexity but also presents challenges for the integration and deployment of lightweight models on edge devices. For instance, recent transformer-based models have excelled in 2D and 3D medical image segmentation due to their extensive receptive fields and high parameter count. However, their effectiveness comes with a risk of overfitting when applied to small datasets and often neglects the vital inductive biases of Convolutional Neural Networks (CNNs), essential for local feature representation. In this work, we propose PMFSNet, a novel medical imaging segmentation model that effectively balances global and local feature processing while avoiding the computational redundancy typical in larger models. PMFSNet streamlines the UNet-based hierarchical structure and simplifies the self-attention mechanism’s computational complexity, making it suitable for lightweight applications. It incorporates a plug-and-play PMFS block, a multi-scale feature enhancement module based on attention mechanisms, to capture long-term dependencies. Extensive comprehensive results demonstrate that even with a model (≤ 1 million parameters), our method achieves superior performance in various segmentation tasks across different data scales. It achieves (IoU) metrics of 84.68%, 82.02%, and 78.82% on public datasets of teeth CT (CBCT), ovarian tumors ultrasound(MMOTU), and skin lesions dermoscopy images (ISIC 2018), respectively. The source code is available at https://***/yykzjh/PMFSNet. Copyright © 2024, The Authors. All rights reserved.

关键词： Medical imaging

Meta-Auto-Decoder: A Meta-Learning Based Reduced Order Model for Solving Parametric Partial Differential Equations

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Ye, Zhanhong Huang, Xiang Liu, Hongsheng Dong, Bin Beijing International Center for Mathematical Research Peking University Beijing China School of Computer Science and Technology University of Science and Technology of China Hefei China Central Software Institute Huawei Technologies Co. Ltd Hangzhou China Center for Machine Learning Research Peking University Beijing China

Many important problems in science and engineering require solving the so-called parametric partial differential equations (PDEs), i.e., PDEs with different physical parameters, boundary conditions, shapes of computational domains, etc. Typical reduced order modeling techniques accelarate solution of the parametric PDEs by projecting them onto a linear trial manifold constructed in the offline stage. These methods often need a predefined mesh as well as a series of precomputed solution snapshots, and may struggle to balance between efficiency and accuracy due to the limitation of the linear ansatz. Utilizing the nonlinear representation of neural networks, we propose Meta-Auto-Decoder (MAD) to construct a nonlinear trial manifold, whose best possible performance is measured theoretically by the decoder width. Based on the meta-learning concept, the trial manifold can be learned in a mesh-free and unsupervised way during the pre-training stage. Fast adaptation to new (possibly heterogeneous) PDE parameters is enabled by searching on this trial manifold, and optionally fine-tuning the trial manifold at the same time. Extensive numerical experiments show that the MAD method exhibits faster convergence speed without losing accuracy than other deep learning-based *** Codes 68T07 © 2023, CC BY.

关键词： Partial differential equations

Task Scheduling Based on Multi-level Hashing and HRRN in cloud computing

学校读者我要写书评

暂无评论

Task Scheduling Based on Multi-level Hashing and HRRN in clo...

IEEE International Symposium on Dependable, Autonomic and Secure Computing (DASC)

作者： Lei Zeng Jiawei Sun Jianhao Ma Qi Liu School of Computer and Software Nanjing University of Information Science and Technology (NUIST) Nanjing China

ISBN: (纸本)9781665421751

Task scheduling plays a crucial role in the cloud data center. An excellent job scheduling system can not only bring a high quality of user service, but also improve the resource utilization rate of the cloud service providers. This paper focuses on makespan and response time in task scheduling to improve quality of service. Under the framework of packing problem, this paper proposes a heterogeneous data center task scheduling algorithm based on multi-level hashing and High Response Ratio Next. Considering the different computing performance of physical hosts in heterogeneous data centers and the different resource demands of tasks, multi-level hashing is adopted to map the tasks to different queues waiting for execution, which can improve the speed of task allocation. When the tasks are waiting to be executed, the high response ratio next algorithm is executed in each queue to improve the service quality. Simulation is conducted by CloudSim, and the experimental results show that the proposed method is better than First-Come-First-Served (FCFS) and Shortest-Job-First (SJF) in the optimization of response time and makespan.

关键词： Heating systems Data centers Cloud computing Scheduling algorithms Computational modeling Scheduling Time factors

Manifold Learning-based Word Representation Refinement Incorporating Global and Local Information 28

学校读者我要写书评

暂无评论

Manifold Learning-based Word Representation Refinement Incor...

28th International Conference on Computational Linguistics, COLING 2020

作者： Zhao, Wenyu Zhou, Dong Li, Lin Chen, Jinjun School of Computer Science and Engineering Hunan University of Science and Technology Hunan Xiangtan411201 China School of Computer Science and Technology Wuhan University of Technology Hubei Wuhan430070 China Department of Computer Science and Software Engineering Swinburne University of Technology Hawthorn MelbourneVIC3122 Australia

ISBN: (纸本)9781952148279

Recent studies show that word embedding models often underestimate similarities between similar words and overestimate similarities between distant words. This results in word similarity results obtained from embedding models inconsistent with human judgment. Manifold learning-based methods are widely utilized to refine word representations by re-embedding word vectors from the original embedding space to a new refined semantic space. These methods mainly focus on preserving local geometry information through performing weighted locally linear combination between words and their neighbors twice. However, these reconstruction weights are easily influenced by different selections of neighboring words and the whole combination process is time-consuming. In this paper, we propose two novel word representation refinement methods leveraging isometry feature mapping and local tangent space respectively. Unlike previous methods, our first method corrects pre-trained word embeddings by preserving global geometry information of all words instead of local geometry information between words and their neighbors. Our second method refines word representations by aligning original and refined embedding spaces based on local tangent space instead of performing weighted locally linear combination twice. Experimental results obtained from standard semantic relatedness and semantic similarity tasks show that our methods outperform various state-of-the-art baselines for word representation refinement. © 2020 COLING 2020 - 28th International Conference on Computational Linguistics, Proceedings of the Conference. All rights reserved.

关键词： Embeddings

Federated Graph Learning for Cross-Domain Recommendation

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Yang, Ziqi Peng, Zhaopeng Wang, Zihui Qi, Jianzhong Chen, Chaochao Pan, Weike Wen, Chenglu Wang, Cheng Fan, Xiaoliang Fujian Key Laboratory of Sensing and Computing for Smart Cities Xiamen University China Key Laboratory of Multimedia Trusted Perception and Efficient Computing Ministry of Education of China Xiamen University China School of Computing and Information Systems The University of Melbourne Australia College of Computer Science and Technology Zhejiang University Hangzhou China College of Computer Science and Software Engineering Shenzhen University Shenzhen China

Cross-domain recommendation (CDR) offers a promising solution to the data sparsity problem by enabling knowledge transfer across source and target domains. However, many recent CDR models overlook crucial issues such as privacy as well as the risk of negative transfer (which negatively impact model performance), especially in multi-domain settings. To address these challenges, we propose FedGCDR, a novel federated graph learning framework that securely and effectively leverages positive knowledge from multiple source domains. First, we design a positive knowledge transfer module that ensures privacy during inter-domain knowledge transmission. This module employs differential privacy-based knowledge extraction combined with a feature mapping mechanism, transforming source domain embeddings from federated graph attention networks into reliable domain knowledge. Second, we design a knowledge activation module to filter out potential harmful or conflicting knowledge from source domains, addressing the issues of negative transfer. This module enhances target domain training by expanding the graph of the target domain to generate reliable domain attentions and fine-tunes the target model for improved negative knowledge filtering and more accurate predictions. We conduct extensive experiments on 16 popular domains of the Amazon dataset, demonstrating that FedGCDR significantly outperforms state-of-the-art methods. © 2024, CC BY.

关键词： Federated learning