检索结果-内蒙古大学图书馆

33rd ACM SIGSOFT International Conference on Software Testing and Analysis (ISSTA)

作者： Wang, Ruixin Lu, Minghai Yu, Cody Hao Lai, Yi-Hsiang Zhang, Tianyi Purdue Univ W Lafayette IN 47907 USA BosonAI Santa Clara CA USA Amazon Web Serv Seattle WA USA

ISBN: (纸本)9798400706127

As deep learning models become increasingly bigger and more complex, it is critical to improve model training and inference efficiency. Though a variety of highly optimized libraries and packages (known as DL kernels) have been developed, it is tedious and time-consuming to figure out which kernel to use, where to use, and how to use them correctly. To address this challenge, we propose an Automated deep learning Ot'Timization approach called ADOPTER. We design a Domain-Specific Language (DSL) to represent DL model architectures and leverage this DSL to specify model transformation rules required to integrate a DL kernel into a model. Given the source code of a DL model and the transformation rules for a set of kernels, ADOPTER first performs inter-procedural analysis to identify and express the model architecture in our DSL. Then, ADOPTER performs scope analysis and sub-sequence matching to identify locations in the model architecture where the transformation rules can be applied. Finally, ADOPTER proposes a synthesis-based code transformation method to apply the transformation rule. We cu rated a benchmark with 199 models from Hugging Face and a diverse set of DL kernels. We found that, compared to a state-of-the-art automated code transformation technique, ADopTot helps improve the precision and recall by 3% and 56%, respectively. An in-depth analysis of 9 models revealed that on average, ADopTiot improved the training speed by 22.7% while decreasing the CPU memory usage by 10.5%.

关键词： Program Transformation deep learning optimization

来源：评论

学校读者我要写书评

暂无评论

Adaptive Regularization via Residual Smoothing in deep learning optimization

引用

IEEE ACCESS 2019年 7卷 122889-122899页

作者： Cho, Junghee Kwon, Junseok Hong, Byung-Woo Seoul Natl Univ Dept Math Sci Seoul 08826 South Korea Chung Ang Univ Comp Sci Dept Seoul 06974 South Korea

We present an adaptive regularization algorithm that can be effectively applied to the optimization problem in deep learning framework. Our regularization algorithm aims to take into account the fitness of data to the current state of model in the determination of regularity to achieve better generalization. The degree of regularization at each element in the target space of the neural network architecture is determined based on the residual at each optimization iteration in an adaptive way. Our adaptive regularization algorithm is designed to apply a diffusion process driven by the heat equation with spatially varying diffusivity depending on the probability density function following a certain distribution of residual. Our data-driven regularity is imposed by adaptively smoothing a simplified objective function in which the explicit regularization term is omitted in an alternating manner between the evaluation of residual and the determination of the degree of its regularity. The effectiveness of our algorithm is empirically demonstrated by the numerical experiments in the application of image classification problems, indicating that our algorithm outperforms other commonly used optimization algorithms in terms of generalization using popular deep learning models and benchmark datasets.

关键词： Adaptive regularization deep learning optimization residual smoothing

来源：评论

学校读者我要写书评

暂无评论

deep learning with optimization Techniques for the Classification of Spoken English Digit 13th

Deep Learning with Optimization Techniques for the Classific...

引用

13th International Conference on Computational Collective Intelligence (ICCCI)

作者： Oruh, Jane Viriri, Serestina Univ KwaZulu Natal Sch Math Stat & Comp Sci Durban South Africa

ISBN: (纸本)9783030880811;9783030880804

Multiclass classification is a fundamental problem for many speech recognition systems. A typical example of multiclass classification in speech recognition is spoken digit classification. This type of classification is generally a challenging task since the signals last for a short period and often some digits are acoustically very similar to each other. To resolve this challenge, an intelligent based system is proposed using an enhanced deep feedforward network technique for the spoken digit classification. In the proposed method, Short Time Fourier Transform (STFT) features were first extracted from audio data and one hot encoding was performed on the audio data to generate the target audio class label. This was then used as input into the deep learning model. The model performance was evaluated using hyper-parameter optimization techniques such as Adam optimization algorithm and Stochastic Gradient Descent (SGD) optimization algorithm to reduce losses and to provide the most accurate results possible. The experimental results show that the system when used with Adam optimization algorithm outperformed the classical SGD optimization algorithm on the well-established public benchmark spoken English digit dataset, PCM and achieved an overall accuracy of 99.65%. The results show that the choice of a good hyper-parameter optimizer like Adam optimization algorithm can achieve optimal accuracy.

关键词： Automatic speech recognition deep learning optimization Short Time Fourier Transform (STFT) One hot encoding Speech classification

来源：评论

学校读者我要写书评

暂无评论

Large Language Model Enhanced Particle Swarm optimization for Hyperparameter Tuning for deep learning Models

IEEE OPEN JOURNAL OF THE COMPUTER SOCIETY

引用

IEEE OPEN JOURNAL OF THE COMPUTER SOCIETY 2025年第1期6卷 574-585页

作者： Hameed, Saad Qolomany, Basheer Belhaouari, Samir Brahim Abdallah, Mohamed Qadir, Junaid Al-Fuqaha, Ala Hamad Bin Khalifa Univ Coll Sci & Engn Div Informat & Comp Technol Doha 5825 Qatar Howard Univ Coll Med Dept Med Washington DC USA Qatar Univ Dept Comp Sci & Engn Doha Qatar

Determining the ideal architecture for deep learning models, such as the number of layers and neurons, is a difficult and resource-intensive process that frequently relies on human tuning or computationally costly optimization approaches. While Particle Swarm optimization (PSO) and Large Language Models (LLMs) have been individually applied in optimization and deep learning, their combined use for enhancing convergence in numerical optimization tasks remains underexplored. Our work addresses this gap by integrating LLMs into PSO to reduce model evaluations and improve convergence for deep learning hyperparameter tuning. The proposed LLM-enhanced PSO method addresses the difficulties of efficiency and convergence by using LLMs (particularly ChatGPT-3.5 and Llama3) to improve PSO performance, allowing for faster achievement of target objectives. Our method speeds up search space exploration by substituting underperforming particle placements with best suggestions offered by LLMs. Comprehensive experiments across three scenarios-(1) optimizing the Rastrigin function, (2) using Long Short-Term Memory (LSTM) networks for time series regression, and (3) using Convolutional Neural Networks (CNNs) for material classification-show that the method significantly improves convergence rates and lowers computational costs. Depending on the application, computational complexity is lowered by 20% to 60% compared to traditional PSO methods. Llama3 achieved a 20% to 40% reduction in model calls for regression tasks, whereas ChatGPT-3.5 reduced model calls by 60% for both regression and classification tasks, all while preserving accuracy and error rates. This groundbreaking methodology offers a very efficient and effective solution for optimizing deep learning models, leading to substantial computational performance improvements across a wide range of applications.

关键词： Computational modeling Tuning Neurons deep learning Convergence Large language models Computational efficiency Accuracy Predictive models Particle swarm optimization deep learning optimization PSO LLM machine learning hyper-parameter optimization

来源：评论

学校读者我要写书评

暂无评论

Semantic-Based optimization of deep learning for Efficient Real-Time Medical Image Segmentation

引用

INTERNATIONAL JOURNAL ON SEMANTIC WEB AND INFORMATION SYSTEMS 2024年第1期20卷 1页

作者： Wei, Zhenkun Liu, Jia Yao, Yu Chinese Acad Sci Chengdu Inst Comp Applicat Chengdu Peoples R China Univ Chinese Acad Sci Beijing Peoples R China China Mobile Ind Res Inst Beijing Peoples R China

In response to the critical need for advanced solutions in medical imaging segmentation, particularly for real-time applications in diagnostics and treatment planning, this study introduces SM-UNet. This novel deep learning architecture efficiently addresses the challenge of real-time, accurate medical image segmentation by integrating convolutional neural network (CNN) with multilayer perceptron (MLP). The architecture uniquely combines an initial convolutional encoder for detailed feature extraction, MLP module for capturing long-range dependencies, and a decoder that merges global features with high-resolution CNN map. Further optimization is achieved through a tokenization approach, significantly reducing computational demands. Its superior performance is confirmed by evaluations on standard datasets, showing interaction times drastically lower than comparable networks-between 1/6 to 1/10, and 1/25 compared to SOTA models. These advancements underscore SM-UNet's potential as a groundbreaking tool for facilitating real-time, precise medical diagnostics and treatment strategies.

关键词： Computational Efficiency Convolutional Neural Networks deep learning optimization Healthcare System Magnetic Resonance Imaging Multilayer Perceptron

来源：评论

学校读者我要写书评

暂无评论

Improving stochastic models by smart denoising and latent representation optimization

引用

INFORMATION SCIENCES 2025年 692卷

作者： Jelencic, Jakob Massri, M. Besher Todorovski, Ljupco Grobelnik, Marko Mladenic, Dunja Jozef Stefan Inst Dept Artificial Intelligence Jamova Cesta 39 Ljubljana 1000 Slovenia Jozef Stefan Int Postgrad Sch Jamova Cesta 39 Ljubljana 1000 Slovenia Univ Ljubljana Fac Math & Phys Jadranska Ulica 19 Ljubljana 1000 Slovenia Qlector Doo Trzaska C 68a Ljubljana 1000 Slovenia

This paper introduces an innovative deep learning-based optimization method specifically designed for data derived from stochastic processes. Addressing the prevalent issue of rapid overfitting in real-world scenarios with limited historical data, our approach focuses on denoising optimization. The method effectively balances the simultaneous optimization of latent data representation and target variables, leading to enhanced model performance. We rigorously test our approach using five diverse real-world datasets. Our study is structured into three parts: an ablation study to validate the individual components of our method, a statistical analysis using the Wilcoxon rank-sum test to confirm the superiority of our method against five research hypotheses, and a detailed exploration of parameter visualization and fine-tuning. The comprehensive evaluation demonstrates that our method not only outperforms existing techniques but also significantly contributes to the advancement of deep learning models for stochastic processes. The findings underscore the potential of our method as a robust solution to the challenges in modeling stochastic processes with deep learning, offering new avenues for efficient and accurate predictions.

关键词： deep learning optimization Stochastic processes

来源：评论

学校读者我要写书评

暂无评论

LoCo: Low-Bit Communication Adaptor for Large-Scale Model Training

引用

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 2025年第6期47卷 4285-4298页

作者： Xie, Xingyu Lin, Zhijie Toh, Kim-Chuan Zhou, Pan Natl Univ Singapore Dept Math Singapore 119077 Singapore Sea AI Lab Singapore 138522 Singapore Singapore Management Univ Sch Comp & Informat Syst Singapore 188065 Singapore

To efficiently train large-scale models, low-bit gradient communication compresses full-precision gradients on local GPU nodes into low-precision ones for higher gradient synchronization efficiency among GPU nodes. However, it often degrades training quality due to compression information loss. To address this, we propose the Low-bit Communication Adaptor (LoCo), which compensates gradients on local GPU nodes before compression, ensuring efficient synchronization without compromising training quality. Specifically, LoCo designs a moving average of historical compensation errors to stably estimate concurrent compression error and then adopts it to compensate for the concurrent gradient compression, yielding a less lossless compression. This mechanism allows it to be compatible with general optimizers like Adam and sharding strategies like FSDP. Theoretical analysis shows that integrating LoCo into full-precision optimizers like Adam and SGD does not impair their convergence speed on non-convex problems. Experimental results show that across large-scale model training frameworks like Megatron-LM and PyTorch's FSDP, LoCo significantly improves communication efficiency, e.g., improving Adam's training speed by 14% to 40% without performance degradation on large language models like LLAMAs and MoEs.

关键词： Training Scalability Computational modeling Sharding Graphics processing units Servers Parallel processing Adaptation models Topology Synchronization Efficient large-scale training large-scale optimization deep learning optimization fully sharded data parallel 4-bit training

来源：评论

学校读者我要写书评

暂无评论

Enhancing Hajj and Umrah Services Through Predictive Social Media Classification

引用

IEEE ACCESS 2025年 13卷 67220-67238页

作者： Chelloug, Samia Allaoua Muthanna, Mohammed Saleh Ali Jamil, Faisal Al-Gaashani, Mehdhar S. A. M. Alhelaly, Soha Aziz, Ahmed Muthanna, Ammar Princess Nourah bint Abdulrahman Univ Coll Comp & Informat Sci Dept Informat Technol POB 84428 Riyadh 11671 Saudi Arabia Tashkent State Univ Econ Dept Int Business Management Tashkent 100066 Uzbekistan Ulster Univ Sch Comp Engn & Intelligent Syst Londonderry BT48 7JL North Ireland Univ Elect Sci & Technol China Sch Resources & Environm Chengdu 610056 Sichuan Peoples R China Saudi Elect Univ Dept Comp Sci Riyadh 13323 Saudi Arabia Benha Univ Fac Comp & Artificial Intelligence Dept Comp Sci Banha 13511 Egypt Cent Asian Univ Engn Sch Tashkent 111221 Uzbekistan Peoples Friendship Univ Russia RUDN Univ Dept Appl Probabil & Informat Moscow 117198 Russia

Each year, millions of individuals embark on the sacred journeys of Hajj and Umrah to Saudi Arabia. Given the diverse needs of these pilgrims and the continuous efforts to enhance their experience, we propose an advanced social media classification system based on predictive deep learning. The primary objective of this system is to efficiently classify and analyze social media content related to Hajj and Umrah services. To improve the effectiveness of this classification model, we introduce a predictive optimization strategy that employs a deep neural network as the learning module and utilizes particle swarm optimization to refine the weighting parameters. Leveraging real-time data from various microblogging platforms Twitter, blogging websites, Facebook, and Instagram, our model classifies individual posts using natural language processing techniques. The classification is based on relevant attributes such as service-level scores. If the dataset contains non-English text, it is first translated into English. Tokenization and preprocessing are then applied to categorize posts into five key areas: religious rites, management, safety, well-being, and services. The labeled posts are subsequently used to train a deep learning model. By incorporating a service-level score algorithm based on the TextBlob NLP library, each post is accurately classified and utilized as a feature in a supervised machine-learning classification system. The model's performance is evaluated using standard metrics, including F-measure, Precision, and Recall. The ultimate objective is to achieve high-accuracy classification, enabling precise evaluation and improved analysis of social media content related to the pilgrimage experience.

关键词： Social networking (online) deep learning Real-time systems Natural language processing Blogs Accuracy Web sites Particle swarm optimization optimization Multimedia communication Hajj and Umrah management deep learning optimization social media classification predictive modelling sentiment analysis information sharing

来源：评论

学校读者我要写书评

暂无评论

Reducing Memory Footprint in deep Network Training by Gradient Space Reutilization 7th

Reducing Memory Footprint in Deep Network Training by Gradie...

引用

7th Chinese Conference on Pattern Recognition and Computer Vision

作者： Dong, Yiming Lin, Zhouchen Peking Univ Sch Intelligence Sci & Technol State Key Lab Gen Artificial Intelligence Beijing Peoples R China Pazhou Lab Huangpu Guangzhou Guangdong Peoples R China

ISBN: (纸本)9789819784899;9789819784905

As deep learning continues to spearhead transformative breakthroughs across various domains, the computational and memory demands for training state-of-the-art models have surged exponentially. This escalation not only challenges the scalability of deep learning systems but also significantly increases the financial cost associated with training. Memory-intensive operations, particularly during the optimization phase of training large models, can drastically inflate budgets, making cutting-edge research and applications less accessible. In response to this challenge, we introduce a novel technique termed gradient space reutilization, aiming at reducing memory usage in deep network training by repurposing the memory allocated for gradients once it is no longer needed in the later computing process. This approach is implemented across modified versions of popular optimizers, with their names AdamW-R, Adan-R, and Lion-R, respectively, demonstrating appreciable memory savings without compromising the performance as they are equivalent to the original algorithms. Extensive experiments demonstrate that our simple engineering trick can achieve up to 25.60% memory savings at the best, providing a practical solution for efficient resource management in deep learning training environments.

关键词： deep learning optimization Efficient deep learning Memory footprint

来源：评论

学校读者我要写书评

暂无评论

A Survey on Edge Computing Systems and Tools

引用

PROCEEDINGS OF THE IEEE 2019年第8期107卷 1537-1562页

作者： Liu, Fang Tang, Guoming Li, Youhuizi Cai, Zhiping Zhang, Xingzhou Zhou, Tongqing Sun Yat Sen Univ Sch Data & Comp Sci Guangzhou 510006 Guangdong Peoples R China Natl Univ Def Technol Key Lab Sci & Technol Informat Syst Engn Changsha 410073 Hunan Peoples R China Hangzhou Dianzi Univ Sch Comp Sci & Technol Hangzhou 310002 Zhejiang Peoples R China Natl Univ Def Technol Coll Comp Changsha 410073 Hunan Peoples R China Chinese Acad Sci Inst Comp Technol State Key Lab Comp Architecture Beijing 100190 Peoples R China

Driven by the visions of Internet of Things and 5G communications, the edge computing systems integrate computing, storage, and network resources at the edge of the network to provide computing infrastructure, enabling developers to quickly develop and deploy edge applications. At present, the edge computing systems have received widespread attention in both industry and academia. To explore new research opportunities and assist users in selecting suitable edge computing systems for specific applications, this survey paper provides a comprehensive overview of the existing edge computing systems and introduces representative projects. A comparison of open-source tools is presented according to their applicability. Finally, we highlight energy efficiency and deep learning optimization of edge computing systems. Open issues for analyzing and designing an edge computing system are also studied in this paper.

关键词： deep learning optimization edge computing systems energy efficiency open-source tools survey

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：