检索结果-内蒙古大学图书馆

7th International Conference on Computer and Communication Technology (ICCCT)

作者： Maurya, Ashish Kumar Tripathi, Anil Kumar Indian Inst Technol BHU Dept Comp Sci & Engn Varanasi Uttar Pradesh India

ISBN: (纸本)9781450353243

Efficient scheduling algorithms play an essential part in heterogeneous computing systems to achieve high performance. The problem of producing an optimal schedule for the precedence-constrained tasks is recognized to be an NP-complete problem. To work out this problem, the researchers have already been proposed various scheduling algorithms in the literature. This paper discusses four well-known list scheduling algorithms such as HEFT, Lookahead, CEFT and peft for heterogeneous computing systems and performs experiments for randomly created application graphs and the application graphs generated from real-world problem for instance molecular dynamic code. The performance of algorithms are evaluated and compared on different scheduling parameters such as scheduling length ratio, efficiency, etc.

关键词： DAG Scheduling HEFT Lookahead CEFT peft Heterogeneous Computing Systems

来源：评论

学校读者我要写书评

暂无评论

IISAN: Efficiently Adapting Multimodal Representation for Sequential Recommendation with Decoupled peft 47

IISAN: Efficiently Adapting Multimodal Representation for Se...

引用

47th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR)

作者： Fu, Junchen Ge, Xuri Xin, Xin Karatzoglou, Alexandros Arapakis, Ioannis Wang, Jie Jose, Joemon M. Univ Glasgow Glasgow Lanark Scotland Shandong Univ Qingdao Peoples R China Amazon Barcelona Spain Telefon Res Barcelona Spain

ISBN: (纸本)9798400704314

Multimodal foundation models are transformative in sequential recommender systems, leveraging powerful representation learning capabilities. While Parameter-efficient Fine-tuning (peft) is commonly used to adapt foundation models for recommendation tasks, most research prioritizes parameter efficiency, often overlooking critical factors like GPU memory efficiency and training speed. Addressing this gap, our paper introduces IISAN (Intra- and Inter-modal Side Adapted Network for Multimodal Representation)1, a simple plug-and-play architecture using a Decoupled peft structure and exploiting both intra- and inter-modal adaptation. IISAN matches the performance of full fine-tuning (FFT) and state-of-the-art peft. More importantly, it significantly reduces GPU memory usage - from 47GB to just 3GB for multimodal sequential recommendation tasks. Additionally, it accelerates training time per epoch from 443s to 22s compared to FFT. This is also a notable improvement over the Adapter and LoRA, which require 37-39 GB GPU memory and 350-380 seconds per epoch for training. Furthermore, we propose a new composite efficiency metric, TPME (Training-time, Parameter, and GPU Memory Efficiency) to alleviate the prevalent misconception that "parameter efficiency represents overall efficiency". TPME provides more comprehensive insights into practical efficiency comparisons between different methods. Besides, we give an accessible efficiency analysis of all peft and FFT approaches, which demonstrate the superiority of IISAN.

关键词： Recommender Systems Parameter-efficient Fine-tuning peft Decoupled peft Fine-tuning Sequential Recommendation IISAN TPME

来源：评论

学校读者我要写书评

暂无评论

peft-based Trade-off Schedule Plan for Execution IoT Applications in Cloud Environment

引用

IETE JOURNAL OF RESEARCH 2023年第9期69卷 6152-6161页

作者： Verma, Amandeep Panjab Univ Univ Inst Engn & Technol Chandigarh India

Internet of Things (IoT) consists of physical things or objects that are connected through a network of sensors, software, and electronics. Mainly, IoT devices are of restricted processing and storage capacities. On the other hand, infinite storage and processing capacities are available over the cloud. Thus by merging these two methods, the mutual advantages have been discussed in the past years. Due to heterogeneous resource requirements of IoT application's task, it is not so easy to schedule these over the cloud. Existing heuristic-based scheduling strategies had mainly developed considering either minimization of makespan or minimization of cost. But, for the present scenario, an efficient schedule plan is required to schedule IoT applications over cloud that minimizes the execution time along with execution cost while preserving the task dependencies constraints as well as user-defined constraints like budget and deadline. In this paper, Cost-time computational intelligent heuristics based upon peft i.e. CTpeft has been proposed to give a trade-off schedule plan between computational cost and time beneath the user-specified deadline and budget constraints. To validate the effectiveness of the proposed heuristic, extensive simulation experiments have been conducted while considering different synthetic IoT application tasks. CTpeft has been compared with our previously proposed algorithm, i.e. BDHEFT as well as with other competing algorithms like BHEFT, and HCPpeft. Simulation results shows that CTpeft heuristic can generate better cost-makespan trade-off plan as compared to BDHEFT, BHEFT, and HCPpeft.

关键词： BDHEFT Cloud HEFT IoT peft Workflow

来源：评论

学校读者我要写书评

暂无评论

Frozen Weights as Prior for Parameter-Efficient Fine-Tuning

引用

IEEE ACCESS 2025年 13卷 24411-24425页

作者： Ma, Xiaolong Liu, Peishun Gao, Haojie Yan, Zikang Ma, Ningning Liu, Wenqiang Wang, Xuefang Tang, Ruichun Ocean Univ China Sch Comp Sci & Engineer Qingdao 266100 Peoples R China Ocean Univ China Sch Math Sci Qingdao 266100 Peoples R China

In the fields of natural language processing and computer vision, the emergence of large pre-trained models has led to the adoption of fine-tuning them for downstream tasks as an important paradigm. However, the full fine-tuning approach often comes with a hefty cost, which is not feasible for many researchers. Therefore, in recent years, numerous fine-tuning methods have been proposed to efficiently learn incremental updates of pre-trained weights in a more parameter-efficient way (e.g., employing low-rank increments or introducing adapters to modify the network architecture). However, most of these methods involve adding a set of incrementally learned parameters from scratch. From the perspective of full fine-tuning, these approaches often fail to fully exploit the connection between incremental changes during fine-tuning and the frozen weights of the pre-trained model. In order to delve into how to more effectively harness the weights of pre-trained models during the fine-tuning process to fully acquire new knowledge, we propose a novel parameter-efficient approach, which reuses the Frozen Weights as a prior (FoWA). We adapt the incremental matrix to the unitary matrix obtained from the singular value decomposition of the frozen weights, and further fine-tune the model by incorporating prior information. Through the frozen weight prior, FoWA can automatically select the appropriate rank and decouple the number of trainable parameters from the rank. We have conducted extensive experiments on various tasks, including natural language processing, question answering, natural language generation, and visual classification, demonstrating the effectiveness of FoWA.

关键词： Adaptation models Tuning Matrix decomposition Vectors Training Computational modeling Singular value decomposition Transfer learning Costs Computer architecture Fine-tuning peft large pre-trained models transfer learning adaptation

来源：评论

学校读者我要写书评

暂无评论

A Survey on Stability of Learning with Limited Labelled Data and its Sensitivity to the Effects of Randomness

引用

ACM COMPUTING SURVEYS 2025年第1期57卷 1-40页

作者： Pecher, Branislav Srba, Ivan Bielikova, Maria Brno Univ Technol Fac Informat Technol Brno Czech Republic Kempelen Inst Intelligent Technol Bratislava Slovakia Slovak AI Bratislava Slovakia

Learning with limited labelled data, such as prompting, in-context learning, fine-tuning, meta-learning, or few-shot learning, aims to effectively train a model using only a small amount of labelled samples. However, these approaches have been observed to be excessively sensitive to the effects of uncontrolled randomness caused by non-determinism in the training process. The randomness negatively affects the stability of the models, leading to large variances in results across training runs. When such sensitivity is disregarded, it can unintentionally, but unfortunately also intentionally, create an imaginary perception of research progress. Recently, this area started to attract research attention and the number of relevant studies is continuously growing. In this survey, we provide a comprehensive overview of 415 papers addressing the effects of randomness on the stability of learning with limited labelled data. We distinguish between four main tasks addressed in the papers (investigate/evaluate, determine, mitigate, benchmark/compare/report randomness effects), providing findings for each one. Furthermore, we identify and discuss seven challenges and open problems together with possible directions to facilitate further research. The ultimate goal of this survey is to emphasise the importance of this growing research area, which so far has not received an appropriate level of attention, and reveal impactful directions for future research.

关键词： Randomness stability sensitivity meta-learning large language mod- els fine-tuning prompting in-context learning instruction-tuning prompt-based learning peft literature survey

来源：评论

学校读者我要写书评

暂无评论

Finetuning Large Language Models for Vulnerability Detection

引用

IEEE ACCESS 2025年 13卷 38889-38900页

作者： Shestov, Aleksei Levichev, Rodion Mussabayev, Ravil Maslov, Evgeny Zadorozhny, Pavel Cheshkov, Anton Mussabayev, Rustam Toleu, Alymzhan Tolegen, Gulmira Krassovitskiy, Alexander Sber AI Lab Moscow 117312 Russia SaluteDevices Moscow 117312 Russia Satbayev Univ AI Res Lab Alma Ata 050000 Kazakhstan Huawei Russian Res Inst Software Dev Tools Cloud Technol Lab Moscow 121099 Russia

This paper presents the results of finetuning large language models (LLMs) for the task of detecting vulnerabilities in Java source code. We leverage WizardCoder, a recent improvement of the state-of-the-art LLM StarCoder, and adapt it for vulnerability detection through further finetuning. To accelerate training, we modify WizardCoder's training procedure, also we investigate optimal training regimes. For the imbalanced dataset with many more negative examples than positive, we also explore different techniques to improve classification performance. The finetuned WizardCoder model achieves improvement in ROC AUC and F1 measures on balanced and imbalanced vulnerability datasets over CodeBERT-like model, demonstrating the effectiveness of adapting pretrained LLMs for vulnerability detection in source code. The key contributions are finetuning the state-of-the-art code LLM, WizardCoder, increasing its training speed without the performance harm, optimizing the training procedure and regimes, handling class imbalance, and improving performance on difficult vulnerability detection datasets. This demonstrates the potential for transfer learning by finetuning large pretrained language models for specialized source code analysis tasks.

关键词： Training Codes Standards Large language models Predictive models Limiting Java Entropy Adaptation models Reviews vulnerability detection cybersecurity finetuning StarCoder WizardCoder peft LoRA

来源：评论

学校读者我要写书评

暂无评论

Automatic Parkinson's disease detection from speech: Layer selection vs adaptation of foundation models

Automatic Parkinson's disease detection from speech: Layer s...

引用

2025 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2025

作者： Purohit, Tilak Ruvolo, Barbara Orozco-Arroyave, Juan Rafael Magimai.-Doss, Mathew Idiap Research Institute Martigny Switzerland EPFL École polytechnique fédérale de Lausanne Switzerland GITA Lab Universidad de Antioquia Colombia

ISBN: (纸本)9798350368741

In this work, we investigate Speech Foundation Models (SFMs) for Parkinson's Disease (PD) detection. We explore two main approaches: (1) using SFMs as frozen feature extractors and, (2) fine-tuning/adapting SFMs for PD detection. We propose a cross-validation-based layer selection methodology to identify the layer effective for PD detection. Additionally, we compare the performance of the layer selection scheme with full fine-tuning and, parameter-efficient fine-tuning (peft) using Low-Rank Adaptation (LoRA). Our results show that layer selection and LoRA-based fine-tuning can perform on par with full fine-tuning, providing a more parameter-efficient alternative. The highest accuracy was achieved by fine-tuning Whisper using LoRA. © 2025 IEEE.

关键词： Fine-tuning Foundation Models LoRA Parkinson's Disease PC-GITA peft Speech for health

来源：评论

学校读者我要写书评

暂无评论

ADEM-VL: Adaptive and Embedded Fusion for Efficient Vision-Language Tuning

引用

INTERNATIONAL JOURNAL OF COMPUTER VISION 2025年 1-17页

作者： Hao, Zhiwei Guo, Jianyuan Shen, Li Luo, Yong Hu, Han Wen, Yonggang Beijing Inst Technol Sch Informat & Elect Beijing Peoples R China City Univ Hong Kong Dept Comp Sci Hong Kong Peoples R China Sun Yat sen Univ Sch Cyber Sci & Technol Shenzhen Campus Shenzhen Peoples R China Wuhan Univ Sch Comp Sci Wuhan Peoples R China Nanyang Technol Univ Sch Comp Sci & Engn Singapore Singapore

Recent advancements in multimodal fusion have witnessed the remarkable success of vision-language (VL) models, which excel in various multimodal applications such as image captioning and visual question answering. However, building VL models requires substantial hardware resources, where efficiency is restricted by two key factors: the extended input sequence of the language model with vision features demands more computational operations, and a large number of additional learnable parameters increase memory complexity. These challenges significantly restrict the broader applicability of such models. To bridge this gap, we propose ADEM-VL, an efficient vision-language method that tunes VL models based on pretrained large language models (LLMs) by adopting a parameter-free cross-attention mechanism for similarity measurements in multimodal fusion. This approach only requires embedding vision features into the language space, significantly reducing the number of trainable parameters and accelerating both training and inference speeds. To enhance representation learning in fusion module, we introduce an efficient multiscale feature generation scheme that requires only a single forward pass through the vision encoder. Moreover, we propose an adaptive fusion scheme that dynamically discards less relevant visual information for each text token based on its attention score. This ensures that the fusion process prioritizes the most pertinent visual features. With experiments on various tasks including visual question answering, image captioning, and instruction-following, we demonstrate that our framework outperforms existing approaches. Specifically, our method surpasses existing methods by an average accuracy of 0.77% on ScienceQA dataset, with reduced training and inference latency, demonstrating the superiority of our framework.

关键词： Multimodal fusion Parameter-free Cross-attention peft LLMs

来源：评论

学校读者我要写书评

暂无评论

Optimizing translation for low-resource languages: Efficient fine-tuning with custom prompt engineering in large language models

MACHINE LEARNING WITH APPLICATIONS

引用

MACHINE LEARNING WITH APPLICATIONS 2025年 20卷

作者： Khoboko, Pitso Walter Marivate, Vukosi Sefara, Joseph Univ Pretoria UP 140 Lunnon Rd Pretoria South Africa Council Sci & Ind Res CSIR Meiring Naude Rd Pretoria South Africa

Training large language models (LLMs) can be prohibitively expensive. However, the emergence of new Parameter-Efficient Fine-Tuning (peft) strategies provides a cost-effective approach to unlocking the potential of LLMs across a variety of natural language processing (NLP) tasks. In this study, we selected the Mistral 7B language model as our primary LLM due to its superior performance, which surpasses that of LLAMA 2 13B across multiple benchmarks. By leveraging peft methods, we aimed to significantly reduce the cost of fine-tuning while maintaining high levels of performance. Despite their advancements, LLMs often struggle with translation tasks for low-resource languages, particularly morphologically rich African languages. To address this, we employed customized prompt engineering techniques to enhance LLM translation capabilities for these languages. Our experimentation focused on fine-tuning the Mistral 7B model to identify the best-performing ensemble using a custom prompt strategy. The results obtained from the fine-tuned Mistral 7B model were compared against several models: Serengeti, Gemma, Google Translate, and No Language Left Behind (NLLB). Specifically, Serengeti and Gemma were fine-tuned using the same custom prompt strategy as the Mistral model, while Google Translate and NLLB Gemma, which are pre-trained to handle English-to-Zulu and English-to-Xhosa translations, were evaluated directly on the test data set. This comparative analysis allowed us to assess the efficacy of the fine-tuned Mistral 7B model against both custom-tuned and pre-trained translation models. LLMs have traditionally struggled to produce high-quality translations, especially for low-resource languages. Our experiments revealed that the key to improving translation performance lies in using the correct prompt during fine-tuning. We used the Mistral 7B model to develop a custom prompt that significantly enhanced translation quality for English-to-Zulu and English-to-Xhosa language

关键词： Large language model (LLM) peft Mistral 7B Prompt Engineering In-Context-Learning (ICL) En-Zul (English-to-Zulu) En-Xh (English-to-Xhosa) BLUE-score F1score G -Eva (mean cosine-similarity score)

来源：评论

学校读者我要写书评

暂无评论

Fine-Tuning Large Language Models for Specialized Use Cases

Mayo Clinic Proceedings: Digital Health

引用

Mayo Clinic Proceedings: Digital Health 2025年第1期3卷 100184-100184页

作者： Anisuzzaman, D.M. Malins, Jeffrey G. Friedman, Paul A. Attia, Zachi I. Department of Cardiovascular Medicine Mayo Clinic Rochester MN

Large language models (LLMs) are a type of artificial intelligence, which operate by predicting and assembling sequences of words that are statistically likely to follow from a given text input. With this basic ability, LLMs are able to answer complex questions and follow extremely complex instructions. Products created using LLMs such as ChatGPT by OpenAI and Claude by Anthropic have created a huge amount of traction and user engagements and revolutionized the way we interact with technology, bringing a new dimension to human-computer interaction. Fine-tuning is a process in which a pretrained model, such as an LLM, is further trained on a custom data set to adapt it for specialized tasks or domains. In this review, we outline some of the major methodologic approaches and techniques that can be used to fine-tune LLMs for specialized use cases and enumerate the general steps required for carrying out LLM fine-tuning. We then illustrate a few of these methodologic approaches by describing several specific use cases of fine-tuning LLMs across medical subspecialties. Finally, we close with a consideration of some of the benefits and limitations associated with fine-tuning LLMs for specialized use cases, with an emphasis on specific concerns in the field of medicine. © 2025

关键词： AI artificial intelligence CoT chain-of-thought GPU graphical processing unit LLM large language model peft parameter-efficient fine-tuning PPO proximal policy optimization QLoRA quantized low-rank adaptation RAG retrieval augmented generation RLHF reinforcement learning from human feedback

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：