检索结果-内蒙古大学图书馆

2023 ACM Turing Award Celebration Conference, china 2023

作者： Yao, Ting Wan, Jiguang Huang, Ping Zhang, Yiwen Xie, Changsheng He, Xubin Huawei Cloud Computing Technologies Co. Ltd. China School of Computer Science and Technology Huazhong University of Science and Technology China College of Science and Technology Temple University United States

No abstract available.

ISBN: (纸本)9798400702334

No abstract available.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Distilling Instruction-following Abilities of Large Language Models with Task-aware Curriculum Planning

arXiv

引用

arXiv 2024年

作者： Yue, Yuanhao Wang, Chengyu Huang, Jun Wang, Peng School of Computer Science Fudan University Shanghai China Alibaba Cloud Computing Hangzhou China

Instruction tuning aims to align large language models (LLMs) with open-domain instructions and human-preferred responses. While several studies have explored autonomous approaches to distilling and annotating instructions from powerful proprietary LLMs, such as ChatGPT, they often neglect the impact of the distributions and characteristics of tasks, together with the varying difficulty of instructions in training sets. This oversight can lead to imbalanced knowledge capabilities and poor generalization powers of student LLMs. To address these challenges, we introduce Task-Aware Curriculum Planning for Instruction Refinement (TAPIR), a multi-round distillation framework that utilizes an oracle LLM to select instructions that are difficult for a student LLM to follow. To balance the student’s capabilities, task distributions in training sets are adjusted with responses automatically refined according to their corresponding tasks. In addition, by incorporating curriculum planning, our approach systematically escalates the difficulty levels of tasks, progressively enhancing the student LLM’s capabilities. We rigorously evaluate TAPIR using several widely recognized benchmarks (such as AlpacaEval 2.0, MT-Bench, etc.) and multiple student LLMs. Empirical results demonstrate that student LLMs, trained with our method and less training data, outperform larger instruction-tuned models and strong distillation baselines. Copyright © 2024, The Authors. All rights reserved.

关键词： Students

来源：评论

学校读者我要写书评

暂无评论

Building a Family of Data Augmentation Models for Low-cost LLM Fine-tuning on the cloud

arXiv

引用

arXiv 2024年

作者： Yue, Yuanhao Wang, Chengyu Huang, Jun Wang, Peng School of Computer Science Fudan University Shanghai China Alibaba Cloud Computing Hangzhou China

Specializing LLMs in various domain-specific tasks has emerged as a critical step towards achieving high performance. However, the construction and annotation of datasets in specific domains are always very costly. Apart from using superior and expensive closed-source LLM APIs to construct datasets, some open-source models have become strong enough to handle dataset construction in many scenarios. Thus, we present a family of data augmentation models designed to significantly improve the efficiency for model fine-tuning. These models, trained based on sufficiently small LLMs, support key functionalities with low inference costs: instruction expansion, instruction refinement, and instruction-response pair expansion. To fulfill this goal, we first construct an automatic data collection system with seed datasets generated from both public repositories and our in-house datasets. This system leverages powerful LLMs to expand, refine and re-write the instructions and responses, incorporating quality assessment techniques. Following this, we introduce the training process of our models, which effectively distills task-solving and text synthesis abilities from teacher LLMs. Finally, we demonstrate how we integrate these functionalities into a machine learning platform to support low-cost LLM fine-tuning from both dataset preparation and training perspectives for users. Experiments and an application study prove the effectiveness of our approach. Copyright © 2024, The Authors. All rights reserved.

关键词：

来源：评论

学校读者我要写书评

暂无评论

A Study on Privacy Protection technology for Cross-border Transactions Based on Blockchain technology 6

A Study on Privacy Protection Technology for Cross-border Tr...

引用

6th International Conference on Computer Information science and Application technology, CISAT 2023

作者： Wang, Wennan Zhu, Linkai Song, Shiyang Academy of Management Guangdong University of Science and Technology Dongguan China Information Technology School Hebei University of Economics and Business Shijiazhuang China Alibaba Cloud Big Data Application College Zhuhai College of Science and Technology Zhuhai China

ISBN: (纸本)9781510668546

With the acceleration of global economic integration, cross-border transactions are becoming more and more frequent. However, traditional cross- border transaction systems generally suffer from inefficiency, high cost and privacy leakage. In this paper, we study the privacy protection problem in cross-border transactions and propose a privacy protection scheme for cross-border transactions based on block-chain technology. First, this paper briefly introduces the basic principles of block-chain technology and its application scenarios in cross-border transactions. Second, to address the shortcomings of existing privacy protection technologies, this paper proposes a privacy protection scheme that combines data encryption technology, zero-knowledge proof and hybrid network. The scheme achieves automatic execution and verification through smart contracts, which effectively protects the privacy of cross-border transaction participants. Finally, the effectiveness of the privacy protection scheme proposed in this paper in improving the level of privacy protection for cross- border transactions is verified through experimental evaluation and result analysis. This study is important for promoting the development of cross- border transactions, improving transaction efficiency and protecting user privacy. © 2023 SPIE.

关键词： Smart contract

来源：评论

学校读者我要写书评

暂无评论

Evolving Knowledge Distillation with Large Language Models and Active Learning 30

Evolving Knowledge Distillation with Large Language Models a...

引用

Joint 30th International Conference on Computational Linguistics and 14th International Conference on Language Resources and Evaluation, LREC-COLING 2024

作者： Liu, Chengyuan Kang, Yangyang Zhao, Fubang Kuang, Kun Jiang, Zhuoren Sun, Changlong Wu, Fei College of Computer Science and Technology Zhejiang University China Institute for Intelligent Computing Alibaba Group China School of Public Affairs Zhejiang University China Shanghai Institute for Advanced Study of Zhejiang University China

ISBN: (纸本)9782493814104

Large language models (LLMs) have demonstrated remarkable capabilities across various NLP tasks. However, their computational costs are prohibitively high. To address this issue, previous research has attempted to distill the knowledge of LLMs into smaller models by generating annotated data. Nonetheless, these works have mainly focused on the direct use of LLMs for text generation and labeling, without fully exploring their potential to comprehend the target task and acquire valuable knowledge. In this paper, we propose EvoKD: Evolving Knowledge Distillation, which leverages the concept of active learning to interactively enhance the process of data generation using large language models, simultaneously improving the task capabilities of small domain model (student model). Different from previous work, we actively analyze the student model's weaknesses, and then synthesize labeled samples based on the analysis. In addition, we provide iterative feedback to the LLMs regarding the student model's performance to continuously construct diversified and challenging samples. Experiments and analysis on different NLP tasks, namely, text classification and named entity recognition show the effectiveness of EvoKD. © 2024 ELRA Language Resource Association: CC BY-NC 4.0.

关键词： Distillation

来源：评论

学校读者我要写书评

暂无评论

Interpreting and improving large language models in arithmetic calculation 24

Interpreting and improving large language models in arithmet...

引用

Proceedings of the 41st International Conference on Machine Learning

作者： Wei Zhang Chaoqun Wan Yonggang Zhang Yiu-ming Cheung Xinmei Tian Xu Shen Jieping Ye University of Science and Technology of China and Alibaba Cloud Alibaba Cloud Hong Kong Baptist University University of Science and Technology of China and Institute of Artificial Intelligence Hefei Comprehensive National Science Center

Large language models (LLMs) have demonstrated remarkable potential across numerous applications and have shown an emergent ability to tackle complex reasoning tasks, such as mathematical computations. However, even for the simplest arithmetic calculations, the intrinsic mechanisms behind LLMs remains mysterious, making it challenging to ensure reliability. In this work, we delve into uncovering a specific mechanism by which LLMs execute calculations. Through comprehensive experiments, we find that LLMs frequently involve a small fraction (< 5%) of attention heads, which play a pivotal role in focusing on operands and operators during calculation processes. Subsequently, the information from these operands is processed through multi-layer perceptrons (MLPs), progressively leading to the final solution. These pivotal heads/MLPs, though identified on a specific dataset, exhibit transferability across different datasets and even distinct tasks. This insight prompted us to investigate the potential benefits of selectively fine-tuning these essential heads/MLPs to boost the LLMs' computational performance. We empirically find that such precise tuning can yield notable enhancements on mathematical prowess, without compromising the performance on non-mathematical tasks. Our work serves as a preliminary exploration into the arithmetic calculation abilities inherent in LLMs, laying a solid foundation to reveal more intricate mathematical tasks.

关键词：

来源：评论

学校读者我要写书评

暂无评论

DiffForward: On Balancing Forwarding Traffic for Modern cloud Block Services via Differentiated Forwarding

引用

Performance Evaluation Review 2023年第1期51卷 11-12页

作者： Zhu, Wenzhe Li, Yongkun Xu, Erci Li, Fei Xu, Yinlong Lui, John C. S. University of Science and Technology of China Hefei China PDL Changsha China Alibaba Group Beijing China Anhui Province Key Laboratory of High Performance Computing USTC Hefei China The Chinese University of Hong Kong Hong Kong Hong Kong

Modern cloud block service provides cloud users with virtual block disks (VDisks), and it usually relies on a forwarding layer consisting of multiple proxy servers to forward the block-level writes from applications to the underlying distributed storage. However, we discover that severe traffic imbalance exists among the proxy servers at the forwarding layer, thus creating a performance bottleneck which severely prolongs the latency of accessing VDisks. Worse yet, due to the diverse access patterns of VDisks, stable traffic and burst traffic coexist at the forwarding layer, and thus making existing load balancing designs inefficient for balancing the traffic at the forwarding layer of VDisks, as they are unaware of and also lack the ability to differentiate the decomposable burst and stable traffic. To this end, we propose a novel traffic forwarding scheme DiffForward for cloud block services. DiffForward differentiates the burst traffic from stable traffic in an accurate and efficient way at the client side, then it forwards the burst traffic to a decentralized distributed log store to realize real-time load balance by writing the data in a round-robin manner and balances the stable traffic by segmentation. DiffForward also judiciously coordinates the stable and burst traffic and preserves strong consistency under differentiated forwarding. Extensive experiments with reallife workloads on our prototype show that DiffForward effectively balances the traffic at the forwarding layer at a fine-grained subsecond level, thus significantly reducing the write latency of VDisks. © 2023 Owner/Author.

关键词： Digital storage

来源：评论

学校读者我要写书评

暂无评论

Learning Neural Volumetric Pose Features for Camera Localization

arXiv

引用

arXiv 2024年

作者： Lin, Jingyu Gu, Jiaqi Wu, Bojian Fan, Lubin Chen, Renjie Liu, Ligang Ye, Jieping University of Science and Technology of China China Alibaba Cloud China Zhejiang University China

We introduce a novel neural volumetric pose feature, termed PoseMap, designed to enhance camera localization by encapsulating the information between images and the associated camera poses. Our framework leverages an Absolute Pose Regression (APR) architecture, together with an augmented NeRF module. This integration not only facilitates the generation of novel views to enrich the training dataset but also enables the learning of effective pose features. Additionally, we extend our architecture for self-supervised online alignment, allowing our method to be used and fine-tuned for unlabelled images within a unified framework. Experiments demonstrate that our method achieves 14.28% and 20.51% performance gain on average in indoor and outdoor benchmark scenes, outperforming existing APR methods with state-of-the-art accuracy. Copyright © 2024, The Authors. All rights reserved.

关键词： Self-supervised learning

来源：评论

学校读者我要写书评

暂无评论

Preserving model privacy for Federated Reinforcement Learning in Complementary Environments 23

Preserving model privacy for Federated Reinforcement Learnin...

引用

15th International Conference on Machine Learning and computing, ICMLC 2023

作者： Xuan, Pengyu Chen, Aiguo Sha, Zexin University of Electronic Science and Technology of China Chengdu China Trusted Cloud Computing and Big Data Key Laboratory of Sichuan Province Chengdu China

ISBN: (纸本)9781450398411

Federated reinforcement learning (FRL) uses data from multiple partners interacting with the environment to train a global decision model while maintaining data privacy. In specific situations, it is necessary to protect not only data privacy but also model privacy. That is, our objective is to collaboratively train a global model that can be used privately and independently by the initiator. We refer to this need for model privacy preservation as model monopolization. It allows the task initiator to ensure the privacy of the model during the training process and to own the trained model alone. To the best of our knowledge, model monopolization protection in Federated Reinforcement Learning has not been studied. We propose monopolized federated reinforcement learning in complementary environments via function transformation. We also provide contribution metrics to ensure the sustainability of monopolized federated reinforcement learning and to provide a quantitative basis for task initiators to give back to task participants. The experimental results show that our method achieved model monopolization by training agents in the Grid-world. © 2023 ACM.

关键词： Reinforcement learning

来源：评论

学校读者我要写书评

暂无评论

Towards an Automated Testing Framework for IoT Devices 2

Towards an Automated Testing Framework for IoT Devices

引用

2022 2nd International Symposium on Artificial Intelligence and Intelligent Manufacturing, AIIM 2022

作者： Wang, Baoping Zhu, Linkai Sun, Di Wang, Wennan Song, Shiyang Peng, Sheng Academy of Management Guangdong University of Science and Technology Dongguan China Information Technology School Hebei University of Economics and Business Shijiazhuang China Shijiazhuang University Applied Technology China Alibaba Cloud Big Data Application College Zhuhai College of Science and Technology Zhuhai China

As the Internet of Things (IoT) devices become increasingly available, there is an increasing demand for automated testing. Starting from the advantages of automated testing in the IoT, this paper first examines that automated testing has significantly improved compared to traditional manual testing in terms of test efficiency, test level, cost savings, and product stability. We then explain in detail the steps and key points of implementing automatic tests. Finally, a variety of automatic client frameworks are developed based on the Python language and connected to an automated testing environment, which has achieved significant benefits in equipment stability testing and performance testing. © Published under licence by IOP Publishing Ltd.

关键词： Internet of things

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：