检索结果-内蒙古大学图书馆

7th IEEE Information Technology, Networking, Electronic and Automation Control Conference, ITNEC 2024

作者： Wan, Guibin Ma, Xiao Zeng, Zhi Gao, Yan Jia, Tianhao China Academy of Launch Vehicle Technology Beijing China

ISBN: (纸本)9798350370782

Compared with static routing algorithms, dynamic routing algorithms are more suitable for network transmission of aircraft with topology changes. Traditional typical routing algorithms do not have the ability to evaluate and analyze survivability. This paper proposes a high survivability routing technology based on resource statistical characteristics. By analyzing the degree of survivability indicators and deducing the QoS indicators, a high survivability routing algorithm is formed. It has advantages in improving network connectivity and reducing routing discovery costs. Besides, it can enhance the dynamic reconfiguration and high connectivity capabilities of routing in adversarial scenarios. © 2024 IEEE.

关键词： dynamic routing algorithms

来源：评论

学校读者我要写书评

暂无评论

A Survey on Inference Optimization Techniques for Mixture of Experts Models

arXiv

引用

arXiv 2024年

作者： Liu, Jiacheng Tang, Peng Wang, Wenfeng Ren, Yuhang Hou, Xiaofeng Heng, Pheng-Ann Guo, Minyi Li, Chao Chinese University of Hong Kong Hong Kong Shanghai Jiao Tong Universtiy Shanghai China

The emergence of large-scale Mixture of Experts (MoE) models represents a significant advancement in artificial intelligence, offering enhanced model capacity and computational efficiency through conditional computation. However, deploying and running inference on these models presents significant challenges in computational resources, latency, and energy efficiency. This comprehensive survey analyzes optimization techniques for MoE models across the entire system stack. We first establish a taxonomical framework that categorizes optimization approaches into model-level, system-level, and hardware-level optimizations. At the model level, we examine architectural innovations including efficient expert design, attention mechanisms, various compression techniques such as pruning, quantization, and knowledge distillation, as well as algorithm improvement including dynamic routing strategies and expert merging methods. At the system level, we investigate distributed computing approaches, load balancing mechanisms, and efficient scheduling algorithms that enable scalable deployment. Furthermore, we delve into hardware-specific optimizations and co-design strategies that maximize throughput and energy efficiency. This survey provides both a structured overview of existing solutions and identifies key challenges and promising research directions in MoE inference optimization. To facilitate ongoing updates and the sharing of cutting-edge advances in MoE inference optimization research, we have established a repository accessible at https://***/MoE-Inf/awesome-moe-inference/. Copyright © 2024, The Authors. All rights reserved.

关键词： dynamic routing algorithms

来源：评论

学校读者我要写书评

暂无评论

MaskMoE: Boosting Token-Level Learning via routing Mask in Mixture-of-Experts

arXiv

引用

arXiv 2024年

作者： Su, Zhenpeng Lin, Zijia Bai, Xue Wu, Xing Xiong, Yizhe Lian, Haoran Ma, Guangyuan Chen, Hui Ding, Guiguang Zhou, Wei Hu, Songlin Institute of Information Engineering Chinese Academy of Sciences China School of Cyber Security University of Chinese Academy of Sciences China Tsinghua University China University of Science and Technology of China China Beihang University China

Scaling the size of a model enhances its capabilities but significantly increases computation complexity. Mixture-of-Experts models (MoE) address the issue by allowing model size to scale up without substantially increasing training or inference costs. In MoE, there is an important module called the router, which is used to distribute each token to the experts. Currently, the mainstream routing methods include dynamic routing and fixed routing. Despite their promising results, MoE models encounter several challenges. Primarily, for dynamic routing methods, the dispersion of training tokens across multiple experts can lead to underfitting, particularly for infrequent tokens. Additionally, though fixed routing methods can mitigate that issue, they compromise on the diversity of representations. In this paper, we propose MaskMoE, a method designed to enhance token-level learning by employing a routing masking technique within the Mixture-of-Experts model. MaskMoE is capable of maintaining representation diversity while achieving more comprehensive training. Experimental results demonstrate that our method outperforms previous dominant Mixture-of-Experts models in terms of both perplexity (PPL) and downstream task performance. © 2024, CC BY.

关键词： dynamic routing algorithms

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：