检索结果-内蒙古大学图书馆

41st International Conference on Machine Learning, ICML 2024

作者： Yang, Nianzu Zeng, Kaipeng Lu, Haotian Wu, Yexin Yuan, Zexin Chen, Danni Jiang, Shengdian Wu, Jiaxiang Wang, Yimin Yan, Junchi School of Artificial Intelligence Department of Computer Science and Engineering MoE Lab of AI Shanghai Jiao Tong University China Guangdong Institute of Intelligence Science and Technology China SEU-ALLEN Joint Center Institute for Brain and Intelligence Southeast University China XVERSE Technology China

Neuronal morphology is essential for studying brain functioning and understanding neurodegenerative disorders. As acquiring real-world morphology data is expensive, computational approaches for morphology generation have been studied. Traditional methods heavily rely on expert-set rules and parameter tuning, making it difficult to generalize across different types of morphologies. Recently, MorphVAE was introduced as the sole learning-based method, but its generated morphologies lack plausibility, i.e., they do not appear realistic enough and most of the generated samples are topologically invalid. To fill this gap, this paper proposes MorphGrower, which mimicks the neuron natural growth mechanism for generation. Specifically, MorphGrower generates morphologies layer by layer, with each subsequent layer conditioned on the previously generated structure. During each layer generation, MorphGrower utilizes a pair of sibling branches as the basic generation block and generates branch pairs synchronously. This approach ensures topological validity and allows for fine-grained generation, thereby enhancing the realism of the final generated morphologies. Results on four real-world datasets demonstrate that MorphGrower outperforms MorphVAE by a notable margin. Importantly, the electrophysiological response simulation demonstrates the plausibility of our generated samples from a neuroscience perspective. Our code is available at https://***/Thinklab-SJTU/MorphGrower. Copyright 2024 by the author(s)

关键词： Neurons

来源：评论

学校读者我要写书评

暂无评论

Evolving Subnetwork Training for Large Language Models 41

Evolving Subnetwork Training for Large Language Models

引用

41st International Conference on Machine Learning, ICML 2024

作者： Li, Hanqi Chen, Lu Ma, Da Wu, Zijian Zhu, Su Yu, Kai X-LANCE Lab Department of Computer Science and Engineering MoE Key Lab of Artificial Intelligence SJTU AI Institute Shanghai Jiao Tong University Shanghai China Suzhou Laboratory Suzhou China AISpeech Co. Ltd. Suzhou China

Large language models have ushered in a new era of artificial intelligence research. However, their substantial training costs hinder further development and widespread adoption. In this paper, inspired by the redundancy in the parameters of large language models, we propose a novel training paradigm: Evolving Subnetwork Training (EST). EST samples subnetworks from the layers of the large language model and from commonly used modules within each layer, Multi-Head Attention (MHA) and Multi-Layer Perceptron (MLP). By gradually increasing the size of the subnetworks during the training process, EST can save the cost of training. We apply EST to train GPT2 model and TinyLlama model, resulting in 26.7% FLOPs saving for GPT2 and 25.0% for TinyLlama without an increase in loss on the pre-training dataset. Moreover, EST leads to performance improvements in downstream tasks, indicating that it benefits generalization. Additionally, we provide intuitive theoretical studies based on training dynamics and Dropout theory to ensure the feasibility of EST. Copyright 2024 by the author(s)

关键词：

来源：评论

学校读者我要写书评

暂无评论

Making Offline RL Online: Collaborative World Models for Offline Visual Reinforcement Learning 38

Making Offline RL Online: Collaborative World Models for Off...

引用

38th Conference on Neural Information Processing Systems, NeurIPS 2024

作者： Wang, Qi Yang, Junming Wang, Yunbo Jin, Xin Zeng, Wenjun Yang, Xiaokang MoE Key Lab of Artificial Intelligence AI Institute Shanghai Jiao Tong University China Ningbo Institute of Digital Twin Eastern Institute of Technology China School of Computer Science and Engineering Southeast University China

Training offline RL models using visual inputs poses two significant challenges, i.e., the overfitting problem in representation learning and the overestimation bias for expected future rewards. Recent work has attempted to alleviate the overestimation bias by encouraging conservative behaviors. This paper, in contrast, tries to build more flexible constraints for value estimation without impeding the exploration of potential advantages. The key idea is to leverage off-the-shelf RL simulators, which can be easily interacted with in an online manner, as the "test bed" for offline policies. To enable effective online-to-offline knowledge transfer, we introduce CoWorld, a model-based RL approach that mitigates cross-domain discrepancies in state and reward spaces. Experimental results demonstrate the effectiveness of CoWorld, outperforming existing RL approaches by large margins. © 2024 Neural information processing systems foundation. All rights reserved.

关键词：

来源：评论

学校读者我要写书评

暂无评论

RL with Balanced Reward and Masking Mechanism for Multi-NUMA Virtual Machine Scheduling

RL with Balanced Reward and Masking Mechanism for Multi-NUMA...

引用

International Conference on Ubiquitous Information Management and Communication (IMCOM)

作者： Yucen Gao Yunlong Cheng Chan Tin Ping Xiaofeng Gao Guihai Chen Department of Computer Science and Engineering MoE Key Lab of Artificial Intelligence Shanghai Jiao Tong University Shanghai China

ISBN: (数字)9798331507817

ISBN: (纸本)9798331507824

With the rapid development of data center and cloud computing, the importance of resource management is increasing in recent years. In this paper, we focus on the virtual machine scheduling problem in resource management, that is, how to allo-cate the limited virtual machine resources when a request arrives. Existing works usually model the virtual machine scheduling problem as a bin-packing problem, but they do not consider the multi-NUMA architecture servers that have been widely used in recent years. Compared with the traditional symmetric multiprocessing architecture, the multi-NUMA architecture has the advantages of faster access time and higher energy efficiency, which is more suitable for dealing with today's scenarios with a large number of computing requirements. In this paper, we first formalize the multi-NUMA virtual machine scheduling problem as a combinatorial optimization problem and explain why it is suitable to be solved by reinforcement learning. A reinforcement learning model called BM-DQN is then proposed, which designs a balanced reward function suitable for the multi-NUMA architecture and adopts a masking mechanism to screen out the optional servers in each step of reinforcement learning, so as to improve the solution performance. Experimental results on the open-source real-world virtual machine scheduling dataset of Huawei Cloud show that BM-DQN is superior to the existing methods in terms of the number of completed requests.

关键词： Cloud computing Processor scheduling Computational modeling Reinforcement learning computer architecture Predictive models Virtual machines Servers Resource management Load modeling

来源：评论

学校读者我要写书评

暂无评论

Quantum Algorithms and Lower Bounds for Finite-Sum Optimization 41

Quantum Algorithms and Lower Bounds for Finite-Sum Optimizat...

引用

41st International Conference on Machine Learning, ICML 2024

作者： Zhang, Yexin Zhang, Chenyi Fang, Cong Wang, Liwei Li, Tongyang School of Electronics Engineering and Computer Science Peking University China Computer Science Department Stanford University United States National Key Lab of General Artificial Intelligence School of Intelligence Science and Technology Peking University China Institute for Artificial Intelligence Peking University China Center on Frontiers of Computing Studies Peking University China School of Computer Science Peking University China

Finite-sum optimization has wide applications in machine learning, covering important problems such as support vector machines, regression, *** this paper, we initiate the study of solving finite-sum optimization problems by quantum ***, let f1, ..., fn : d → be -smooth convex functions and ψ: d → be a µ-strongly convex proximal *** goal is to find an ϵ-optimal point for F(x) = n1 Pni=1 fi(x) + ψ(x).We give a quantum algorithm with complexity Õ(Equation presented) 1 improving the classical tight bound (Equation presented).We also prove a quantum lower bound Ω˜(n + n3/4(/µ)1/4) when d is large *** our quantum upper and lower bounds can extend to the cases where ψ is not necessarily strongly convex, or each fi is Lipschitz but not necessarily *** addition, when F is nonconvex, our quantum algorithm can find_an ϵ-critial point using (Equation presented) queries. Copyright 2024 by the author(s)

关键词： Quantum computers

来源：评论

学校读者我要写书评

暂无评论

Double-Bounded Optimal Transport for Advanced Clustering and Classification

arXiv

引用

arXiv 2024年

作者： Shi, Liangliang Shen, Zhaoqi Yan, Junchi Department of Computer Science and Engineering MoE Key Lab of Artificial Intelligence Shanghai Jiao Tong University China

Optimal transport (OT) is attracting increasing attention in machine learning. It aims to transport a source distribution to a target one at minimal cost. In its vanilla form, the source and target distributions are predetermined, which contracts to the real-world case involving undetermined targets. In this paper, we propose Doubly Bounded Optimal Transport (DB-OT), which assumes that the target distribution is restricted within two boundaries instead of a fixed one, thus giving more freedom for the transport to find solutions. Based on the entropic regularization of DB-OT, three scaling-based algorithms are devised for calculating the optimal solution. We also show that our DB-OT is helpful for barycenter-based clustering, which can avoid the excessive concentration of samples in a single cluster. Then we further develop DB-OT techniques for long-tailed classification which is an emerging and open *** first propose a connection between OT and classification, that is, in the classification task, training involves optimizing the Inverse OT to learn the representations, while testing involves optimizing the OT for predictions. With this OT perspective, we first apply DB-OT to improve the loss, and the Balanced Softmax is shown as a special case. Then we apply DB-OT for inference in the testing process. Even with vanilla Softmax trained features, our extensive experimental results show that our method can achieve good results with our improved inference scheme in the testing stage. Copyright © 2024, The Authors. All rights reserved.

关键词： Machine learning

来源：评论

学校读者我要写书评

暂无评论

Improving Few-Shot Learning for Talking Face System with TTS Data Augmentation 48

Improving Few-Shot Learning for Talking Face System with TTS...

引用

48th IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2023

作者： Chen, Qi Ma, Ziyang Liu, Tao Tan, Xu Lu, Qu Yu, Kai Chen, Xie Shanghai Jiao Tong University X-LANCE Lab Department of Computer Science and Engineering MoE Key Lab of Artificial Intelligence Ai Institute China Microsoft Research Asia China Shanghai Media Tech China

ISBN: (纸本)9781728163277

Audio-driven talking face has attracted broad interest from academia and industry recently. However, data acquisition and labeling in audio-driven talking face are labor-intensive and costly. The lack of data resource results in poor synthesis effect. To alleviate this issue, we propose to use TTS (Text-To-Speech) for data augmentation to improve few-shot ability of the talking face system. The misalignment problem brought by the TTS audio is solved with the introduction of soft-DTW, which is first adopted in the talking face task. Moreover, features extracted by HuBERT are explored to utilize underlying information of audio, and found to be superior over other features. The proposed method achieves 17%, 14%, 38% dominance on MSE score, DTW score and user study preference repectively over the baseline model, which shows the effectiveness of improving few-shot learning for talking face system with TTS augmentation. © 2023 IEEE.

关键词： few-shot HuBERT low-resource soft-DTW

来源：评论

学校读者我要写书评

暂无评论

Semantic-Aware Pseudo-labeling for Unsupervised Meta-Learning

引用

IEEE Transactions on Pattern Analysis and Machine intelligence 2025年第7期47卷 5475-5488页

作者： Ouyang, Tianran Dong, Xingping Ye, Mang Du, Bo Shao, Ling Shen, Jianbing Wuhan University School of Computer Science National Engineering Research Center for Multimedia Software Institute of Artificial Intelligence Hubei Key Laboratory of Multimedia and Network Communication Engineering Wuhan430072 China University of Chinese Academy of Sciences UCAS-Terminus AI Lab Beijing101408 China University of Macau State Key Laboratory of Internet of Things for Smart City Department of Computer and Information Science 314100 China

In unsupervised meta-learning, the clustering-based pseudo-labeling approach is an attractive framework, since it is model-agnostic, allowing it to synergize with supervised algorithms to learn from unlabeled data. However, the pseudo-labels suffer from clustering noise and semantic chaos problems, further impacting the effectiveness of meta-learning. In this paper, we analyze and optimize the pseudo-labeling process, including encoding and clustering, aiming to generate semantic-like pseudo-labels to narrow the gap between unsupervised and supervised meta-learning. First, during the encoding, we observe that the embedding space of existing methods lacks clustering-friendly properties, which is the primary reason for clustering noise. To address this issue, we minimize the inter-to-intra-class similarity ratio to generate clustering-friendly embedding features and validate our approach through comprehensive experiments. Then, during the clustering, we find that the semantic quality of pseudo-labels is not adequately controlled, resulting in semantic chaos of pseudo-labels. We propose a semantic-stability index to measure the semantic quality of pseudo-labels quantitatively. Based on this index, we propose the Semantic-aware Pseudo-label Reassignment mechanism to generate semantic-like pseudo-labels for all samples. Our approach is model-agnostic and can easily be integrated into existing supervised methods. To demonstrate its generalization ability, we integrate it into two representative algorithms: MAML and EP. The results on three main few-shot benchmarks clearly show that the proposed method achieves significant improvement compared to state-of-the-art models. Notably, our approach also outperforms the corresponding supervised method in three tasks. © 1979-2012 IEEE.

关键词： Latent semantic analysis

来源：评论

学校读者我要写书评

暂无评论

Exploring Effective Distillation of Self-Supervised Speech Models for Automatic Speech Recognition

Exploring Effective Distillation of Self-Supervised Speech M...

引用

2023 IEEE Automatic Speech Recognition and Understanding Workshop, ASRU 2023

作者： Wang, Yujin Tang, Changli Ma, Ziyang Zheng, Zhisheng Chen, Xie Zhang, Wei-Qiang Tsinghua University Department of Electronic Engineering China Ai Institute X-LANCE Lab Shanghai Jiao Tong University MoE Key Lab of Artificial Intelligence Department of Computer Science and Engineering Shanghai China Peng Cheng Laboratory Shenzhen China

ISBN: (纸本)9798350306897

Self-supervised learning (SSL) has achieved great success in speech processing, but always with a large model size to increase the modeling capacity. This may limit its potential applications due to the expensive computation and memory costs introduced by the oversize model. Compression for SSL models has become an important research direction of practical value. To this end, we explore the effective distillation of HuBERT-based SSL models for automatic speech recognition. First, a comprehensive study of different student model structures is conducted. On top of this, as a supplement to the regression loss widely adopted in previous works, a discriminative loss is introduced for HuBERT to enhance the distillation performance, especially in low-resource scenarios. In addition, we design a simple and effective algorithm to distill the front-end input from waveform to Fbank feature, resulting in 17% parameter reduction and doubling inference speed, at marginal performance degradation. © 2023 IEEE.

关键词： Distillation

来源：评论

学校读者我要写书评

暂无评论

PointMC: Multi-instance Point Cloud Registration based on Maximal Cliques 41

PointMC: Multi-instance Point Cloud Registration based on Ma...

引用

41st International Conference on Machine Learning, ICML 2024

作者： Wu, Yue Hu, Xidao Yuan, Yongzhe Fan, Xiaolong Gong, Maoguo Li, Hao Zhang, Mingyang Miao, Qiguang Ma, Wenping MoE Key Lab of Collaborative Intelligence Systems Xidian University Xi'an China School of Computer Science and Technology Xidian University Xi'an China School of Electronic Engineering Xidian University Xi'an China School of Artificial Intelligence Xidian University Xi'an China

Multi-instance point cloud registration is the problem of estimating multiple rigid transformations between two point clouds. Existing solutions rely on global spatial consistency of ambiguity and the time-consuming clustering of high-dimensional correspondence features, making it difficult to handle registration scenarios where multiple instances overlap. To address these problems, we propose a maximal clique based multi-instance point cloud registration framework called PointMC. The key idea is to search for maximal cliques on the correspondence compatibility graph to estimate multiple transformations, and cluster these transformations into clusters corresponding to different instances to efficiently and accurately estimate all poses. PointMC leverages a correspondence embedding module that relies on local spatial consistency to effectively eliminate outliers, and the extracted discriminative features empower the network to circumvent missed pose detection in scenarios involving multiple overlapping instances. We conduct comprehensive experiments on both synthetic and real-world datasets, and the results show that the proposed PointMC yields remarkable performance improvements. Copyright 2024 by the author(s)

关键词： Cloud platforms

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：