检索结果-内蒙古大学图书馆

Neuromodulated Meta-Learning

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Wang, Jingyao Guo, Huijie Qiang, Wenwen Li, Jiangmeng Zheng, Changwen Xiong, Hui Hua, Gang University of Chinese Academy of Sciences Beijing China National Key Laboratory of Space Integrated Information System Institute of Software Chinese Academy of Sciences Beijing China Beijing Advanced Innovation Center for Big Data and Brain Computing Beihang University Beijing China Thrust of Artificial Intelligence Hong Kong University of Science and Technology Guangzhou China Department of Computer Science and Engineering The Hong Kong University of Science and Technology Hong Kong Wormpex AI Research LLC BellevueWA98004 United States

Humans excel at adapting perceptions and actions to diverse environments, enabling efficient interaction with the external world. This adaptive capability relies on the biological nervous system (BNS), which activates different brain regions for distinct tasks. Meta-learning similarly trains machines to handle multiple tasks but relies on a fixed network structure, not as flexible as BNS. To investigate the role of flexible network structure (FNS) in meta-learning, we conduct extensive empirical and theoretical analyses, finding that model performance is tied to structure, with no universally optimal pattern across tasks. This reveals the crucial role of FNS in meta-learning, ensuring meta-learning to generate the optimal structure for each task, thereby maximizing the performance and learning efficiency of meta-learning. Motivated by this insight, we propose to define, measure, and model FNS in meta-learning. First, we define that an effective FNS should possess frugality, plasticity, and sensitivity. Then, to quantify FNS in practice, we present three measurements for these properties, collectively forming the structure constraint with theoretical supports. Building on this, we finally propose Neuromodulated Meta-Learning (NeuronML) to model FNS in meta-learning. It utilizes bi-level optimization to update both weights and structure with the structure constraint. Extensive theoretical and empirical evaluations demonstrate the effectiveness of NeuronML on various tasks. Code is publicly available at https://***/WangJingyao07/NeuronML. Copyright © 2024, The Authors. All rights reserved.

关键词： Contrastive Learning

Multi-task representation learning with multi-view graph convolutional networks

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Huang, Hong Song, Yu Wu, Yao Shi, Jia Xie, Xia Jin, Hai National Engineering Research Center for Big Data Technology Service Computing Technology and System Lab Cluster and Grid Computing Lab School of Computer Science and Technology Huazhong University of Science and Technology Wuhan430074 China

Link prediction and node classification are two important downstream tasks of network representation learning. Existing methods have achieved acceptable results but they perform these two tasks separately, which requires a lot of duplication of work and ignores the correlations between tasks. Besides, conventional models suffer from the identical treatment of information of multiple views, thus they fail to learn robust representation for downstream tasks. To this end, we tackle link prediction and node classification problems simultaneously via multi-task multi-view learning in this paper. We first explain the feasibility and advantages of multi-task multi-view learning for these two tasks. Then we propose a novel model named as MT-MVGCN to perform link prediction and node classification tasks simultaneously. More specifically, we design a multi-view graph convolutional network to extract abundant information of multiple views in a network, which is shared by different tasks. We further apply two attention mechanisms: view attention mechanism and task attention mechanism to make views and tasks adjust the view fusion process. Moreover, view reconstruction can be introduced as an auxiliary task to boost the performance of the proposed model. Experiments on real-world network datasets demonstrate that our model is efficient yet effective, and outperforms advanced baselines in these two tasks. © 2021, CC BY.

关键词： data mining

MC3D-AD: A Unified Geometry-aware Reconstruction Model for Multi-category 3D Anomaly Detection

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Cheng, Jiayi Gao, Can Zhou, Jie Wen, Jiajun Dai, Tao Wang, Jinbao College of Computer Science and Software Engineering Shenzhen University Shenzhen China National Engineering Laboratory for Big Data System Computing Technology Shenzhen University China Guangdong Provincial Key Laboratory of Intelligent Information Processing Shenzhen China

3D Anomaly Detection (AD) is a promising means of controlling the quality of manufactured products. However, existing methods typically require carefully training a task-specific model for each category independently, leading to high cost, low efficiency, and weak generalization. Therefore, this paper presents a novel unified model for Multi-Category 3D Anomaly Detection (MC3D-AD) that aims to utilize both local and global geometry-aware information to reconstruct normal representations of all categories. First, to learn robust and generalized features of different categories, we propose an adaptive geometry-aware masked attention module that extracts geometry variation information to guide mask attention. Then, we introduce a local geometry-aware encoder reinforced by the improved mask attention to encode group-level feature tokens. Finally, we design a global query decoder that utilizes point cloud position embeddings to improve the decoding process and reconstruction ability. This leads to local and global geometry-aware reconstructed feature tokens for the AD task. MC3D-AD is evaluated on two publicly available Real3D-AD and Anomaly-ShapeNet datasets, and exhibits significant superiority over current state-of-the-art single-category methods, achieving 3.1% and 9.3% improvement in object-level AUROC over Real3D-AD and Anomaly-ShapeNet, respectively. The source code will be released upon acceptance. Copyright © 2025, The Authors. All rights reserved.

关键词： Geometry

Neural Multi-Objective Combinatorial Optimization with Diversity Enhancement

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Chen, Jinbiao Zhang, Zizhen Cao, Zhiguang Wu, Yaoxin Ma, Yining Ye, Te Wang, Jiahai School of Computer Science and Engineering Sun Yat-sen University China School of Computing and Information Systems Singapore Management University Singapore Department of Industrial Engineering & Innovation Sciences Eindhoven University of Technology Netherlands Department of Industrial Systems Engineering & Management National University of Singapore Singapore Key Laboratory of Machine Intelligence and Advanced Computing Ministry of Education Sun Yat-sen University China Guangdong Key Laboratory of Big Data Analysis and Processing Guangzhou China

Most of existing neural methods for multi-objective combinatorial optimization (MOCO) problems solely rely on decomposition, which often leads to repetitive solutions for the respective subproblems, thus a limited Pareto set. Beyond decomposition, we propose a novel neural heuristic with diversity enhancement (NHDE) to produce more Pareto solutions from two perspectives. On the one hand, to hinder duplicated solutions for different subproblems, we propose an indicator-enhanced deep reinforcement learning method to guide the model, and design a heterogeneous graph attention mechanism to capture the relations between the instance graph and the Pareto front graph. On the other hand, to excavate more solutions in the neighborhood of each subproblem, we present a multiple Pareto optima strategy to sample and preserve desirable solutions. Experimental results on classic MOCO problems show that our NHDE is able to generate a Pareto front with higher diversity, thereby achieving superior overall performance. Moreover, our NHDE is generic and can be applied to different neural methods for MOCO. Copyright © 2023, The Authors. All rights reserved.

关键词： Deep learning

Insulating materials for realising carbon neutrality:Opportunities,remaining issues and challenges

学校读者我要写书评

暂无评论

High Voltage 2022年第4期7卷 610-632页

作者： Chuanyang Li Yang Yang Guoqiang Xu Yao Zhou Mengshuo Jia Shaolong Zhong Yu Gao Chanyeop Park Qiang Liu Yalin Wang Shakeel Akram Xiaoliang Zeng Yi Li Fangwei Liang Bin Cui Junpeng Fang Lingling Tang Yulin Zeng Xingtao Hu Jiachen Gao Giovanni Mazzanti Jinliang He Jianxiao Wang Davide Fabiani Gilbert Teyssedre Yang Cao Feipeng Wang Yunlong Zi The State Key Laboratory of Power System Department of Electrical EngineeringTsinghua UniversityBeijingChina Department of Mechanical and Automation Engineering The Chinese University of Hong KongHong KongChina Simpson Querrey Institute Northwestern UniversityEvanstonIllinoisUSA Department of Materials Science and Engineering The Pennsylvania State UniversityUniversity ParkPennsylvaniaUSA Department of Information Technology and Electrical Engineering ETH ZürichZürichSwitzerland School of Electrical and Information Engineering Tianjin UniversityTianjinChina Department of Electrical and Computer Engineering Mississippi State UniversityMississippi StateMississippiUSA Electrical and Electronic Engineering The University of ManchesterManchesterUK Department of Electrical Engineering School of Electronic Information and Electrical EngineeringShanghai Jiao Tong UniversityShanghaiChina Key Laboratory of Control of Power Transmission and Conversion(SJTU) Ministry of EducationShanghaiChina College of Electrical Engineering Sichuan UniversityChengduChina Shenzhen Institute of Advanced Technology Chinese Academy of Sciences ShenzhenShenzhenChina School of Electrical Engineering Wuhan UniversityWuhanChina School of Integrated Circuits Tsinghua UniversityBeijingChina Jiangsu Jinxin Electric Appliance Co.Ltd. YangzhouChina Tai'an Tiancheng Safety Evaluation Co. Ltd.Tai'anChina Department Electrical Electronic and Information EngineeringUniversity of BolognaBolognaItaly National Engineering Laboratory for Big Data Analysis and Applications Peking UniversityBeijingChina Laplace Paul Sabatier Universityand CNRSToulouseFrance Electrical and Computer Engineering University of ConnecticutStorrsConnecticutUSA State Key Laboratory of Power Transmission Equipment&System Security and New Technology School of Electrical EngineeringChongqing UniversityChongqingChina Thrust of Sustainable Energy and Environment The Hong Kong University of Science and Technology(Guangzhou)GuangzhouChina De

The 2050 carbon-neutral vision spawns a novel energy structure revolution,and the construction of the future energy structure is based on equipment *** material,as the core of electrical power equipment and electrified transportation asset,faces unprecedented challenges and *** goal of carbon neutral and the urgent need for innovation in electric power equipment and electrification assets are first *** engineering challenges constrained by the insulation system in future electric power equipment/devices and electrified transportation assets are *** materials,including intelligent insulating material,high thermal conductivity insulating material,high energy storage density insulating material,extreme environment resistant insulating material,and environmental-friendly insulating material,are cat-egorised with their scientific issues,opportunities and challenges under the goal of carbon neutrality being *** the context of carbon neutrality,not only improves the understanding of the insulation problems from a macro level,that is,electrical power equipment and electrified transportation asset,but also offers opportunities,remaining issues and challenges from the insulating material *** is hoped that this paper en-visions the challenges regarding design and reliability of insulations in electrical equipment and electric vehicles in the context of policies towards carbon neutrality *** authors also hope that this paper can be helpful in future development and research of novel insulating materials,which promote the realisation of the carbon-neutral vision.

关键词： neutral insulation extreme

DepGraph: A Dependency-Driven Accelerator for Efficient Iterative Graph Processing

学校读者我要写书评

暂无评论

DepGraph: A Dependency-Driven Accelerator for Efficient Iter...

IEEE Symposium on High-Performance Computer Architecture

作者： Yu Zhang Xiaofei Liao Hai Jin Ligang He Bingsheng He Haikun Liu Lin Gu National Engineering Research Center for Big Data Technology and System Services Computing Technology and System Lab Cluster and Grid Computing Lab School of Computer Science and Technology Huazhong University of Science and Technology China University of Warwick United Kingdom National University of Singapore Singapore

Many graph processing systems have been recently developed for many-core processors. However, for iterative graph processing, due to the dependencies between vertices' states, the propagations of new states of vertices are inherently conducted along graph paths sequentially and are also dependent on each other. Despite the years' research effort, existing solutions still severely underutilize many-core processors to quickly propagate the new states of vertices, suffering from slow convergence speed. In this paper, we propose a dependency-driven programmable accelerator, DepGraph, which couples with the core architecture of the many-core processor and can fundamentally alleviate the challenge of dependencies for faster state propagation. Specifically, we propose an effective dependency-driven asynchronous execution approach into novel microarchitecture designs for faster state propagations. DepGraph prefetches the vertices for the core on-the-fly along the dependency chains between their states and the active vertices' new states, aiming to effectively accelerate the propagations of the active vertices' new states and also ensure better data locality. Through transforming the dependency chains along the frequently-used paths into direct ones at runtime and maintaining these calculated direct dependencies as a set of fast shortcuts, called hub index, DepGraph further accelerates most state propagations. Also, many propagations do not need to wait for the completion of other propagations, which enables more propagations to be effectively conducted along the paths with higher degree of parallelism. The experimental results show that for iterative graph processing on a simulated 64-core processor, a cutting-edge software graph processing system can achieve 5.0-22.7 times speedup after integrating with our DepGraph while incurring only 0.6% area cost. In comparison with three state-of-the-art hardware solutions, i.e., HATS, Minnow, and PHI, DepGraph improves the performan

关键词： Program processors Prefetching Computer architecture Parallel processing Software Acceleration Indexes

iGniter: Interference-Aware GPU Resource Provisioning for Predictable DNN Inference in the Cloud

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Xu, Fei Xu, Jianian Chen, Jiabin Chen, Li Shang, Ruitao Zhou, Zhi Liu, Fangming The Shanghai Key Laboratory of Multidimensional Information Processing School of Computer Science and Technology East China Normal University 3663 N. Zhongshan Road Shanghai200062 China The School of Computing and Informatics University of Louisiana at Lafayette 301 East Lewis Street LafayetteLA70504 United States The Guangdong Key Laboratory of Big Data Analysis and Processing School of Computer Science and Engineering Sun Yat-sen University 132 E. Waihuan Road Guangzhou510006 China The National Engineering Research Center for Big Data Technology and System The Services Computing Technology and System Lab Cluster and Grid Computing Lab School of Computer Science and Technology Huazhong University of Science and Technology 1037 Luoyu Road Wuhan430074 China

GPUs are essential to accelerating the latency-sensitive deep neural network (DNN) inference workloads in cloud datacenters. To fully utilize GPU resources, spatial sharing of GPUs among co-located DNN inference workloads becomes increasingly compelling. However, GPU sharing inevitably brings severe performance interference among co-located inference workloads, as motivated by an empirical measurement study of DNN inference on EC2 GPU instances. While existing works on guaranteeing inference performance service level objectives (SLOs) focus on either temporal sharing of GPUs or reactive GPU resource scaling and inference migration techniques, how to proactively mitigate such severe performance interference has received comparatively little attention. In this paper, we propose iGniter, an interference-aware GPU resource provisioning framework for cost-efficiently achieving predictable DNN inference in the cloud. iGniter is comprised of two key components: (1) a lightweight DNN inference performance model, which leverages the system and workload metrics that are practically accessible to capture the performance interference;(2) A cost-efficient GPU resource provisioning strategy that jointly optimizes the GPU resource allocation and adaptive batching based on our inference performance model, with the aim of achieving predictable performance of DNN inference workloads. We implement a prototype of iGniter based on the NVIDIA Triton inference server hosted on EC2 GPU instances. Extensive prototype experiments on four representative DNN models and datasets demonstrate that iGniter can guarantee the performance SLOs of DNN inference workloads with practically acceptable runtime overhead, while saving the monetary cost by up to 25% in comparison to the state-of-the-art GPU resource provisioning strategies. Copyright © 2022, The Authors. All rights reserved.

关键词： Deep neural networks

PoE: a Panel of Experts for Generalized Automatic Dialogue Assessment

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Zhang, Chen D'Haro, Luis Fernando Zhang, Qiquan Friedrichs, Thomas Li, Haizhou The Human Language Technology Group at Electrical & Computer Engineering Department National University of Singapore Singapore Spain Pte Ltd Singapore The Guangdong Provincial Key Laboratory of Big Data Computing The Chinese University of Hong Kong Shenzhen China

Chatbots are expected to be knowledgeable across multiple domains, e.g. for daily chit-chat, exchange of information, and grounding in emotional situations. To effectively measure the quality of such conversational agents, a model-based automatic dialogue evaluation metric (ADEM) is expected to perform well across multiple domains. Despite significant progress, an ADEM that works well in one domain does not necessarily generalize to another. This calls for a dedicated network architecture for domain generalization. To tackle the multi-domain dialogue evaluation task, we propose a Panel of Experts (PoE), a multitask network that consists of a shared transformer encoder and a collection of lightweight adapters. The shared encoder captures the general knowledge of dialogues across domains, while each adapter specializes in one specific domain and serves as a domain expert. To validate the idea, we construct a high-quality multi-domain dialogue dataset leveraging data augmentation and pseudo labeling. The PoE network is comprehensively assessed on 16 dialogue evaluation datasets spanning a wide range of dialogue domains. It achieves state-of-the-art performance in terms of mean Spearman correlation over all the evaluation datasets. It exhibits better zero-shot generalization than existing state-of-the-art ADEMs and the ability to easily adapt to new domains with few-shot transfer learning. Copyright © 2022, The Authors. All rights reserved.

关键词： Network architecture

High-Frequency Stereo Matching Network

学校读者我要写书评

暂无评论

High-Frequency Stereo Matching Network

Conference on Computer Vision and Pattern Recognition (CVPR)

作者： Haoliang Zhao Huizhou Zhou Yongjun Zhang Jie Chen Yitong Yang Yong Zhao Text Computing & Cognitive Intelligence Engineering Research Center of National Education Ministry State Key Laboratory of Public Big Data College of Computer Science and Technology Institute of Artificial Intelligence Guizhou University Guiyang Guizhou China Ghost-Valley AI Technology Shenzhen Guangdong China School of Physics and Optoelectronic Engineering Guangdong University of Technology Guangzhou China The Key Laboratory of Integrated Microsystems Shenzhen Graduate School Peking University China

In the field of binocular stereo matching, remarkable progress has been made by iterative methods like RAFT-Stereo and CREStereo. However, most of these methods lose information during the iterative process, making it difficult to generate more detailed difference maps that take full advantage of high-frequency information. We propose the Decouple module to alleviate the problem of data coupling and allow features containing subtle details to transfer across the iterations which proves to alleviate the problem significantly in the ablations. To further capture high-frequency details, we propose a Normalization Refinement module that unifies the disparities as a proportion of the disparities over the width of the image, which address the problem of module failure in cross-domain scenarios. Further, with the above improvements, the ResNet-like feature extractor that has not been changed for years becomes a bottleneck. Towards this end, we proposed a multi-scale and multi-stage feature extractor that introduces the channel-wise self-attention mechanism which greatly addresses this bottleneck. Our method (DLNR) ranks 1st on the Middlebury leaderboard, significantly outperforming the next best method by 13.04%. Our method also achieves SOTA performance on the KITTI-2015 benchmark for D1-fg. Code and demos are available at: https://***/David-Zhao-1997/High-frequency-Stereo-Matching-Network.

关键词：