检索结果-内蒙古大学图书馆

10th IEEE Joint International Information technology and Artificial Intelligence Conference, ITAIC 2022

作者： Liu, Yang Lu, Jun College of Computer Science and Technology Heilongjiang University Harbin150080 China Heilongjiang University Key Laboratory of Database and Parallel Computing of Heilongjiang Province Harbin150080 China

ISBN: (数字)9781665422079

ISBN: (纸本)9781665422079

Neural architecture search aims to automatically design network architectures by machines, which is expected to bring about a new revolution in machine learning. Despite high expectations, the effectiveness and efficiency of existing neural architecture search solutions are unclear, and the inefficiency of neural architecture search solutions is mainly attributed to inaccurate architecture evaluation. In this work, this paper divides the large search space of neural architecture search into multiple modules, effectively reducing the error caused by shared parameters to ensure that the candidate architectures in the modules are fully trained. By adopting a block search method, this paper can Evaluate all candidate architectures in a module, addressing the problem of inaccurate evaluation. In addition, high-quality data is necessary. Under the guidance of the teacher model, combined with the ground truth, the neural architecture search is supervised by the double loss function. Under the constraint of the same amount of computation, the method in this paper achieves 77.3% top-level accuracy on ImageNet., which is 1.0% higher than the efficient network EfficientNet-B0, and 0.3% higher than the DNA method without additional computation. Therefore, the double loss block neural architecture search model proposed in this paper significantly improves the effectiveness of neural architecture search. © 2022 IEEE.

关键词： Distillation

来源：评论

学校读者我要写书评

暂无评论

Offline Imitation Learning Using Reward-free Exploratory Data 22

Offline Imitation Learning Using Reward-free Exploratory Dat...

引用

Proceedings of the 2022 5th International Conference on Algorithms, Computing and Artificial Intelligence

作者： Hao Wang Dawei Feng Bo Ding Wei Li College of Computer National University of Defense Technology China National Laboratory for Parallel and Distributed Processing National University of Defense Technology China Independent Researcher China

ISBN: (纸本)9781450398336

Offline imitative learning(OIL) is often used to solve complex continuous decision-making tasks. For these tasks such as robot control, automatic driving and etc., it is either difficult to design an effective reward for learning or very expensive and time-consuming for agents to collect data interactively with the environment. However, the data used in previous OIL methods are all gathered by reinforcement learning algorithms guided by task-specific rewards, which is not a true reward-free premise and still suffers from the problem of designing an effective reward function in real tasks. To this end, we propose the reward-free exploratory data driven offline imitation learning (ExDOIL) framework. ExDOIL first trains an unsupervised reinforcement learning agent by interacting with the environment, and collects enough unsupervised exploration data during training; Then, a task independent yet simple and efficient reward function is used to relabel the collected data; Finally, an agent is trained to imitate the expert to complete the task through a conventional RL algorithm such as TD3. Extensive experiments on continuous control tasks demonstrate that the proposed framework can achieve better imitation performance(28% higher episode returns on average) comparing with previous SOTA method(ORIL) without any task-specific rewards.

关键词： dataset

来源：评论

学校读者我要写书评

暂无评论

TeeSwap: Private Data Exchange using Smart Contract and Trusted Execution Environment 23

TeeSwap: Private Data Exchange using Smart Contract and Trus...

引用

23rd IEEE International Conference on High Performance Computing and Communications, 7th IEEE International Conference on Data science and Systems, 19th IEEE International Conference on Smart City and 7th IEEE International Conference on Dependability in Sensor, Cloud and Big Data Systems and Applications, HPCC-DSS-SmartCity-DependSys 2021

作者： Chen, Peng Shi, Peichang Xu, Jie Fu, Xiang Li, Linhui Zhong, Tao Xiang, Liangliang Kong, Jinzhu College of Computer National University of Defense Technology National Key Laboratory of Parallel and Distributed Processing Changsha410073 China Zhejiang Lab Hangzhou311100 China Kylin Software Co. Ltd Changsha410073 China

ISBN: (纸本)9781665494571

With importance of data value is approved, data-sharing will create more and greater value has become consensus. However, data exchange has to use a trusted third party(TTP) as an intermediary in an untrusted network environment, which may lead to leakage of data privacy. In this paper, we design a system named TeeSwap, which uses smart contract to complete data exchange, and at the same time uses cryptographic algorithms and trusted execution environment (TEE) to ensure privacy in process of data transmission, verification and contract execution. Meanwhile, TeeSwap can ensure the credibility, fairness, integrity and privacy while data are shared in our system. In the end, we implement contracts about data market and data exchange. When confirming the data transaction, data market contract can automatically generate and deploy exchange contract, which can complete the data exchange authoritatively. © 2021 IEEE.

关键词： Data privacy Smart contracts Data communication Cryptography

来源：评论

学校读者我要写书评

暂无评论

ShuffleVITNet: Mobile-Friendly Vision Transformer With Less-Memory

ShuffleVITNet: Mobile-Friendly Vision Transformer With Less-...

引用

International Joint Conference on Neural Networks (IJCNN)

作者： Xinyu Zhao Jun Lu College of Computer Science and Technology Heilongjiang University Harbin China Key Laboratory of Database and Parallel Computing of Heilongjiang Province Harbin China

ISBN: (数字)9798350359312

ISBN: (纸本)9798350359329

Traditional Convolutional Neural Networks (CNNs) can efficiently acquire local features, while ViT uses a Transformer structure that captures global contextual information. In this paper, a new high-performance lightweight target detector is designed by combining the respective advantages of CNN and ViT: ShuffleViTNet. In this paper, we improve the VIT Block in MobileVIT, firstly, we use channel-by-channel convolution to replace the normal convolution to reduce the memory access cost (MAC), number of parameters (Parmas) and floating-point operations (FLOPs) in the model inference. The 1×1 convolution is then used to fuse the local features learned by the convolution with the global features learned by the Transofmer, which further lightens the model and allows for easier modification of the number of channels and migration to other lightweight neural networks. Finally a lightweight network friendly to mobile devices is constructed using the improved VIT Block. On the ImageNet-1k dataset, ShuffleViTNet achieves 75.2% Top-1 accuracy, the model’s floating-point operation is 536.5 MFLOPs, which is 44.0% less compared to MobileViT-XS, and the cost of memory accesses for model inference is saved by about 25.6%, and the required memory is saved by about 66.37%. For the object detection task on the MS-COCO dataset, ShuffleViTNet obtains higher mAP accuracy. The experimental results show that ShuffleViTNet is a high-performance lightweight network and can be well applied to downstream tasks.

关键词： Accuracy Costs Convolution Computational modeling Neural networks Memory management Object detection

来源：评论

学校读者我要写书评

暂无评论

Contrastive Enhanced Filter Model Based on Bidirectional Transformer for Sequential Recommendation

Contrastive Enhanced Filter Model Based on Bidirectional Tra...

引用

Image Processing, Electronics and computers (IPEC), Asia-Pacific Conference on

作者： Anpeng Sun Jun Lu College of Computer Science and Technology Heilongjiang University Harbin China Key Laboratory of Database and Parallel Computing of Heilongjiang Province Harbin China

ISBN: (数字)9798350374407

ISBN: (纸本)9798350374414

Transformer-based sequential recommendation is highly powerful as it can capture short-term and long-term sequential recommendation. It plays a crucial role in personalized recommendation systems, aiming to extract dynamic and diverse preference information from users’ realtime growing behaviors. This is mainly due to its unique selfattention mechanism, which utilizes item-item interactions within the sequences. However, in online platforms, due to the interweaving of items in user behavior, user behavior data inevitably contains noise. Currently, most methods model user preferences in the time domain, overlooking the influence of frequency patterns on user behavior. To address this issue, this paper shifts the perspective to the frequency domain and proposes a novel contrast-enhanced filtering model for sequential recommendation based on the bidirectional Transformer, called CEFRec. Specifically, the paper designs a filtering layer on top of the bidirectional Transformer, allowing learnable filters to operate on different layers’ spectra. This enables the model to capture various frequency patterns and adaptively attenuate noise information within the frequency domain. Next, the model combines the cloze task mask and dropout masks to generate high-quality positive samples and utilizes contrastive learning to enhance the quality of user embeddings learned from the frequency domain. Finally, experimental results on three public benchmark datasets demonstrate that the CEFRec model outperforms state-of-theart sequential recommendation models.

关键词： Adaptation models Time-frequency analysis Electric potential Filters Image processing Noise Contrastive learning Transformers Time-domain analysis Recommender systems

来源：评论

学校读者我要写书评

暂无评论

BIVD: Improved Side Adapter Network based on BiFormer

BIVD: Improved Side Adapter Network based on BiFormer

引用

Image Processing, Electronics and computers (IPEC), Asia-Pacific Conference on

作者： Liyuan Hu Jun Lu College of Computer Science and Technology Heilongjiang University Harbin China Key Laboratory of Database and Parallel Computing of Heilongjiang Province Harbin China

ISBN: (数字)9798350374407

ISBN: (纸本)9798350374414

Currently, most open vocabulary models rely on CLIP (Contrastive Language-Image Pre-training) to classify masked regions, but CLIP models still have many drawbacks. The traditional transformer model is context aware through self attention, but this local context based mechanism is limited by the sequence length of information, it can also impact model effect. Inspired by the article BiFormer, This article replaces the traditional transformer model in the Side Adapter Network (SAN) model with a variant module BIVD block based on BiFormer blocks. This model follows the latest transformer model and extracts features using a four layer feature pyramid structure, For the reduced spatial resolution caused by patch merging, add a Visual-Decoder to reshape the visual vector sequence into a spatial feature map, and restore it to the original resolution through convolution and upsampling. This can enhance the extracted feature information through a feature pyramid without affecting the fusion with CLIP layer features. But the cost is increasing the cost of the model. This article was tested on 5 commonly used datasets and showed a stable improvement in accuracy.

关键词： Adaptation models Vocabulary Visualization Costs Semantic segmentation Feature extraction Transformers Decoding Spatial resolution Context modeling

来源：评论

学校读者我要写书评

暂无评论

FedNAT: Byzantine-robust Federated Learning through Activation-based Attention Transfer

FedNAT: Byzantine-robust Federated Learning through Activati...

引用

IEEE International Conference on Data Mining Workshops (ICDM Workshops)

作者： Mengxin Wang Liming Fang Kuiqi Chen College of Computer Science and Technology Nanjing University of Aeronautics and Astronautics Nanjing China Nanjing University of Aeronautics and Astronautics Shenzhen Research Institute Shenzhen China Science and Technology on Parallel and Distributed Processing Laboratory (PDL) Changsha China

Federated learning (FL) is a decentralized machine learning framework that prioritizes privacy by allowing clients to train statistical models without sharing their private data, thus eliminating the impact of data fortresses. However, the presence of Byzantine attacks, such as data poisoning and backdoor attack, threatens the robustness of FL schemes. Currently, existing mainstream defense methods are susceptible to multiple adaptive attacks, some of which even violate the privacy principle of FL. Furthermore, these defense schemes become less robust when subjected to targeted poisoning attacks with highly non-IID data distributions. In this work, we propose FedNAT, a novel Byzantine-robust FL framework for whittling away these limitations mentioned above. Specifically, FedNAT first performs a privacy-respecting attention refinement on the activation layer outputs of the local uploads. Then, the server scores the local attentions by calculating their Wasserstein distances and clusters them through the k-median algorithm for global attention aggregation, thus rejecting poisoned local attentions for untargeted attacks. After this process, the global attention is transferred to local attention through the FedNAT loss function, which erases backdoors through the distillation concept. We conduct a comprehensive experimental evaluation to demonstrate that FedNAT significantly outperforms existing robust FL schemes in defending against Byzantine poisoning attacks under both IID and highly non-IID data proportions.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Cross-Modality Encoder Representations Based On External Attention Mechanism

Cross-Modality Encoder Representations Based On External Att...

引用

Neural Networks, Information and Communication Engineering (NNICE), International Conference on

作者： Yudong Zheng Jun Lu College of Computer Science and Technology Heilongjiang University Harbin China College of Computer Science and Technology Heilongjiang University Key Laboratory of Database and Parallel Computing of Heilongjiang Province Harbin China

With the prevalence of deep learning, people use multi-modality information for interpretation and reasoning. In this paper, a cross-modality encoder CMEEA (cross-modality encoder representation based on external attention mechanism) is adopted to improve the accuracy of the model. The two external storage units of CMEEA can be regarded as a dictionary of the whole dataset to improve the performance of the network, while learning more representative features of the input and reducing the computational cost. This paper uses five pretraining tasks that help the model learn internal modal and cross-modal relationships. It also verifies the generalizability of the model by applying the pretraining cross-modal model to the visual inference task NLVR 2 , and improves the previous results by 0.1%. By fine-tuning the pretrain parameters, the model improves by 1.3% on visual question answering (VQA).

关键词： Training Deep learning Analytical models Visualization Dictionaries Computational modeling Question answering (information retrieval)

来源：评论

学校读者我要写书评

暂无评论

A Class of Fast and Accurate Multi-layer Block Summation and Dot Product Algorithms 18th

A Class of Fast and Accurate Multi-layer Block Summation a...

引用

18th IFIP WG 10.3 International Conference on Network and parallel Computing, NPC 2021

作者： He, Kang Barrio, Roberto Chen, Lin Jiang, Hao Liu, Jie Gu, Tongxiang Qi, Jin Science and Technology on Parallel and Distributed Processing Laboratory National University of Defense Technology Changsha410073 China Department of Applied Mathematics University of Zaragoza ZaragozaE50009 Spain College of Computer National University of Defense Technology Changsha410073 China Institute of Applied Physics and Computational Mathematics Beijing100000 China

ISBN: (纸本)9783030935702

Basic recursive summation and common dot product algorithm have a backward error bound that grows linearly with the vector dimension. Blanchard [1] proposed a class of fast and accurate summation and dot product algorithms respectively called FABsum and FABdot, which trades off the calculation accuracy and speed by the block size. Castaldo [2] proposed a multi-layer block summation and dot product algorithm called SuperBlocksum and SuperBlockdot that can increase the accuracy while adding almost no additional calculations. We combine the idea of [1] with the multi-layer block structure to propose SuperFABsum (for "super fast and accurate block summation") and SuperFABdot (for "super fast and accurate block dot product"). Our algorithms have two variants, one is SuperFAB(within), the other is SuperFAB(outside). Our algorithms further improve accuracy and speed compared with FAB and SuperBlock. We conducted accuracy and speed tests on the high-performance FT2000+ processor. Experimental results show that SuperFABdot(within) algorithm is more accurate than FABdot and SuperBlockdot. Compared with FABdot, SuperFABdot(outside) algorithm can achieve up to 1.2 × performance speedup while ensuring similar accuracy. © 2022, IFIP International Federation for Information Processing.

关键词： Artificial intelligence

来源：评论

学校读者我要写书评

暂无评论

Merak: An Efficient distributed DNN Training Framework with Automated 3D parallelism for Giant Foundation Models

arXiv

引用

arXiv 2022年

作者： Lai, Zhiquan Li, Shengwei Tang, Xudong Ge, Keshi Liu, Weijie Duan, Yabo Qiao, Linbo Li, Dongsheng The National Laboratory for Parallel and Distributed Processing College of Computer National University of Defense Technology in Changsha Hunan China

Foundation models are in the process of becoming the dominant deep learning technology. Pretraining a foundation model is always time-consuming due to the large scale of both the model parameter and training dataset. Besides being computing-intensive, the pretraining process is extremely memory- and communication-intensive. These challenges make it necessary to apply 3D parallelism, which integrates data parallelism, pipeline model parallelism, and tensor model parallelism, to achieve high training efficiency. However, current 3D parallelism frameworks still encounter two issues: i) they are not transparent to model developers, requiring manual model modification to parallelize training, and ii) their utilization of computation resources, GPU memory, and network bandwidth is insufficient. We propose Merak, an automated 3D parallelism deep learning training framework with high resource utilization. Merak automatically deploys 3D parallelism with an automatic model partitioner, which includes a graph-sharding algorithm and proxy node-based model graph. Merak also offers a non-intrusive API to scale out foundation model training with minimal code modification. In addition, we design a high-performance 3D parallel runtime engine that employs several techniques to exploit available training resources, including a shifted critical path pipeline schedule that increases computation utilization, stage-aware recomputation that makes use of idle worker memory, and sub-pipelined tensor model parallelism that overlaps communication and computation. Experiments on 64 GPUs demonstrate Merak's capability to speed up training performance over state-of-the-art 3D parallelism frameworks of models with 1.5, 2.5, 8.3, and 20 billion parameters by up to 1.42, 1.39, 1.43, and 1.61×, respectively. The code for Merak has been open-sourced at https://***/hpdl-group/Merak. Copyright © 2022, The Authors. All rights reserved.

关键词： Pipelines

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：