检索结果-内蒙古大学图书馆

Pre-training molecular representation model with spatial geometry for property prediction

COMPUTATIONAL BIOLOGY AND CHEMISTRY 2024年 109卷 108023页

作者： Li, Yishui Wang, Wei Liu, Jie Wu, Chengkun Natl Univ Def Technol Lab Digitizing Software Frontier Equipment Deya Rd Changsha 410073 Peoples R China Natl Univ Def Technol Natl Key Lab Parallel & Distributed Comp Deya Rd Changsha 410073 Peoples R China Natl Supercomp Ctr Tianjin TEDA Sixth St Tianjin 300450 Peoples R China

AI-enhanced bioinformatics and cheminformatics pivots on generating increasingly descriptive and generalized molecular representation. Accurate prediction of molecular properties needs a comprehensive description of molecular geometry. We design a novel Graph Isomorphic Network (GIN) based model integrating a threelevel network structure with a dual-level pre-training approach that aligns the characteristics of molecules. In our Spatial Molecular Pre-training (SMPT) Model, the network can learn implicit geometric information in layers from lower to higher according to the dimension. Extensive evaluations against established baseline models validate the enhanced efficacy of SMPT, with notable accomplishments in classification tasks. These results emphasize the importance of spatial geometric information in molecular representation modeling and demonstrate the potential of SMPT as a valuable tool for property prediction.

关键词： Molecular representation Graph Isomorphic Network Molecular property prediction

来源：评论

学校读者我要写书评

暂无评论

Rotation Invariant Quantization for Model Compression

arXiv

引用

arXiv 2023年

作者： Kampeas, Joseph Nahshan, Yury Kremer, Hanoch Lederman, Gil Zaloshinski, Shira Li, Zheng Haleva, Emir Distributed and Parallel Software Lab Huawei Technologies China

Post-training Neural Network (NN) model compression is an attractive approach for deploying large, memory-consuming models on devices with limited memory resources. In this study, we investigate the rate-distortion tradeoff for NN model compression. First, we suggest a Rotation-Invariant Quantization (RIQ) technique that utilizes a single parameter to quantize the entire NN model, yielding a different rate at each layer, i.e., mixed-precision quantization. Then, we prove that our rotation-invariant approach is optimal in terms of compression. We rigorously evaluate RIQ and demonstrate its capabilities on various models and tasks. For example, RIQ facilitates ×19.4 and ×52.9 compression ratios on pre-trained VGG dense and pruned models, respectively, with © 2023, CC BY-NC-ND.

关键词： Neural networks

来源：评论

学校读者我要写书评

暂无评论

Online knowledge distillation with elastic peer

引用

INFORMATION SCIENCES 2022年 583卷 1-13页

作者： Tan, Chao Liu, Jie Natl Univ Def Technol Sci & Parallel & Distributed Proc Lab Changsha 410073 Peoples R China Natl Univ Def Technol Lab Software Engn Complex Syst Changsha 410073 Peoples R China

Knowledge distillation is a highly effective method for transferring knowledge from a cum-bersome teacher network to a lightweight student network. However, teacher networks are not always available. An alternative method called online knowledge distillation, which applies a group of peer networks to learn collaboratively with each other, has been pro -posed previously. In this study, we revisit online knowledge distillation and find that the existing training strategy limits the diversity among peer networks. Thus, online knowledge distillation cannot achieve its full potential. To address this, a novel online knowledge dis-tillation with elastic peer (KDEP) strategy is introduced here. The entire training process is divided into two phases by KDEP. In each phase, a specific training strategy is applied to adjust the diversity to an appropriate degree. Extensive experiments have been conducted on individual benchmarks, including CIFAR-100, CINIC-10, Tiny ImageNet, and Caltech-UCSD Birds. The results demonstrate the superiority of KDEP. For example, when the peer networks are ShuffleNetV2-1.0 and ShuffleNetV2-0.5, the target peer network ShuffleNetV2-0.5 achieves 57:00% top-1 accuracy on Tiny ImageNet via KDEP. (c) 2021 Elsevier Inc. All rights reserved.

关键词： Neural network compression Knowledge distillation Knowledge transfer

来源：评论

学校读者我要写书评

暂无评论

Hard Contrastive Learning for Video Captioning 5

Hard Contrastive Learning for Video Captioning

引用

IEEE 5th International Conference on Electronics and Communication Engineering (ICECE)

作者： Wu, Lilei Liu, Jie Natl Univ Def Technol Sci & Technol Parallel & Distributed Proc Lab Lab Software Engn Complex Syst Changsha Peoples R China

ISBN: (纸本)9781665487894

Maximum likelihood estimation has been widely adopted along with the encoder-decoder framework for video captioning. However, it ignores the structure of sentences and restrains the diversity and distinction of generated captions. To address this issue, we propose a hard contrastive learning (HCL) method for video captioning. Specifically, built on the encoderdecoder framework, we introduce mismatched pairs to learn a reference distribution of video descriptions. The target model on the matched pairs is learned on top the reference model, which improves the distinctiveness of generated captions. In addition, we further boost the distinctiveness of the captions by developing a hard mining technique to select the hardest mismatched pairs within the contrastive learning framework. Finally, the relationships among multiple relevant captions for each video is consider to encourage the diversity of generated captions. The proposed method generates high quality captions which effectively capture the specialties in individual videos. Extensive experiments on two benchmark datasets, i.e., MSVD and MSR-VTT, show that our approach outperforms state-ofthe-art methods.

关键词： Hard Sample Mining Contrastive Learning

来源：评论

学校读者我要写书评

暂无评论

An efficient heterogeneous parallel password recovery system on MT-3000

引用

JOURNAL OF SUPERCOMPUTING 2025年第1期81卷 1-28页

作者： Luo, Yongtao Liu, Jie Gong, Chunye Li, Tun Natl Univ Def Technol Lab Digitizing Software Frontier Equipment Changsha 410073 Peoples R China Natl Univ Def Technol Natl Key Lab Parallel & Distributed Comp Changsha 410073 Peoples R China Natl Univ Def Technol Comp Sch Changsha 410073 Peoples R China Natl Supercomp Ctr Tianjin Tianjin 300457 Peoples R China

Password-based recovery is a widely used method for regaining access to applications or services when passwords are lost or forgotten. It is commonly used in electronic forensics by law enforcement agencies, information acquisition in the commercial sector, and data recovery for individuals. However, as encryption algorithms and complex passwords become more prevalent for security purposes, traditional CPU-based and GPU-based password recovery systems are struggling to meet the time-sensitive requirements for deciphering, and there is an urgent need for a more efficient password recovery system. Therefore, this paper presents an efficient heterogeneous parallel password recovery system based on the MT-3000 multi-zone processor. According to the architectural features of MT-3000, this system adopts a heterogeneous multi-level parallelism strategy, including inter-acceleration cluster data parallelism through MPI, intra-acceleration core data parallelism through the hthreads APIs, and instruction-level parallelism through a very long instruction word manner. Additionally, this system utilizes a unified task allocation mechanism that assigns the initialization and comparison verification tasks to the CPU side, while the accelerator side executes the hash iteration. This approach ensures that the system achieves optimal performance while maintaining its efficiency. The experimental analysis and results confirm that the proposed system significantly improves the recovery efficiency compared to traditional CPU-based and GPU-based systems, and also has an advantage in deciphering speed compared to the most popular hybrid CPU-FPGA-based system. Furthermore, it offers superior scalability, allowing for expansion to more compute nodes, making it a good solution for large-scale password recovery needs.

关键词： MT-3000 Password recovery Heterogeneous computing SIMD

来源：评论

学校读者我要写书评

暂无评论

Accelerated Selective Algebraic Multigrid Method for Fully-Coupled Incompressible Flow Solver

引用

INTERNATIONAL JOURNAL OF COMPUTATIONAL FLUID DYNAMICS 2025年

作者： Liang, Yuechao Guo, Xiao-Wei Zhang, Qingyang Li, Chao Liu, Jie Natl Univ Def Technol Sci & Technol Parallel & Distributed Proc Lab Changsha Peoples R China Natl Univ Def Technol Lab Digitizing Software Frontier Equipment Changsha Peoples R China Natl Univ Def Technol Coll Comp Sci & Technol Changsha Peoples R China

The fully coupled pressure-based algorithm is widely recognised for its superior convergence and robustness in solving incompressible flow problems. However, the increased scale of equations and the difficulty in solving linear systems have limited the widespread use of this algorithm in large-scale simulations. This paper presents an optimised block selective algebraic multigrid method significantly reducing computational complexity. Our approach employs a parallel modified independent set algorithm, allowing each process to perform matrix coarsening individually. Furthermore, an aggressive coarsening strategy is introduced to reduce complexity and enable the solution of larger-scale problems. Numerical experiments demonstrate that the solution time is shortened by 14% to 49% compared to the latest existing methods and outperforms the segregated algorithm. By addressing the computational challenges associated with the selective algebraic multigrid solver, this work unleashes the superior convergence properties of the fully coupled method.

关键词： Fully coupled method velocity-pressure coupling selective algebraic multigrid multigrid parallelisation incompressible flow simulation

来源：评论

学校读者我要写书评

暂无评论

GGN: a model-free, data-driven deep learning framework for reconstructing gene regulatory networks 23

GGN: a model-free, data-driven deep learning framework for r...

引用

13th International Conference on Bioscience, Biochemistry and Bioinformatics (ICBBB)

作者： Mao, Guo Pang, Zhengbin Zuo, Ke Liu, Jie Natl Univ Def Technol Sci & Technol Parallel & Distributed Proc Lab Changsha Peoples R China Natl Univ Def Technol Lab Software Engn Complex Syst Changsha Peoples R China

ISBN: (纸本)9781450398190

Reconstructing gene regulatory networks based on time-series gene expression data is a huge challenge in the field of systems biology. However, the accuracy of traditional methods can be further improved. In practical situations, the gene regulatory network topology and the dynamic rules of genes are not observable. Therefore, reconstructing the underlying network structure and dynamics from observed time-series gene expression data is an important task. In this work, we introduce a new framework, Gumbel Graph Network (GGN), a model-free, data-driven deep learning framework for accomplishing the reconstruction of gene regulatory networks and the reconstruction of gene dynamics. Our model consists of two co-trained parts: a network generator, which generates a discrete network using Gumbel Softmax technique;and a dynamic learner, which uses the generated network and single-step trajectory values to predict the state at the next moment. According to the experimental results on multiple simulated and real datasets, GGN has better performance and stability compared with other state-of-the-art algorithms.

关键词： Gumbel Graph Network Network Generator Gumbel Softmax Technique Gynamic Learner Gene Regulatory Network

来源：评论

学校读者我要写书评

暂无评论

LINEAR LOG-NORMAL ATTENTION WITH UNBIASED CONCENTRATION

arXiv

引用

arXiv 2023年

作者： Nahshan, Yury Kampeas, Joseph Haleva, Emir Distributed and Parallel Software Lab Huawei Technologies United States

Transformer models have achieved remarkable results in a wide range of applications. However, their scalability is hampered by the quadratic time and memory complexity of the self-attention mechanism concerning the sequence length. This limitation poses a substantial obstacle when dealing with long documents or high-resolution images. In this work, we study the self-attention mechanism by analyzing the distribution of the attention matrix and its concentration ability. Furthermore, we propose instruments to measure these quantities and introduce a novel self-attention mechanism, Linear Log-Normal Attention, designed to emulate the distribution and concentration behavior of the original self-attention. Our experimental results on popular natural language benchmarks reveal that our proposed Linear Log-Normal Attention outperforms other linearized attention alternatives, offering a promising avenue for enhancing the scalability of transformer models. © 2023, CC BY.

关键词： Scalability

来源：评论

学校读者我要写书评

暂无评论

ST-PINN: A Self-Training Physics-Informed Neural Network for Partial Differential Equations

ST-PINN: A Self-Training Physics-Informed Neural Network for...

引用

International Joint Conference on Neural Networks (IJCNN)

作者： Yan, Junjun Chen, Xinhai Wang, Zhichao Zhoui, Enqiang Liu, Jie Natl Univ Def Technol Sci & Technol Parallel & Distributed Proc Lab Changsha 410073 Peoples R China Natl Univ Def Technol Lab Digitizing Software Frontier Equipment Changsha 410073 Peoples R China

ISBN: (纸本)9781665488679

Partial differential equations (PDEs) are an essential computational kernel in physics and engineering. With the advance of deep learning, physics-informed neural networks (PINNs), as a mesh-free method, have shown great potential for fast PDE solving in various applications. To address the issue of low accuracy and convergence problems of existing PINNs, we propose a self-training physics-informed neural network, ST-PINN. Specifically, ST-PINN introduces a pseudo label based self-learning algorithm during training. It employs governing equation as the pseudo-labeled evaluation index and selects the highest confidence examples from the sample points to attach the pseudo labels. To our best knowledge, we are the first to incorporate a self-training mechanism into physics-informed learning. We conduct experiments on five PDE problems in different fields and scenarios. The results demonstrate that the proposed method allows the network to learn more physical information and benefit convergence. The ST-PINN outperforms existing physics-informed neural network methods and improves the accuracy by a factor of 1.33x-2.54x.

关键词： partial differential equations physics-informed neural networks pseudo label self-training

来源：评论

学校读者我要写书评

暂无评论

HyperPart: A Hypergraph-Based Abstraction for Deduplicated Storage Systems

引用

IEEE TRANSACTIONS ON CLOUD COMPUTING 2025年第1期13卷 46-60页

作者： Cheng, Geyao Xia, Junxu Luo, Lailong Mi, Haibo Guo, Deke Ma, Richard T. B. Natl Univ Def Technol Coll Informat & Commun State Key Lab Complex & Crit Software Environm Wuhan 430019 Peoples R China Natl Univ Def Technol Natl Key Lab Informat Syst Engn Changsha 410073 Peoples R China Natl Univ Def Technol Natl Lab Parallel & Distributed Proc Changsha 410073 Peoples R China Natl Univ Singapore Sch Comp Singapore 119077 Singapore

Currently, deduplication techniques are utilized to minimize the space overhead by deleting redundant data blocks across large-scale servers in data centers. However, such a process exacerbates the fragmentation of data blocks, causing more cross-server file retrievals with plummeting retrieval throughput. Some attempts prefer better file retrieval performance by confining all blocks of a file to one single server, resulting in non-trivial space consumption for more replicated blocks across servers. An ideal network storage system, in effect, should take both the deduplication and retrieval performance into account by implementing reasonable assignment of the detected unique blocks. Such a fine-grained assignment requires an accurate and comprehensive abstraction of the files, blocks, and the file-block affiliation relationships. To achieve this, we innovatively design the weighted hypergraph to profile the multivariate data correlations. With this delicate abstraction in place, we propose HyperPart, which elegantly transforms this complex block allocation problem into a hypergraph partition problem. For more general scenarios with dynamic file updates, we further propose a two-phase incremental hypergraph repartition scheme, which mitigates the performance degradation with minimal migration volume. We implement a prototype system of HyperPart, and the experiment results validate that it saves around 50% of the storage space and improves the retrieval throughput by approximately 30% of state-of-the-art methods under the balance constraints.

关键词： Servers Fingerprint recognition Throughput Metadata Resource management Switches Data centers Indexing Ethernet Correlation Network storage systems data deduplication hypergraph partition

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：