The attention mechanism has become a pivotal component in artificial intelligence, significantly enhancing the performance of deep learning applications. However, its quadratic computational complexity and intricate c...
详细信息
The attention mechanism has become a pivotal component in artificial intelligence, significantly enhancing the performance of deep learning applications. However, its quadratic computational complexity and intricate computations lead to substantial inefficiencies when processing long sequences. To address these challenges, we introduce Attar, a resistive random access memory(RRAM)-based in-memory accelerator designed to optimize attention mechanisms through software-hardware co-optimization. Attar leverages efficient Top-k pruning and quantization strategies to exploit the sparsity and redundancy of attention matrices, and incorporates an RRAM-based in-memory softmax engine by harnessing the versatility of the RRAM crossbar. Comprehensive evaluations demonstrate that Attar achieves a performance improvement of up to 4.88× and energy saving of 55.38% over previous computing-in-memory(CIM)-based accelerators across various models and datasets while maintaining comparable accuracy. This work underscores the potential of in-memory computing to enhance the efficiency of attention-based models without compromising their effectiveness.
Domain-adaptive point cloud semantic segmentation (PCSS) is crucial for high-level autonomous driving. However, supervised deep learning methods are often constrained by training data and suffer from poor generalizati...
详细信息
order to help investors understand the credit status of target corporations and reduce investment risks,the corporate credit rating model has become an important evaluation tool in the financial *** models are based o...
详细信息
order to help investors understand the credit status of target corporations and reduce investment risks,the corporate credit rating model has become an important evaluation tool in the financial *** models are based on statistical learning,machine learning and deep learning especially graph neural networks(GNNs).However,we found that only few models take the hierarchy,heterogeneity or unlabeled data into account in the actual corporate credit rating ***,we propose a novel framework named hierarchical heterogeneous graph neural networks(HHGNN),which can fully model the hierarchy of corporate features and the heterogeneity of relationships between *** addition,we design an adversarial learning block to make full use of the rich unlabeled samples in the financial *** experiments conducted on the public-listed corporate rating dataset prove that HHGNN achieves SOTA compared to the baseline methods.
Sparse Knowledge Graph(KG)scenarios pose a challenge for previous Knowledge Graph Completion(KGC)methods,that is,the completion performance decreases rapidly with the increase of graph *** problem is also exacerbated ...
详细信息
Sparse Knowledge Graph(KG)scenarios pose a challenge for previous Knowledge Graph Completion(KGC)methods,that is,the completion performance decreases rapidly with the increase of graph *** problem is also exacerbated because of the widespread existence of sparse KGs in practical *** alleviate this challenge,we present a novel framework,LR-GCN,that is able to automatically capture valuable long-range dependency among entities to supplement insufficient structure features and distill logical reasoning knowledge for sparse *** proposed approach comprises two main components:a GNN-based predictor and a reasoning path *** reasoning path distiller explores high-order graph structures such as reasoning paths and encodes them as rich-semantic edges,explicitly compositing long-range dependencies into the *** step also plays an essential role in densifying KGs,effectively alleviating the sparse ***,the path distiller further distills logical reasoning knowledge from these mined reasoning paths into the *** two components are jointly optimized using a well-designed variational EM *** experiments and analyses on four sparse benchmarks demonstrate the effectiveness of our proposed method.
In the daily application of an iris-recognition-at-a-distance(IAAD)system,many ocular images of low quality are *** the iris part of these images is often not qualified for the recognition requirements,the more access...
详细信息
In the daily application of an iris-recognition-at-a-distance(IAAD)system,many ocular images of low quality are *** the iris part of these images is often not qualified for the recognition requirements,the more accessible periocular regions are a good complement for *** further boost the performance of IAAD systems,a novel end-to-end framework for multi-modal ocular recognition is *** proposed framework mainly consists of iris/periocular feature extraction and matching,unsupervised iris quality assessment,and a score-level adaptive weighted fusion ***,ocular feature reconstruction(OFR)is proposed to sparsely reconstruct each probe image by high-quality gallery images based on proper feature ***,a brand new unsupervised iris quality assessment method based on random multiscale embedding robustness is *** from the existing iris quality assess-ment methods,the quality of an iris image is measured by its robustness in the embedding *** last,the fusion strategy exploits the iris quality score as the fusion weight to coalesce the complementary information from the iris and periocular *** experi-mental results on ocular datasets prove that the proposed method is obviously better than unimodal biometrics,and the fusion strategy can significantly improve therecognition performance.
Path optimization problem is a classical issue in urban infrastructure. Heuristic strategies may not provide immediate qualitative solutions to complex problems instantly. This paper gave a novel reinforcement learnin...
详细信息
For the diagnostics and health management of lithium-ion batteries,numerous models have been developed to understand their degradation *** models typically fall into two categories:data-driven models and physical mode...
详细信息
For the diagnostics and health management of lithium-ion batteries,numerous models have been developed to understand their degradation *** models typically fall into two categories:data-driven models and physical models,each offering unique advantages but also facing ***-informed neural networks(PINNs)provide a robust framework to integrate data-driven models with physical principles,ensuring consistency with underlying physics while enabling generalization across diverse operational *** study introduces a PINN-based approach to reconstruct open circuit voltage(OCV)curves and estimate key ageing parameters at both the cell and electrode *** parameters include available capacity,electrode capacities,and lithium inventory *** proposed method integrates OCV reconstruction models as functional components into convolutional neural networks(CNNs)and is validated using a public *** results reveal that the estimated ageing parameters closely align with those obtained through offline OCV tests,with errors in reconstructed OCV curves remaining within 15 *** demonstrates the ability of the method to deliver fast and accurate degradation diagnostics at the electrode level,advancing the potential for precise and efficient battery health management.
Lithium plating is a detrimental phenomenon in lithium-ion cells that compromises both functionality and *** study investigates electro-chemo-mechanical behaviors of lithium plating in lithium iron phosphate pouch cel...
详细信息
Lithium plating is a detrimental phenomenon in lithium-ion cells that compromises both functionality and *** study investigates electro-chemo-mechanical behaviors of lithium plating in lithium iron phosphate pouch cells under different external *** force microscopy nanoindentation is performed on the graphite electrode to analyze the influence of external pressure on solid-electrolyte interphase(SEI),revealing that the mechanical strength of SEI,indicated by Young's modulus,increases with the presence of external ***,an improved phase field model for lithium plating is developed by incorporating electrochemical parameterization based on nonequilibrium *** results demonstrate that higher pressure promotes lateral lithium deposition,covering a larger area of ***,electrochemical impedance spectroscopy and thickness measurements of the pouch cells are conducted during overcharge,showing that external pressure suppresses gas generation and thus increases the proportion of lithium deposition among galvanostatic overcharge *** integrating experimental results with numerical simulations,it is demonstrated that moderate pressure mitigates SEI damage during lithium plating,while both insufficient and excessive pressure may exacerbate *** study offers new insights into optimizing the design and operation of lithium iron phosphate pouch cells under external pressures.
The prediction of molecular properties is a fundamental task in the field of drug ***,graph neural networks(GNNs)have been gaining prominence in this *** a molecule tends to have multiple correlated properties,there i...
详细信息
The prediction of molecular properties is a fundamental task in the field of drug ***,graph neural networks(GNNs)have been gaining prominence in this *** a molecule tends to have multiple correlated properties,there is a great need to develop the multi-task learning ability of ***,limited by expensive and time-consuming human annotations,collecting complete labels for each task is *** a result,most existing benchmarks involve many missing labels in training data,and the performance of GNNs is impaired due to the lack of sufficient supervision *** overcome this obstacle,we propose to improve multi-task molecular property prediction by missing label ***,a bipartite graph is first introduced to model the molecule-task co-occurrence ***,the imputation of missing labels is transformed into predicting missing edges on this bipartite *** predict the missing edges,a graph neural network is devised,which can learn the complex molecule-task co-occurrence *** that,we select reliable pseudo labels according to the uncertainty of the prediction *** with enough and reliable supervision information,our approach achieves state-of-the-art performance on a variety of real-world datasets.
Non-volatile memories(NVMs)provide lower latency and higher bandwidth than block ***,NVMs are byte-addressable and provide persistence that can be used as memory-level storage devices(non-volatile main memory,NVMM).Th...
详细信息
Non-volatile memories(NVMs)provide lower latency and higher bandwidth than block ***,NVMs are byte-addressable and provide persistence that can be used as memory-level storage devices(non-volatile main memory,NVMM).These features change storage hierarchy and allow CPU to access persistent data using load/store ***,we can directly build a file system on ***,traditional file systems are designed based on slow block *** use a deep and complex software stack to optimize file system *** design results in software overhead being the dominant factor affecting NVMM file ***,scalability,crash consistency,data protection,and cross-media storage should be reconsidered in NVMM file *** survey existing work on optimizing NVMM file ***,we analyze the problems when directly using traditional file systems on NVMM,including heavy software overhead,limited scalability,inappropriate consistency guarantee techniques,***,we summarize the technique of 30 typical NVMM file systems and analyze their advantages and ***,we provide a few suggestions for designing a high-performance NVMM file system based on real hardware Optane DC persistent memory ***,we suggest applying various techniques to reduce software overheads,improving the scalability of virtual file system(VFS),adopting highly-concurrent data structures(e.g.,lock and index),using memory protection keys(MPK)for data protection,and carefully designing data placement/migration for cross-media file system.
暂无评论