The attention mechanism has become a pivotal component in artificial intelligence, significantly enhancing the performance of deep learning applications. However, its quadratic computational complexity and intricate c...
详细信息
The attention mechanism has become a pivotal component in artificial intelligence, significantly enhancing the performance of deep learning applications. However, its quadratic computational complexity and intricate computations lead to substantial inefficiencies when processing long sequences. To address these challenges, we introduce Attar, a resistive random access memory(RRAM)-based in-memory accelerator designed to optimize attention mechanisms through software-hardware co-optimization. Attar leverages efficient Top-k pruning and quantization strategies to exploit the sparsity and redundancy of attention matrices, and incorporates an RRAM-based in-memory softmax engine by harnessing the versatility of the RRAM crossbar. Comprehensive evaluations demonstrate that Attar achieves a performance improvement of up to 4.88× and energy saving of 55.38% over previous computing-in-memory(CIM)-based accelerators across various models and datasets while maintaining comparable accuracy. This work underscores the potential of in-memory computing to enhance the efficiency of attention-based models without compromising their effectiveness.
The accelerated expansion of the Internet of Things (IoT) has raised critical challenges associated with privacy, security, and data integrity, specifically in infrastructures such as smart cities or smart manufacturi...
详细信息
AI is increasingly being used to moderate player behaviour in online multiplayer games, working to identify and respond to toxic and problematic conduct with greater efficiency and accuracy than existing automated sys...
详细信息
We often compare time-dependent data elastically such that some compression or dilation along the time dimension can be ignored, for example, spatial trajectories of vehicles moving at different speeds or acceleromete...
详细信息
Instance segmentation is a critical component of medical image analysis, enabling tasks such as tissue and organ delineation, and disease detection. This paper provides a detailed comparative analysis of two fine-tune...
详细信息
While video sensing, performed by resource-constrained pervasive devices, is a key enabler of many machine intelligence applications, the high energy and bandwidth overheads of streaming video transmission continue to...
详细信息
The utilization of visual attention enhances the performance of image classification *** attentionbased models have demonstrated notable performance,but many of these models exhibit reduced accuracy when confronted wi...
详细信息
The utilization of visual attention enhances the performance of image classification *** attentionbased models have demonstrated notable performance,but many of these models exhibit reduced accuracy when confronted with inter-class and intra-class similarities and ***-Controlled Differential Equations(N-CDE’s)and Neural Ordinary Differential Equations(NODE’s)are extensively utilized within this ***’s possesses the capacity to effectively illustrate both inter-class and intra-class similarities and differences with enhanced *** this end,an attentive neural network has been proposed to generate attention maps,which uses two different types of N-CDE’s,one for adopting hidden layers and the other to generate attention *** distinct attention techniques are implemented including time-wise attention,also referred to as bottom N-CDE’s;and element-wise attention,called topN-CDE’***,a trainingmethodology is proposed to guarantee that the training problem is sufficiently *** classification tasks including fine-grained visual classification andmulti-label classification,are utilized to evaluate the *** proposedmethodology is employed on five publicly available datasets,including CUB-200-2011,ImageNet-1K,PASCAL VOC 2007,PASCAL VOC 2012,and MS *** obtained visualizations have demonstrated that N-CDE’s are better appropriate for attention-based activities in comparison to conventional NODE’s.
People are increasingly concerned about their mental health wellness. Scientific studies suggest that online counselling for anxiety and depression is just as effective as in-person treatment. Additionally, journaling...
详细信息
Evaluating physicians’ performance is one of the fundamental pillars of improving the quality of healthcare in medical institutions, as it contributes to measuring their ability to provide appropriate treatment, inte...
详细信息
The evaluation of disease severity through endoscopy is pivotal in managing patients with ulcerative colitis,a condition with significant clinical ***,endoscopic assessment is susceptible to inherent variations,both w...
详细信息
The evaluation of disease severity through endoscopy is pivotal in managing patients with ulcerative colitis,a condition with significant clinical ***,endoscopic assessment is susceptible to inherent variations,both within and between observers,compromising the reliability of individual *** study addresses this challenge by harnessing deep learning to develop a robust model capable of discerning discrete levels of endoscopic disease *** initiate this endeavor,a multi-faceted approach is embarked *** dataset is meticulously preprocessed,enhancing the quality and discriminative features of the images through contrast limited adaptive histogram equalization(CLAHE).A diverse array of data augmentation techniques,encompassing various geometric transformations,is leveraged to fortify the dataset’s diversity and facilitate effective feature extraction.A fundamental aspect of the approach involves the strategic incorporation of transfer learning principles,harnessing a modified ResNet-50 *** augmentation,informed by domain expertise,contributed significantly to enhancing the model’s classification *** outcome of this research endeavor yielded a highly promising model,demonstrating an accuracy rate of 86.85%,coupled with a recall rate of 82.11%and a precision rate of 89.23%.
暂无评论