The attention mechanism has become a pivotal component in artificial intelligence, significantly enhancing the performance of deep learning applications. However, its quadratic computational complexity and intricate c...
详细信息
The attention mechanism has become a pivotal component in artificial intelligence, significantly enhancing the performance of deep learning applications. However, its quadratic computational complexity and intricate computations lead to substantial inefficiencies when processing long sequences. To address these challenges, we introduce Attar, a resistive random access memory(RRAM)-based in-memory accelerator designed to optimize attention mechanisms through software-hardware co-optimization. Attar leverages efficient Top-k pruning and quantization strategies to exploit the sparsity and redundancy of attention matrices, and incorporates an RRAM-based in-memory softmax engine by harnessing the versatility of the RRAM crossbar. Comprehensive evaluations demonstrate that Attar achieves a performance improvement of up to 4.88× and energy saving of 55.38% over previous computing-in-memory(CIM)-based accelerators across various models and datasets while maintaining comparable accuracy. This work underscores the potential of in-memory computing to enhance the efficiency of attention-based models without compromising their effectiveness.
In this paper,we study a class of online continuous optimization *** each round,the utility function is the sum of a weakly diminishing-returns(DR)submodular function and a concave function,certain cost associated wit...
详细信息
In this paper,we study a class of online continuous optimization *** each round,the utility function is the sum of a weakly diminishing-returns(DR)submodular function and a concave function,certain cost associated with the action will occur,and the problem has total limited *** the two methods,the penalty function and Frank-Wolfe strategies,we present an online method to solve the considered *** appropriate stepsize and penalty parameters,the performance of the online algorithm is guaranteed,that is,it achieves sub-linear regret bound and certain mild constraint violation bound in expectation.
As a result of its aggressive nature and late identification at advanced stages, lung cancer is one of the leading causes of cancer-related deaths. Lung cancer early diagnosis is a serious and difficult challenge that...
详细信息
Solar flares are one of the strongest outbursts of solar activity,posing a serious threat to Earth’s critical infrastructure,such as communications,navigation,power,and ***,it is essential to accurately predict solar...
详细信息
Solar flares are one of the strongest outbursts of solar activity,posing a serious threat to Earth’s critical infrastructure,such as communications,navigation,power,and ***,it is essential to accurately predict solar flares in order to ensure the safety of human ***,the research focuses on two directions:first,identifying predictors with more physical information and higher prediction accuracy,and second,building flare prediction models that can effectively handle complex observational *** terms of flare observability and predictability,this paper analyses multiple dimensions of solar flare observability and evaluates the potential of observational parameters in *** flare prediction models,the paper focuses on data-driven models and physical models,with an emphasis on the advantages of deep learning techniques in dealing with complex and high-dimensional *** reviewing existing traditional machine learning,deep learning,and fusion methods,the key roles of these techniques in improving prediction accuracy and efficiency are *** prevailing challenges,this study discusses the main challenges currently faced in solar flare prediction,such as the complexity of flare samples,the multimodality of observational data,and the interpretability of *** conclusion summarizes these findings and proposes future research directions and potential technology advancement.
Diabetes is a long-term illness that results in a variety of chronic body damage, such as kidney failure, heart problems, eye damage, depression, and nerve damage. This disease is caused by several risk factors, ...
详细信息
The emergence of multimodal disease risk prediction signifies a pivotal shift towards healthcare by integrating information from various sources and enhancing the reliability of predicting susceptibility to specific d...
详细信息
The disease that contains the highest mortality and morbidity across the world is cardiac disease. Annually millions of people are affected and deaths take place due to cardiac diseases worldwide. There are various di...
详细信息
In the realm of decision-making for defense and security applications,it is paramount to swiftly and accurately identify the intentions of incoming *** identification methods predominantly focus on single-target appli...
详细信息
In the realm of decision-making for defense and security applications,it is paramount to swiftly and accurately identify the intentions of incoming *** identification methods predominantly focus on single-target applications and overlook the perturbations introduced by measurement *** this study,we propose a novel concept:the Dynamic Distribution Probability(DDP)image,which is constructed using the estimated state and its covariance *** grayscale pixel value within the image signifies the probability of the presence of the agent within the *** proposed identification scheme integrates the use of Extended Kalman Filter(EKF),Convolutional Neural Network(CNN),Back Propagation(BP)network,and Gated Recurrent Unit(GRU)***,the DDP image is processed through a CNN to distill the formation characteristics,and the estimated swarm state from EKF is inputted into a BP network to deduce the kinematic *** outputs from both networks are summed and subsequently channeled into a GRU network to capture temporal *** numerical simulations and flight experiments are presented to demonstrate the robust anti-noise capability of the proposed scheme compared with conventional methods,as well as its superior training efficiency.
Recent progress in maximizing submodular functions with a cardinality constraint through centralized and streaming modes has demonstrated a wide range of applications and also developed comprehensive theoretical *** s...
详细信息
Recent progress in maximizing submodular functions with a cardinality constraint through centralized and streaming modes has demonstrated a wide range of applications and also developed comprehensive theoretical *** submodularity was investigated to capture the diversity and representativeness of the utilities,and the monotonicity has the advantage of improving the *** submodular optimization models were developed in the latest studies(such as a house on fire),which aimed to sieve subsets with constraints to optimize regularized *** study is motivated by the setting in which the input stream is partitioned into several disjoint parts,and each part has a limited size constraint.A first threshold-based bicriteria(1/3,2/3/)-approximation for the problem is provided.
Online social networks are becoming more and more popular, according to recent trends. The user's primary concern is the secure preservation of their data and privacy. A well-known method for preventing individual...
详细信息
暂无评论