With the increasing number of airborne targets and the shift towards clustered combat modes, radar systems are facing the challenge of dealing with a large number of targets within limited time. Effectively utilizing ...
详细信息
Quantizing the activation, weight, and gradient to 4-bit is promising to accelerate neural network training. However, existing 4-bit training methods require custom numerical formats which are not supported by contemp...
详细信息
ISBN:
(纸本)9781713899921
Quantizing the activation, weight, and gradient to 4-bit is promising to accelerate neural network training. However, existing 4-bit training methods require custom numerical formats which are not supported by contemporary hardware. In this work, we propose a training method for transformers with all matrix multiplications implemented with the INT4 arithmetic. Training with an ultra-low INT4 precision is challenging. To achieve this, we carefully analyze the specific structures of activation and gradients in transformers to propose dedicated quantizers for them. For forward propagation, we identify the challenge of outliers and propose a Hadamard quantizer to suppress the outliers. For backpropagation, we leverage the structural sparsity of gradients by proposing bit splitting and leverage score sampling techniques to quantize gradients accurately. Our algorithm achieves competitive accuracy on a wide range of tasks including natural language understanding, machine translation, and image classification. Unlike previous 4-bit training methods, our algorithm can be implemented on the current generation of GPUs. Our prototypical linear operator implementation is up to 2.2 times faster than the FP16 counterparts and speeds up the training by 17.8% on average for sufficiently large models. Our code is available at https://***/xijiu9/Train_Transformers_with_INT4.
Vibration-based damage detection has become one of the principal practices to prevent structural collapses in civil, mechanical, and other engineering disciplines. Meanwhile, with the advancement of computing technolo...
详细信息
ISBN:
(纸本)9780791886274
Vibration-based damage detection has become one of the principal practices to prevent structural collapses in civil, mechanical, and other engineering disciplines. Meanwhile, with the advancement of computing technology, various machine learning (ML) approaches have been applied toward structural damage detection through the application of post-processingalgorithms. To accurately predict damages with ML, large amounts of structural response data are collected from a series of sensors attached to the structure. Therefore, the damage diagnosis requires high computational efforts. To address such an issue, this paper presents a revolutionary approach utilizing an image-based pre-trained convolutional neural network (CNN) to detect bridge damage locations and severities. Our research adopted scalograms from wavelet transform to convert structure acceleration data into image data. Compared with the traditional frequency analysis derived from the Fourier transform, the new method maintains both spatial and temporal information from the original structural behaviors. To generate CNN learning features, six channels of acceleration data are gathered from six strategically selected points of a finite element (FE) bridge model. Two pre-trained CNN, AlexNet and Resnet, are selected to conduct transfer machine learning for higher training efficiency. The performances of the proposed method are assessed with various damage scenarios. The prediction accuracies of AlexNet and Resnet are 98% and 100%, respectively.
Objective image quality assessment measures were extensively used to evaluate the performance of different imageprocessing and analysis algorithms. However, they are application-driven. In contrast, subjective assess...
详细信息
The proceedings contain 87 papers. The topics discussed include: Auto-FP: an experimental study of automated feature preprocessing for tabular data;Data-CASE: grounding data regulations for compliant data processing s...
ISBN:
(纸本)9783893180950
The proceedings contain 87 papers. The topics discussed include: Auto-FP: an experimental study of automated feature preprocessing for tabular data;Data-CASE: grounding data regulations for compliant data processingsystems;data coverage for detecting representation bias in image datasets: a crowdsourcing approach;balancing utility and fairness in submodular maximization;stateful entities: object-oriented cloud applications as distributed dataflows;learning over sets for databases;a new PET for data collection via forms with data minimization, full accuracy and informed consent;adaptive compression for databases;analysis of open government datasets from a data design and integration perspective;fine-grained geo-obfuscation to protect workers’ location privacy in time-sensitive spatial crowdsourcing;and a framework to evaluate early time-series classification algorithms.
The proceedings contain 180 papers. The topics discussed include: analysis of medical data and machine-learning algorithms from the perspective of public-goods models of data-provision decision making;price competitio...
The proceedings contain 180 papers. The topics discussed include: analysis of medical data and machine-learning algorithms from the perspective of public-goods models of data-provision decision making;price competition for service provision with different bargaining abilities;survey on stakeholder cooperative behavior for designing voluntary medical data provision motivation mechanisms;analysis of excellent service systems from co-creation and emergent synthesis perspective;meta-heuristic scheduling auction applying distributed genetic algorithm;the impact of characteristic function of Shapley value mechanism in distributed machine learning environment for equipment diagnosis;automatic measurement of timber diameter using imageprocessing;intelligent scheduling based on discrete-time simulation using machine learning;and a fuzzy synthesis approach for hierarchical decision analysis to select optimum repair technique.
In traditional interval-set information systems (ISISs), each attribute is single-scale. However, processing and analyzing data at different scales is often necessary for practical applications. This paper introduces ...
详细信息
Visual pollution is a significant obstacle in the modern era, where the world is advancing towards increasingly diverse inventions. These inventions require a suitable environment to achieve accurate outcomes. Artific...
详细信息
ISBN:
(纸本)9783031664304;9783031664311
Visual pollution is a significant obstacle in the modern era, where the world is advancing towards increasingly diverse inventions. These inventions require a suitable environment to achieve accurate outcomes. Artificial intelligence has already permeated all fields and interests of life;similarly, visual pollution also needs to be addressed properly. Visual pollution often creates obstacles in performing various tasks. To mitigate these issues, an artificial intelligence-based model will play a vital role. This work deals with detecting visual pollution using an artificial intelligence-based algorithm to apply practical solutions that enhance urban public scenery. In the first step, a dataset is chosen from an authorized organization;specifically, the data is sourced from Mendeley, named the Saudi Arabia Public Roads Visual Pollution Dataset 2023. The second step involves data scaling and background removal from training images to facilitate learning in AI models. In the third step, the dataset is processed using Random Forest and support vector machine algorithms to visualize the model's accuracy results. The support vector machine demonstrates better performance compared to the Random Forest.
The rapid development of signal processing (SP) technology, especially the continuous innovation of audio signal recognition and analysis algorithms, has led to a wide variety of application areas. Under this backgrou...
详细信息
With the increasing availability of unmanned aerial vehicles (UAV), their potential misuse has become a serious concern, posing a threat to public security. Existing tracking methods have limitations in detecting UAV ...
详细信息
暂无评论