Big data management refers to the processes and technologies used to collect, store, organize, and analyses large and complex data sets. This includes data ingestion, storage, and processing to data governance, securi...
详细信息
large language models (LLMs) have garnered significant attention in both the AI community and beyond. Among these, the Generative Pre-trained Transformer (GPT) has emerged as the dominant architecture, spawning numero...
详细信息
ISBN:
(纸本)9798350387117;9798350387124
large language models (LLMs) have garnered significant attention in both the AI community and beyond. Among these, the Generative Pre-trained Transformer (GPT) has emerged as the dominant architecture, spawning numerous variants. However, these variants have undergone pre-training under diverse conditions, including variations in input data, data preprocessing, and training methodologies, resulting in a lack of controlled comparative studies. Here we meticulously examine two prominent open-sourced GPT architectures, GPT-NeoX and LLaMA, leveraging the computational power of Frontier, the world's first Exascale supercomputer. Employing the same materials science text corpus and a comprehensive end-to-end pipeline, we conduct a comparative analysis of their training and downstream performance. Our efforts culminate in achieving state-of-the-art performance on a challenging materials science benchmark. Furthermore, we investigate the computation and energy efficiency, and propose a computationally efficient method for architecture design. To our knowledge, these pre-trained models represent the largest available for materials science. Our findings provide practical guidance for building LLMs on HPC platforms.
In the structure from motion, the viewing graph is a graph where the vertices correspond to cameras (or images) and the edges represent the fundamental matrices. We provide a new formulation and an algorithm for deter...
详细信息
In the structure from motion, the viewing graph is a graph where the vertices correspond to cameras (or images) and the edges represent the fundamental matrices. We provide a new formulation and an algorithm for determining whether a viewing graph is solvable, i.e., uniquely determines a set of projective cameras. The known theoretical conditions either do not fully characterize the solvability of all viewing graphs, or are extremely difficult to compute because they involve solving a system of polynomial equations with a large number of unknowns. The main result of this paper is a method to reduce the number of unknowns by exploiting cycle consistency. We advance the understanding of solvability by (i) finishing the classification of all minimal graphs up to 9 nodes, (ii) extending the practical verification of solvability to minimal graphs with up to 90 nodes, (iii) finally answering an open research question by showing that finite solvability is not equivalent to solvability, and (iv) formally drawing the connection with the calibrated case (i.e., parallel rigidity). Finally, we present an experiment on real data that shows that unsolvable graphs may appear in practice.
large scale wildfires, intensified by climate change, cause severe threats to human life, property, and ecosystems, with potential for secondary damages. Accurate detection and calculation wildfire areas is important,...
详细信息
ISBN:
(纸本)9798350360332;9798350360325
large scale wildfires, intensified by climate change, cause severe threats to human life, property, and ecosystems, with potential for secondary damages. Accurate detection and calculation wildfire areas is important, necessitating efficient monitoring through satellite imagery and deep learning. However, the application of deep learning models has been limited, lacking comprehensive quantitative performance evaluation reports. This study focuses on a comparative analysis of performance improvement through model and data design. Utilizing U-Net, HRNet and Swin transformer, we created a model to predict burned area in California, USA. To improve detection performance, transfer learning was applied, and spectral indices like NDVI and NBR, considering vegetation fertility and ground moisture, were used as input images. This deep learning methodology, if further developed, is expected to serve as a foundation for swift wildfire identification and recovery plan establishment.
This work presents an approach for the automatic detection of locally turbulent vortices within turbulent 2D flows such as instabilites. First, given a time step of the flow, methods from Topological dataanalysis (TD...
详细信息
ISBN:
(数字)9798331516925
ISBN:
(纸本)9798331516932
This work presents an approach for the automatic detection of locally turbulent vortices within turbulent 2D flows such as instabilites. First, given a time step of the flow, methods from Topological dataanalysis (TDA) are leveraged to extract the geometry of the vortices. Specifically, the enstrophy of the flow is simplified by topological persistence, and the vortices are extracted by collecting the basins of the simplified enstrophy 's Morse complex. Next, the local kinetic energy power spectrum is computed for each vortex. We introduce a set of indicators based on the kinetic energy power spectrum to estimate the correlation between the vortex's behavior and that of an idealized turbulent vortex. Our preliminary experiments show the relevance of these indicators for distinguishing vortices which are turbulent from those which have not yet reached a turbulent state and thus known as laminar.
The scarcity of a sufficiently large and representative hyperspectral image dataset is a substantial obstacle to the effective development of algorithms for remote sensing applications. Hyperspectral images can provid...
详细信息
ISBN:
(纸本)9798350360332;9798350360325
The scarcity of a sufficiently large and representative hyperspectral image dataset is a substantial obstacle to the effective development of algorithms for remote sensing applications. Hyperspectral images can provide rich spectral information for various tasks, such as land cover classification, vegetation monitoring, and environmental assessment. However, the limited availability of diverse and well-annotated hyperspectral datasets hinders the development and optimization of these models in this domain. For this purpose, the generation of synthetic hyperspectral images has emerged as a pivotal area of research. This paper aims to introduce a preliminary analysis of various AI-based methodologies specifically crafted to generate synthetic PRISMA hyperspectral images derived from Sentinel-2 data. By exploring innovative approaches, this study aims to develop novel techniques for creating synthetic datasets, providing valuable insights into the potential of synthetic hyperspectral imagery for algorithm training and evaluation in the absence of extensive realworld hyperspectral datasets.
The architectural design-space exploration (or DSE) process-whether manual or automated-benefits greatly from knowing the limits of the metrics of interest in advance. data movement is rapidly emerging as a critical m...
详细信息
ISBN:
(纸本)9798350326598;9798350326581
The architectural design-space exploration (or DSE) process-whether manual or automated-benefits greatly from knowing the limits of the metrics of interest in advance. data movement is rapidly emerging as a critical metric for DSE due to its increasing impact on both performance and energy efficiency. Unfortunately, the commonly used algorithmic minimum (or "compulsory misses") limit for data movement is extremely loose, limiting its utility in design-space search. In this paper, we present Orojenesis, an approach to compute data movement limits (or bounds) for tensor algorithms. Unlike algorithmic-minimum bounds, Orojenesis comprehends reuse and the ability of a buffer (such as a cache or scratchpad) to exploit reuse to reduce data movement. Orojenesis provides a bound that no dataflow or mapping can possibly exceed under varying on-chip buffer capacity constraints, including mappings that fuse a sequence of tensor operations to exploit producer-consumer reuse. Orojenesis produces a plot that shows the relationship between a buffer's size and the lower data movement limit to/from the next level in a memory hierarchy. This plot, dubbed a ski-slope diagram, allows designers to gain critical insights into the behavior of a workload as a function of storage capacity. This analysis can inform early high-level design decisions before embarking on thorough design space searches. We use Orojenesis to analyze a set of valuable tensor algorithms including batched and grouped matrix multiplications, convolutions, and sequences of operations in large Language Models (LLMs). Our analysis reveals a range of architectural insights, including the fact that attainable data movement can be orders-of-magnitude higher than algorithmic minimum, that there exists a sweet spot between SRAM and compute resource provisioning for optimal throughput, and that up to 5.6x data movement reduction can be achieved with fusion with a buffer capacity of 320MB for the GPT-3-6.7b LLM.
Financial sentiment analysis is the task of evaluating and quantifying the emotions and opinions expressed in financial news, reports, or social media to help investors and institutions make informed decisions. Financ...
详细信息
An important danger to both the environment and human health is air pollution, necessitating the need for reliable prediction models. This study introduces a novel approach to enhancing air quality prediction using LS...
详细信息
暂无评论