Adept data management, as well as analysis, hold an increasing significant role in fast progressing filed of financial modeling. This paper is based on major elements such as relational databases, big data and cloud c...
详细信息
ISBN:
(纸本)9798350368130
Adept data management, as well as analysis, hold an increasing significant role in fast progressing filed of financial modeling. This paper is based on major elements such as relational databases, big data and cloud computing, each of them is presented from a perspective concerning financial modeling. The proposed solution is based on our investigation into the usability of each technology, optimal contexts for their application, and the methods for effectively interconnecting them. Each element is described in terms of operational efficiency, integration and overall performance, with machine learning techniques as means of assessment. Besides the thorough presentation of these technologies, the paper includes an analysis of their implementation so as to improve financial modeling of financial institutions. The interconnectivity and the correct utilization of the application of this technologies may prove of utmost importance because of relational databases, big data, and cloud computing, which represent major data-driven elements for the business environment.
The proceedings contain 17 papers. The topics discussed include: large-scale graphs community detection using spark GraphFrames;design and development of XiveNet: a hybrid CAN research testbed;SmartShip: data decoding...
ISBN:
(纸本)9798350369199
The proceedings contain 17 papers. The topics discussed include: large-scale graphs community detection using spark GraphFrames;design and development of XiveNet: a hybrid CAN research testbed;SmartShip: data decoding optimization for onboard AI anomaly detection;a methodology for computing irrational numbers;voxelization of moving deformable geometries on GPU;high-performance simulations for urban planning: implementing paralleldistributed multi-agent systems in MATSim;towards end-to-end compression in Lustre;distributed, continuous and real-time trajectory similarity search;toward profiling IoT processes for remote service attestation;and real-time routing in traffic simulations: a distributed event processing approach.
We introduce UnifyFS, a user-level file system that aggregates node-local storage tiers available on high performance computing (HPC) systems and makes them available to HPC applications under a unified namespace. Uni...
详细信息
ISBN:
(纸本)9798350337662
We introduce UnifyFS, a user-level file system that aggregates node-local storage tiers available on high performance computing (HPC) systems and makes them available to HPC applications under a unified namespace. UnifyFS employs transparent I/O interception, so it does not require changes to application code and is compatible with commonly used HPC I/O libraries. The design of UnifyFS supports the predominant HPC I/O workloads and is optimized for bulk-synchronous I/O patterns. Furthermore, UnifyFS provides customizable file system semantics to flexibly adapt its behavior for diverse I/O workloads and storage devices. In this paper, we discuss the unique design goals and architecture of UnifyFS and evaluate its performance on a leadership-class HPC system. In our experimental results, we demonstrate that UnifyFS exhibits excellent scaling performance for write operations and can improve the performance of application checkpoint operations by as much as 3x versus a tuned configuration.
As accurate and stable time transfer continues to increase its footprint across many market verticals, more so over convergence infrastructures such as data centers, applications such as 5G workloads, accurate packet ...
详细信息
ISBN:
(纸本)9798350366112;9798350366129
As accurate and stable time transfer continues to increase its footprint across many market verticals, more so over convergence infrastructures such as data centers, applications such as 5G workloads, accurate packet scheduling, distributeddatabases, and synchronized collective communications may coexist. One of the key considerations is to further improve the overall performance of the Precision Time Protocol (PTP) by optimizing the control loop performance. This paper proposes a solution to assess optimal Proportional-Integral (PI) controller values using a Bayesian Optimization based method. We show a relative improvement of 12% in timing error, compared to the default parameters baseline.
Verifying user attributes to provide fine-grained access control to databases is fundamental to an attributebased authentication system. In such systems, either a single (central) authority verifies all attributes, or...
详细信息
ISBN:
(纸本)9798350382853;9798350382846
Verifying user attributes to provide fine-grained access control to databases is fundamental to an attributebased authentication system. In such systems, either a single (central) authority verifies all attributes, or multiple independent authorities verify individual attributes distributedly to allow a user to access records stored on the servers. While a central setup is more communication cost efficient, it causes privacy breach of all user attributes to a central authority. Recently, Jafarpisheh et al. studied an information theoretic formulation of the distributed multi-authority setup with N non-colluding authorities, N attributes and K possible values for each attribute, called an (N, K) distributed attribute-based private access control (DAPAC) system, where each server learns only one attribute value that it verifies, and remains oblivious to the remaining N-1 attributes. We show that off-loading a subset of attributes to a central server for verification improves the achievable rate from 1/2K in Jafarpisheh et al. to 1/K+1 in this paper, thus almost doubling the rate for relatively large K, while sacrificing the privacy of a few possibly non-sensitive attributes.
EDDIS is a novel distributed deep learning library designed to efficiently utilize heterogeneous GPU resources for training deep neural networks (DNNs), addressing scalability and conuminication challenges in distribu...
详细信息
ISBN:
(纸本)9798350364613;9798350364606
EDDIS is a novel distributed deep learning library designed to efficiently utilize heterogeneous GPU resources for training deep neural networks (DNNs), addressing scalability and conuminication challenges in distributed training environments. It offers three training modes (synchronous, asynchronous, and hybrid) and supports TensorFlow and PyTorch frameworks. EDDIS significantly accelerates DNN training in heterogeneous GPU settings, achieving up to 17.5x faster training with 16 nodes compared to a single node. Remarkably, the Hybrid training mode surpasses Horovod, achieving training speeds 2.53 times faster for the ResNet50 model.
Scientific workflows consist of multiple, connected applications, with data and results flowing from one to another in a pipeline. Traditionally, such workflows are executed in sequential order, storing intermediate d...
详细信息
ISBN:
(纸本)9798350311990
Scientific workflows consist of multiple, connected applications, with data and results flowing from one to another in a pipeline. Traditionally, such workflows are executed in sequential order, storing intermediate data in storage disks. Co-scheduling application workflows concurrently on the same compute nodes would greatly reduce the cost of moving data to/from storage and allow real-time analysis of intermediate results. Nevertheless, most parallel programming runtimes do not allow seamless integration of various applications in a scientific workflow, in part due to the complexity of managing data and resources. The situation is even more complicated for heterogeneous systems. In this work we extend the Minos Computing Library (MCL) runtime to accelerate pipe-lined and parallel workloads where multiple applications are running in the same system. MCL's asynchronous task library and runtime dynamically manages resources to allow co-scheduling of multiple processes sharing heterogeneous resources. In addition, we design a custom extension of the Open Compute Language (OpenCL) to enable multiple processes to share device memory. We enable MCL to coordinate these shared buffers to allow for easy, fast data sharing between applications. Using malleable micro-benchmarks and two application workflows that combine scientific simulation and AI-based analysis, we show that our method outperforms traditional approaches.
Applications that fuse machine learning and simulation can benefit from the use of multiple computing resources, with, for example, simulation codes running on highly parallel supercomputers and AI training and infere...
详细信息
ISBN:
(纸本)9798350311990
Applications that fuse machine learning and simulation can benefit from the use of multiple computing resources, with, for example, simulation codes running on highly parallel supercomputers and AI training and inference tasks on specialized accelerators. Here, we present our experiences deploying two AI-guided simulation workflows across such heterogeneous systems. A unique aspect of our approach is our use of cloud-hosted management services to manage challenging aspects of cross-resource authentication and authorization, function-as-a-service (FaaS) function invocation, and data transfer. We show that these methods can achieve performance parity with systems that rely on direct connection between resources. We achieve parity by integrating the FaaS system and data transfer capabilities with a system that passes data by reference among managers and workers, and a user-configurable steering algorithm to hide data transfer latencies. We anticipate that this ease of use can enable routine use of heterogeneous resources in computational science.
Given the inherent reliance of distributedsystems on concurrent programming, coupled with increased hardware concurrency and diversity, ensuring their reliability, safety, and security without compromising performanc...
详细信息
ISBN:
(纸本)9783031753794;9783031753800
Given the inherent reliance of distributedsystems on concurrent programming, coupled with increased hardware concurrency and diversity, ensuring their reliability, safety, and security without compromising performance has become exceedingly challenging. This necessitates scalable verification methods that can accurately capture the behavior of concurrent and distributedsystems while providing robust guarantees of compliance with specific requirements. The Scalable Verification and Validation of Concurrent and distributedsystems (ScaVeri) track is dedicated to presenting and discussing advancements in formal methods tailored to these systems. Emphasizing scalable techniques and models that have been validated through real-world case studies, the track covers subtopics such as generating correct parallel code, compositional verification with assume-guarantee contracts, enhanced analysis for large-scale systems, and combinations of various analysis techniques, collectively aiming to improve the assurance of diverse and complex distributed computing environments.
Deep learning has revolutionized various fields but faces challenges such as limitations owing to vanishing gradients and the inability to handle non-differentiable loss functions and layers. Our study proposes Parall...
详细信息
暂无评论