检索结果-内蒙古大学图书馆

Filling the Void: Data-Driven Machine Learning-based Reconstruction of Sampled Spatiotemporal Scientific Simulation Data 24

Filling the Void: Data-Driven Machine Learning-based Reconst...

引用

proceedings of the SC '24 Workshops of the International Conference on High Performance computing, Network, Storage, and analysis

作者： Ayan Biswas Aditi Mishra Meghanto Majumder Subhashis Hazarika Alexander Most Juan Castorena Christopher Bryan Patrick McCormick James Ahrens Earl Lawrence Aric Hagberg Los Alamos National Lab Los Alamos U.S.A. Arizona State University Tempe U.S.A. Fujitsu Research of America Inc. Santa Clara U.S.A. Los Alamos National Lab Los Alamos U.S.A

ISBN: (纸本)9798350355543

As high-performance computing systems continue to advance, the gap between computing performance and I/O capabilities is widening. This bottleneck limits the storage capabilities of increasingly large-scale simulations, which generate data at never-before-seen granularities while only being able to store a small subset of the raw data. Recently, strategies for data-driven sampling have been proposed to intelligently sample the data in a way that achieves high data reduction rates while preserving important regions or features with high fidelity. However, a thorough analysis of how such intelligent samples can be used for data reconstruction is lacking. We propose a data-driven machine learning approach based on training neural networks to reconstruct full-scale datasets based on a simulation's sampled output. Compared to current state-of-the-art reconstruction approaches such as Delaunay triangulation-based linear interpolation, we demonstrate that our machine learning-based reconstruction has several advantages, including reconstruction quality, time-to-reconstruct, and knowledge transfer to unseen timesteps and grid resolutions. We propose and evaluate strategies that balance the sampling rates with model training (pretraining and fine-tuning) and data reconstruction time to demonstrate how such machine learning approaches can be tailored for both speed and quality for the reconstruction of grid-based datasets.

关键词： computing methodologies

来源：评论

学校读者我要写书评

暂无评论

Utilizing Risk Number and Program Slicing to Improve Human-Machine Pair Inspection 27

Utilizing Risk Number and Program Slicing to Improve Human-M...

引用

27th International Conference on Engineering of Complex Computer systems, ICECCS 2023

作者： Dai, Yujun Liu, Shaoying Xu, Guangquan Liu, Ai Hiroshima University Graduate School of Advanced Science and Engineering Hiroshima Japan Tianjin University School of Cybersecurity College of Intelligence and Computing Tianjin China

ISBN: (纸本)9798350340044

Human-Machine Pair Inspection (HMPI) is a novel code inspection technology proposed in our previous work, which is the style that machine will intelligently guide the programmer to carry out inspections of the program code during programming. For large-scale software projects, the efficiency of HMPI needs to be improved due to the inaccurate measurement of the code structure and the excessive inspection scope. In this paper, to alleviate the above deficiencies, we propose the Risk Number, a code evaluation metric generated based on historical error data. The Risk Number is calculated by a statistical tool called regression analysis, which more accurately indicates the relationship between the nested structure of the code and the likelihood of containing bugs than Cognitive Complexity. Additionally, HMPI is supported by utilizing Risk Number to point out high-risk code and program slicing techniques to extract statements that have dependencies on the code to generate checklists, thereby reducing the scope of inspection. We describe a case study to evaluate the performance of this method by comparing its inspection time and number of detected errors with our previous work. The result shows that the method is likely to guide the programmer to inspect the faulty code earlier and be more efficient in detecting defects than HMPI based on Cognitive Complexity. © 2023 IEEE.

关键词： Inspection

来源：评论

学校读者我要写书评

暂无评论

Scalable Privacy-preserving Geo-distance Evaluation for Precision Agriculture IoT systems

引用

acm TRANSACTIONS ON SENSOR NETWORKS 2021年第4期17卷 1–30页

作者： Yan, Qiben Lou, Jianzhi Vuran, Mehmet C. Irmak, Suat Michigan State Univ 28 S Shaw LaneRoom 3115 E Lansing MI 48824 USA Univ Nebraska 214 Schorr Ctr Lincoln NE 68588 USA Penn State Univ Dept Agr & Biol Engn 105 Agr Engn BldgShortlidge Rd University Pk PA 16802 USA

Precision agriculture has become a promising paradigm to transform modern agriculture. The recent revolution in big data and Internet-of-Things (IoT) provides unprecedented benefits including optimizing yield, minimizing environmental impact, and reducing cost. However, the mass collection of farm data in IoT applications raises serious concerns about potential privacy leakage that may harm the farmers' welfare. In this work, we propose a novel scalable and private geo-distance evaluation system, called SPRIDE, to allow application servers to provide geographic-based services by computing the distances among sensors and farms privately. The servers determine the distances without learning any additional information about their locations. The key idea of SPRIDE is to perform efficient distance measurement and distance comparison on encrypted locations over a sphere by leveraging a homomorphic cryptosystem. To serve a large user base, we further propose SPRIDE+ with novel and practical performance enhancements based on pre-computation of cryptographic elements. Through extensive experiments using real-world datasets, we show SPRIDE1 achieves private distance evaluation on a large network of farms, attaining 3+ times runtime performance improvement over existing techniques. We further show SPRIDE+ can run on resource-constrained mobile devices, which offers a practical solution for privacy-preserving precision agriculture IoT applications.

关键词： Privacy-preserving data analysis distance evaluation precision agriculture IoT

来源：评论

学校读者我要写书评

暂无评论

On the Effective Horizon of Inverse Reinforcement Learning 25

On the Effective Horizon of Inverse Reinforcement Learning

引用

proceedings of the 24th International Conference on Autonomous Agents and Multiagent systems

作者： Yiqing Xu Finale Doshi-Velez David Hsu School of Computing National University of Singapore Singapore Singapore Department of Computer Science Harvard University Cambridge MA USA School of Computing Smart System Institute National University of Singapore Singapore Singapore

ISBN: (纸本)9798400714269

Inverse reinforcement learning (IRL) algorithms often rely on (forward) reinforcement learning or planning, over a given time horizon, to compute an approximately optimal policy for a hypothesized reward function; they then match this policy with expert demonstrations. The time horizon plays a critical role in determining both the accuracy of reward estimates and the computational efficiency of IRL algorithms. Interestingly, an effective time horizon shorter than the ground-truth value often produces better results faster. This work formally analyzes this phenomenon and provides an explanation: the time horizon controls the complexity of an induced policy class and mitigates overfitting with limited data. This analysis provides a guide for the principled choice of the effective horizon for IRL. It also prompts us to re-examine the classic IRL formulation: it is more natural to learn jointly the reward and the effective horizon rather than the reward alone with a given horizon. To validate our findings, we implement a cross-validation extension and the experimental results support the theoretical analysis. The https://***/ project page and https://***/eeching/Effective_Horizon_IRL code are publicly available.

关键词： imitation learning

来源：评论

学校读者我要写书评

暂无评论

EndWatch: A Practical Method for Detecting Non-Termination in Real-World Software 23

EndWatch: A Practical Method for Detecting Non-Termination i...

引用

proceedings of the 38th IEEE/acm International Conference on Automated Software Engineering

作者： Yao Zhang Xiaofei Xie Yi Li Sen Chen Cen Zhang Xiaohong Li College of Intelligence and Computing Tianjin University School of Computing and Information Systems Singapore Management University School of Computer Science and Engineering Nanyang Technological University

ISBN: (纸本)9798350329964

Detecting non-termination is crucial for ensuring program correctness and security, such as preventing denial-of-service attacks. While termination analysis has been studied for many years, existing methods have limited scalability and are only effective on small programs. To address this issue, we propose a practical termination checking technique, called EndWatch, for detecting non-termination caused by infinite loops through testing. Specifically, we introduce two methods to generate non-termination oracles based on checking state revisits, i.e., if the program returns to a previously visited state at the same program location, it does not terminate. The non-termination oracles can be incorporated into testing tools (e.g., AFL used in this paper) to detect non-termination in large programs. For linear loops, we perform symbolic execution on individual loops to infer State Revisit Conditions (SRCs) and instrument SRCs into target loops. For non-linear loops, we instrument target loops for checking concrete state revisits during execution. We evaluated EndWatch on standard benchmarks with small-sized programs and real-world projects with large-sized programs. The evaluation results show that EndWatch is more effective than the state-of-the-art tools on standard benchmarks (detecting 87% of non-terminating programs while the best baseline detects only 67%), and useful in detecting non-termination in real-world projects (detecting 90% of known non-termination CVEs and 4 unknown bugs).

关键词： non-termination detection

来源：评论

学校读者我要写书评

暂无评论

Octopus: Experiences with a Hybrid Event-Driven Architecture for Distributed Scientific computing 24

Octopus: Experiences with a Hybrid Event-Driven Architecture...

引用

proceedings of the SC '24 Workshops of the International Conference on High Performance computing, Network, Storage, and analysis

作者： Haochen Pan Ryan Chard Sicheng Zhou Alok Kamatar Rafael Vescovi Valérie Hayot-Sasson André Bauer Maxime Gonthier Kyle Chard Ian Foster Department of Computer Science University of Chicago Chicago IL USA Data Science and Learning Division Argonne National Laboratory Lemont IL USA Department of Computer Science Southern University of Science and Technology Guangdong China Department of Computer Science Illinois Institute of Technology Chicago IL USA Department of Computer Science University of Chicago Chicago IL USA and Data Science and Learning Division Argonne National Laboratory Lemont IL USA

ISBN: (纸本)9798350355543

Scientific research increasingly relies on distributed computational resources, storage systems, networks, and instruments, ranging from HPC and cloud systems to edge devices. Event-driven architecture (EDA) benefits applications targeting distributed research infrastructures by enabling the organization, communication, processing, reliability, and security of events generated from many sources. To support the development of scientific EDA, we introduce Octopus, a hybrid, cloud-to-edge event fabric designed to link many local event producers and consumers with cloud-hosted brokers, and to provide a fabric for developing resilient applications. Octopus can be scaled to meet demand, permits the deployment of highly available Triggers for automatic event processing, and enforces fine-grained access control. We identify requirements in self-driving laboratories, scientific data automation, online task scheduling, epidemic modeling, and dynamic workflow management use cases, and present results demonstrating Octopus' ability to meet those requirements. Octopus supports producing and consuming events at a rate of over 4.2 M and 9.6 M events per second, respectively, from distributed clients.

关键词： Event-driven architecture

来源：评论

学校读者我要写书评

暂无评论

proceedings of the 7th Workshop on Micro Aerial Vehicle Networks, systems, and Applications, DroNet 2021

Proceedings of the 7th Workshop on Micro Aerial Vehicle Netw...

引用

7th Workshop on Micro Aerial Vehicle Networks, systems, and Applications, DroNet 2021

ISBN: (纸本)9781450385992

The proceedings contain 6 papers. The topics discussed include: toward a marketplace for aerial computing;POD: a smartphone that flies;a comparative study of virtual infrastructure management solutions for UAV networks;persistent airborne surveillance using semi-autonomous drone swarms;towards adaptive, self-configuring networked unmanned aerial vehicles;and an experimental analysis on drone-mounted access points for improved latency-reliability.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Scalable and Consistent Graph Neural Networks for Distributed Mesh-based Data-driven Modeling 24

Scalable and Consistent Graph Neural Networks for Distribute...

引用

proceedings of the SC '24 Workshops of the International Conference on High Performance computing, Network, Storage, and analysis

作者： Shivam Barwey Riccardo Balin Bethany Lusch Saumil Patel Ramesh Balakrishnan Pinaki Pal Romit Maulik Venkatram Vishwanath Transportation and Power Systems Argonne National Laboratory Lemont USA Leadership Computing Facility Argonne National Laboratory Lemont USA Computational Science Division Argonne National Laboratory Lemont USA Information Sciences and Technology Pennsylvania State University University Park USA Mathematics and Computer Science Argonne National Laboratory Lemont USA

ISBN: (纸本)9798350355543

This work develops a distributed graph neural network (GNN) methodology for mesh-based modeling applications using a consistent neural message passing layer. As the name implies, the focus is on enabling scalable operations that satisfy physical consistency via halo nodes at sub-graph boundaries. Here, consistency refers to the fact that a GNN trained and evaluated on one rank (one large graph) is arithmetically equivalent to evaluations on multiple ranks (a partitioned graph). This concept is demonstrated by interfacing GNNs with NekRS, a GPU-capable exascale CFD solver developed at Argonne National Laboratory. It is shown how the NekRS mesh partitioning can be linked to the distributed GNN training and inference routines, resulting in a scalable mesh-based data-driven modeling workflow. We study the impact of consistency on the scalability of mesh-based GNNs, demonstrating efficient scaling in consistent GNNs for up to O(1B) graph nodes on the Frontier exascale supercomputer.

关键词： graph neural networks

来源：评论

学校读者我要写书评

暂无评论

SCTuner: An Autotuner Addressing Dynamic I/O Needs on Supercomputer I/O Subsystems 6

SCTuner: An Autotuner Addressing Dynamic I/O Needs on Superc...

引用

IEEE/acm 6th International Parallel Data systems Workshop (PDSW)

作者： Tang, Houjun Xie, Bing Byna, Suren Carns, Philip Koziol, Quincey Kannan, Sudarsun Lofstead, Jay Oral, Sarp Oak Ridge Natl Lab Oak Ridge TN USA Lawrence Berkeley Natl Lab Berkeley CA 94720 USA Argonne Natl Lab Lemont IL USA Rutgers State Univ New Brunswick NJ USA Sandia Natl Labs POB 5800 Albuquerque NM 87185 USA

ISBN: (纸本)9781665418379

In high-performance computing (HPC), scientific applications often manage a massive amount of data using I/O libraries. These libraries provide convenient data model abstractions, help ensure data portability, and, most important, empower end users to improve I/O performance by tuning configurations across multiple layers of the HPC I/O stack. We propose SCTuner, an autotuner integrated within the I/O library itself to dynamically tune both the I/O library and the underlying I/O stack at application runtime. To this end, we introduce a statistical benchmarking method to profile the behaviors of individual supercomputer I/O subsystems with varied configurations across I/O layers. We use the benchmarking results as the built-in knowledge in SCTuner, implement an I/O pattern extractor, and plan to implement an online performance tuner as the SCTuner runtime. We conducted a benchmarking analysis on the Summit supercomputer and its GPFS file system Alpine. The preliminary results show that our method can effectively extract the consistent I/O behaviors of the target system under production load, building the base for I/O autotuning at application runtime.

关键词： Runtime Tuners File systems Production Benchmark testing Supercomputers Libraries

来源：评论

学校读者我要写书评

暂无评论

DeSQL: Interactive Debugging of SQL in Data-Intensive Scalable computing

Proceedings of the ACM on Software Engineering

引用

proceedings of the acm on Software Engineering 2024年第FSE期1卷 767-788页

作者： Sabaat Haroon Chris Brown Muhammad Ali Gulzar Virginia Tech Blacksburg USA

SQL is the most commonly used front-end language for data-intensive scalable computing (DISC) applications due to its broad presence in new and legacy workflows and shallow learning curve. However, DISC-backed SQL introduces several layers of abstraction that significantly reduce the visibility and transparency of workflows, making it challenging for developers to find and fix errors in a query. When a query returns incorrect outputs, it takes a non-trivial effort to comprehend every stage of the query execution and find the root cause among the input data and complex SQL query. We aim to bring the benefits of step-through interactive debugging to DISC-powered SQL with DeSQL. Due to the declarative nature of SQL, there are no ordered atomic statements to place a breakpoint to monitor the flow of data. DeSQL’s automated query decomposition breaks a SQL query into its constituent sub queries, offering natural locations for setting breakpoints and monitoring intermediate data. However, due to advanced query optimization and translation in DISC systems, a user query rarely matches the physical execution, making it challenging to associate subqueries with their intermediate data. DeSQL performs fine-grained taint analysis to dynamically map the subqueries to their intermediate data, while also recognizing subqueries removed by the optimizers. For such subqueries, DeSQL efficiently regenerates the intermediate data from a nearby subquery’s data. On the popular TPC-DC benchmark, DeSQL provides a complete debugging view in 13% less time than the original job time while incurring an average overhead of 10% in addition to retaining Apache Spark’s scalability. In a user study comprising 15 participants engaged in two debugging tasks, we find that participants utilizing DeSQL identify the root cause behind a wrong query output in 74% less time than the de-facto, manual debugging.

关键词： Debugging SQL data-intensive scalable computing

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：