检索结果-内蒙古大学图书馆

S$^\text{3}$Attention: Improving Long Sequence Attention With Smoothed Skeleton Sketching

学校读者我要写书评

暂无评论

IEEE Journal of Selected Topics in Signal Processing 2024年第6期18卷 985-996页

作者： Xue Wang Tian Zhou Jianqing Zhu Jialin Liu Kun Yuan Tao Yao Wotao Yin Rong Jin HanQin Cai Alibaba Group Bellevue WA USA Computer Electrical and Mathematical Science and Engineering Division King Abdullah University of Science and Technology Thuwal Saudi Arabia Department of Statistics and Data Science University of Central Florida Orlando FL USA Center for Machine Learning Research Peking University Beijing China Antai College of Economics and Management Shanghai Jiao Tong University Shanghai China Meta Menlo Park CA USA Department of Statistics and Data Science and the Department of Computer Science University of Central Florida Orlando FL USA

Attention based models have achieved many remarkable breakthroughs in numerous applications. However, the quadratic complexity of Attention makes the vanilla Attention based models hard to apply to long sequence tasks. Various improved Attention structures are proposed to reduce the computation cost by inducing low rankness and approximating the whole sequence by sub-sequences. The most challenging part of those approaches is maintaining the proper balance between information preservation and computation reduction: the longer sub-sequences used, the better information is preserved, but at the price of introducing more noise and computational costs. In this paper, we propose a smoothed skeleton sketching based Attention structure, coined S $^{3}$ Attention, which significantly improves upon the previous attempts to negotiate this trade-off. S $^{3}$ Attention has two mechanisms to effectively minimize the impact of noise while keeping the linear complexity to the sequence length: a smoothing block to mix information over long sequences and a matrix sketching method that simultaneously selects columns and rows from the input matrix. We verify the effectiveness of S $^{3}$ Attention both theoretically and empirically. Extensive studies over Long Range Arena (LRA) datasets and six time-series forecasting show that S $^{3}$ Attention significantly outperforms both vanilla Attention and other state-of-the-art variants of Attention structures.

关键词： Computational modeling Complexity theory Transformers Task analysis Skeleton Kernel Noise

Can Automated Metadata Extraction Make Scientific data More Navigable?

学校读者我要写书评

暂无评论

Can Automated Metadata Extraction Make Scientific Data More ...

IEEE International Conference on e-science and Grid Computing

作者： Tyler J. Skluzacek Kyle Chard Ian Foster Oak Ridge National Lab National Center for Computational Sciences Oak Ridge TN Department of Computer Science University of Chicago Chicago IL Data Science and Learning Division Argonne National Lab Lemont IL

FAIR principles require that scientific data be findable, discoverable, and reusable by users. To enable FAIRness, practioners of a science repository will often construct a rich, searchable index of metadata derived from the data. Unfortunately, manual metadata annotation methods do not scale to the many data files generated by many projects; and instead automated extraction systems are needed to scalably parse these files—often with nonstandard schema requiring specialized parsing strategies—and deposit representative metadata into a search index. In this work, we evaluate whether, and the extent to which, automatically extracted metadata make research repositories more navigable. We present a two-part user study conducted with scientists at two U.S. national laboratories from projects spanning spectroscopy and battery modeling. We constructed research indexes automatically by using the Xtract metadata extraction system. In the first part of our study, we learned about each user's role and identified key navigation concerns for scientists. We found that participants wished to navigate for purposes of discovery, retrieval, and organization. In the second part, participants completed simulated research data navigation tasks crafted to reflect real-world navigability concerns. We found that regardless of the interface used, participants consistently solved navigation tasks with high degrees of confidence and correctness, and significantly ( $1.2\mathrm{X}-50\times$ ) faster than via their alternative methods (e.g., manual directory scans or designing a customized navigational tool).

关键词：

DEEP learning METHODS FOR DRUG RESPONSE PREDICTION IN CANCER: PREDOMINANT AND EMERGING TRENDS

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Partin, Alexander Brettin, Thomas S. Zhu, Yitan Narykov, Oleksandr Clyde, Austin Overbeek, Jamie Stevens, Rick L. Division of Data Science and Learning Argonne National Laboratory LemontIL United States Department of Computer Science The University of Chicago ChicagoIL United States

Cancer claims millions of lives yearly worldwide. While many therapies have been made available in recent years, by in large cancer remains unsolved. Exploiting computational predictive models to study and treat cancer holds great promise in improving drug development and personalized design of treatment plans, ultimately suppressing tumors, alleviating suffering, and prolonging lives of patients. A wave of recent papers demonstrates promising results in predicting cancer response to drug treatments while utilizing deep learning methods. These papers investigate diverse data representations, neural network architectures, learning methodologies, and evaluations schemes. However, deciphering promising predominant and emerging trends is difficult due to the variety of explored methods and lack of standardized framework for comparing drug response prediction models. To obtain a comprehensive landscape of deep learning methods, we conducted an extensive search and analysis of deep learning models that predict the response to single drug treatments. A total of 60 deep learning-based models have been curated and summary plots were generated. Based on the analysis, observable patterns and prevalence of methods have been revealed. This review allows to better understand the current state of the field and identify major challenges and promising solution paths. © 2022, CC BY.

关键词： Forecasting

Machine learning Materials Properties with Accurate Predictions, Uncertainty Estimates, Domain Guidance, and Persistent Online Accessibility

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Jacobs, Ryan Schultz, Lane E. Scourtas, Aristana Schmidt, K.J. Price-Skelly, Owen Engler, Will Foster, Ian Blaiszik, Ben Voyles, Paul M. Morgan, Dane Department of Materials Science and Engineering University of Wisconsin-Madison MadisonWI United States Globus University of Chicago ChicagoIL United States Data Science and Learning Division Argonne National Laboratory LemontIL United States Department of Computer Science University of Chicago ChicagoIL United States

One compelling vision of the future of materials discovery and design involves the use of machine learning (ML) models to predict materials properties and then rapidly find materials tailored for specific applications. However, realizing this vision requires both providing detailed uncertainty quantification (model prediction errors and domain of applicability) and making models readily usable. At present, it is common practice in the community to assess ML model performance only in terms of prediction accuracy (e.g., mean absolute error), while neglecting detailed uncertainty quantification and robust model accessibility and usability. Here, we demonstrate a practical method for realizing both uncertainty and accessibility features with a large set of models. We develop random forest ML models for 33 materials properties spanning an array of data sources (computational and experimental) and property types (electrical, mechanical, thermodynamic, etc.). All models have calibrated ensemble error bars to quantify prediction uncertainty and domain of applicability guidance enabled by kernel-density-estimate-based feature distance measures. All data and models are publicly hosted on the Garden-AI infrastructure, which provides an easy-to-use, persistent interface for model dissemination that permits models to be invoked with only a few lines of Python code. We demonstrate the power of this approach by using our models to conduct a fully ML-based materials discovery exercise to search for new stable, highly active perovskite oxide catalyst materials. © 2024, CC BY.

关键词： Machine learning

Linking the Dynamic PicoProbe Analytical Electron-Optical Beam Line / Microscope to Supercomputers

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Brace, Alexander Vescovi, Rafael Chard, Ryan Saint, Nickolaus D. Ramanathan, Arvind Zaluzec, Nestor J. Foster, Ian Data Science and Learning Division Argonne National Laboratory LemontIL United States Computer Science Department University of Chicago ChicagoIL United States Photon Sciences Division Argonne National Laboratory LemontIL United States Physical Sciences Division University of Chicago ChicagoIL United States

The Dynamic PicoProbe at Argonne National Laboratory is undergoing upgrades that will enable it to produce up to 100s of GB of data per day. While this data is highly important for both fundamental science and industrial applications, there is currently limited on-site infrastructure to handle these high-volume data streams. We address this problem by providing a software architecture capable of supporting large-scale data transfers to the neighboring supercomputers at the Argonne Leadership Computing Facility. To prepare for future scientific workflows, we implement two instructive use cases for hyperspectral and spatiotemporal datasets, which include: (i) off-site data transfer, (ii) machine learning/artificial intelligence and traditional data analysis approaches, and (iii) automatic metadata extraction and cataloging of experimental results. This infrastructure supports expected workloads and also provides domain scientists the ability to reinterrogate data from past experiments to yield additional scientific value and derive new insights. © 2023, CC BY.

关键词： data transfer

EAIRA: Establishing a Methodology for Evaluating AI Models as Scientific Research Assistants

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Cappello, Franck Madireddy, Sandeep Underwood, Robert Getty, Neil Chia, Nicholas Lee-Ping Ramachandra, Nesar Nguyen, Josh Keçeli, Murat Mallick, Tanwi Li, Zilinghan Ngom, Marieme Zhang, Chenhui Yanguas-Gil, Angel Antoniuk, Evan Kailkhura, Bhavya Tian, Minyang Du, Yufeng Ting, Yuan-Sen Wells, Azton Nicolae, Bogdan Maurya, Avinash Mustafa Rafique, M. Huerta, Eliu Li, Bo Foster, Ian Stevens, Rick Mathematics and Computer Science Division Argonne National Laboratory United States Data Science and Learning Division Argonne National Laboratory United States Computational Science Division Argonne National Laboratory United States Applied Materials Division Argonne National Laboratory United States Department of Computer Science The University of Chicago United States University of Pennsylvania United States Lawrence Livermore National Laboratory United States The Ohio State University United States Rochester Institute of Technology United States Massachusetts Institute of Technology United States

Recent advancements have positioned AI, and particularly Large Language Models (LLMs) as transformative tools for scientific research, capable of addressing complex tasks that require reasoning, problem-solving, and decision-making. Their exceptional capabilities suggest their potential as scientific research assistants, but also highlight the need for holistic, rigorous, and domain-specific evaluation to assess effectiveness in real-world scientific applications. This paper describes a multifaceted methodology for Evaluating AI models as scientific Research Assistants (EAIRA) developed at Argonne National Laboratory. This methodology incorporates four primary classes of evaluations. 1) Multiple Choice Questions to assess factual recall;2) Open Response to evaluate advanced reasoning and problem-solving skills;3) Lab-Style Experiments involving detailed analysis of capabilities as research assistants in controlled environments;and 4) Field-Style Experiments to capture researcher-LLM interactions at scale in a wide range of scientific domains and applications. These complementary methods enable a comprehensive analysis of LLM strengths and weaknesses with respect to their scientific knowledge, reasoning abilities, and adaptability. Recognizing the rapid pace of LLM advancements, we designed the methodology to evolve and adapt so as to ensure its continued relevance and applicability. This paper describes the methodology’s state at the end of February 2025. Although developed within a subset of scientific domains, the methodology is designed to be generalizable to a wide range of scientific domains. Copyright © 2025, The Authors. All rights reserved.

关键词： Decision making

Quantum Advantage in Distributed Sensing with Noisy Quantum Networks

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Zang, Allen Kolar, Alexander Gonzales, Alvin Chung, Joaquin Gray, Stephen K. Kettimuthu, Rajkumar Zhong, Tian Saleem, Zain H. Pritzker School of Molecular Engineering University of Chicago ChicagoIL United States Mathematics and Computer Science Division Argonne National Laboratory LemontIL United States Data Science and Learning Division Argonne National Laboratory LemontIL United States Center for Nanoscale Materials Argonne National Laboratory LemontIL United States

It is critically important to analyze the achievability of quantum advantage under realistic imperfections. In this work, we show that quantum advantage in distributed sensing can be achieved with noisy quantum networks which can only distribute noisy entangled states. We derive a closed-form expression of the quantum Fisher information (QFI) for estimating the average of local parameters using GHZ-diagonal probe states, an important distributed sensing prototype. From the QFI we obtain the necessary condition to achieve quantum advantage over the optimal local sensing strategy, which can also serve as an optimization-free entanglement detection criterion for multipartite states. In addition, we prove that genuine multipartite entanglement is neither necessary nor sufficient through explicit examples of depolarized and dephased GHZ states. We further explore the impacts from imperfect local entanglement generation and local measurement constraint, and our results imply that the quantum advantage is more robust against quantum network imperfections than local operation errors. Finally, we demonstrate that the probe state with potential for quantum advantage in distributed sensing can be prepared by a three-node quantum network using practical protocol stacks through simulations with SeQUeNCe, an open-source, customizable quantum network simulator. © 2024, CC BY.

关键词： Quantum entanglement

Recurrent and Spiking Modeling of Sparse Surgical Kinematics 2020

学校读者我要写书评

暂无评论

Recurrent and Spiking Modeling of Sparse Surgical Kinematics

2020 International Conference on Neuromorphic Systems, ICONS 2020

作者： Getty, Neil Zhao, Zixuan Gruessner, Stephan Chen, Liaohai Xia, Fangfang Argonne National Laboratory Data Science and Learning Division Department of Surgery University of Illinois at Chicago

ISBN: (纸本)9781450388511

Robot-assisted minimally invasive surgery is improving surgeon performance and patient outcomes. This innovation is also turning what has been a subjective practice into motion sequences that can be precisely measured. A growing number of studies have used machine learning to analyze video and kinematic data captured from surgical robots. In these studies, models are typically trained on benchmark datasets for representative surgical tasks to assess surgeon skill levels. While they have shown that novices and experts can be accurately classified, it is not clear whether machine learning can separate highly proficient surgeons from one another, especially without video data. In this study, we explore the possibility of using only kinematic data to predict surgeons of similar skill levels. We focus on a new dataset created from surgical exercises on a simulation device for skill training. A simple, efficient encoding scheme was devised to encode kinematic sequences so that they were amenable to edge learning. We report that it is possible to identify surgical fellows receiving near perfect scores in the simulation exercises based on their motion characteristics alone. Further, our model could be converted to a spiking neural network to train and infer on the Nengo simulation framework with no loss in accuracy. Overall, this study suggests that building neuromorphic models from sparse motion features may be a potentially useful strategy for identifying surgeons and gestures with chips deployed on robotic systems to offer adaptive assistance during surgery and training with additional latency and privacy benefits. © 2020 ACM.

关键词： Kinematics

Adversarial Predictions of data Distributions Across Federated Internet-of-Things Devices

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Rajani, Samir Dematties, Dario Hudson, Nathaniel Chard, Kyle Ferrier, Nicola Sankaran, Rajesh Beckman, Peter Northwestern Argonne Institute of Science and Engineering Northwestern University EvanstonIL United States Mathematics and Computer Science Division Argonne National Laboratory LemontIL United States Department of Computer Science University of Chicago ChicagoIL United States Data Science and Learning Division Argonne National Laboratory LemontIL United States

Federated learning (FL) is increasingly becoming the default approach for training machine learning models across decentralized Internet-of-Things (IoT) devices. A key advantage of FL is that no raw data are communicated across the network, providing an immediate layer of privacy. Despite this, recent works have demonstrated that data reconstruction can be done with the locally trained model updates which are communicated across the network. However, many of these works have limitations with regard to how the gradients are computed in backpropagation. In this work, we demonstrate that the model weights shared in FL can expose revealing information about the local data distributions of IoT devices. This leakage could expose sensitive information to malicious actors in a distributed system. We further discuss results which show that injecting noise into model weights is ineffective at preventing data leakage without seriously harming the global model accuracy. © 2023, CC BY.

关键词： Sensor networks