检索结果-内蒙古大学图书馆

Context-Scaling versus Task-Scaling in In-Context Learning

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Abedsoltan, Amirhesam Radhakrishnan, Adityanarayanan Wu, Jingfeng Belkin, Mikhail Department of Computer Science and Engineering UC San Diego United States Eric and Wendy Schmidt Center Broad Institute of MIT and Harvard United States School of Engineering and Applied Sciences Harvard University United States Halicioglu Data Science Institute UC San Diego United States Simons Institute UC Berkeley United States

Transformers exhibit In-Context Learning (ICL), where these models solve new tasks by using examples in the prompt without additional training. In our work, we identify and analyze two key components of ICL: (1) context-scaling, where model performance improves as the number of in-context examples increases and (2) task-scaling, where model performance improves as the number of pre-training tasks increases. While transformers are capable of both context-scaling and task-scaling, we empirically show that standard Multi-Layer Perceptrons (MLPs) with vectorized input are only capable of task-scaling. To understand how transformers are capable of context-scaling, we first propose a significantly simplified transformer architecture without key, query, value weights. We show that it performs ICL comparably to the original GPT-2 model in various statistical learning tasks including linear regression, teacher-student settings. Furthermore, a single block of our simplified transformer can be viewed as data dependent "feature map" followed by an MLP. This feature map on its own is a powerful predictor that is capable of context-scaling but is not capable of task-scaling. We show empirically that concatenating the output of this feature map with vectorized data as an input to MLPs enables both context-scaling and task-scaling. This finding provides a simple setting to study context and task-scaling for ICL. © 2024, CC0.

关键词： Distribution transformers

Simulation of CO2 Storage using a Parameterization Method for Essential Trapping Physics: FluidFlower Benchmark Study

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Wang, Yuhang Zhang, Ziliang Vuik, Cornelis Hajibeygi, Hadi School of Environmental Studies China University of Geosciences Wuhan430078 China Faculty of Electrical Engineering Mathematics and Computer Science Department of Applied Mathematics Delft University of Technology Delft2628XE Netherlands Faculty of Civil Engineering and Geosciences Department of Geoscience and Engineering Delft University of Technology Delft2628CV Netherlands

An efficient compositional framework is developed for simulation of CO2 storage in saline aquifers during a full-cycle injection, migration and post-migration processes. Essential trapping mechanisms, including structural, dissolution, and residual trapping, which operate at different time scales are accurately captured in the presented unified framework. In particular, a parameterization method is proposed to efficiently describe the relevant physical processes. The proposed framework is validated by comparing the dynamics of gravity-induced convective transport with that reported in the literature. Results show good agreement for both the characteristics of descending fingers and the associated dissolution rate. The developed simulator is then applied to study the FluidFlower benchmark model. An experimental setup with heterogeneous geological layers is discretized into a two-dimensional computational domain where numerical simulation is performed. Impacts of hysteresis and the diffusion of CO2 in liquid phase on the migration and trapping of CO2 plume are investigated. Inclusion of the hysteresis effect does not affect plume migration in this benchmark model, whereas diffusion plays an important role in promoting convective mixing. This work casts a promising approach to predict the migration of the CO2 plume, and to assess the amount of trapping from different mechanisms for long-term CO2 storage. © 2023, CC BY-NC-ND.

关键词： Porous materials

Key Exchange in the Quantum Era: Evaluating a Hybrid System of Public-Key Cryptography and Physical-Layer Security

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Staat, Paul Dörpinghaus, Meik Sheikholeslami, Azadeh Paar, Christof Fettweis, Gerhard Goeckel, Dennis Max Planck Institute for Security and Privacy Bochum Germany Vodafone Chair Mobile Communications Systems Technische Universität Dresden Dresden Germany Math and Computer Science Department Suffolk University Boston United States Dept. Electrical and Computer Engineering University of Massachusetts Amherst United States

Today’s information society relies on cryptography to achieve security goals such as confidentiality, integrity, authentication, and non-repudiation for digital communications. Here, public-key cryptosystems play a pivotal role to share encryption keys and create digital signatures. However, quantum computers threaten the security of traditional public-key cryptosystems as they can tame computational problems underlying the schemes, i.e., discrete logarithm and integer factorization. The prospective arrival of capable-enough quantum computers already threatens today’s secret communication in terms of their long-term secrecy when stored to be later decrypted. Therefore, researchers strive to develop and deploy alternative schemes. In this work, evaluate a key exchange protocol based on combining public-key schemes with physical-layer security, anticipating the prospect of quantum attacks. If powerful quantum attackers cannot immediately obtain private keys, legitimate parties have a window of short-term secrecy to perform a physical-layer jamming key exchange (JKE) to establish a long-term shared secret. Thereby, the protocol constraints the computation time available to the attacker to break the employed public-key cryptography. In this paper, we outline the protocol, discuss its security, and point out challenges to be resolved. © 2024, CC BY-NC-SA.

关键词： Quantum computers

Discovery: Virtual Implementation of Inquiry-Based Remote Learning for Secondary STEM Students During the COVID-19 Pandemic

学校读者我要写书评

暂无评论

Biomedical engineering education 2021年第1期1卷 87-94页

作者： Neal I Callaghan Sukhmani Khaira Angelina Ouyang Jose L Cadavid Huntley H Chang Ileana L Co Patrick Diep Nicolas Ivanov Guijin Li Nancy T Li Nhien Tran-Nguyen Corinna Smith Locke Davenport Huyer Dawn M Kilkenny Institute of Biomedical Engineering Faculty of Applied Science & Engineering University of Toronto Toronto ON Canada. Department of Chemical Engineering Faculty of Applied Science & Engineering University of Toronto Toronto ON Canada. Department of Electrical & Computer Engineering Faculty of Applied Science & Engineering University of Toronto Toronto ON Canada. Institute for Studies in Transdisciplinary Engineering Education & Practice Faculty of Applied Science & Engineering University of Toronto Toronto ON Canada.

A theoretical framework for the assessment of water fraction-dependent longitudinal decay rates and magnetisation transfer in membrane lipid phantoms

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Neeb, Heiko Schyboll, Felix Shaharabani, Rona Mezer, Aviv A. Shtangel, Oshrat Department of Mathematics Computer Science and Technology University of Applied Sciences Koblenz RheinAhrCampus Remagen Germany Institute for Medical Engineering and Information Processing - MTI Mittelrhein University of Koblenz Germany Edmond and Lily Safra Center for Brain Sciences Hebrew University of Jerusalem Israel

Motivation Phantom systems consisting of liposome suspensions are widely employed to investigate quantitative MRI parameters mimicking cellular membranes. The proper physical understanding of the measurement results, however, requires proper models for liposomes and their interaction with the surrounding water molecules. Molecular dynamics (MD) simulations allow for investigating the basic lipid-water interaction and determining quantitative MR parameters such as R1 = 1/T1. Here, we present an MD-based approach for the theoretical prediction of R1, the dependence of R1 on water concentration and the magnetization exchange between lipids and interacting water layer in lipids and lipid mixtures. Moreover, a new parameter is introduced which quantitatively measures the amount of hydration water based on conventional spoiled gradient echo MR acquisitions. Methods Molecular dynamics simulations were performed to determine the native R1 rates in three lipids and their interacting water pools. Employing a three-pool exchange model between lipid, hydration water and free water, the hydration water fraction, fHW, as a new parameter as well as the magnetization transfer rate between hydration water and lipids, kHW,L, were quantitatively determined, from which the water concentration dependence of R1 was predicted for all liposome systems investigated. Results Both fHW and kHW,L were determined quantitatively from spoiled gradient echo data by taking the MD-determined relaxation rates into account. Liposome systems behaved similarly, apart from PLPC which showed both lower hydration water fraction and lower exchange rate. The extracted parameters accurately predicted the measured water fraction-dependent R1 rates. Conclusion Hydration water fraction and magnetisation transfer between lipids and water can be determined by a combination of spoiled gradient echo acquisitions and MD-derived relaxation rates. The parameters enable a theoretical understanding of MR parameters in lip

关键词： Phantoms

Digital Twins for Energy-Efficient Manufacturing

学校读者我要写书评

暂无评论

Digital Twins for Energy-Efficient Manufacturing

Annual IEEE Systems Conference

作者： Nader Mohamed Sanja Lazarova-Molnar Jameela Al-Jaroodi Department of Computer Science and Information Systems Pennsylvania Western University California PA USA Institute of Applied Informatics and Formal Description Methods Karlsruhe Institute of Technology Karlsruhe Germany Faculty of Engineering University of Southern Denmark Odense Denmark Department of Engineering Robert Morris University Pittsburgh PA USA

Some manufacturing industries consume massive energy for their manufacturing and logistics processes. The costs of manufacturing can increase significantly as the energy cost increases. Such increase amplifies the final cost of the created products on the consumers. This makes finding different approaches that utilize innovative technologies to improve energy efficiency in manufacturing extremely important. This paper investigates how digital twins can be utilized to enable improved energy efficiency in manufacturing. Different applications of digital twins for energy-efficient manufacturing are investigated. In addition, the paper discusses some challenges of utilizing digital twins for this purpose. A framework for utilizing digital twins for energy-efficient manufacturing is also discussed.

关键词： Manufacturing industries Costs Manufacturing processes Buildings Energy efficiency Raw materials Manufacturing

Combining Dense and Sparse Rewards to Improve Deep Reinforcement Learning Policies in Reach-Avoid Games with Faster Evaders in Two vs. One Scenarios

学校读者我要写书评

暂无评论

Combining Dense and Sparse Rewards to Improve Deep Reinforce...

International Conference on Control, Decision and Information Technologies (CoDIT)

作者： Jefferson Silveira Kalena McCloskey Camille-Alain Rabbath Craig Williams Sidney Givigi Department of Electrical & Computer Engineering The Ingenuity Labs Research Institute Queen’s University Kingston Canada Department of Applied Mathematics and Engineering Queen’s University Kingston Canada School of Computing Queen’s University Kingston Canada Department of Physics and Space Science Royal Military College of Canada Kingston Canada School of Computing and the Ingenuity Labs Research Institute Queen’s University Kingston Canada

ISBN: (数字)9798350373974

ISBN: (纸本)9798350373981

This paper investigates a variation of the reach-avoid game, a multi-agent pursuit and evasion scenario applicable to aerial defense, with faster evaders. Using Deep Reinforcement Learning techniques, the study proposes a different reward function that combines dense (distance-based) and sparse (outcome-based) rewards. Focused on the defender’s perspective in aerial defense, this new reward function resulted in effective learned policies against faster evaders, outperforming traditional differential game and DRL strategies with dense-only rewards. Moreover, the learned policy demonstrated versatility across different instances of the problem, including changes in pursuer speeds and winning radii, illustrating its versatility in unseen situations during training.

关键词： Training Games Differential games Deep reinforcement learning Information technology

Obligations and permissions, algebraically

学校读者我要写书评

暂无评论

arXiv 2024年

作者： De Domenico, Andrea Farjami, Ali Manoorkar, Krishna Palmigiano, Alessandra Panettiere, Mattia Wang, Xiaolong School of Business and Economics Vrije Universiteit Amsterdam Netherlands School of Computer Engineering Iran University of Science and Technology Tehran Iran Department of Mathematics and Applied Mathematics University of Johannesburg South Africa School of Philosophy and Social Development Shandong University Jinan China

We further develop the algebraic approach to input/output logic initiated in [1], where subordination algebras and a family of their generalizations were proposed as a semantic environment of various input/output logics. In particular, we consider precontact algebras as a suitable algebraic environment for negative permission, and we characterize properties of several types of permission (negative, static, dynamic), as well as their interactions with normative systems, by means of suitable modal languages encoding outputs. © 2024, CC BY.

关键词： Algebra

Synthesis of iron-doped zinc oxide (Fe-ZnO) nanoparticles by using several stabilizers 1

学校读者我要写书评

暂无评论

Synthesis of iron-doped zinc oxide (Fe-ZnO) nanoparticles by...

1st South East Asia science, Technology, engineering and mathematics International Conference, SEA- STEM 2020

作者： Muktaridha, O. Adlim, M. Suhendrayatna, S. Ismail, I. Graduate School of Mathematics and Applied Science Universitas Syiah Kuala Darussalam Banda Aceh23111 Indonesia Chemical Engineering Department Faculty of Engineering Universitas Syiah Kuala Darussalam Banda Aceh23111 Indonesia Physics Department Faculty of Math and Natural Science Universitas Syiah Kuala Darussalam Banda Aceh23111 Indonesia

Studies on the effect of the stabilizer of chitosan, citric acid and urea for preparation Fe-doped ZnO have been conducted. Both ZnO and Fe were prepared from each salt precursor solution. The Zn2+ was added fist and subsequently Fe2+ by dropwise method into each stabilizer solution before converted into the hydroxide compounds before annealing. All metal ion solutions precipitated out after 7 minutes of NaOH addition except in urea stabilizer, where the solution remained in the colloidal form for more than 30 minutes. The hydroxyl ions are required for ZnO formation, but it weakens chitosan and citric acid stabilizing capacity, which instigated ZnO particles to be highly agglomerated. Each stabilizer generated a smaller ZnO crystallite size compared to the control sample. Fe-ZnO crystallite size prepared using chitosan, citric acid and urea as the stabilizer was 14.75, 23.29 and 22.51 nm, respectively. The crystallite size of Fe-ZnO that was prepared without stabilizer (control) was 32.89 nm. © Published under licence by IOP Publishing Ltd.

关键词： Urea