Zeroth-order (ZO) methods, which use the finite difference of two function evaluations (also called ZO gradient) to approximate first-order gradient, have attracted much attention recently in machine learning because ...
详细信息
Zeroth-order (ZO) methods, which use the finite difference of two function evaluations (also called ZO gradient) to approximate first-order gradient, have attracted much attention recently in machine learning because of their broad applications. the accuracy of the ZO gradient highly depends on how many finite differences are averaged, which are intrinsically determined by the number of perturbations randomly drawn from a distribution. Existing ZO methods try to learn a data-driven distribution for sampling the perturbations to improve the efficiency of ZO optimization (ZOO) algorithms. In this paper, we explore a new and parallel direction, i.e., learn an optimal sampling policy instead of using a random strategy to generate perturbations based on the techniques of reinforcement learning (RL), which makes it possible to approximate the gradient with only two function evaluations. Specifically, we first formulate the problem of learning a sampling policy as a Markov decision process. then, we propose our ZO-RL algorithm, i.e., using deep deterministic policy gradient, an actor-critic RL algorithm to learn a sampling policy that can guide the generation of perturbed vectors in getting ZO gradients as accurately as possible. Importantly, the existing ZOO algorithms for learning a distribution can be plugged in to improve the exploration of ZO-RL. Experimental results with different ZO estimators show that our ZO-RL algorithm can effectively reduce the query complexity of ZOO algorithms and converge faster than existing ZOO algorithms, especially in the later stage of the optimization process.
the complexity of irrigation management necessitates the use of professional experience for effective decision-making. Expert systems (ES) are specifically engineered to mimic human expertise efficiently, allowing cog...
详细信息
ISBN:
(纸本)9783031686597;9783031686603
the complexity of irrigation management necessitates the use of professional experience for effective decision-making. Expert systems (ES) are specifically engineered to mimic human expertise efficiently, allowing cognitive mechanisms to be replicated at the irrigation level. the value of combining human knowledge with cutting-edge technologies such as fog computing, cloud computing, blockchain, IoT, and machine learning has recently come to be recognized more and more. the objective is to create an integrated platform that increases irrigation systems' resilience and adaptability while also increasing their accuracy, efficiency, and capacity for adaptive decision-making in light of agricultural environments' changing conditions. With a focus on ES, fuzzy logic, and machine learningalgorithms, this paper examines the research trends and applicability of intelligent approaches for optimal irrigation management scenarios. It also provides a thorough analysis of how these approaches contribute to the overall success of smart irrigation management.
Model-based offline reinforcement learning (RL) algorithms have emerged as a promising paradigm for offline RL. these algorithms usually learn a dynamics model from a static dataset of transitions, use the model to ge...
详细信息
ISBN:
(纸本)1577358872
Model-based offline reinforcement learning (RL) algorithms have emerged as a promising paradigm for offline RL. these algorithms usually learn a dynamics model from a static dataset of transitions, use the model to generate synthetic trajectories, and perform conservative policy optimization within these trajectories. However, our observations indicate that policy optimization methods used in these model-based offline RL algorithms are not effective at exploring the learned model and induce biased exploration, which ultimately impairs the performance of the algorithm. To address this issue, we propose Offline Conservative ExplorAtioN (OCEAN), a novel rollout approach to model-based offline RL. In our method, we incorporate additional exploration techniques and introduce three conservative constraints based on uncertainty estimation to mitigate the potential impact of significant dynamic errors resulting from exploratory transitions. Our work is a plug-in method and can be combined with classical model-based RL algorithms, such as MOPO, COMBO, and RAMBO. Experiment results of our method on the D4RL MuJoCo benchmark show that OCEAN significantly improves the performance of existing algorithms.
As an important branch of geophysics, the nonlinear identification method has played a unique role, while deep learning is still in the initial stage of geophysical research. To identify the geoelectric model of the d...
详细信息
作者:
Mpofu, K.T.Mthunzi-Kufa, P.Biophotonics
Council for Scientific and Industrial Research National Laser Centre Gauteng Pretoria0001 South Africa University of Cape Town
Division of Biomedical Engineering Department of Human Biology Western Cape Cape Town7701 South Africa
College of Graduate Studies University of South Africa Pretoria South Africa
Research on quantum computing is still in its infancy, but it has a lot of potential uses. One topic with potential is machine learning, namely in the field of reinforcement learning. this work examines the integratio...
详细信息
ISBN:
(纸本)9798350387988
Research on quantum computing is still in its infancy, but it has a lot of potential uses. One topic with potential is machine learning, namely in the field of reinforcement learning. this work examines the integration of parametrized quantum circuits (PQC) into reinforcement learning (RL) algorithms, assessing the potential of quantum-enhanced models to solve classical RL tasks. It closely follows the example found on the TensorFlow website. this paper reviews applications of quantum reinforcement learning (QRL). We examine PQCs in a standard RL scenario, the CartPole-v1 environment from Gym, using TensorFlow Quantum and Cirq, to evaluate the relative performance of quantum versus conventional models. In comparison to conventional deep neural network (ONN) models, PQCs show slower convergence and higher processing needs, even if they are still able to learn the task and perform competitively. After they are fully trained, the quantum models show unique difficulties during the early training stages and reach a performance stability level like classical methods. this study sheds light on the present constraints as well as possible uses of quantum computing in reinforcement learning, particularly in situations with intricate, high-dimensional settings that prove difficult for classical computers to handle effectively. As we look to the future, we suggest that investigating hybrid quantum-classical algorithms, developing quantum hardware, and using quantum RL for increasingly difficult tasks are essential first steps. the study presents findings from both a classical reinforcement learning algorithm and a quantum integrated reinforcement learning algorithm. To provide a reliable comparison between quantum reinforcement algorithms and their classical equivalents, further work remains. this work lays the groundwork for future advances in the field by investigating the viability and use of quantum algorithms in reinforcement learning, even if it is not particularly unique
the proceedings contain 360 papers. the topics discussed include: comparative analysis of machine learning and optimization techniques in chest X-ray Image Analysis for medical diagnosis;leveraging Mongo DB for effici...
ISBN:
(纸本)9798350350357
the proceedings contain 360 papers. the topics discussed include: comparative analysis of machine learning and optimization techniques in chest X-ray Image Analysis for medical diagnosis;leveraging Mongo DB for efficient data storage in MERN;authentication for online fraud detection through hidden Markov model;deep learning approach for optimal segmentation and classification of multi-class skin cancer;a machine learning system for predicting severity under single transmission line outages;survey on clustering problems using metaheuristic algorithms;an approach to recognize efficient deep learning model for pattern recognition;designing and development of AR based application for studying magnetism;a survey on blockchain for rental lease management;automated multi-page document classification and information extraction for insurance applications using deep learning techniques;and global perspectives in facial recognition for cyber security: a bibliometeric analysis.
the integration of Quantum Approximate optimization Algorithm (QAOA) and Quantum Annealing (QA) offers a promising approach to addressing energy management and load balancing challenges in modern power systems. this s...
详细信息
this paper focuses on the security consensus problem of multi-agent systems under cyber-attack. Two types of cyber-attack are considered in our research, this situation is more common in real applications. these two t...
详细信息
ISBN:
(纸本)9798350377859;9798350377842
this paper focuses on the security consensus problem of multi-agent systems under cyber-attack. Two types of cyber-attack are considered in our research, this situation is more common in real applications. these two types of network attacks (deception and DoS attacks) may occur alternately, resulting in the infeasibility to obtain actual output measurements. To reach the system security, a novel iterative learning control strategy is designed to analyze the issue. then, sufficient conditions are proposed from the view of the trajectory tracking, and our achieved convergence conditions are relatively simpler in comparison to the existing literature. Furthermore, the theoretical result in this paper is an extension of existing research. Finally, a numerical simulation is presented for illustration.
One of the main challenges in cloud computing is resource management, the ability to schedule workloads and services over the infrastructure in the most automated way. By optimizing cloud assignment and resource usage...
详细信息
ISBN:
(纸本)9783031779404;9783031779411
One of the main challenges in cloud computing is resource management, the ability to schedule workloads and services over the infrastructure in the most automated way. By optimizing cloud assignment and resource usage, energy can be saved, production incident can be anticipated and services QoS improved. Withthe recent years emergence of light virtualisation, known as containerization, the resource allocation problem was brought back, notably to support containers elasticity, hence the dynamic allocation of ressource at runtime at a single service scale. In this paper we show that using an hybrid loop system, which combines unsupervised learning and optimization techniques, our algorithm provides and iteratively improves scheduling solutions to containers resource assignment, enabling capacity planning over dynamic resource loads. Within our benchmarks, these solutions outperform state of the art algorithms, by an average of 6.3%, while providing more expressivity and control over input parameters. We describe also the implementation of this method, through an open source Python library called HOTS, which allows hybrid optimization for time series based use cases.
the Co-Evolutionary algorithms for Feature Selections delve into feature selection in the context of data-rich environments. the study aims to identify and implement a suitable co-evolutionary feature selection method...
详细信息
暂无评论