We propose a theoretical framework for approximate planning and learning in partially observed systems. Our framework is based on the fundamental notion of information state. We provide two definitions of information ...
详细信息
We propose a theoretical framework for approximate planning and learning in partially observed systems. Our framework is based on the fundamental notion of information state. We provide two definitions of information state--i) a function of history which is sufficient to compute the expected reward and predict its next value; ii) a function of the history which can be recursively updated and is sufficient to compute the expected reward and predict the next observation. An information state always leads to a dynamic programming decomposition. Our key result is to show that if a function of the history (called approximate information state (AIS)) approximately satisfies the properties of the information state, then there is a corresponding approximate dynamic program. We show that the policy computed using this is approximately optimal with bounded loss of optimality. We show that several approximations in state, observation and action spaces in literature can be viewed as instances of AIS. In some of these cases, we obtain tighter bounds. A salient feature of AIS is that it can be learnt from data. We present AIS based multi-time scale policy gradient algorithms and detailed numerical experiments with low, moderate and high dimensional environments.
In this paper, we present an experimental study of L10-FePt granular films with crystalline/amorphous boron nitride (BN) grain boundary materials for heat assisted magnetic recording (HAMR). It is found that an adequa...
详细信息
Sorting and searching are critical processes for effecttive data analysis. In this paper, we evaluate the performance of various sorting and searching algorithms and compare their time and space complexities on both s...
Sorting and searching are critical processes for effecttive data analysis. In this paper, we evaluate the performance of various sorting and searching algorithms and compare their time and space complexities on both sorted and unsorted data. The algorithms we analyzed include six common sorting algorithms (insertion, radix, bucket, merge, bubble, and quick sort) and three search algorithms (linear, binary, and jump search). The results of our study provide insights into the best algorithms to use for different input sizes and types of data. It was found that for small input sizes, all algorithms perform similarly, but for larger input sizes, insertion and radix sorts are better for time complexity while bubble sort is better for space complexity. Additionally, jump search outperformed linear and binary search algorithms in both time and space complexity. Besides, difference between time and space complexity of sorted and unsorted data was significant.
A linear fiber laser system for measurements of paracetamol concentration is experimentally demonstrated. The cavity is based on a fiber loop mirror and an FBG centered at 1567.8 nm. The sensing head corresponds to a ...
A linear fiber laser system for measurements of paracetamol concentration is experimentally demonstrated. The cavity is based on a fiber loop mirror and an FBG centered at 1567.8 nm. The sensing head corresponds to a refractometric sensor, whose which principle of operation is based on Fresnel reflection in the fiber tip (FBG side). The system works at detected variations of paracetamol concentrations with a sensitivity of $[(8.74\pm 0.34)\times 10^{-5}]\ \ \mu \mathrm{W}/(\mathrm{g}/\text{kg})$ and a resolution of 2.77 g/kg. The results prove that the fiber laser system could be an asset for processing industries, specifically for non-invasive and real-time measurements of concentration.
This paper investigates the energy-efficient beamforming design in a simultaneous transmission and reflection-reconfigurable intelligent surface (STAR-RIS) assisted wireless communication system, where the antenna sel...
This paper investigates the energy-efficient beamforming design in a simultaneous transmission and reflection-reconfigurable intelligent surface (STAR-RIS) assisted wireless communication system, where the antenna selection scheme is adopted. An energy efficiency (EE) maximization problem is formulated by optimizing the transmit beamformers and the phase shift vectors subject to the power budget constraint of the base station (BS), the maximum transmit power constraint per antenna and the users' data rate requirements. An alternating optimization-based algorithm is proposed to tackle the coupled variables, and the quadratic transform is used to deal with the fractional formulations. Simulation results demonstrate that the antenna selection scheme can significantly improve the EE performance by suppressing the energy consumption due to massive antennas. With the assistance of the STAR-RIS, the EE performance is further enhanced.
Immunotherapy utilizes the potential of the patient's immune system to fight disease. T-lymphocytes have become the central focus of the immune system in the fight against cancer. One of the fundamental studies in...
Immunotherapy utilizes the potential of the patient's immune system to fight disease. T-lymphocytes have become the central focus of the immune system in the fight against cancer. One of the fundamental studies in immunotherapy is how to accurately identify various types of immune cells. Therefore, the study of immune-related cells will be an important reference for the treatment of COVID19 or other related diseases. In this paper, a series of experiments were conducted with COVID-19 patients as the starting point to design a strategy to identify the important subtypes of CD$4^{+}$T cells. In this paper, an intelligent machine learning algorithm is proposed according to the results of different experiments of linear and nonlinear dimension reduction algorithms, integrating the important characteristics of these two algorithms and improving the performance. Our experimental results show that the algorithm designed based on the front-end experiments can effectively distinguish the important lymphocytes. The proposed algorithm not only improves the performance, but also the dimension reduction procedure keeps the important features of both global and regional data. The experimental results of this paper will be beneficial for the subsequent analysis of the clustering in identifying important subtypes of CD$4^{+}$T cells. Most importantly, this wm significantly improve the procedure of manually integrating and labeling the recorded data after several tedious and monotonous analyses with external software.
This paper studies a class of strongly monotone games involving non-cooperative agents that optimize their own time-varying cost functions. We assume that the agents can observe other agents' historical actions an...
This paper studies a class of strongly monotone games involving non-cooperative agents that optimize their own time-varying cost functions. We assume that the agents can observe other agents' historical actions and choose actions that best respond to other agents' previous actions; we call this a best response scheme. We start by analyzing the convergence rate of this best response scheme for standard time-invariant games. Specifically, we provide a sufficient condition on the strong monotonicity parameter of the time-invariant games under which the proposed best response algorithm achieves exponential convergence to the static Nash equilibrium. We further illustrate that this best response algorithm may oscillate when the proposed sufficient condition fails to hold, which indicates that this condition is tight. Next, we analyze this best response algorithm for time-varying games where the cost functions of each agent change over time. Under similar conditions as for time-invariant games, we show that the proposed best response algorithm stays asymptotically close to the evolving equilibrium. We do so by analyzing both the equilibrium tracking error and the dynamic regret. Numerical experiments on economic market problems are presented to validate our analysis.
Miniaturized autonomous unmanned aerial vehicles (UAVs) are an emerging and trending topic. With their form factor as big as the palm of one hand, they can reach spots otherwise inaccessible to bigger robots and safel...
Miniaturized autonomous unmanned aerial vehicles (UAVs) are an emerging and trending topic. With their form factor as big as the palm of one hand, they can reach spots otherwise inaccessible to bigger robots and safely operate in human surroundings. The simple electronics aboard such robots (sub-100 mW) make them particularly cheap and attractive but pose significant challenges in enabling onboard sophisticated intelligence. In this work, we leverage a novel neural architecture search (NAS) technique to automatically identify several Pareto-optimal convolutional neural networks (CNNs) for a visual pose estimation task. Our work demonstrates how reallife and field-tested robotics applications can concretely leverage NAS technologies to automatically and efficiently optimize CNNs for the specific hardware constraints of small UAVs. We deploy several NAS-optimized CNNs and run them in closed-loop aboard a 27-g Crazyflie nano-UAV equipped with a parallel ultra-low power System-on-Chip. Our results improve the State-of-the-Art by reducing the in-field control error of 32% while achieving a real-time onboard inference-rate of ~10Hz@10mW and ~50Hz@90mW.
In this paper, we propose a dynamic hybrid active-passive reconfigurable intelligent surface (HRIS) to enhance multiple-input-single-output (MISO) communications, leveraging the property of dynamically placing active ...
详细信息
Traditional machine learning is generally treated as a black-box optimization problem and does not typically produce interpretable functions that connect inputs and outputs. However, the ability to discover such inter...
详细信息
暂无评论