检索结果-内蒙古大学图书馆

Simplicity bias, algorithmic probability, and the random logistic map

PHYSICA D-NONLINEAR PHENOMENA 2024年 463卷

作者： Hamzi, Boumediene Dingle, Kamaludin Caltech Dept Comp & Math Sci Pasadena CA 91125 USA Alan Turing Inst London England Gulf Univ Sci & Technol Ctr Appl Math & Bioinformat Dept Math & Nat Sci Mubarak Al Abdullah Kuwait

Simplicity bias is an intriguing phenomenon prevalent in various input-output maps, characterized by a preference for simpler, more regular, or symmetric outputs. Notably, these maps typically feature high- probability outputs with simple patterns, whereas complex patterns are exponentially less probable. This bias has been extensively examined and attributed to principles derived from algorithmic information theory and algorithmic probability. In a significant advancement, it has been demonstrated that the renowned logistic map x(k+1) = mu x(k)(1 - x(k)), a staple in dynamical systems theory, and other one-dimensional maps exhibit simplicity bias when conceptualized as input-output systems. Building upon this work, our research delves into the manifestations of simplicity bias within the random logistic map, specifically focusing on scenarios involving additive noise. This investigation is driven by the overarching goal of formulating a comprehensive theory for the prediction and analysis of time series. Our primary contributions are multifaceted. We discover that simplicity bias is observable in the random logistic map for specific ranges of mu and noise magnitudes. Additionally, we find that this bias persists even with the introduction of small measurement noise, though it diminishes as noise levels increase. Our studies also revisit the phenomenon of noise-induced chaos, particularly when mu = 3.83, revealing its characteristics through complexity-probability plots. Intriguingly, we employ the logistic map to illustrate a paradoxical aspect of data analysis: more data adhering to a consistent trend can occasionally lead to reduced confidence in extrapolation predictions, challenging conventional wisdom. We propose that adopting a probability-complexity perspective in analyzing dynamical systems could significantly enrich statistical learning theories related to series prediction and analysis. This approach not only facilitates a deeper understanding of simplici

关键词： Random dynamical systems algorithmic probability Simplicity bias Time series Machine learning

来源：评论

学校读者我要写书评

暂无评论

Multiclass classification utilising an estimated algorithmic probability prior

引用

PHYSICA D-NONLINEAR PHENOMENA 2023年 448卷

作者： Dingle, Kamaludin Batlle, Pau Owhadi, Houman CALTECH Dept Comp & Math Sci Pasadena CA 91125 USA Gulf Univ Sci & Technol Ctr Appl Math & Bioinformat Dept Math & Nat Sci Hawally Kuwait

Methods of pattern recognition and machine learning are applied extensively in science, technology, and society. Hence, any advances in related theory may translate into large-scale impact. Here we explore how algorithmic information theory, especially algorithmic probability, may aid in a machine learning task. We study a multiclass supervised classification problem, namely learning the RNA molecule sequence-to-shape map, where the different possible shapes are taken to be the classes. The primary motivation for this work is a proof of concept example, where a concrete, well-motivated machine learning task can be aided by approximations to algorithmic probability. Our approach is based on directly estimating the class (i.e., shape) probabilities from shape complexities, and using the estimated probabilities as a prior in a Gaussian process learning problem. Naturally, with a large amount of training data, the prior has no significant influence on classification accuracy, but in the very small training data regime, we show that using the prior can substantially improve classification accuracy. To our knowledge, this work is one of the first to demonstrate how algorithmic probability can aid in a concrete, real-world, machine learning problem.& COPY;2023 Elsevier B.V. All rights reserved.

关键词： Machine learning Multiclass classification Gaussian processes algorithmic probability RNA

来源：评论

学校读者我要写书评

暂无评论

Predicting phenotype transition probabilities via conditional algorithmic probability approximations

引用

JOURNAL OF THE ROYAL SOCIETY INTERFACE 2022年第197期19卷 20220694-20220694页

作者： Dingle, Kamaludin Novev, Javor K. Ahnert, Sebastian E. Louis, Ard A. Univ Cambridge Dept Chem Engn & Biotechnol Cambridge CB2 1TN Cambs England CALTECH Dept Comp & Math Sci Pasadena CA 91125 USA Gulf Univ Sci & Technol Ctr Appl Math & Bioinformat CAMB Dept Math & Nat Sci Mubarak Al Abdullah 32093 Kuwait Univ Oxford Rudolf Peierls Ctr Theoret Phys Dept Phys Oxford OX1 2JD Oxon England

Unravelling the structure of genotype-phenotype (GP) maps is an important problem in biology. Recently, arguments inspired by algorithmic information theory (AIT) and Kolmogorov complexity have been invoked to uncover simplicity bias in GP maps, an exponentially decaying upper bound in phenotype probability with the increasing phenotype description] complexity. This means that phenotypes with many genotypes assigned via the GP map must be simple, while complex phenotypes must have few genotypes assigned. Here, we use similar arguments to bound the probability P(x -> y) that phenotype x, upon random genetic mutation, transitions to phenotype y. The bound is P(x -> y) less than or similar to 2(-a (K) over tilde (y vertical bar x)-b), where (K) over tilde (y vertical bar x) is the estimated conditional complexity of y given x, quantifying how much extra information is required to make y given access to x. This upper bound is related to the conditional form of algorithmic probability from AIT. We demonstrate the practical applicability of our derived bound by predicting phenotype transition probabilities (and other related quantities) in simulations of RNA and protein secondary structures. Our work contributes to a general mathematical understanding of GP maps and may facilitate the prediction of transition probabilities directly from examining phenotype themselves, without utilizing detailed knowledge of the GP map.

关键词： genotype-phenotype maps evolution complexity algorithmic probability

来源：评论

学校读者我要写书评

暂无评论

A Generalized Characterization of algorithmic probability

引用

THEORY OF COMPUTING SYSTEMS 2017年第4期61卷 1337-1352页

作者： Sterkenburg, Tom F. Ctr Wiskunde & Informat Algorithms & Complex Grp Amsterdam Netherlands Univ Groningen Fac Philosophy Groningen Netherlands

An a priori semimeasure (also known as "algorithmic probability" or "the Solomonoff prior" in the context of inductive inference) is defined as the transformation, by a given universal monotone Turing machine, of the uniform measure on the infinite strings. It is shown in this paper that the class of a priori semimeasures can equivalently be defined as the class of transformations, by all compatible universal monotone Turing machines, of any continuous computable measure in place of the uniform measure. Some consideration is given to possible implications for the association of algorithmic probability with certain foundational principles of statistics.

关键词： algorithmic probability A priori semimeasure Semicomputable semimeasures Monotone turing machines Principle of indifference Occam's razor

来源：评论

学校读者我要写书评

暂无评论

A note on a priori forecasting and simplicity bias in time series

引用

PHYSICA A-STATISTICAL MECHANICS AND ITS APPLICATIONS 2023年 609卷

作者： Dingle, Kamaludin Kamal, Rafiq Hamzi, Boumediene Gulf Univ Sci & Technol Ctr Appl Math & Bioinformat Dept Math & Nat Sci Mubarak Al Abdullah Kuwait Gulf Univ Sci & Technol Dept English Mubarak Al Abdullah Kuwait Caltech Dept Comp & Math Sci Pasadena CA USA

To what extent can we forecast a time series without fitting to historical data? Can universal patterns of probability help in this task? Deep relations between pattern Kolmogorov complexity and pattern probability have recently been used to make a priori probability predictions in a variety of systems in physics, biology and engineering. Here we study simplicity bias (SB) - an exponential upper bound decay in pattern probability with increasing complexity - in discretised time series extracted from the World Bank Open Data collection. We predict upper bounds on the probability of discretised series patterns, without fitting to trends in the data. Thus we perform a kind of 'forecasting without training data', predicting time series shape patterns a priori, but not the actual numerical value of the series. Additionally we make predictions about which of two discretised series is more likely with accuracy of & SIM;80%, much higher than a 50% baseline rate, just by using the complexity of each series. These results point to a promising perspective on practical time series forecasting and integration with machine learning methods. & COPY;2022 The Author(s). Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://***/licenses/by-nc-nd/4.0/).

关键词： Time series Simplicity bias Kolmogorov complexity algorithmic probability Forecasting

来源：评论

学校读者我要写书评

暂无评论

The Semimeasure Property of algorithmic probability - "Feature" or "Bug"?

The Semimeasure Property of Algorithmic Probability - "Featu...

引用

Ray Solomonoff 85th Memorial Conference

作者： Campbell, Douglas Univ Canterbury Philosophy Programme Christchurch 8041 New Zealand

ISBN: (纸本)9783642449581;9783642449574

An unknown process is generating a sequence of symbols, drawn from an alphabet, A. Given an initial segment of the sequence, how can one predict the next symbol? Ray Solomonoff's theory of inductive reasoning rests on the idea that a useful estimate of a sequence's true probability of being outputted by the unknown process is provided by its algorithmic probability (its probability of being outputted by a species of probabilistic Turing machine). However algorithmic probability is a "semimeasure": i.e., the sum, over all x is an element of A, of the conditional algorithmic probabilities of the next symbol being x, may be less than 1. Solomonoff thought that algorithmic probability must be normalized, to eradicate this semimeasure property, before it can yield acceptable probability estimates. This paper argues, to the contrary, that the semimeasure property contributes substantially, in its own right, to the power of an algorithmic-probability-based theory of induction, and that normalization is unnecessary.

关键词： algorithmic probability sequence prediction inductive reasoning Solomonoff induction Solomonoff normalization semimeasure convergence theorem

来源：评论

学校读者我要写书评

暂无评论

Practical algorithmic probability: An Image Inpainting Example

Practical Algorithmic Probability: An Image Inpainting Examp...

引用

6th International Conference on Machine Vision (ICMV)

作者： Potapov, Alexey Scherbakov, Oleg Zhdanov, Innokentii Natl Res Univ Informat Technol St Petersburg Russia

ISBN: (纸本)9780819499967

Possibility of practical application of algorithmic probability is analyzed on an example of image inpainting problem that precisely corresponds to the prediction problem. Such consideration is fruitful both for the theory of universal prediction and practical image inpaiting methods. Efficient application of algorithmic probability implies that its computation is essentially optimized for some specific data representation In this paper, we considered one image representation, namely spectral representation, for which an image inpainting algorithm is proposed based on the spectrum entropy criterion. This algorithm showed promising results in spite of very simple representation The same approach can be used for introducing ALP-based criterion for more powerful image representations.

关键词： algorithmic probability image inpainting spectrum entropy

来源：评论

学校读者我要写书评

暂无评论

On the Application of algorithmic probability to Autoregressive Models

On the Application of Algorithmic Probability to Autoregress...

引用

Ray Solomonoff 85th Memorial Conference

作者： Solomonoff, Ray J. Saleeby, Elias G. Alfaisal Univ Riyadh Saudi Arabia

ISBN: (纸本)9783642449581;9783642449574

Typically, the first step in carrying out predictions is to develop an inductive model. In many instances, the best model is used, and it is often selected based on certain relevant criteria that may possibly ignore some of the model uncertainties. The Bayesian approach to model selection uses a weighted average of a class of models thereby overcoming some of the uncertainties associated with selecting the best model. This approach is referred to in the literature as the Bayesian Model Averaging (BMA) approach. It turns out that this approach has significant overlap with the theory of algorithmic probability (ALP) developed by R. J. Solomonoff in the early 1960s. The purpose of this article is to first highlight this connection by applying ALP to a set of nested stationary autoregressive time series models, and to give an algorithm to compute "relative weights" of models. This reveals, although empirically, a model weight that can be compared with the Schwarz Bayesian Selection criterion (BIC or SIC). We then develop an elementary algorithm of the Monte Carlo type to evaluate multidimensional integrals over stability domains, and use it to compute what we call the "trimmed weights".

关键词： algorithmic probability predictive probability model weights information criteria autoregressive time series Monte Carlo method

来源：评论

学校读者我要写书评

暂无评论

Queueing-Inventory Models with Batch Demands and Positive Service Times

引用

AUTOMATION AND REMOTE CONTROL 2020年第4期81卷 713-730页

作者： Chakravarthy, S. R. Kettering Univ Dept Ind & Mfg Engn Flint MI 48504 USA Kettering Univ Dept Math Flint MI 48504 USA

We consider queueing-inventory models in which customers arrive according to a point process and each customer demands for one or more items but not exceeding a predetermined (finite) value, say, N. The demands of the customers require positive service times. Replenishments are based on (s, S)-type policy and the lead times are assumed to be random. We consider two models. In Model 1, any arriving customer finding the inventory level to be zero will be lost. In Model 2, the loss of customers occur in two ways. First, an arriving customer finding the inventory level to be zero with the server being idle will be lost, and secondly, the customers, if any, present at a service completion with zero inventory will all be lost. We assume that in both the models the demands may be met partially based on the requests and the available inventory levels at that time. The inventory level is reduced by the amount to meet (fully or partially) the requisite demand of the customer at the beginning of the service. Under the assumption that all underlying random variables are exponential, we perform the steady-state analysis of the models using the classical matrix-analytic methods. Illustrative examples comparing the two models are presented.

关键词： queueing-inventory systems algorithmic probability batch demands lead times matrix-analytic methods

来源：评论

学校读者我要写书评

暂无评论

Confidence intervals, significance values, maximum likelihood estimates, etc. sharpened into Occam's razors

引用

COMMUNICATIONS IN STATISTICS-THEORY AND METHODS 2020年第11期49卷 2703-2712页

作者： Bickel, David R. Univ Ottawa Ottawa Inst Syst Biol Dept Biochem Microbiol & Immunol Dept Math & Stat 451 Smyth Rd Ottawa ON K1H 8M5 Canada

Confidence sets, p values, maximum likelihood estimates, and other results of non-Bayesian statistical methods may be adjusted to favor sampling distributions that are simple compared to others in the parametric famil... 详细信息

关键词： algorithmic probability confidence distribution differential entropy fiducial inference Kolmogorov complexity probability matching prior distribution

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：