检索结果-内蒙古大学图书馆

Sequence complexity and work extraction

JOURNAL OF STATISTICAL MECHANICS-THEORY AND EXPERIMENT 2015年第6期000卷 P06037-P06037页

作者： Merhav, Neri Technion Israel Inst Technol Dept Elect Engn IL-32000 Haifa Israel

We consider a simplified version of a solvable model by Mandal and Jarzynski, which constructively demonstrates the interplay between work extraction and the increase of the Shannon entropy of an information reservoir which is in contact with a physical system. We extend Mandal and Jarzynski's main findings in several directions: first, we allow sequences of correlated bits rather than just independent bits. Secondly, at least for the case of binary information, we show that, in fact, the Shannon entropy is only one measure of complexity of the information that must increase in order for work to be extracted. The extracted work can also be upper bounded in terms of the increase in other quantities that measure complexity, like the predictability of future bits from past ones. Third, we provide an extension to the case of non-binary information (i.e. a larger alphabet), and finally, we extend the scope to the case where the incoming bits (before the interaction) form an individual sequence, rather than a random one. In this case, the entropy before the interaction can be replaced by the Lempel-Ziv (LZ) complexity of the incoming sequence, a fact that gives rise to an entropic meaning of the LZ complexity, not only in information theory, but also in physics.

关键词： rigorous results in statistical mechanics exact results exact results source and channel coding

来源：评论

学校读者我要写书评

暂无评论

Variational Bayes and a problem of reliable communication: II. Infinite systems

引用

JOURNAL OF STATISTICAL MECHANICS-THEORY AND EXPERIMENT 2012年第11期2012卷 P11008-P11008页

作者： Newton, Nigel J. Mitter, Sanjoy K. Univ Essex Sch Comp Sci & Elect Engn Colchester CO4 3SQ Essex England MIT Informat & Decis Syst Lab Cambridge MA 02139 USA MIT Dept Elect Engn & Comp Sci Cambridge MA 02139 USA

We consider a family of estimation problems not admitting conventional analysis because of singularity and measurability issues. We define posterior distributions for the family by a variational technique analogous to that used to define Gibbs measures in statistical mechanics. The family of estimation problems, which arise in the asymptotic analysis of error-control codes, is parametrized by a code rate, R is an element of (0, infinity);this is shown to be analogous to the absolute temperature of statistical mechanics. The family undergoes an (Ehrenfest) first-order phase transition at a critical code rate C (the channel capacity), where there is a convex set of posterior distributions. At all other code rates, there is only one posterior distribution;if R < C, this is the Dirac measure located at the source sequence, whereas if R > C it has infinite support. In a result reflecting the Dobrushin construction, we show that these posterior distributions are asymptotically consistent with those of families of finite-sequence error-control codes.

关键词： exact results source and channel coding statistical inference

来源：评论

学校读者我要写书评

暂无评论

Typical reconstruction limits for distributed compressed sensing based on l_2,1-norm minimization and Bayesian optimal reconstruction

引用

JOURNAL OF STATISTICAL MECHANICS-THEORY AND EXPERIMENT 2015年第5期000卷 P05029-P05029页

作者： Shiraki, Yoshifumi Kabashima, Yoshiyuki NTT Corp NTT Commun Sci Lab Atsugi Kanagawa 2430198 Japan Tokyo Inst Technol Dept Computat Intelligence & Syst Sci Yokohama Kanagawa 2268502 Japan

The distributed compressed sensing framework provides an efficient compression scheme of multichannel signals that are sparse in some domains and highly correlated with one another. In particular, a signal model called the joint sparse model 2 (JSM-2) or multiple measurement vector problem, in which all sparse signals share their support, is important for dealing with practical problems such as magnetic resonance imaging and magnetoencephalography. In this paper, we investigate the typical reconstruction performance of JSM-2 problems for two schemes. One is l(2,1)-norm minimization reconstruction and the other is Bayesian optimal reconstruction. Employing the replica method, we show that the reconstruction performance of both schemes which exploit the knowledge of the sharing of the signal support overcomes that of their corresponding approaches for the single-channel compressed sensing problem. We also develop a computationally feasible approximate algorithm for performing the Bayes optimal scheme to validate our theoretical estimation. Our replica-based analysis numerically indicates that the spinodal point of the Bayesian reconstruction disappears, which implies that a fundamental reconstruction limit can be achieved by the BP-based approximate algorithm in a practical amount of time when the number of channels is sufficiently large. The results of the numerical experiments of both reconstruction schemes agree excellently with the theoretical evaluation.

关键词： message-passing algorithms source and channel coding communication supply and information networks

来源：评论

学校读者我要写书评

暂无评论

Stochastic optimization by message passing

引用

JOURNAL OF STATISTICAL MECHANICS-THEORY AND EXPERIMENT 2011年第11期2011卷

作者： Altarelli, F. Braunstein, A. Ramezanpour, A. Zecchina, R. Politecn Torino Dept Phys I-10129 Turin Italy Politecn Torino Ctr Computat Sci I-10129 Turin Italy Coll Carlo Alberto I-10024 Moncalieri Italy Human Genet Fdn I-10126 Turin Italy

Most optimization problems in applied sciences realistically involve uncertainty in the parameters defining the cost function, of which only statistical information is known beforehand. Here we provide an in-depth discussion of how message passing algorithms for stochastic optimization based on the cavity method of statistical physics can be constructed. We focus on two basic problems, namely the independent set problem and the matching problem, for which we display the general method and caveats for the case of the so called two-stage problem with independently distributed stochastic parameters. We compare the results with some greedy algorithms and briefly discuss the extension to more complicated stochastic multi-stage problems.

关键词： message-passing algorithms random graphs networks source and channel coding error correcting codes

来源：评论

学校读者我要写书评

暂无评论

A typical reconstruction limit for compressed sensing based on L_p-norm minimization

引用

JOURNAL OF STATISTICAL MECHANICS-THEORY AND EXPERIMENT 2009年第09期2009卷 L09003-L09003页

作者： Kabashima, Y. Wadayama, T. Tanaka, T. Tokyo Inst Technol Dept Computat Intelligence & Syst Sci Yokohama Kanagawa 2268502 Japan Nagoya Inst Technol Dept Comp Sci Nagoya Aichi 4668555 Japan Kyoto Univ Dept Syst Sci Kyoto 6068501 Japan

We consider the problem of reconstructing an N-dimensional continuous vector x from P constraints which are generated from its linear transformation under the assumption that the number of non-zero elements of x is typically limited to rho N (0 <= rho <= 1). Problems of this type can be solved by minimizing a cost function with respect to the L-p-norm parallel to x parallel to(p) = lim(epsilon ->+0) Sigma(N)(i=1) vertical bar x(i)vertical bar(p+epsilon), subject to the constraints under an appropriate condition. For several values of p, we assess a typical case limit alpha(c)(rho), which represents a critical relation between alpha = P/N and rho for successfully reconstructing the original vector by the minimization for typical situations in the limit N, P -> infinity while keeping alpha finite, utilizing the replica method. For p = 1, alpha(c)(rho) is considerably smaller than its worst case counterpart, which has been rigorously derived in the existing literature on information theory.

关键词： analysis of algorithms robust and stochastic optimization source and channel coding communication supply and information networks

来源：评论

学校读者我要写书评

暂无评论

Survey propagation as local equilibrium equations

引用

JOURNAL OF STATISTICAL MECHANICS-THEORY AND EXPERIMENT 2004年第06期2004卷 P06007-P06007页

作者： Braunstein, A Zecchina, R SISSA I-34100 Trieste Italy Abdus Salaam Int Ctr Theoret Phys I-34100 Trieste Italy

It has been shown experimentally that a decimation algorithm based on survey propagation (SP) equations allows one to solve efficiently some combinatorial problems over random graphs. We show that these equations can be derived as sum - product equations for the computation of marginals in an extended space where the variables are allowed to take an additional value - - when they are not forced by the combinatorial constraints. An appropriate 'local equilibrium condition' cost/energy function is introduced and its entropy is shown to coincide with the expected logarithm of the number of clusters of solutions as computed by SP. These results may help to clarify the geometrical notion of clusters assumed by SP for random K-SAT or random graph colouring ( where it is conjectured to be exact) and help to explain which kind of clustering operation or approximation is enforced in general/small sized models in which it is known to be inexact.

关键词： cavity and replica method message-passing algorithms source and channel coding

来源：评论

学校读者我要写书评

暂无评论

Analogy between gambling and measurement-based work extraction

引用

JOURNAL OF STATISTICAL MECHANICS-THEORY AND EXPERIMENT 2016年第4期2016卷 043403-043403页

作者： Vinkler, Dror A. Permuter, Haim H. Merhav, Neri Ben Gurion Univ Negev Dept Elect & Comp Engn POB 653 IL-84105 Beer Sheva Israel Technion Dept Elect Engn IL-32000 Haifa Israel

In information theory, one area of interest is gambling, where mutual information characterizes the maximal gain in wealth growth rate due to knowledge of side information;the betting strategy that achieves this maximum is named the Kelly strategy. In the field of physics, it was recently shown that mutual information can characterize the maximal amount of work that can be extracted from a single heat bath using measurement-based control protocols, i. e. using 'information engines'. However, to the best of our knowledge, no relation between gambling and information engines has been presented before. In this paper, we briefly review the two concepts and then demonstrate an analogy between gambling, where bits are converted into wealth, and information engines, where bits representing measurements are converted into energy. From this analogy follows an extension of gambling to the continuous-valued case, which is shown to be useful for investments in currency exchange rates or in the stock market using options. Moreover, the analogy enables us to use well-known methods and results from one field to solve problems in the other. We present three such cases: maximum work extraction when the probability distributions governing the system and measurements are unknown, work extraction when some energy is lost in each cycle, e. g. due to friction, and an analysis of systems with memory. In all three cases, the analogy enables us to use known results in order to obtain new ones.

关键词： exact results source and channel coding statistical inference communication supply and information networks

来源：评论

学校读者我要写书评

暂无评论

Amorphous packings of hard spheres for large space dimension

引用

JOURNAL OF STATISTICAL MECHANICS-THEORY AND EXPERIMENT 2006年第3期2006卷 P03017-P03017页

作者： Parisi, G Zamponi, F Univ Roma La Sapienza Dipartimento Fis I-00185 Rome Italy Univ Roma La Sapienza Ist Nazl Fis Nucl INFM CRSSMC I-00185 Rome Italy Ecole Normale Super Phys Theor Lab F-75231 Paris 05 France

In a recent paper we derived an expression for the replicated free energy of a liquid of hard spheres based on the hypernetted chain ( HNC) free energy functional. An approximate equation of state for the glass and an estimate of the random close packing density were obtained for d = 3. Here we show that the HNC approximation is not needed: the same expression can be obtained from the full diagrammatic expansion of the replicated free energy. Then, we consider the asymptotics of this expression when the space dimension d is very large. In this limit, the entropy of the hard sphere liquid has been computed exactly. Using this solution, we derive asymptotic expressions for the glass transition density and for the random close packing density for hard spheres for large space dimension.

关键词： cavity and replica method disordered systems (theory) structural glasses (theory) source and channel coding

来源：评论

学校读者我要写书评

暂无评论

Relative entropy via non-sequential recursive pair substitution

引用

JOURNAL OF STATISTICAL MECHANICS-THEORY AND EXPERIMENT 2010年第09期2010卷 P09010-P09010页

作者： Benedetto, D. Caglioti, E. Cristadoro, G. Esposti, M. Degli Univ Roma La Sapienza Dipartimento Matemat I-00185 Rome Italy Univ Bologna Dipartmento Matemat I-40126 Bologna Italy

The entropy of an ergodic source is the limit of properly rescaled oneblock entropies of sources obtained applying successive non-sequential recursive pair substitutions (NSRPS). In this paper we prove that the cross-... 详细信息

关键词： source and channel coding

来源：评论

学校读者我要写书评

暂无评论

Passage-time coding with a timing kernel inferred from irregular cortical spike sequences

引用

JOURNAL OF STATISTICAL MECHANICS-THEORY AND EXPERIMENT 2013年第03期2013卷 P03004-P03004页

作者： Tsubo, Yasuhiro Isomura, Yoshikazu Fukai, Tomoki RIKEN Brain Sci Inst Lab Neural Circuit Theory Wako Saitama 3510198 Japan Tamagawa Univ Brain Sci Inst Machida Tokyo 1948610 Japan JST CREST Kawaguchi Saitama 3320012 Japan

In vivo cortical neurons exhibit highly irregular spike sequences. However, the computational implications of this irregular firing with respect to neural codings are not yet fully understood. Recently, we formulated neuronal firing as a stochastic process to translate the firing rate determined by synaptic input into an irregular sequence of inter-spike intervals (ISIs). We previously determined that steady-state distributions of ISIs obey a power-law in the majority of neurons recorded from the sensorimotor cortex of a rat performing a forelimb-movement task. This observation led us to a hypothesis for a neural code with irregular spiking, which we termed as 'constrained maximization of firing-rate entropy' (CMFE). This hypothesis asserts that noisy neuronal activity maximizes steady-state firing-rate entropy under the joint constraints of energy consumption and uncertainty over output spike trains. CMFE previously assumed that the values of the rate parameters in two successive ISIs are random and uncorrelated with each other. However, this is an oversimplification and is unrealistic. In addition, the CMFE hypothesis seems to contradict our perception that the firing-rate distribution is dependent on external stimuli. Therefore, it is necessary to explore a rate-coding scheme that can resolve these issues. In this study, we review the concept of CMFE and extend it to incorporate the correlated nature of a firing rate sequence in a biological nervous system. We introduced passage-time coding of stimulus with the timing kernel inferred from irregular cortical spike sequences in our previous study. We showed how the timing kernel determines the amount of information in spike sequences.

关键词： stochastic processes (theory) source and channel coding neural code computational neuroscience

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：