Regularities Unseen, Randomness Observed: Levels of Entropy Convergence
Citation
J.P. Crutchfield and D. P. Feldman
Regularities Unseen, Randomness Observed: Levels of Entropy Convergence.
Chaos, 2003. 15: 25-54. 2003.
cond-mat
0102181.
Abstract
We study how the Shannon entropy of sequences produced by an
information source converges to the source's entropy rate. We
synthesize several phenomenological approaches to applying
information theoretic measures of randomness and memory to
stochastic and deterministic processes by using successive
derivatives of the Shannon entropy growth curve. This leads, in
turn, to
natural measures of apparent memory stored in a source and the
amounts of information that must be extracted from observations
of
a source in order for it to be optimally predicted and for an
observer to synchronize to it. One consequence of ignoring these
structural properties is that the missed regularities are
converted to apparent randomness. We demonstrate that this
problem arises
particularly for small data sets; e.g., in settings where one
has access only to short measurement sequences.
Download the Paper:
[Back to Publications Page]