Original Blog Entry: NSB Entropy Estimation and related information theoretic quantities from undersampled discrete data

edge — Thu, 16 Jul 2009 04:03:43 GMT

We will be adding entropy studies into random numerical/pattern sequences research.

This research sparked my interest as it pertains directly to undersampled data.

Reference 3 points out possible applications of this algorithm:

Dynamical systems

1. Complexity of dynamics

2. Dimensions of strange attractors

as well as

rare events statistics

Reference 3, is actually extremely interesting even outside of theme of study we are doing (random games, random sequences), in fact, NSB Entropy is used by bio-physicist to study emergence of intelligent behaviour...

References:

1. The nsb-entropy project is devoted to implementation and practical use of the NSB algorithm for estimation of entropy and related information-theoretic quantities

from undersampled discrete data

http://nsb-entropy.sourceforge.net/

2. (Formal paper) Entropy and inference, revisited by Ilya Nemenman, Fariel Shafee, William Bialek

http://xxx.lanl.gov/abs/physics/0108025

3. (Power Point Presentation) A Bayesian Estimator of Entropies in a Severely Undersampled Regime: Theory and Applications to the Neural Code) by Ilya Nemenman

http://www.menem.com/~ilya/wiki/images/2/2a/LANL-D1.pdf

... [ More ]