Sign in to follow this  
Puzzler183

Calculating information entropy

Recommended Posts

After much Googling, I wasn't able to find a good source of information on information entropy and how to calculate it from a stream of binary data. A few of the sites I read lead me to beleive that this would work (C#):
			float chiSq = 0;
			float expected = (float) length / freq.Length;
			for (int i = 0; i < freq.Length; ++i)
				chiSq += (freq[i] - expected) * (freq[i] - expected) / expected;
			Console.WriteLine("Length: " + length);
			Console.WriteLine("Chi-Squared: " + chiSq);
			Console.WriteLine("Entropy: " + (float) length / chiSq);


Where freq is the frequency table (usually 256 elements), and length is the length of the input. However, this value seems to grow arbitrarily and I was wondering how entropy bits per actual byte is calculated?

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this