Sign in to follow this  
Crazyfool

Base 256

Recommended Posts

Howdy - I'd say I'm a beginner but I've been using base 256 a lot (turning a number into a char symbol to reduce packet size) and decided, for fun, to develope a Base256 class (not completely done) and was wondering if this would be something useful for network programmers? I know it's not very hard to implement, and most often people wont be dealing with too many numbers larger than 256, but I came upon a problem resulting from data length being larger than 256 (I used one byte to tell packet size) and thought of ways to handle it. So as of now, It converts a piece of data up to, 256^6 + 256^5 + ... + 255 (adding 1 would make it 7 bytes long) and I think this would really handle nicely for those sending large numbers through the network.. and possibly other uses. What do you think?

Share this post


Link to post
Share on other sites
Quote:
So as of now, It converts a piece of data up to, 256^6 + 256^5 + ... + 255 (adding 1 would make it 7 bytes long) and I think this would really handle nicely for those sending large numbers through the network.. and possibly other uses.


How many bytes would an int (32-bit) occupy if stored in base 256?

Share this post


Link to post
Share on other sites
I guess I had things wrong..

I thought that it sent stuff digit by digit, i.e. 12345 would be 5 bytes, but 1 is only 1 byte..

Meh, cant blame me for trying to help =/

Edit: Btw, my original idea was to sacrifice memory/processing in favor of bandwith.

Share this post


Link to post
Share on other sites
You want to use minimum number of bits (base-2 notation) to send things over network.

For example, let's say your numbers are in range 500-1000 only.

Your range is 500, or, it takes 9 bits to fully represent such number.

Then, you need to specify how many bits you used. This is a bit harder, since you need to use variable number. Ideally, you don't even send this number across the wire, but is defined by protocol.

So, instead of sending 32 bits over network, you send only 9, or less than 30%.

The other option is to not rely on such tweaks, since you need to do a lot of weird shifting, and simply pass it through some entropy compressor, and achieve 50-75% compression.

Last option, is not to compress data. This isn't as invalid as it sounds, especially if content is very dense.

Share this post


Link to post
Share on other sites
See, I didnt even know you could send bits at a time (though I dont think you'd wanna send only a few bits in one send, but rather 100 bytes compressed to 30% of what they would use, because of the packet's header and such, right?).

Heh, well it was a fun thought of being able to help others ;) Some day I'll contribute :D

Thanks.

Share this post


Link to post
Share on other sites
The trick question I asked above: Every data stored in memory is stored in base 256, if you assume the memory is composed of 8-bit blocks. Each byte in this case represents one base-256 digit.

So if you open a file in hex editor, the hex chars are basically base 256 digits.

So there is no need to convert the numbers, you just use pointer into memory.

Share this post


Link to post
Share on other sites
My binary packet tutorial. Ideally if you design your own you can set it up to allow for a number of bits. For instance, my C++ packet can easily be updated to do just that. The boolean data type in the packet takes 1 bit, and the others use the normal 32 bit "default" sizes shown in most C++ books.
http://gpwiki.org/index.php/Binary_Packet

I reccomend learning bit operators. They are a life saver for bandwidth optimization.
Here's a tutorial. Have fun!
http://www.gmonline.demon.co.uk/cscene/CS9/CS9-02.html

Share this post


Link to post
Share on other sites
Anth/Sirisian, thank you both very much. Anth, yes, you pretty much made me realize everything is basically base 256 (I was thinking, it seemed too good NOT to be the case now)) and Sirisian thanks for pointing out the links.. I have em both open now :)

Share this post


Link to post
Share on other sites
Quote:
Original post by Antheus
The trick question I asked above: Every data stored in memory is stored in base 256, if you assume the memory is composed of 8-bit blocks. Each byte in this case represents one base-256 digit.

So if you open a file in hex editor, the hex chars are basically base 256 digits.

So there is no need to convert the numbers, you just use pointer into memory.



No, a hex editor displays in base-16. (Hex is short for hexadecimal -- 16)
http://en.wikipedia.org/wiki/Hexadecimal

If you had a 256 based system you would need 256 seperate symbols for the digits (and using ascii chars even would have problems as many are unprintable control codes). 256 different symbols for digits would likewise be rather difficult to memorize (for most people).

I once made a printable code system for transmitting binary data via Emails and it was base 64 ( a-z A-Z 0-9 + a few punctuation chars to round out to 64).
I didnt use a standard one like 'Base64' because I included my own encryptor
and a base 128 system like 'uuencoding' had problems with the Email mangling the Ascii.


http://www.natural-innovations.com/binhex/

Share this post


Link to post
Share on other sites
Quote:
Original post by Crazyfool
I guess I had things wrong..

I thought that it sent stuff digit by digit, i.e. 12345 would be 5 bytes, but 1 is only 1 byte..

Meh, cant blame me for trying to help =/

Edit: Btw, my original idea was to sacrifice memory/processing in favor of bandwith.
That's still base 10, but uses a encoding similiar to BCD, only half as efficient.
BCD only used half a byte per digit, whereas you were using a whole byte per digit. Then of course you have to have a way of indicating when there are no more digits in each number. Basically it is a string, except not using the ASCII character set. So you should either just use a string, or just transmit it as the original bytes.

Share this post


Link to post
Share on other sites
Quote:
Original post by wodinoneeye

No, a hex editor displays in base-16. (Hex is short for hexadecimal -- 16)
http://en.wikipedia.org/wiki/Hexadecimal

If you had a 256 based system you would need 256 seperate symbols for the digits (and using ascii chars even would have problems as many are unprintable control codes). 256 different symbols for digits would likewise be rather difficult to memorize (for most people).

I once made a printable code system for transmitting binary data via Emails and it was base 64 ( a-z A-Z 0-9 + a few punctuation chars to round out to 64).
I didnt use a standard one like 'Base64' because I included my own encryptor
and a base 128 system like 'uuencoding' had problems with the Email mangling the Ascii.


http://www.natural-innovations.com/binhex/



Most hex editors provide provide data dump as well, which is ascii character. Some of them provide complete 8-bit ASCII set.

And your inability to display the characters doesn't change the encoding. Unicode is an example. If it can't display the glyph, that doesn't mean the character has been misteriously converted into something else - its value is still there.

And although ASCII has unprintable characters, this doesn't change the fact that each character is unique - even if you can't see it.

It's also possible to draw n-dimensional objects on computer screen (where n > 3). And despite our screens being limited to 2D, the object rendered is still n-dimensional, although we can't represent it.

Think of hex viewer as a 2D renderer for 4D objects. It can't display everything, but it displays base 256 number accurately.

Share this post


Link to post
Share on other sites
Quote:
Original post by Antheus
Quote:
Original post by wodinoneeye

No, a hex editor displays in base-16. (Hex is short for hexadecimal -- 16)
http://en.wikipedia.org/wiki/Hexadecimal

If you had a 256 based system you would need 256 seperate symbols for the digits (and using ascii chars even would have problems as many are unprintable control codes). 256 different symbols for digits would likewise be rather difficult to memorize (for most people).

I once made a printable code system for transmitting binary data via Emails and it was base 64 ( a-z A-Z 0-9 + a few punctuation chars to round out to 64).
I didnt use a standard one like 'Base64' because I included my own encryptor
and a base 128 system like 'uuencoding' had problems with the Email mangling the Ascii.


http://www.natural-innovations.com/binhex/



Most hex editors provide provide data dump as well, which is ascii character. Some of them provide complete 8-bit ASCII set.

And your inability to display the characters doesn't change the encoding. Unicode is an example. If it can't display the glyph, that doesn't mean the character has been misteriously converted into something else - its value is still there.

And although ASCII has unprintable characters, this doesn't change the fact that each character is unique - even if you can't see it.

It's also possible to draw n-dimensional objects on computer screen (where n > 3). And despite our screens being limited to 2D, the object rendered is still n-dimensional, although we can't represent it.

Think of hex viewer as a 2D renderer for 4D objects. It can't display everything, but it displays base 256 number accurately.



I forgot about the ascii dump. But its just so you can spot text in the data easily and is not itself a proper base256 representation (unprintables...).
It would be like trying to use a text editor where a quarter of the characters are randomly blank . Do you remember which x-ascii code was the 'spade' character???


Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this