be creative or use a lib(20mil of them for pac/unpac) =/

If by "pack" you really mean "compress", use: http://www.zlib.org

One way I've done it before (back in the 16-bit daze of DOS) was using RLE - run length encoding..

something like this in binary(decimal):

126,128,128,128,0,0,0,0,4,4,4,4,4,5,3,2
=16 bytes
I would convert each to:

[data][count], so:

126,1, 128,2, 0,4, 4,5, 5,1, 3,1, 2,1
=14 bytes
savings: 2 bytes, roughly 12% compressed

Count up to 255 repetitions, since 255 is largest one byte is

It's a start, there are more advanced methods which include higher math levels than what I have..


I fseek, therefore I fam.

[edited by - drarem on August 16, 2003 9:27:42 AM]

[edited by - drarem on August 16, 2003 9:28:36 AM]

quote:Original post by programering
But I meant how do I do?
How do I program it?

"it"? - data compression is a whole, very large and varied field of computing rather than a single simple method, there are 1000s of different compression algorithms, variations etc.

You could start at: http://www.datacompression.info

quote:Is it something with bitwise operators & and shifting <<, >>

An implementation of some compression algorithms might use bitshifting. It isn''t mandatory though. It depends on how YOU as the programmer decide to implement that specific algorithm. If you deem them best to use, use them...

You''ll most likely use bitshifts in the output part of a variable bit length coder such as Huffman, and maybe with a look up table to save walking the binary tree to get every code.

quote:losses and gains?
compressing a long variable to an Uint8.

you can''t think of data compression terms of single items of data, most schemes concentrate on redundancy and repeatability in large streams. Lossless compression versus lossy compression makes a difference here too - it depends what''s stored in that "long" variable - if it''s a 32bit ARGB colour value for example, then you could palettise (usually involving quantisation) and store an 8 bit index into a palette.

Most methods take the data stream as a whole though (though adaptive schemes do it in a piecewise way).

quote:Do I have to figure it out myself?

Ask a more precise question and you''ll get a more precise answer. Follow the link I''ve posted. There''s over 50 years of research already been done into data compression so no, you don''t need to figure it out yourself.

Follow the ZLIB link I posted. That contains information on how their algorithm works AND full source code. Take a look at their source code and decide if you still want to re-invent the wheel...

quote:Or do you just not know how to do it?

Hmm... search the forums for my previous answer to a post on Huffman Encoding...

quote:If so, how hard is it?

How good are you as a programmer?
How good is your knowledge of basic algorithms like binary trees?
How good is your knowledge of statistical maths?
How good is your knowledge of calculus?
How good is your knowldege of signal processing theory?

If you''re an "expert" at all of those, then most data compression methods will be pretty easy.

If you don''t have a clue about any of the above, then most data compression methods will be VERY difficult.

An average programmer should find RLE schemes such as that already posted pretty easy.

An average programmer should find basic Huffman trees (canonical & adaptive) relatively easy - provided you know about tree data structure operations and can follow the algorithm description.

Likewise the average programmer should be able to cope ok with algorithms like LZ77 etc with a bit of hard work and be able to combine with Huffman to be able to come up with variations on LZH and the like.

Move on to things like Arithmetic Coding, block quantisation, Fourier transforms to compress in other spaces etc and you''re going to find it tough without some more advanced knowledge.

I believe the OP is referring to packing values into a buffer for the purpose of transmitting over a network.