Sign in to follow this  

BZIP2

This topic is 3740 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

In my opinion, the documentation for libbz2 isn't great. I'm trying to compress an image, but the output is all zeros. I'm passing BZ_RUN and BZ_FINISH. Since I don't quite understand the documentation, I don't quite get the purpose of BZ_FLUSH, which I think might be the problem, because I didn't use it anywhere. Here's my code:
int CBzip2::Compress( IStream *in, IStream *out, size_t *uncomplen, size_t *complen )
{
	char  *inbuf;
	char  *outbuf;
	size_t uncomplen2;
	size_t complen2;
	int    state;
	int    r;
	
	// Init
	m_bz.bzalloc = NULL;
	m_bz.bzfree  = NULL;
	m_bz.opaque  = NULL;
	
	if( BZ2_bzCompressInit( &m_bz, 1, 0, 30 ) != BZ_OK )
		return -1;
	
	inbuf      = (char*) malloc( BUFSIZE );
	outbuf     = (char*) malloc( BUFSIZE );
	uncomplen2 = 0;
	complen2   = 0;
	state      = BZ_RUN;
	
	// Compression loop
	for(;;)
	{
		if( state == BZ_RUN )
		{
			m_bz.next_in  = inbuf;
			m_bz.avail_in = in->Read( inbuf, BUFSIZE );
			
			uncomplen2 += m_bz.avail_in;
			
			if( !m_bz.avail_in )
				state = BZ_FINISH;
		}
		
		m_bz.next_out  = outbuf;
		m_bz.avail_out = BUFSIZE;
		
		r = BZ2_bzCompress( &m_bz, state );
		
		if( r == BZ_STREAM_END ) break;
		
		else if( ( r != BZ_RUN_OK ) && ( r != BZ_FINISH_OK ) )
		{
			free( inbuf );
			free( outbuf );
			BZ2_bzCompressEnd( &m_bz );
			
			return -2;
		}
		
		else
		{
			out->Write( outbuf, m_bz.avail_out );
			complen2 += m_bz.avail_out;
		}
	}
	
	// Finish
	free( inbuf );
	free( outbuf );
	BZ2_bzCompressEnd( &m_bz );
	
	if( *uncomplen == -1 ) *uncomplen = uncomplen2;
	*complen = complen2;
	
	return 0;
}

I know my code isn't pretty, but it's not a super long function, so hopefully you can read it. The output says: Original length: 36.46 KB Compressed length: 37.00 KB The buffer size is 1024 bytes, which is why the compressed length was rounded up. Of course, it should be smaller than the original size!! AHHHH! And like I said before, the output file is 37 KB worth of 0x00. Any thoughts? Any help is appreciated.

Share this post


Link to post
Share on other sites
ok, this is some old code that I wrote quite some time ago, but I do recall it working. hope it helps; my code doesn't use stream compression, instead I use the BuffToBuff methods and I just create a background thread and do it asynchronously.


static void _MultiThread_compress( void* arg )
{
#ifdef MANAGED_ENABLE_COMPRESSION
CompressionCommandMT *cmd = static_cast<CompressionCommandMT*>(arg);
thisType::CompressionHeap& ch = cmd->_this->compressedHeap;
thisType::MemoryBlockIter mb = cmd->_this->memoryBlocks.begin();

IterSeek< MemoryBlockContainer >( cmd->memoryBlockIndex, mb );

unsigned int destLen = ch.heapSize - ch.cur;
unsigned int sourceLen = mb->sizeBytes;
char *dest = (char*) ch.heap + ch.cur;
char *source = (char*) mb->address;

int result = BZ2_bzBuffToBuffCompress( dest, &destLen, source, sourceLen,
cmd->blockSize, 0, cmd->workFactor );

switch (result)
{
case BZ_OK:
std::cout << mb->sizeBytes << " compressed to: " << destLen << std::endl;
ch.memoryBlocks.back().address = ch.heap + ch.cur;
ch.memoryBlocks.back().sizeBytes = destLen;
ch.cur += destLen;
break;
case BZ_OUTBUFF_FULL: break;
case BZ_MEM_ERROR: break;
}
#endif
}


Share this post


Link to post
Share on other sites
I definitely like the idea of compressing with one function call. Here's my new code:


int CBzip2::Compress( IStream *in, IStream *out, size_t *uncomplen, size_t *complen )
{
char *inbuf;
char *outbuf;

*complen = *uncomplen + ( *uncomplen / 100 ) + 600;

inbuf = (char*) malloc( *uncomplen );
outbuf = (char*) malloc( *complen );

in->Read( inbuf, *uncomplen );
BZ2_bzBuffToBuffCompress( outbuf, complen, inbuf, *uncomplen, 1, 0, 30 );
out->Write( outbuf, *complen );

free( inbuf );
free( outbuf );

return 0;
}

int CBzip2::Decompress( IStream *in, IStream *out, size_t *uncomplen, size_t *complen )
{
char *inbuf;
char *outbuf;

inbuf = (char*) malloc( *complen );
outbuf = (char*) malloc( *uncomplen );

in->Read( inbuf, *complen );
BZ2_bzBuffToBuffDecompress( outbuf, uncomplen, inbuf, *complen, 1, 0 );
out->Write( outbuf, *uncomplen );

free( inbuf );
free( outbuf );

return 0;
}



I haven't added any error checking yet, but I have a weird problem. My test program compresses an image file from "in.jpg" to "compressed", the decompresses it again to "out.jpg". The output reads:

Original length: 36.46 KB
Compressed length: 0.04 KB
Extracted length: 36.46 KB

It made me happy to see this. But then I checked out.jpg and once again I got all zeros. I checked the compressed file, and it seems odd that the image was compressed down to 46 bytes. But I viewed it with a hex editor and there was some real data in there as opposed to zeros. Is there something wrong with my CBzip2::Decompress function? Thanks again for the help.

Share this post


Link to post
Share on other sites
try changing the 2nd to last param on this call to 0:
BZ2_bzBuffToBuffDecompress( outbuf, uncomplen, inbuf, *complen, 1, 0 );

If I recall it's a flag indicating whether or not to use some kind of alternate decompression algorithm, setting it to 0 should make it use the default settings..I think...

Share this post


Link to post
Share on other sites
also, there's something fishy about your length calculations, you need to check for error flags, its quite important with compression systems.

int result = BZ2_bzBuffToBuffCompress( ... );

switch (result)
{
case BZ_OK:
break;
case BZ_OUTBUFF_FULL:
throw std::runtime_exception("BZ_OUTBUFF_FULL");
break;
case BZ_MEM_ERROR:
throw std::runtime_exception("BZ_MEM_ERROR");
break;
}

Share this post


Link to post
Share on other sites
With a bit of thinking I found that the problem was due to a really idiotic mistake I made earlier on. Seriously, this is a really stupid mistake. I used my CFile::GetSize to determine the length of "in.jpg", and it used fseek and ftell. The thing is this function didn't fseek back to the previous position.

But now another problem (surprise). Here's the output:

Original length: 36.46 KB
Compressed length: 36.52 KB
Extracted length: 36.46 KB

The file is 64 bytes larger compressed. Grrrr..here's the new code:


int CBzip2::Compress( IStream *in, IStream *out, size_t *uncomplen, size_t *complen )
{
char *inbuf;
char *outbuf;

*complen = *uncomplen + ( *uncomplen / 100 ) + 600;

inbuf = (char*) malloc( *uncomplen );
outbuf = (char*) malloc( *complen );

if( ( !inbuf ) || ( !outbuf ) )
return -1;

if( in->Read( inbuf, *uncomplen ) < *uncomplen )
return -2;

if( BZ2_bzBuffToBuffCompress( outbuf, complen, inbuf, *uncomplen, 1, 0, 30 ) != BZ_OK )
return -3;

if( out->Write( outbuf, *complen ) < *complen )
return -4;

free( inbuf );
free( outbuf );

return 0;
}

int CBzip2::Decompress( IStream *in, IStream *out, size_t *uncomplen, size_t *complen )
{
char *inbuf;
char *outbuf;

inbuf = (char*) malloc( *complen );
outbuf = (char*) malloc( *uncomplen );

if( ( !inbuf ) || ( !outbuf ) )
return -1;

if( in->Read( inbuf, *complen ) < *complen )
return -2;

if( BZ2_bzBuffToBuffDecompress( outbuf, uncomplen, inbuf, *complen, 0, 0 ) != BZ_OK )
return -3;

if( out->Write( outbuf, *uncomplen ) < *uncomplen )
return -4;

free( inbuf );
free( outbuf );

return 0;
}


Share this post


Link to post
Share on other sites
That's not that unlikely.

JPEG is already compressed data - which means that you are unlikely to be able to compress it any further. BZIP needs to put some headers on the front of the file to proceed with the unzip process and in this case it appears that the jpeg is so compressed already that bzip can't shave off more bytes than it adds.

This is fairly common when trying to compress already compressed data. Try compressing a bitmap or something - those aren't already compressed and so should get much smaller.

Share this post


Link to post
Share on other sites

This topic is 3740 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this