ZLIB vs JPG and PNG

This topic is 4177 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

Recommended Posts

I am considering the idea of creating my own bitmap file format for my game that would greatly simplify its design and easily allow me to impliment custom I/O functions that will handle all I/O (loading from a random file, the resource/pack file, or RAM will all operate on the same interface without the user needing to know the source). Anyway, the chances of me coming up with a graphics compression algorithm that matches JPG or PNG is slim, and I don't plan on writting code to do what they do either. Instead I hope to use ZLib compression to compress raw bitmap data before it is stored in a file. There is a long list of benefits that I can imagine using my own format. Of course I have plans on making a standalone conversion tool to go from JPG/PNG to my format. I imagine one nice (although the least significant) benefit would be the ability to use a single buffer for loading texture data that is only destoyed when all image I/O is done. That would save the costs of allocating and deallocating the RAM for data destined for storage in graphics memory. Regardless, my question is, how much worse (in terms of file size, although compression speed matters too) would ZLib be compared with JPG and PNG.

Share on other sites
JPEG is a lossy format based on Fourier Transformations. It will always be smaller than any lossless formats, but well... you loose information. Should be avoided for games in my opinion.

PNG already uses ZLIB for its data chunks, but there's an additional filter employed that transforms the actual image data to be better compressible. You'll have a hard time to outperform it.

Said that, I don't think you should even bother. You could as well use an uncompressed format and rely on ZIP and the likes when delivering it. It's not worth the hassle. Write your own or use an existing format, just as you like. I suggest using an existing format, converting all data to own formats will soon get an annoying and repetive task.

Bye, Thomas

Share on other sites
Quote:
 [...] creating my own bitmap file format for my game that would greatly simplify its design and easily allow me to impliment custom I/O functions that will handle all I/O (loading from a random file, the resource/pack file, or RAM will all operate on the same interface without the user needing to know the source).
Using custom I/O-functions as you describe is perfectly possible with libPNG and libJPEG. That said, I don't think your desgin will be simpler when using Zlib than using libPNG or libJPEG.
That, including the fact that your images will probably be larger leads me to believe that you have little to gain by using your proposed format.
Quote:
 Currenlty in Iraq using my limited free internet time.
My support for doing a tough job there.

Share on other sites
First, it's funny that you want to compare zlib vs. PNG, because PNG uses zlib for compression.

Quote:
 Original post by T1OracleI am considering the idea of creating my own bitmap file format for my game that would greatly simplify its design and easily allow me to impliment custom I/O functions that will handle all I/O (loading from a random file, the resource/pack file, or RAM will all operate on the same interface without the user needing to know the source).

Anyway, if this the real reason for creating your own format, then I recommend against it. Instead, you could write your custom I/O functions so they will work for any kind of file (including bitmaps), and then find a way to get an existing image library to use your I/O functions. The worst case would be that you would have to write your own loading code.

One problem with a custom bitmap format is that you have no way to edit the bitmap, though I guess you could write a program to convert a standard format to your format.

The standard reinventing-the-wheel argument also applies.

Finally, if I haven't convinced you, here is an excerpt from the libPNG manual about customizing libPNG I/O:
Quote:
 from libpng version 1.2.5: A description on how to use and modify libpngInput/Output in libpng is done through png_read() and png_write(), which currently just call fread() and fwrite(). The FILE * is stored in png_struct and is initialized via png_init_io(). If you wish to change the method of I/O], the library supplies callbacks that you can set through the function png_set_read_fn() and png_set_write_fn() at run time, instead of calling the png_init_io() function. These functions also provide a void pointer that can be retrieved via the function png_get_io_ptr(). For example: png_set_read_fn(png_structp read_ptr, voidp read_io_ptr, png_rw_ptr read_data_fn) png_set_write_fn(png_structp write_ptr, voidp write_io_ptr, png_rw_ptr write_data_fn, png_flush_ptr output_flush_fn); voidp read_io_ptr = png_get_io_ptr(read_ptr); voidp write_io_ptr = png_get_io_ptr(write_ptr); The replacement I/O functions must have prototypes as follows: void user_read_data(png_structp png_ptr, png_bytep data, png_size_t length); void user_write_data(png_structp png_ptr, png_bytep data, png_size_t length); void user_flush_data(png_structp png_ptr);

Share on other sites
As JohnBolton said, you do not have to create your own bitmap format. But i assume graphics are not the only data you will use. You could consider using zlib to compress the other files (like maps, etc).

Share on other sites
You need to consider that you need to create the images somehow. No one
will like another boring image format ...

However, it may be perfectly legal to use an own representation if you want
to speed things up. This would be possible some 'dump' of your internal
structures, which you can later read in without needing to convert or adapt
anything. This whole blob of data can (and should) of course be zipped ...

So - the tip is to use a common format for construction, and use whatever you
are comfortable with in the 'runtime' of the game.

Share on other sites
There is certainly nothing wrong with creating your own image format, and its definately a good learning experience. It has other side effect benefits like (marginally) better resouce protection, if thats a goal.

Of course as has been said before, as far as compression and speed goes, you will be hard pressed to beat png, or jpeg, or many others. They have been developed for years (and decades). So if it is just purely for better compression, then stick to png, or similar.

Share on other sites
Since you want to abstract your resource loading from actual source code used in the project, then there's really nothing preventing you from doing this in a scripting language, and providing the scripting interface to users. This lets you use an existing texture package right now, but change the backend later.

e.g. consider if you have the following function:

texture* LoadTexture(string filename){   string ext = GetExtension(filename);   switch (ext)      case "png":         return load_png_using_libpng_functions(filename);      case "your_extension_here":         return load_image_using_your_functions(filename);}

...it doesn't really matter what you do inside LoadTexture(), as long as the return result is formatted in whatever texture format your game uses internally. This allows you to expose the LoadTexture() function in scripting to other users, or however you like, without needing to worry about what format filename is in.

Once you have the internal black-box processor working, you can add any format you like.

As for your actual question (sorry for going off on a tangent!), I would stick with png/zlib. If you have text or high detail in your textures, it may be sacrificed due to jpeg's lossy nature.

Share on other sites
From an enducational persective I'd recommend you to implement your own image format. I even wrote my own fully functional JPEG decoder and encoder from scratch. It was a very tedious process, but it was well worth it considering the amount of knowledge I gained from that experience. However, if you don't want to spend a lot of time on developing image decoders, use the libs mentioned earlier.

Share on other sites
Quote:
 Original post by T1OracleOf course I have plans on making a standalone conversion tool to go from JPG/PNG to my format.

I wonder why no one read that line, or how some missed...
Quote:
 Original post by T1Oracle my question is, how much worse (in terms of file size, although compression speed matters too) would ZLib be compared with JPG and PNG.

I accept that I won't beat PNG compression, I just hope that the trade off is worth it. I wrote more for this post originally but I had to log off and I didn't rewrite all of my reasons, regardless...

Quote:
Original post by JohnBolton
Finally, if I haven't convinced you, here is an excerpt from the libPNG manual about customizing libPNG I/O:
Quote:
 from libpng version 1.2.5: A description on how to use and modify libpngInput/Output in libpng is done through png_read() and png_write(), which currently just call fread() and fwrite(). The FILE * is stored in png_struct and is initialized via png_init_io(). If you wish to change the method of I/O], the library supplies callbacks that you can set through the function png_set_read_fn() and png_set_write_fn() at run time, instead of calling the png_init_io() function. These functions also provide a void pointer that can be retrieved via the function png_get_io_ptr(). For example: png_set_read_fn(png_structp read_ptr, voidp read_io_ptr, png_rw_ptr read_data_fn) png_set_write_fn(png_structp write_ptr, voidp write_io_ptr, png_rw_ptr write_data_fn, png_flush_ptr output_flush_fn); voidp read_io_ptr = png_get_io_ptr(read_ptr); voidp write_io_ptr = png_get_io_ptr(write_ptr); The replacement I/O functions must have prototypes as follows: void user_read_data(png_structp png_ptr, png_bytep data, png_size_t length); void user_write_data(png_structp png_ptr, png_bytep data, png_size_t length); void user_flush_data(png_structp png_ptr);

Well, that part was helpful so thank you. If I can cover 8 bit grayscale, 24 bit RGB, and 32 RGBA in PNG using Photoshop CS2 then that may be a good solution.

As for reinventing the wheel. If it takes all of 15 minutes to do then it's really not a issue. Binary formats are easy I've done plenty of them.

Quote:
 Original post by starmoleHowever, it may be perfectly legal to use an own representation if you wantto speed things up. This would be possible some 'dump' of your internal structures, which you can later read in without needing to convert or adaptanything. This whole blob of data can (and should) of course be zipped ...So - the tip is to use a common format for construction, and use whatever youare comfortable with in the 'runtime' of the game.

Thank you, that is one of the things that I considered.

Quote:
 Original post by MatrixCubedscripting language

Thanks but no thank you. That would be over complicating the problem. I have plans for scripting and scripts will be kept very limited.

While I will look into the libPNG thing, I still see benefits to my idea.
1) Support for custom bitmap formats which will be needed in my engine such as:
- 8 bit alpha, 8 bit red/blue/green-scale (breaking up color components could be use), 16 bit grayscale + alpha
2) As starmole suggested the data in the format will plug right into my texture format with out added conversions (not that I expect a significant performance in conversion).
3) Also related to #2, if my textures need data not contained in a PNG, (such as 8 bit alpha instead of grayscale) then I will not have to load from a separate file to get it.

Share on other sites
Check out DevIL at openil.sourceforge.net/

It is built on top of ZLib, libPNG, libJPEG and many others...

Share on other sites
DevIL is overkill and in my experience it adds noise to JPG's when it loads them. The intel library is much better for JPG's although they no longer offer that for free.

Anyway, I've thoroughly read through the libPNG manual and researched the PNG format, and have determined that I should be able to hack it into providing all of the features I need. I just need to write an OO wrapper for it. Although that solution still leaves me with a problem.

PNG files support many color formats, of those I have need for:
(8 bits/channel for all formats) grayscale, rgb, rgba, gray alpha

The problem is I do not know if there is image editing software that supports those formats. I have Adobe Photoshop CS2 at home (wich is far from here...) and I do believe that it supports 32-bit RGBA.

Anyway, I can still make a conversion tool to get to all of the formats that I need, although native support in Photoshop would be awesome. I've searched to see if CS2 support all of those PNG formats, but I have yet to find any confirmation.