Jump to content

  • Log In with Google      Sign In   
  • Create Account


What is the fastes way to load textures in opengl ?


Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

  • You cannot reply to this topic
22 replies to this topic

#1 Kario   Members   -  Reputation: 127

Like
0Likes
Like

Posted 02 August 2011 - 05:46 AM

Loading textures takes about 2/3 - 3/4 of initialization time in our engine. Lot of time is spent in driver for texture compression, and in our code for generating mipmaps for normal maps (down sample, normalize each vector). Can this be speed up ? Is there a way to make the driver create all mipmaps for normalmaps ? Can I load already compressed textures to save time ?

Sponsor:

#2 MrDaaark   Members   -  Reputation: 3551

Like
0Likes
Like

Posted 02 August 2011 - 06:00 AM

Use a file format that supports compression and mip maps, like DDS.

#3 mhagain   Crossbones+   -  Reputation: 7592

Like
1Likes
Like

Posted 02 August 2011 - 06:01 AM

Use DDS. It's precompressed (to one of the S3TC/DXT formats), already in a format that's most likely native to your GPU, and can contain a full mipchain in each file. Don't be put off by the fact that you normally see it mentioned in conjunction with D3D; it's just a binary file format and all you need is a loader for it.

http://www.opengl.org/registry/specs/EXT/texture_compression_s3tc.txt

It appears that the gentleman thought C++ was extremely difficult and he was overjoyed that the machine was absorbing it; he understood that good C++ is difficult but the best C++ is well-nigh unintelligible.


#4 Danny02   Members   -  Reputation: 271

Like
0Likes
Like

Posted 02 August 2011 - 06:04 AM

I havn't investigated into this topic that much,
but couldn't the loadtime be speedup by Pixel Buffer Objects?

#5 Kario   Members   -  Reputation: 127

Like
0Likes
Like

Posted 02 August 2011 - 06:10 AM

Can I use DDS with OpenGL ? Is so how do I load the data into OpenGL ?

#6 SimonForsman   Crossbones+   -  Reputation: 5965

Like
0Likes
Like

Posted 02 August 2011 - 07:09 AM

Can I use DDS with OpenGL ? Is so how do I load the data into OpenGL ?


http://www.codesampler.com/oglsrc/oglsrc_4.htm#ogl_dds_texture_loader
I don't suffer from insanity, I'm enjoying every minute of it.
The voices in my head may not be real, but they have some good ideas!

#7 V-man   Members   -  Reputation: 805

Like
0Likes
Like

Posted 02 August 2011 - 10:30 AM

I use DevIL to read the DDS file and the compressed DDS texture goes straight to GL. Look at

mhagain Posted Image post for the extension to use.


Sig: http://glhlib.sourceforge.net
an open source GLU replacement library. Much more modern than GLU.
float matrix[16], inverse_matrix[16];
glhLoadIdentityf2(matrix);
glhTranslatef2(matrix, 0.0, 0.0, 5.0);
glhRotateAboutXf2(matrix, angleInRadians);
glhScalef2(matrix, 1.0, 1.0, -1.0);
glhQuickInvertMatrixf2(matrix, inverse_matrix);
glUniformMatrix4fv(uniformLocation1, 1, FALSE, matrix);
glUniformMatrix4fv(uniformLocation2, 1, FALSE, inverse_matrix);

#8 Kario   Members   -  Reputation: 127

Like
-1Likes
Like

Posted 07 August 2011 - 10:47 AM

I have played a little with this issue. I cannot use DDS files, because I want source data to be as small as possible to minimize download time. What I found is:
- OpenGL compresses textures very slowly, maybe I should write my own compression code.
- If I switch compression off glTexImage2D is still relatively slow.
- gluBuild2DMipmaps is much slower than generating mipmaps on client side and calling glTexImage2D for each mipmap level.

So my questions now are:
- Is there a third party library of some kind that can compress textures fast ?
- Why is glTexImage2D so slow ? it takes 3 sec for a ~2500 calls (with mipmaps and with out compresson) that is extreamly slow for just coping data.

#9 mhagain   Crossbones+   -  Reputation: 7592

Like
1Likes
Like

Posted 07 August 2011 - 01:05 PM

glTexImage2D does more than just copy data. It needs to allocate video RAM too, possibly do some moving of already allocated data around to deal with fragmentation, possibly do some swapping, possibly allocate backup copies in system memory, and very probably convert data from non-GPU-native formats to GPU-native formats.

You'll get the most mileage on the latter point, and this relates to the 'format' parameter to glTexImage2D. If you're using GL_RGB you should investigate switching to GL_BGRA. In most cases this will be considerably faster, and I've personally benchmarked it up to 6 times faster on NVIDIA and up to 40 (!!!) times faster on Intel. Likewise if you're using GL_BGR you should still consider switching to GL_BGRA. For further info see here: http://www.opengl.or...and_pixel_reads

Of course this means that you need to store images in a format that supports BGRA, otherwise you'll need to convert on the CPU during texture loading. And that in turn means TGA is the only really viable option, which means bigger image files. So you're in tradeoff country right away.

That brings me to DDS again. I'm quite surprised that you found DDS files to be excessively large. It will give you between 8:1 and 4:1 compression over TGA, so unless you're using something like PNG as your image format (which will also contribute to slowness as it needs to be decompressed before being sent through glTexImage2D) DDS should be optimal.

It appears that the gentleman thought C++ was extremely difficult and he was overjoyed that the machine was absorbing it; he understood that good C++ is difficult but the best C++ is well-nigh unintelligible.


#10 Trienco   Crossbones+   -  Reputation: 2085

Like
0Likes
Like

Posted 07 August 2011 - 10:26 PM

I have played a little with this issue. I cannot use DDS files, because I want source data to be as small as possible to minimize download time. What I found is:


Maybe at this point it would be interesting to know what format you are using for your textures, if compressed DDS files are too large. And don't say "jpg".

Also, did you zip them and compare _that_ size? Because that's the only size that matters for downloads.
f@dzhttp://festini.device-zero.de

#11 V-man   Members   -  Reputation: 805

Like
1Likes
Like

Posted 08 August 2011 - 07:29 AM

- OpenGL compresses textures very slowly, maybe I should write my own compression code


You are suppose to send the DDS raw data to GL. GL will not decompress or recompress. It is fast.
Sig: http://glhlib.sourceforge.net
an open source GLU replacement library. Much more modern than GLU.
float matrix[16], inverse_matrix[16];
glhLoadIdentityf2(matrix);
glhTranslatef2(matrix, 0.0, 0.0, 5.0);
glhRotateAboutXf2(matrix, angleInRadians);
glhScalef2(matrix, 1.0, 1.0, -1.0);
glhQuickInvertMatrixf2(matrix, inverse_matrix);
glUniformMatrix4fv(uniformLocation1, 1, FALSE, matrix);
glUniformMatrix4fv(uniformLocation2, 1, FALSE, inverse_matrix);

#12 Kario   Members   -  Reputation: 127

Like
0Likes
Like

Posted 15 August 2011 - 06:34 AM

Thanks for replays. I'll investigate GL_BGRA for sure as I'm using RGBA at this point. Most of textures are in jpeg format so switch to DDS means much larger files. One odf the features of DDS files is it contains mip maps but that means +50% larger files.

#13 mhagain   Crossbones+   -  Reputation: 7592

Like
0Likes
Like

Posted 15 August 2011 - 07:32 AM

Thanks for replays. I'll investigate GL_BGRA for sure as I'm using RGBA at this point. Most of textures are in jpeg format so switch to DDS means much larger files. One odf the features of DDS files is it contains mip maps but that means +50% larger files.


That's your tradeoff - faster loading or smaller files - pick one.

Using JPEGs will definitely give you much slower loading times, by the way. First of all you have to decompress the JPEG to RGBA (or BGRA), then glTexImage2D it, then generate all of your miplevels by hand and glTexImage2D each of them. The sole benefit is that it comes off the disk faster, but once in memory it will be orders of magnitude slower.

It appears that the gentleman thought C++ was extremely difficult and he was overjoyed that the machine was absorbing it; he understood that good C++ is difficult but the best C++ is well-nigh unintelligible.


#14 V-man   Members   -  Reputation: 805

Like
1Likes
Like

Posted 16 August 2011 - 08:34 AM

Not all DDS files contain mipmaps. You need to check if there are mipmaps present at load time. If there aren't , have gl generate them.
Sig: http://glhlib.sourceforge.net
an open source GLU replacement library. Much more modern than GLU.
float matrix[16], inverse_matrix[16];
glhLoadIdentityf2(matrix);
glhTranslatef2(matrix, 0.0, 0.0, 5.0);
glhRotateAboutXf2(matrix, angleInRadians);
glhScalef2(matrix, 1.0, 1.0, -1.0);
glhQuickInvertMatrixf2(matrix, inverse_matrix);
glUniformMatrix4fv(uniformLocation1, 1, FALSE, matrix);
glUniformMatrix4fv(uniformLocation2, 1, FALSE, inverse_matrix);

#15 L. Spiro   Crossbones+   -  Reputation: 12873

Like
0Likes
Like

Posted 16 August 2011 - 08:23 PM

I have played a little with this issue. I cannot use DDS files, because I want source data to be as small as possible to minimize download time. What I found is:
- OpenGL compresses textures very slowly, maybe I should write my own compression code.
- If I switch compression off glTexImage2D is still relatively slow.
- gluBuild2DMipmaps is much slower than generating mipmaps on client side and calling glTexImage2D for each mipmap level.

So my questions now are:
- Is there a third party library of some kind that can compress textures fast ?
- Why is glTexImage2D so slow ? it takes 3 sec for a ~2500 calls (with mipmaps and with out compresson) that is extreamly slow for just coping data.


I have to admit I am surprised at this result, because OpenGL is insanely faster than DirectX when it comes to generating mipmaps (which is the main bottleneck).
But I am using ::glTexParameteri( GL_TEXTURE_2D, GL_GENERATE_MIPMAP_SGIS, GL_TRUE );.

gluBuild2DMipmaps() is most likely trying to create mipmaps that are of high quality, which is the primary reason for slowdown in Direct3D.
If the quality of the mipmaps is not so important, they can be generated nearly instantaneously with GL_GENERATE_MIPMAP_SGIS.
Also try ::glHint( GL_GENERATE_MIPMAP_HINT_SGIS, GL_FASTEST );.


L. Spiro
It is amazing how often people try to be unique, and yet they are always trying to make others be like them. - L. Spiro 2011
I spent most of my life learning the courage it takes to go out and get what I want. Now that I have it, I am not sure exactly what it is that I want. - L. Spiro 2013
I went to my local Subway once to find some guy yelling at the staff. When someone finally came to take my order and asked, “May I help you?”, I replied, “Yeah, I’ll have one asshole to go.”
L. Spiro Engine: http://lspiroengine.com
L. Spiro Engine Forums: http://lspiroengine.com/forums

#16 swiftcoder   Senior Moderators   -  Reputation: 9759

Like
0Likes
Like

Posted 16 August 2011 - 11:57 PM

But I am using ::glTexParameteri( GL_TEXTURE_2D, GL_GENERATE_MIPMAP_SGIS, GL_TRUE );

The SGI version of this token has been deprecated for eons. For OpenGL 2 and before, use the standard GL_GENERATE_MIPMAP token in its place.

For OpenGL 3+, use the glGenerateMipmap(GLenum target) function instead.

Tristam MacDonald - Software Engineer @Amazon - [swiftcoding]


#17 mhagain   Crossbones+   -  Reputation: 7592

Like
0Likes
Like

Posted 17 August 2011 - 03:26 AM


I have played a little with this issue. I cannot use DDS files, because I want source data to be as small as possible to minimize download time. What I found is:
- OpenGL compresses textures very slowly, maybe I should write my own compression code.
- If I switch compression off glTexImage2D is still relatively slow.
- gluBuild2DMipmaps is much slower than generating mipmaps on client side and calling glTexImage2D for each mipmap level.

So my questions now are:
- Is there a third party library of some kind that can compress textures fast ?
- Why is glTexImage2D so slow ? it takes 3 sec for a ~2500 calls (with mipmaps and with out compresson) that is extreamly slow for just coping data.


I have to admit I am surprised at this result, because OpenGL is insanely faster than DirectX when it comes to generating mipmaps (which is the main bottleneck).
But I am using ::glTexParameteri( GL_TEXTURE_2D, GL_GENERATE_MIPMAP_SGIS, GL_TRUE );.

gluBuild2DMipmaps() is most likely trying to create mipmaps that are of high quality, which is the primary reason for slowdown in Direct3D.
If the quality of the mipmaps is not so important, they can be generated nearly instantaneously with GL_GENERATE_MIPMAP_SGIS.
Also try ::glHint( GL_GENERATE_MIPMAP_HINT_SGIS, GL_FASTEST );.


L. Spiro


That comparison is misleading as it's comparing a software process with a hardware process. One could just as easily claim that D3D is insanely faster than OpenGL at building mipmaps because D3DUSAGE_AUTOGENMIPMAP with GenerateMipSublevels is faster than gluBuild2DMipmaps. ;)

Speaking of which, gluBuild2DMipmaps is slow, slow, SLOW. Check the documentation for it here to see why. Rescaling dimensions: check. Building proxy textures: check. Readback from the GPU: check. None of those are going to give you the best performance, which is why it's generally not recommended.

(It doesn't build high quality mipmaps by the way; it just uses a simple box filter - although I've found myself that this filter often produces results that look better than any other).

It appears that the gentleman thought C++ was extremely difficult and he was overjoyed that the machine was absorbing it; he understood that good C++ is difficult but the best C++ is well-nigh unintelligible.


#18 Tachikoma   Members   -  Reputation: 552

Like
0Likes
Like

Posted 17 August 2011 - 03:47 AM

That brings me to DDS again. I'm quite surprised that you found DDS files to be excessively large. It will give you between 8:1 and 4:1 compression over TGA, so unless you're using something like PNG as your image format (which will also contribute to slowness as it needs to be decompressed before being sent through glTexImage2D) DDS should be optimal.


Using JPEGs will definitely give you much slower loading times, by the way


Depends where your bottle neck is. Sometimes PNG or JPEG decompression is faster than reading larger files off disk. In my experience JPEG decompression can be especially fast if the lib is hardware assisted.

I do have a question regarding DDS though, anyone know licensing issues and restrictions for using this format in commercial applications?
Latest project: Sideways Racing on the iPad

#19 V-man   Members   -  Reputation: 805

Like
0Likes
Like

Posted 17 August 2011 - 04:20 AM


I have played a little with this issue. I cannot use DDS files, because I want source data to be as small as possible to minimize download time. What I found is:
- OpenGL compresses textures very slowly, maybe I should write my own compression code.
- If I switch compression off glTexImage2D is still relatively slow.
- gluBuild2DMipmaps is much slower than generating mipmaps on client side and calling glTexImage2D for each mipmap level.

So my questions now are:
- Is there a third party library of some kind that can compress textures fast ?
- Why is glTexImage2D so slow ? it takes 3 sec for a ~2500 calls (with mipmaps and with out compresson) that is extreamly slow for just coping data.


I have to admit I am surprised at this result, because OpenGL is insanely faster than DirectX when it comes to generating mipmaps (which is the main bottleneck).
But I am using ::glTexParameteri( GL_TEXTURE_2D, GL_GENERATE_MIPMAP_SGIS, GL_TRUE );.

gluBuild2DMipmaps() is most likely trying to create mipmaps that are of high quality, which is the primary reason for slowdown in Direct3D.
If the quality of the mipmaps is not so important, they can be generated nearly instantaneously with GL_GENERATE_MIPMAP_SGIS.
Also try ::glHint( GL_GENERATE_MIPMAP_HINT_SGIS, GL_FASTEST );.


L. Spiro


There is an app for that
http://www.opengl.org/wiki/Common_Mistakes#gluBuild2DMipmaps
http://www.opengl.org/wiki/Common_Mistakes#Automatic_mipmap_generation
Sig: http://glhlib.sourceforge.net
an open source GLU replacement library. Much more modern than GLU.
float matrix[16], inverse_matrix[16];
glhLoadIdentityf2(matrix);
glhTranslatef2(matrix, 0.0, 0.0, 5.0);
glhRotateAboutXf2(matrix, angleInRadians);
glhScalef2(matrix, 1.0, 1.0, -1.0);
glhQuickInvertMatrixf2(matrix, inverse_matrix);
glUniformMatrix4fv(uniformLocation1, 1, FALSE, matrix);
glUniformMatrix4fv(uniformLocation2, 1, FALSE, inverse_matrix);

#20 V-man   Members   -  Reputation: 805

Like
1Likes
Like

Posted 17 August 2011 - 04:22 AM

I do have a question regarding DDS though, anyone know licensing issues and restrictions for using this format in commercial applications?


DDS is cool. Use it.
Sig: http://glhlib.sourceforge.net
an open source GLU replacement library. Much more modern than GLU.
float matrix[16], inverse_matrix[16];
glhLoadIdentityf2(matrix);
glhTranslatef2(matrix, 0.0, 0.0, 5.0);
glhRotateAboutXf2(matrix, angleInRadians);
glhScalef2(matrix, 1.0, 1.0, -1.0);
glhQuickInvertMatrixf2(matrix, inverse_matrix);
glUniformMatrix4fv(uniformLocation1, 1, FALSE, matrix);
glUniformMatrix4fv(uniformLocation2, 1, FALSE, inverse_matrix);




Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.



PARTNERS