how to speed up texture loading?

Started by
23 comments, last by MasterWorks 18 years, 10 months ago
Quote:Original post by Jiia
Quote:Original post by streamer
I just don't know how will DXLoadTextureFromMemory behave in my program. But this seams to be a right point. Again, lot of changes in code. I think this will be one more sleepless night

It's incredibly simple. Instead of passing a filename, you just pass the memory pointer and byte size. The hard part would be combining all of your files as you update art [smile]


ineresting [smile].
I rewroted a texture load proc like this:

D3DXIMAGE_INFO imageInfo;		HRESULT hr = E_FAIL;		char* buffer;	int fh;		if( ( fh = _open( buf, _O_RDONLY | _O_BINARY ) ) != -1 )	{		// file length		long fl = _filelength( fh );		buffer = new char[fl];		// read binary data		if ( _read( fh, buffer, fl ) > 0)		{			// read from memory			hr = D3DXCreateTextureFromFileInMemoryEx(m_pd3dDevice,buffer,fl,0,0,0,0,				D3DFMT_UNKNOWN, D3DPOOL_MANAGED, D3DX_DEFAULT,				D3DX_DEFAULT, dwCol, &imageInfo, NULL, &m_pTexture);			// save image dimensions			m_imageWidth=imageInfo.Width;			m_imageHeight=imageInfo.Height;		}						}	

and seems to be a a little bit faster to load texure as binary file, and then load it as texture from memory [rolleyes]
Advertisement
You mean it's faster to do this with just a single file? That is interesting. Are you running in release mode? DirectX libraries may be doing a lot of in-between calls when opening files, which may slow the routine down in debug mode. Or they may just be taking a lot of precautions. It's hard to tell.

By the way, out of curiousity, how are you saving your images? Are you exporting them from a renderer? How do you get them saved with individual sizes? Do you have an automatic cropping tool?
Quote:Original post by Jiia
You mean it's faster to do this with just a single file? That is interesting. Are you running in release mode? DirectX libraries may be doing a lot of in-between calls when opening files, which may slow the routine down in debug mode. Or they may just be taking a lot of precautions. It's hard to tell.

By the way, out of curiousity, how are you saving your images? Are you exporting them from a renderer? How do you get them saved with individual sizes? Do you have an automatic cropping tool?


Yeah it is faster with single file and I'm running it in debug mode. Release mode is just faster.
I don't think I understand the second part. Screenshoots from game I grab with CTRL+PrtScr and Paste it in Paint[smile]. If you are thinking about game animation well that is harder part. I render 3D model with 3DMax onto isometric viewport. Output file have several properties which you can define in file output properties. In this section I defined a custom color for backround(sprites and everything in game is alpha-blended)and output size. To get a proper character size inside a texture, in Max you can load an picture as background. Then you need just to set up proper disteance of camera and voala 3DMax is doing the rest. Same thing for objects.
If I get you right, you wan't to SPEED THINGS UP. To do that you must sacrifice memory. Get rid of the image compression. Uncompression takes time. Next: do you need all texturefiles at once? If not, load them as you go on. Load the ones you need at the start and load the others in a thread in the background. Or better yet, if you know that you will only need a certain texture at a certain point in the game (say animating a boss monster or whatever), load when you need it and then dump it.

Storing all files in one big binary/archive file will do one thing for you: getting rid of 6999 file open and close. The rest is pretty much the same.

Load from memory: Well you gotta gett it into memory first don't you?

a 520 frame animation?? thats 22 secs of video, or I would say 64 secs of ingame animation (or whatever you have there). Did you ever think about streaming video instead?

Putting all animations in one file could be good, but then again: isn't that just a video file? I like to collect game data (like textures) in larger files though, but not because it loads faster...

"I could be off endlessly perfecting my 3d engine, but instead I am cursed with the task of actually finishing something, it sucks."
Quote:Original post by Ninebit
Get rid of the image compression. Uncompression takes time.

I know this seems like a simple fact, but in most cases, it's not. Reading more data from the hard drive (or worse, the CD) is usually much slower than reading a (smaller) compressed version of that data and uncompressing it in memory. This can also be tested by comparing loading speeds of PNG vs BMP, which PNG passes with flying colors on my system.
Quote:Original post by Ninebit
If I get you right, you wan't to SPEED THINGS UP. To do that you must sacrifice memory. Get rid of the image compression. Uncompression takes time. Next: do you need all texturefiles at once? If not, load them as you go on. Load the ones you need at the start and load the others in a thread in the background. Or better yet, if you know that you will only need a certain texture at a certain point in the game (say animating a boss monster or whatever), load when you need it and then dump it.


Unfortunetly I can't. I need them all. It is mostly background graphic. Animations is another thing. And textures are in PNG format which have very good compression ratio and also simple decompress routine. And it is fast.

Quote:
Load from memory: Well you gotta gett it into memory first don't you?


Yes I need to get it into memory first. But! I think that DirectX is messing up with something when DXCreateTextureFromFile is called...anyway it slows a loading sequence...and as I said it seems to be a faster to load a texture as binary file in memory and then create a texture from pointer. I don't understand too why it is so. It is just a little bit faster.

Quote:
a 520 frame animation?? thats 22 secs of video, or I would say 64 secs of ingame animation (or whatever you have there). Did you ever think about streaming video instead?


If you played Diablo or any other RPG you would notice that character animations is created from texture sequences. Streaming video is used for internet video streaming, not for games.

Quote:
Putting all animations in one file could be good, but then again: isn't that just a video file? I like to collect game data (like textures) in larger files though, but not because it loads faster...


But the point of this topic is that I'm writing an MMORPG game and loading sequence is too long, and THAT is annoying thing. When I need to restart a game several times because I looking up for some bug, and when I wait 20-30sec per restart then I start to think that maybe a faster game start is better idea.
It sounds a little strange to need all the 7000 files at once? are you sure you really need that many?, and if the problem is that debugging takes too long, I'd suggest replacing many of the files with the same file while debugging? or are you trying to debug the animations / backgrounds ? it probably be easier to do it in some other way, then to run the game each time.
Quote:and seems to be a a little bit faster to load texure as binary file, and then load it as texture from memory [rolleyes]

I think I may understand why this is true. The file-opening routine probably feeds a memory pointer (pointer to the hard drive) to the same function that you're using to load from memory. If so, the file-opening version would end up reading bit by bit from the hard drive, instead of reading the entire file at once. Reading a big chunk at once should be a bit faster.

Not that any of this is important. Just thought I would share my guess.
It looks like you are passing 0 for Mip Map field. Do you really need to generate a full mip-map chain for every texture? That right there could be killing your performance. Try chaning it to 1 to see what kind of speed boost you get...
Hi streamer,
In your code:

a. Trying to open the file, load it into a buffer just to pass it into D3DXCreateTextureFromFileInMemoryEx is a loss of time, the function can do it for you. But the problem is not that. The problem is that you are reserving blocks of memory each time you do:
long fl = _filelength( fh );

But I don't see you releasing that memory. I'd recommend using a buffer with enough size to hold the data. That will be way faster.

But your worst problem is in the Filter and MipFilter parameters. The default filters can be really slow because they try to be a general solution in order to give you the best result when you try to stretch the texture on load time.

Say you are trying to load a texture with non pow2 dimensions, then DX will try to adjust it into a pow 2. I.E. If you load a 100x100 texture, DX will load it into a 128x128 texture. The vertical and horizontal relation with the original texture is set in the image_info structure. So, as the the bmp is being stretched, the filters are used in order to compensate. That makes your loading time slower.

In order to avoid the use of filters, create or pack your artwork using pow2 dimensions. Anyway, if your artist has created a texture, you can assume its a final and optimal version so any form of stretching will destroy the original presentation. So its wise to create your textures in pow2 formats. Once you do that the texture can be loaded using 0,0 in the Width and Height parameters and you will be sure they will have the same dimensions as your original art.

Now theres other big problem in your program. If you use 0 in the miplevels, a complete mipchain will be created for each texture. That means a 33% more in memory (no big deal), but the most important pat is that EACH miplevel is loaded stretching the image with the MipFilter you selected. If you read the rest of this message that means a lot of work if you aren't using mips.

Finally, using a color key is really slow. Try using a format that supports alpha like TGA, PNG or one of the DXTn formats. Colorkey usage is deprecated and you have a lot of better effects that can be achieved by using an alpha map. Usage of the color key slows you down because each image (and mipmap) should be processed in order to transform the color key into an alpha level.

So, I post you a suggested way to do this (in case you dont need mips and your textures are properly packaged):

D3DXIMAGE_INFO imageInfo;
HRESULT hr = E_FAIL;

// read from memory
hr = D3DXCreateTextureFromFileEx(m_pd3dDevice, buf,0,0,1,0,
D3DFMT_UNKNOWN, D3DPOOL_MANAGED, D3DX_FILTER_NONE,
D3DX_FILTER_NONE, 0, &imageInfo, NULL, &m_pTexture);
// save image dimensions
m_imageWidth=imageInfo.Width;
m_imageHeight=imageInfo.Height;


Luck!
Guimo

This topic is closed to new replies.

Advertisement