Sign in to follow this  
CyberSlag5k

Can I make this any faster?

Recommended Posts

I'm loading in about 3MB of vertex data from a file and constructing an object based on that data using openGL. It takes about 5/6 seconds to load in all the data. I'm wondering if there are any optimization techniques I can implement to lower this load time. The program is a Win32 console application, so I'm using ReadFiles to load in all my data at once. I make a single call to ReadFile and pass in a pointer with space allocated for the many instances of the struct I'm using to store the data (it's a bunch of floats and a short int). I'm using #pragma pack(1), but considering using 2 as the object is constructed of many 4 byte member and a single 2 byte member. Would that help optimize things, perhaps? What else could I do to decrease the load time? Thanks in advance.

Share this post


Link to post
Share on other sites
The only thing that I could suggest to make it load faster would be to stream the file in to the object you are creating instead of all at once. I do not know if this would be better than ReadFiles though. The method I would suggest is use the Read function of 'fstream' or maybe the fread of 'stdio'. If you are just reading it in to an array, then I think you should be able to load it in chuncks and the either memcpy it over or set the pointer yourself to where it should read.

I know you guys are thinking what the hell is he talking about, wouldn't that make it slower? Possibily for this case, but maybe not. I used the technique I am describing in my XML parser class. It can handle a file of *any* size because of how I stream the data to memory. I worked with mult-GB files with no problems at all. I was able to process them rather quickly because of how I used the Read function. I noticed that if I specified 32767byte blocks, I had the best performance.

Another advantage of this method is how you are working with smaller pieces of a large file. Like I said before, in this case, 3MB is probabally not enough to make it load well, but if you were working with something, like let's say 20MB, then this would be perfect. What do you think?

- Drew

Share this post


Link to post
Share on other sites
One thing that would probably speed it up quite a bit would be some form of compression. I/O is likely taking 99% of the time with the CPU having very little to do while you wait for the I/O to finish. If you could reduce I/O at the cost of increased CPU usage, you'd likey still have a win. If it is vertex data, it might be possible to use something simple for compression like storing them less accurately (instead of a 3 4-byte floats for example, perhaps you could use 2 0.16 fixed point structures and get the z using the fact that the vector is normalized and then use a single 8 bit number to get the magnitude {might want to make it fixed point depending on scale, so perhapse 6.2 or 4.4}).

Share this post


Link to post
Share on other sites
If you're doing a single ReadFile, the overhead will not be in that. Are you profiling the single call to ReadFile or the overall time it takes to load the data from disk and pass it to OpenGL?

MSN

Share this post


Link to post
Share on other sites
Reading in 3mb definately shouldn't take 5 or 6 seconds. For example, my computer is 3.5 years old and the hard drive averages about 40MB/s for reading. Could you post your loading code? I'd like to take a look.

Share this post


Link to post
Share on other sites
When I came into work this morning, the file loaded almost instantaneously. It was as if there was a big, neon sign that said "You forgot to free up the allocated memory when you exit the program!" Problem solved. Thanks all.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this