[source lang="cpp"}
FILE *filenamex,*stream;
...
...
...
stream =fopen (filename, "r");
if( stream != NULL )
{
fseek( stream, 0L, SEEK_SET );
for (i=0;i<160000;i++)
{
fscanf( stream, "%d", &example );
}
fclose(stream);
fopen+fscanf is there any better way?
Author
hi
i have to load large data files (like code below)
and i want to know if this fopen+fscanf method is
the only way or if there is any better way
(its part of a landscape loader.. normally i got to load 25 time
with a code like this below..takes 8-10seconds...i was doing a simple
test with an opengl texture loader and then grab data from it..
this method was only a bit over an second..so there must be anything better)
simple example what iam using at moment
greetings and thx for any hint :)
Use stream:
Google yields more information on all of those.
Hope that helps,
Dave
#include <ifstream>#include <ofstream>std::ifstream & std::ofstream for input and output filestreams|------------------#include <iostream>std::cout and std::cin for character input and outputGoogle yields more information on all of those.
Hope that helps,
Dave
Instead of your 160000 iteration for-loop on fscanf, try this:
fread(example, sizeof(int), 160000, stream);
C++ streams usually aren't faster than cstdio.
If you want to do it really fast, the best way probably is memory-mapping the file and using a special purpose parser.
If you want to do it really fast, the best way probably is memory-mapping the file and using a special purpose parser.
And remember, it's more efficient to read binary data than numerical data from a file, so a texture loader will run faster, because it can read in a very large buffer at once, without the need to process and convert data types. You might want to look into writing your data files out to a binary format and then youc an load it much simpler with fread or similar functionality.
FILE *stream;stream =fopen (filename, "r"); if( stream == NULL ) return;fseek( stream, 0L, SEEK_SET );for (i=0;i<160000;i++){ fscanf( stream, "%d", &example );}fclose(stream);//Now lets save this to a binary format!stream = fopen(outfilename, "wb");if (stream== NULL) return;//I'm at work, check if i have these in the correct order!fwrite(example,sizeof(example_struct),160000,stream);fclose(stream);//Now we have a binary representation of our file, which will load MUCH faster!!/**To Load **/FILE *stream;stream =fopen (filename, "r"); if( stream == NULL ) return;fseek( stream, 0L, SEEK_SET );fread(example,sizeof(example_struct),160000,stream); //Load it all in one shot!fclose(stream);Quote:Original post by Ready4Dis
And remember, it's more efficient to read binary data than numerical data from a file, so a texture loader will run faster, because it can read in a very large buffer at once, without the need to process and convert data types.
Binary files are more compact than text representations, so loading reading them is quicker, yeah. Converting data doesn't need to be a bottleneck, a harddisc tops at 100 MB/s, so you have around 20 cycles per byte to convert your data, that's more than enough for most reasonable ways of encoding data.
Bulk reading is quicker in most circumstances, so you should try it even if you still need to convert the data you read.
Author
wow
thats perfect
i got much much more help here that i ever think to get
the "loader" is a part of an landscape terrain loader
including height(1xchar) and color data(1xchar)
(i put that into an landscape texture,only 256different colors the
terrain engine calculates the landscape soft)
my landscape got size 2000x2000 and its divided
in 25 pieces (each file holds 400x400) .. every time the position gets over
middel+-400 the offset moves and new landscape parts
are reloaded so i got a seamless landscape running
(walking or with horse it takes a little time to reach this border..
but when flying a dragon the data reloader is often needed )
much thx to all here
thats perfect
i got much much more help here that i ever think to get
the "loader" is a part of an landscape terrain loader
including height(1xchar) and color data(1xchar)
(i put that into an landscape texture,only 256different colors the
terrain engine calculates the landscape soft)
my landscape got size 2000x2000 and its divided
in 25 pieces (each file holds 400x400) .. every time the position gets over
middel+-400 the offset moves and new landscape parts
are reloaded so i got a seamless landscape running
(walking or with horse it takes a little time to reach this border..
but when flying a dragon the data reloader is often needed )
much thx to all here
This topic is closed to new replies.
Advertisement
Popular Topics
Advertisement
