Archived

This topic is now archived and is closed to further replies.

Saving/loading supah-fast

This topic is 5790 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Hello! This is probably a very simple question, but I learned C++ the hard way (switching over from VB) and I''m not clear on some topics... I''ve finally got my 3DS model loading stuff working and computing normals, and it''s all stored into a big struct containing lots of little structs. However, this program only makes the file format to be imported in the actual game. Example (this isn''t really what it looks like...): struct myObject { int vertCount; int faceCount; int texVertCount; int materialID; int flags; Vector3 *Vertices; Vector3 *Normals; Vector2 *TexVertices; Face *Faces; }; struct my3DModel { int objectCount; int materialCount; vector Material; vector Object; }; ...etc. What''s the best way to save ''my3DModel'' to a file? Would I have to loop through all the info or can I just dump it all in one line? I want to load my model information as fast as possible in the game, even if it means making the files a bit larger. Thanks!

Share this post


Link to post
Share on other sites
If you want fast access, I recommend to read and write the data in a buffer before sending and retrieving into/from a file.
It is faster to read/write 1 block of 256kbytes than read/write 1024 blocks of 256bytes.
Unless your model is too big (say, more than 16Mbytes) I would not recommend to read/write the file in more than one block.
Though, this needs to interpret the buffer later. But it''s just like reading the file : you have to increase the pointer into the buffer instead of increasing the cursor into the file. And it is way faster to increase a pointer than to advance a file cursor

that''s one of the things I would recommend. I bet there exists other optimizations too.
hope that helps.

Share this post


Link to post
Share on other sites
ok...

I'll just rip this from my own app...
in here, the class String is just an extended CString object with more usablility. ERR_CHK is just a debug macro thing I do.

            

void CDataFileWrite::WriteDataToFile(String FileName, BYTE * pData, int iDataLength)
{
ERR_CHK
FILE *File;
File = fopen(FileName, "ab");

if (File==NULL) //file may not exist

{
if (!CreateFile(sFileName)) //try and create a file

{
String error="Unable to create or write to file:\n";
error+=sFileName;
throw Exception(BAD_FILE,error);
}
else
{
//try again (now file is created)

File = fopen(sFileName, "at");
if (File==NULL)
{
String error="Unable to create then write to file:\n";
error+=sFileName;
throw Exception(BAD_FILE,error);
}
};
};

fwrite( pData, 1, iDataLength, File );

fclose(File);
}




now, to save the data, you'd just do this:
(assuming the objects are setup right...)
      
myObject object;
//blah blah


DataFileWrite.WriteDataToFile("SavedFile.dat",(BYTE*)&object,sizeof(myObject));


then... use a data loading routine...

like such:


  
bool CDataFileRead::ReadByteData(String sFileName, BYTE *data, unsigned int iDataByteLength)
{
ERR_CHK
FILE *in = fopen(sFileName, "rb");

if (!in)
{
String error="Unable to read from file:\n";
error+=sFileName;
throw Exception(BAD_FILE,error);
}

fread(data, 1, iDataByteLength, in);
fclose(in);
return true;
}



so.... you'd load it something like this (this is all off the top of my head code btw)

           
myObject object;

BYTE *data=new BYTE[sizeof(myObject)];

DataFileRead.ReadByteData("SavedFile.dat",data,sizeof(myObject));

memcpy(&object,data,sizeof(myObject));

delete [] data;

you could probably do it without the temp buffer, but thats the way I'd do it...


argh. modifying replys is a nightmare

Edited by - RipTorn on February 1, 2002 3:52:32 AM

Share this post


Link to post
Share on other sites
hehe
well, because the Visual studio call stack isn''t always very reliable, it actually (if I set it to) keeps an exact log of all calls made... it''s quite slow (not as bad as one would think) - but it can be invaluable for strange crashes that VS can''t find.

I''ll copy the macro... it''s a bit of a mess though

  

#ifdef _DEBUG
#define _DEBUG_APP_CALL_STACK_LEVELS 25
#define _DEBUG_APP_ACTIVE_CALL_STACK "data/call_stack.dt"

#ifdef _DEBUG

#define ERR_CHK \
int _StackItem_;\
String _a_((char*)typeid(*this).name());_a_+="(";_a_+=__LINE__; _a_+=")->"; \
FILE *_in_ = fopen(_DEBUG_APP_ACTIVE_CALL_STACK, "rb");\
fread((BYTE*)&_StackItem_, 4, 1, _in_);fclose(_in_);\
_StackItem_++; if (_StackItem_==_DEBUG_APP_CALL_STACK_LEVELS)_StackItem_=0; \
FILE *_File_ = fopen(String(_DEBUG_APP_ACTIVE_CALL_STACK)+_StackItem_, "wt"); \
fwrite( _a_ ,_a_.GetLength(), 1, _File_ ); \
fclose(_File_);\
_File_=fopen(_DEBUG_APP_ACTIVE_CALL_STACK, "wb");\
fwrite((BYTE*)&_StackItem_, 4, 1, _File_);\
fclose(_File_);

#else

#define ERR_CHK

#endif


:D

its a hang of a lot faster than you might think...

obviously doesn''t do anything in release compiles.

Share this post


Link to post
Share on other sites
Hmm, I'm having problems...

My data varies in size; it contains lines (sort of like):
  
class Some3DVector
{
public:
float x, y, z;
};

struct SomeObject
{
Some3DVector *Vertices;
};

struct MyThing
{
vector<SomeObject> Objects;
};

MyThing ThisThing;


Can I still dump it all to a file?
sizeof(MyThing) won't return the real size of ThisThing.

Agh! Please excuse the non-descript class names.

Edited by - pollier on February 5, 2002 9:27:47 PM

Edited by - pollier on February 5, 2002 9:29:39 PM

Share this post


Link to post
Share on other sites
thats because the vector container uses memory pointers to keep a dynamic list of objects... so you will have the size of the memory pointers being returned by sizeof()...

you will have to create a buffer sized at the number of elements * the size of SOmeObject+ 4...

then, whack in an int as the first 4 bytes, where that int is the number of elements (so you know when loading)

then copy in the objects 1 by 1...


You are also using a memory pointer in the SomeObject object.. this will be returned as 4 bytes (the size of a pointer)... not how ever many it needs to be. You should make the object constant size (ie, no pointers or arrays - which are pointers anyway)


eg:
(you should always use classes, much more flexible than a struct)

class Vector
{
float x,y,z;
};


class MyOject
{
Vector a,b,c,d;
};


now...

to copy an array of these,

say, there are 10 objects in an array.,
(this is all off the top of my head here)

BYTE * buffer= new BYTE[10*sizeof(MyObject)+sizeof(int))]; // sizeof(int) is 4...
BYTE * pBuff=buffer;

int number=10;

memcpy(pBuff,&number,sizeof(int));

pBuff+=sizeof(int); //increment this pointer...

--> loop through all 10 objects...

memcpy(pBuff, &Objects[n], sizeof(MyObject));
pBuff+=sizeof(MyObject);

<--

save the buffer to a file..

delete [] buffer; // note: do not delete pBuff, that pointer has been changed...


then to load, do the opposite, memcpy'ing from a loaded buffer...

eg:

load file into buffer...

int num;
MyObject * objects;

memcpy(&num, pBuff ,sizeof(int));
pBuff+=sizeof(int);

objects=new MyObject[num];

and so on...

ohh, yeah, be very carful to check the file you are loading..

if the value num turns out to be -114325423 after loading, the file is likly corrupt



Edited by - RipTorn on February 5, 2002 11:49:36 PM

Share this post


Link to post
Share on other sites
in a case like that... it''s likly best to whack a function inside of MyObject that creates a buffer for you.. but this can get mighty complex, and you need to be careful with memory allocation if you do..

the best thing is to save each myObject in it''s own file. assuming there arn''t 10,000 of em...

Share this post


Link to post
Share on other sites
don''t use pointers, use index''s... these can be saved with much greater ease.. and you can use unsigned shorts, which save 2 bytes.

you should NEVER save a memory pointer to a file. as it will NEVER be the same the next time you run the app. (unless some crazy miricle occurs)

Share this post


Link to post
Share on other sites