Archived

This topic is now archived and is closed to further replies.

Dynamic Memory Techniques

This topic is 5009 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Hi guys, I''m reading an unknown amount of data from a stream so I want to allocate some memory, then add more if necessary, then free the remainder that''s unused. Is this legal: BYTE* pBuffer=(BYTE*)GlobalAlloc(GPTR,1024); // need some more memory so add another block pBuffer=(BYTE*)GlobalReAlloc(pBuffer,2048,GMEM_ZEROINIT); // ok, only use half of added block so total memory size should be 1024(original)+1024(added)-512(unused)=1536 // unsure about this part: GlobalFree(pBuffer+1536); Maybe I should use ''new'' instead ? The above code is in a loop so I can''t just see how much I need at the start. Any help is appreciated.

Share this post


Link to post
Share on other sites
im not familear with the golobalalloc functions but, this method here should work:

BYTE * mem = new BYTE [ 1024 ];

// then if you need more, alloate the new size, and copy the old into it
BYTE * temp = new BYTE [ 2048 ];
memcpy ( temp, mem );
delete [ ] mem;
mem = temp;


// then when your done just do
delete [ ] mem;

good luck, and just remember dynamic memory is SUPER slow, it should never ever be used inside the game loop at any cost. all allocating and deleteing of memory needs to be done in the loading or unloading section of the game.

Share this post


Link to post
Share on other sites
ReAlloc does pretty much the same thing as what hlivingstone describes. You''ll probably be better off tearing off a chunk larger than you need - before the loop as hlivingstone points out - and not worrying if some of the memory goes unused. Why quibble over less than 1kb on a machine with 128 Mb of Ram?

At any rate - regarding GlobalFree(pBuffer+1536); You don''t need to adjust the pointer to free the memory, GlobalFree(pBuffer);, will work just fine.

Share this post


Link to post
Share on other sites
Hi guys, thanks for the response.

Alright, the 1024 bytes I wrote was for the example, I''m really going to be allocating 50K-100K at a time. Also, this is not for a game loop so it doesn''t have to be super fast but I still need it fast; the stream is an Internet connection that I''ll be getting data from, and it comes in through a loop, usually about 10 iterations for the complete data set.

The reason I was thinking of doing this:

GlobalFree(pBuffer+1024);

Would be to free whatever was allocated over the 1024 that I didn''t need to save, for example:

BYTE* pBuffer=(BYTE*)GlobalFree(GPTR,300);
// I only needed 200 of that so free the rest:
GlobalFree(pBuffer+200);
// That can''t work can it ?

How does GlobalFree() know how many bytes to free in it''s regular usage ? Same with delete [] pBuffer ? It must put some stuff in the registers.

I really don''t like the idea of allocating another smaller buffer and then copying to it, it seems inelegant and inefficient, but maybe there isn''t a better way because that''s probably what GlobalReAlloc() does anyway.

Share this post


Link to post
Share on other sites
If you genuinely have no idea how big the stream will be, and you can't do any intermediate processing until it's totally finished, there is no better algorithm than repeatedly reallocating the memory and moving the data if you want it all in one big lump.

If of course, you don't mind splitting it into several lumps, you could just allocate them one lump at a time, and then you'd have it all without any copying.

For instance creating a linked list of structs that contain big char arrays as their data.

If you're using C++ I'd think that using new is a more consistent way of allocating memory than using low level win32 API to do it. In any case, consistency is important; memory allocated with one type of memory allocation (be it C, C++ or win32) cannot be freed with another, so try to be consistent. I would recommend "new" because it will be the method that most C++ libraries will use internally. And it's the way that C++ programmers reading your code would expect it to do it.

---

As a result, one possible method would be using a STL list of structs containing a block of data each:


const int BlockSize = 1024*1024;
struct {
int datalen;
char data[BlockSize];
} MyDataBlock;

std::list<MyDataBlock *> datablocks;

// ...

// pseudo-code

// Read data from network

// If current block is not full, add to current block

// If current block is full, create new block

// push new block on to datablocks list

// ...



Or for a more basic technique just whack the data into a fat std::vector<char> (which is very similar to a std::string)

Mark

[edited by - markr on March 28, 2004 5:58:07 PM]

[edited by - markr on March 28, 2004 5:59:11 PM]

Share this post


Link to post
Share on other sites
Hi again,

How do you reallocate with 'new' ? If I GlobalReAlloc() the original data stays intact, like this:

BYTE* pBuffer=(BYTE*)GlobalAlloc(GPTR,sizeof(BYTE)*256);
for(int j=0;j<256;j++)
pBuffer[j]=(BYTE)10;

pBuffer=(BYTE*)GlobalReAlloc(pBuffer,sizeof(BYTE)*512,GMEM_ZEROINIT);

for(j=256;j<512;j++)
pBuffer[j]=(BYTE)20;

The values I set in the first loop stay the same, I don't have to copy anything. At the very end I would have to get rid of any extra memory so I can just GlobalReAlloc() to a smaller size.

I would rather use 'new' though, will it work the same way ? Thanks guys.

[edited by - Endemoniada on March 28, 2004 6:53:01 PM]

[edited by - Endemoniada on March 28, 2004 6:53:50 PM]

Share this post


Link to post
Share on other sites
quote:
Original post by Endemoniada
How does GlobalFree() know how many bytes to free in it''s regular usage ? Same with delete [] pBuffer ? It must put some stuff in the registers.



Basically, the operating system maintains a set of data structures that it uses to manage the heap. The pointer passed to GlobalFree is used to look up the corresponding allocation information and free the correct amount of memory. The delete function typically wraps a "free" function. You can google for sample implementations of delete. No doubt some stuff is put in the registers - a whole lot of stuff and quite frequently - but not this kind of information. Perhaps you meant "registry"?

Share this post


Link to post
Share on other sites