• Advertisement
Sign in to follow this  

How much memory do pointers use?

This topic is 3875 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Im trying to write a mesh system to avoid redundent memory but I think I might be causing some redundent pointer usage.

typedef struct KMESH
{
	float			x, y, z;
	float			rX, rY, rZ;

        KMESHLIST*              parent;
	KMESH*			next;
	
} KMESH;

typedef struct KMESHLIST
{
	LPCSTR			filename;
	void*			meshData;
	KTEXTURELIST*	        texList;
	DWORD			numMaterials;

	KMESH*			instanceList;
	KMESHLIST*		next;
	
} KMESHLIST;


perty much its a binary tree where all left values are meshs and all right values are the instances of those meshes. So if I go to load a mesh that I have already loaded it just creates anougher instance of the mesh. Now what I want to know is by linking back to the parent mesh is this causing redundent data. Or do pointers take up so little data it doesnt matter?

Share this post


Link to post
Share on other sites
Advertisement
The size of a pointer is platform dependant, but on a 32-bit system they are only 4 bytes (not enough to matter, basically).

Share this post


Link to post
Share on other sites
One parent pointer is 4 bytes; maybe 8. 1000+ instances is 4k+ maybe 8k+. I dunno, is 4k a lot of memory for the computer you're running? I doubt even modern phones would have problems with 4k.

Share this post


Link to post
Share on other sites
ok I should be fine... Im just trying to cut out as much overhead as possiable im trying to make a shell for DirectX and dont want to slow it down.

Share this post


Link to post
Share on other sites
First, don't start by optimizing ... in the end, when the program is written and working, if you only ever end up needing the next pointer, not the parent pointer, then remove it ... until then it is far easier to assume this dramatically useful pointer will earn its keep.

The beautiful thing about statically typed OO programs is you can put simple things in classes, write the program, then remove (comment out) things you think are not in use ... and if you are wrong the compiler will tell you so ... and where.

The beautiful thing about modern computers, its not the small stuff that matters anymore (4 bytes wasted per struct), its the big things, like core design decisions, algorithm choices, and fundamental data structures used.

The beautiful thing about modern tools is, the profiler will tell you exactly where most of your time is being spent, so you can optimize where it counts IN A REAL PROGRAM, not where some academic thinks it might matter someday.

good luck

Share this post


Link to post
Share on other sites

Well, memory certainly becomes an issue at some point and careful consideration to the data structures must be maintained. Especially true this is, for instance, in applications dealing with large amounts of data like databases, imaging etc.

You could write somethnig like sizeof(void*) to find out the size of pointer in your system (almosta universally the same as sizeof(int)). Also, if you are writing structures and really thinking about optimisations, you should pay close attention to the data types you use, their paddins and also the order in which you declare them. It may very well be that the members of your struct are laid out in the same order as you have declared them.

Here are a couple of links explaining things in more detail, including padding and the importance of declaration order.
Wikipedia: Data structure aligment
MSDN: Structure Aligment Examples
Optimizing C and C++ code by EventHelix
and finally resource from AMD also Optimizing Your C/C++ Applications, Part 2

I don't know then if the compiler is allowed to change the order of the fields defined in structures. If they are, it most likely has to be declared explicitly by the user. As it happens to be, paddings are in general machine architecture (and occasionally compiler) spesific and sometimes code has to be shared. In that case, it is vital that if a struct is being shared, bot sides expect the same aligment.

Share this post


Link to post
Share on other sites
Quote:
Original post by Naurava kulkuri
You could write somethnig like sizeof(void*) to find out the size of pointer in your system (almosta universally the same as sizeof(int)).


That's not true. On 64 bit architectures, int will tend to be 4 bytes and pointers will be 8 bytes.

Share this post


Link to post
Share on other sites

You are correct in that phil_t (And I should know better. Old habits. :-)). Anyway one could write also sizeof(intptr_t) to get the correct answer regarding to the size of the pointer and use the information. Actually, intptr_t legalises a popular yet non-portable tradition of converting data pointers to integral types, and vice versa in a portable way.

If someone wonders, intrptr_t is a part of C99, but not a part C++ standard. Not yet at least, but it has been included in TR1 already and some systems has included a support for it already and C++09 includes much more. Boost also includes portable type definitions.

Share this post


Link to post
Share on other sites
Quote:
Original post by Naurava kulkuri
I don't know then if the compiler is allowed to change the order of the fields defined in structures. If they are, it most likely has to be declared explicitly by the user. As it happens to be, paddings are in general machine architecture (and occasionally compiler) spesific and sometimes code has to be shared. In that case, it is vital that if a struct is being shared, bot sides expect the same aligment.


Indeed, compilers are not allowed to change the order of fields—although they are required to insert padding (as little as necessary) in-between fields for alignment purposes and yet they propose non-standard "do not pad" extensions. Other details vary between platforms, such as endianness or basic type sizes. The general approach is simlpy never to serialize anything larger than a byte.

Share this post


Link to post
Share on other sites
Quote:
Original post by Naurava kulkuri
(almosta universally the same as sizeof(int)

Actually, it's more like almost universally the same as sizeof(long). But, still, sizeof(void*) is one thing and sizeof(long) is something else.

Share this post


Link to post
Share on other sites
Quote:
Original post by Jaiminho
Quote:
Original post by Naurava kulkuri
(almosta universally the same as sizeof(int)

Actually, it's more like almost universally the same as sizeof(long). But, still, sizeof(void*) is one thing and sizeof(long) is something else.


Well, in general the pointer size is something that fits conveniently in the processor register. So, with processors having 32 bits wide registers, the pointers are in general 32 bits in size. This is my experience from several architectures and according to my knowledge of processor architecture. Wikipedia has an article about this also. So, the size of a pointer variable is often, but not always, the widest integer that the hardware supports directly. Of course, exceptions do exist -- especially in embedded world -- even though I'm too lazy to actually point to one.

The aforementioned would be true if only the hardware architecture was incfluencing on this. From MSDN one can see, for instance, that int and long are defined to be the same size. And indeed, in standard and in compiler manuals something like this is phrased
Type int is an integral type that is larger than or equal to the size of type short int, and shorter than or equal to the size of type long.


It is true that sizeof(void*) and sizeof(int) are in some sense a different thing, but a long standing concensus regarding pointers has been that in general the size of the pointer is the size that of the int. Mostly attributable to the fact that int has been defined to the native word size in processors (someone correct me if I'm wrong).

Share this post


Link to post
Share on other sites
Quote:
Original post by Naurava kulkuri
It is true that sizeof(void*) and sizeof(int) are in some sense a different thing,

Not only in some sense. They're two completely different things.

Quote:
Original post by Naurava kulkuri
but a long standing concensus regarding pointers has been that in general the size of the pointer is the size that of the int.

Well, this is an incorrect assumption on the (ever growing, and eventually dominating) x64 platform. So whatever informal consensus existed in the past, it certainly doesn't exist anymore. As I mentioned above, the size of a pointer and of an int are two completely unrelated things. If you sssume the opposite, or rely on some shady unofficial consensus, then you'll have a very funny time when porting your 32bit apps to 64bit...

Share this post


Link to post
Share on other sites
A size of a pointer is the size of a void* because a void*, just
like int*, char* etc are all just pointers.

The size of a pointer is system dependent, and is useually the size of
the data bus width for a given processor. This is useually also
the size of an int (Hence came the concept of a pointer=sizeof int.,
however this is not always true.

On 64 bit machines, a pointer may = sizeof long
On 32 bit machines, a pointer may = sizeof int,
On 16 bit machines, it may = sizeof unsigneed short;
On 8 bit machines = sizeof unsigned char;

It all depends on the architecture you are building for.
It is useually 1,2,4,8,16,32,or 64 bits wide.

If size matters alot, why not print out the value of sizeof (*void)?
This way, you will know the size of a pointer on your system.

Share this post


Link to post
Share on other sites
Quote:
Original post by Yann L
Quote:
Original post by Naurava kulkuri
It is true that sizeof(void*) and sizeof(int) are in some sense a different thing,

Not only in some sense. They're two completely different things.

Well, I took a more machine level position of what actually a pointer is. In any case, the compiler (or more likely the linker) treats all the pointers the same after type checking [EDIT: (An address in memory, an integer.)]. That is, it uses the same internal representation for all the pointers regardless of the type of the entities they point to. (Eventually, though, everything is bits.)

But if we take that position, then also std::basic_string and std::string are different things. I can see your point and agree with it. My thinking was on a lower level since I was actually thinking pointers and optimisations regarding them. :-)


Quote:
Original post by Yann L
Quote:
Original post by Naurava kulkuri
but a long standing concensus regarding pointers has been that in general the size of the pointer is the size that of the int.

Well, this is an incorrect assumption on the (ever growing, and eventually dominating) x64 platform. So whatever informal consensus existed in the past, it certainly doesn't exist anymore. As I mentioned above, the size of a pointer and of an int are two completely unrelated things. If you sssume the opposite, or rely on some shady unofficial consensus, then you'll have a very funny time when porting your 32bit apps to 64bit...


As I was refering to the 32-bit architecture earlier, I'm wondering if we are talking the same thing..? At least from your post I get the impression you are writing about the 64-bit architecture exlusively. In any case,it hasn't been just some "shady unofficial consensus" but a real and a valid concensus. See, for instance, Danny Kalev writing in his article C99 Core Features in C++0X discussing regarding this:
Any valid pointer to an object (the original C99 text speaks of "any valid pointer to void") can be converted to this type, and then converted back safely to void *. The result must compare equal to the original pointer. In other words, intptr_t legalizes a popular yet non-portable tradition of converting data pointers to integral types, and vice versa. The POSIX API defines several functions that rely on such conversions; similarly, Windows frameworks often store the this pointer as an integer. C++0x will support intptr_t, too.


And I have seen this happen also. And actually, I have seen bug in Metrowerks CodeWarrior compiler once that was related to this issues and invalid handling of virtual tables and multiple inheritance (a compiler bug that is).


I hope this won't end up in quibbling as English isn't my native tongue. But my point was that the whole issue of integer sizes as native processor register sizes is a bit shady these days, as you pointed out, with the coming of x64 architectures. That is why there has been some real consideration on standardising on some defined type names like int8_t, int16_t, int32_t, int64_t, and in fact, approach like this has been in use a long time in embedded world for instance.

Oh yes, and Danny also make a point concerning intptr_t too.

Also to those, who might be more interested between interaction of Windows operating system and the machine level regarding these 64-bit issues, there is an article x64 Primer:
Everything You Need To Know To Start Programming 64-Bit Windows Systems
available.

Share this post


Link to post
Share on other sites
Quote:
Original post by Naurava kulkuriAs I was refering to the 32-bit architecture earlier, I'm wondering if we are talking the same thing..?


I thought the conversation was about the C programming language (and also, indirectly, about the C++ programming language), not about some vaguely defined architecture I will assume was a 32-bit i386.

Listen. I've been around a while. I've been through migrating software from a PDP to a VAX, from an 8-bit world of 6502's and Z80s to a 16-bit world of TMS9900 and 8086s and onwards to 32-bit 68K and i386. I've just recently been through the 32-to-64 bit conversion process. There is no "consensus" on the size of an int or the size of a pointer. The two are not interchangable, and software that converts from one to the other arbitrarily was, is, and will be broken. People who have written code that makes any assumption about the size of integral or pointer values have had their names cursed and their souls are damned to an eternity buried up to their necks in boiling sewage.

The size of pointers and integral types are not the only difference: many architectures have different alignment requirements and many architectures have a different register set for addresses and for data.

Assuming all programs will always run on 32-bit Windows is like assuming all computer users speak American English.

--smw

Share this post


Link to post
Share on other sites
Quote:
Now what I want to know is by linking back to the parent mesh is this causing redundent data.


The "redundant" data depends on your use.

Do you need for every node to know about it's parent. For some operations, this complicates the updates, for others, it makes it simpler.

If you don't need parent information, then this is indeed redundant data - not because it takes up space, but because you don't use it.

Going off topic...

KMESH has double responsibility. It not only maintains state about the vertex, but it also does book-keeping for the container it's in. Generally, this is bad for two reasons:
- You cannot recycle vertex information (share coordinates among several meshes)
- You will quickly run into problems when managing the tree

If you want memory optimal solution, then use arrays (I'll assume C, not C++):

typedef struct KVERTEX {
float x, y, z;
float rX, rY, rZ;
} KVertex;
typedef struct KMESH {
KVertex *vertices;
int n;
} KMesh;
typedef struct KMESHLIST {
KMesh *meshData;
int n;
}
or equivalent C++ solution:
struct KVertex {
float x, y, z;
float rX, rY, rZ;
};
typedef std::vector<KVertex> VertexList;
typedef std::vector<VertexList> MeshList;

Share this post


Link to post
Share on other sites
Quote:
Original post by Naurava kulkuri
Well, I took a more machine level position of what actually a pointer is.

No, you the position of "what a pointer happens to be on my computer".
It's sometimes an integer with a different size than int (on x64, it's 64 bit, where an int is still 32), or it may not be an integer at all. on segmented memory systems, pointers aren't "just an integer".

But that still doesn't matter if your code is C or C++. Then a pointer is what C/C++ says it is. And C/C++ says that a pointer is different from an integer regardless of underlying hardware.


Quote:
In any case, the compiler (or more likely the linker) treats all the pointers the same after type checking [EDIT: (An address in memory, an integer.)].

As I said, an address in memory may not just be an integer. And I'm not even sure that all architectures treat every pointer the same.

Quote:
But if we take that position, then also std::basic_string and std::string are different things.

Certainly. std::basic_string is a class template, and std::string is a typedef of std::basic_string<char>

If you meant that basic_string<char> and std::string were different things, then... Well, no. According to the language, std::string is simply a synonym for std::basic_string<char>, and as such they are exactly the same thing.
That is unlike pointers and integers, which the language specs say are different, and on which different sets of operations are defined. (Most arithmetic operations are undefined on pointer types. They might still seem to work on your systems, because of your system's underlying architecture which does indeed treat pointers and ints the same, but you're still in undefined behavior land)

Share this post


Link to post
Share on other sites
As an aside, the in order member placement requirement mentioned in this thread in C for member variables in a struct is relaxed in C++. In C++, member variables declared with the same access specifier must be allocated in order. So in:

class A {
public:
int a;
short b;
int c;
private:
short d;
};

There may be no waste space/padding on your typical 32 bit x86 compiler as the compiler can move d between b and c. Also, the restriction is per access specifier, not per access level so if you add redundant access specifiers you can give the compiler complete freedom in how to allocate the members. So in this example, you would have no idea what order the variables will appear in the class:

class A {
public: int a;
public: short b;
public: int c;
private: short d;
};

Share this post


Link to post
Share on other sites
Quote:
Original post by SiCrane
As an aside, the in order member placement requirement mentioned in this thread in C for member variables in a struct is relaxed in C++.


As an aside to your aside, could you remind me what the guarantees exist on the order of initialization? I think I remember that the order of appearance in the definition determines that.

Share this post


Link to post
Share on other sites
It's strictly by order of declaration in the class definition.

Share this post


Link to post
Share on other sites

All right, I admit I should have been more careful in what I have been writing. Especially since I included this link in my previous post. It reads there that pointers in 32-bit systems and in 64-bit systems can't be treated in the same way due to the fact that instruction encoding has its limitations (that behaves like segmentation and thus isn't an integer at all).

Yes, I know that pointer has some additional qualities that integer doesn't have. For instance, adding one to a pointer makes it point to a memory address that is the address it currenty points to plus sizeof(type). Also doing pointer arithemtic with incomplete types (e.g. void*) isn't allowed. I suspect there might be some cases where C and C++ standards differ in what they define.

I also agree with Bregma that making a cast from pointer value to integer is no good. What I meant with this "concensus" was that there is a great deal of people doing work in some platform writing memory operations related code with the assumption that sizeof(int) is the same as sizeof(void*) in order to ensure it to be big enough to hold the needed memory reference values. Are we in disagreement regarding this? I hope not.

I'll try to justify this: Passing some values to functions that operate on Assembly level it is quite common to take in a memory reference (e.g. a pointer) and do some arithemtics with that value. As it happens, this value is usually passed via register (or stack, but it ends up in the register). Great many people assume that in C/C++ int represents something that has the size of a native register width. If this is coupled with the information that the size of a pointer (sizeof(void*)) is the same as sizeof(int), it justifies the reasoning that casting a pointer value to integer is eligble in some situations. Of course, we can point out that the value isn't of type int either (but some people do arithmetics with this value and cast it back to pointer then).

Anyway, for beginners processing array or something, it's quite common to forget that "adding one" to such a value just crashes the program usually since there is a misaligment. So the usual addition and such doesn't hold anymore -- as does hold with pointers.

To clarify this issue about this "vaguely defined architecture": As I wrote in response to phil_t earlier, he made a valid point when I had written "You could write somethnig like sizeof(void*) to find out the size of pointer in your system (almosta universally the same as sizeof(int))" that it doesn't hold water today with 64-bit machines. So, I had in essence an assumption stuck in my about 32-architecture where it was very common to interpret sizeof(int) == sizeof(void*). And indeed, we have today intptr_t to tell the size of the pointer variable.

And if think of pointers being a different thing than their underlying reprsentation of int (if we remember the context it was written in), then it feels strange that string and basic_string wouldn't be different either. That is, when taking that machine level view, all that pointer represents is a value in the register that is as wide as is sizeof(int) according to the wide beliefs. As we have hopefully concluded already, and as Spoonbender and Bregma clarified and was written in the MSDN article (and actually, which I should have been thinking about earlier when I posted), pointer isn't actually just an int in the sense that to maintain pointer semantics, it can't be treated as a simple integral value in the register level either. Not at least if we want to be on the safe side on this.

Share this post


Link to post
Share on other sites
Quote:
Original post by Crypter
On 64 bit machines, a pointer may = sizeof long
On 32 bit machines, a pointer may = sizeof int,
On 16 bit machines, it may = sizeof unsigneed short;
On 8 bit machines = sizeof unsigned char;


Those sizes depend on implementation. For example, in VS2005, this line...

std::cout << sizeof(long) << std::endl << sizeof(int) << std::endl;


...prints...

4
4

That caught me off guard a while back, because I had just always assumed that VS used 64-bit long types. You hae to declare something long long to get 64-bits.

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement