Hello! I''m currently in a quest for a solution to one of the most evil classes of bugs (IMHO) that can haunt you: using invalid pointers. I''m not concerned about the evil memory leak (at least not yet), wich can be solved with auto_ptr or similar implementation of smart pointers. I''m terrified at the fact that a pointed "object" (may it be a class, struct, int, whatever) may be destroyed without the pointer (smart or not) that points to it becoming aware of the fact. So, when you dereference the now invalid, nonzero pointer, you could be writing at some other data structure (wich could be also a pointer, and now your bugs can write at your own code!), wich will start to silently corrupt the program an cause a random, weird, un-traceable exception five minutes later. Ok, maybe I should be more careful when using pointers. Maybe I should have designed and implemented my program better. And I sincerely belive that a seasoned programmer has a fair chance of preventing any of these bugs from manifesting through development of an entire 2-year project. But as program complexity grows and deadlines draw near, wouldn''t it be nice to have an _optional_ feature wich would make every single pointer in an application become invalid when the pointed object was destroyed? I wonder if anybody here knows of an already-implemented solution for this? I already tried implementing a couple of solutions. They work, but they are cumbersome. One requires the use of lots of evil macros, wich is ugly, very tiresome to use and error-prone in itself. I''m currently trying to overload global operators "new" and "delete". But I''m having a hard time. And also, my program is in fact a DLL loaded by a 3D engine. And I believe the main program, in the engine, already has it''s own custom behavior of "new" and "delete" operators. Can I "append" some custom code to these operators, without erasing the previous ones? Also, I think I would need to override the global operators "&" (get address of) and -> (dereference a pointer). Is it possible at all? (this is very advanced stuff for me, as I''m on C++ for little more than a year) Any info or toughts on this whole topic? Thanks! -- Fabio Reis Cecin Amok Entertainment

Well... the app would need to keep track of all pointers in it. Which implies some sort of special pointer class that registers itself with a central registry. You would also need your delete operator to be able to tell this registry that something has just been deleted. This would then need to go and alter all these pointers. But I can''t see this being safe at all... not to mention issues with modifying pointers inside const objects, references, and so on.

Better to fix the root of the problem than address the symptoms. If you have trouble ensuring that nothing points to deleted objects, you should consider making some changes. Here are some ideas for anyone who doesn''t know what I''m getting at:

1) Replace pointers with other forms of reference. For example, some sort of key value (integer id number, perhaps). You use an intermediate function to look up an object within a container based on its key. This might slow your program down, but guarantees that you will either get a valid or a NULL pointer. This is basically how modern relational databases work.

2) Simplify allocation/deallocation in your program. Ideally perhaps, you never call new/delete anywhere in ''client'' code. Instead, you have some interface functions that deal with creating and destroying your objects. This way, you can put other code into these functions, such as ensuring the pointer is gone from all lists, etc.

3) Reduce interconnections. Ideally, the only pointers in the program will be from an Owner class pointing to Owned classes. This way, when you delete the Owner, it deletes the Owned classes in turn, leaving no dangling pointers and no memory leaks. In cases where 2 entities need to be linked to each other and can vary independently, try seeing if you can remove pointers from one of them and instead use some sort of lookup function if it needs to know which of the other entities refers to it.

4) Use library code wherever possible. For example, using the STL list class instead of embedding previous/next pointers in your classes to make a linked list means that you have 2 fewer pointers to worry about, and well-tested code in its place.

Similar issues to this one arise when using the observer pattern. Since a subject cannot know who is watching it, how do you notify all observers when it is destroyed?

Solutions to this are discussed in-depth in the design patterns book, and there have been numerous articles in various magazines no doubt.

Try a web-search for keywords

Incidentally, IIRC, the best solution proposed by the design patterns book was a mediator object which subjects and observers all registered with, using signatures to identify themselves.

--

Get a stripper on your desktop!

You can use a reference counted pointer. In that case, the object pointed to will never be destroyed unless the last pointer pointing to it is destroyed.

It gives you 2 advantages :

1. You never have to use delete.
2. You know your pointers are always valid.

here is my implementation.

        /*-----------------------------------------------------------------------------	Description :				Reference counted pointer object.				it is a smart pointer to help manage heap object;				It WILL call delete once there is no more				RCPtr referencing that pointer-----------------------------------------------------------------------------*/#ifndef RCPTR_H#define RCPTR_Htemplate<class T>class RCPtr{private:	struct RefCount;	{		T* myPtr;		unsigned int myCount;		RefCount( T* thePtr )			:myCount(0)		{			myPtr = thePtr;		}	};public:		RCPtr( T* thePtr )		:myRefCount( new RefCount( thePtr ) )	{		myRefCount->myCount++;	}	RCPtr( const RCPtr<T>& theRCPtr )		:myRefCount( theRCPtr.myRefCount )	{		myRefCount->myCount++;	}	~RCPtr()	{				CountDown();			}	const RCPtr<T>& operator=( const RCPtr<T>& theRCPtr )	{		CountDown();		myRefCount = theRCPtr.myRefCount;		myRefCount->myCount++;		return (*this);	}	T* operator->()	{		return myRefCount->myPtr;	}	const T* operator->() const	{		return myRefCount->myPtr;	}	T& operator*()	{		return *(myRefCount->myPtr);	}	const T& operator*() const	{		return *(myPtr->myPtr);	}	friend operator==( const RCPtr<T>& theRCPtr,const T* val )	{		return ( val == (theRCPtr.myRefCount->myPtr) );	}				friend operator==( const  T* val, const RCPtr<T>& theRCPtr )	{		return (theRCPtr == val);	}	friend operator==( const RCPtr<T>& theRCPtr, const RCPtr<T>& theRCPtr2 )	{		return ((theRCPtr.myRefCount->myPtr) == 			    (theRCPtr2.myRefCount->myPtr)    );	}	private:	void CountDown()	{		myRefCount->myCount--;		if ( myRefCount->myCount == 0 )		{			delete myRefCount->myPtr;			delete myRefCount;		}	}	RefCount*  myRefCount;	};	#endif        

It has some shortcomings like You need to pass a pointer to it. So that means you have to do new yourself. This was done so that any constructors could be used.

So Clever people would use it like that
RCPtr ptr = new myClass(//whatever contructor);

but not so clever people could do that :

myClass* myc = new myClass();

RCPtr ptr = myc;

delete myc

ptr->myClassfunc() //doh!!!





Edited by - Gorg on June 9, 2001 12:46:46 AM

After a little webcrawling and lots of thinking (hmm what''s that smell) I came up with this sort of conclusion. What do you think?:

This problem is actually two issues:

1 - When you dereference any pointer in C, you are taking a risk. Since currently no big program can be fully proven bugless (in a timely fashion, that is), and a bug can make a pointer invalid, and C doesn''t let you know when you do that, it would be really nice to have a "invalid pointer use" trap in C as an optional feature (Delphi has some sort of this check, I just tested!), as slow as it could get (but totally transparent for the "new/delete" user), for working your apps in "debug mode". So you could actually TRACE for the source of the invalid-pointer-use in question, wich would then be fixed for the release version of your app (and possibly make you rethink your design). This would be very nice (to cover all pointers in a transparent fashion), but likely to be a task for the people who actually write languages (or at least not for me. yet). So C doesn''t have it. Too bad. Deal with it.

2 - If you designed your program with an explicit reference from one object to the other, work it out with good design with Kylotan''s tips for instance, like holding an ID and asking for the reference to some other intermediary entity, or using Gorg''s reference-counted pointer instead of a "naked" pointer. Or just making sure you don''t screw up (now that''s a nirvana!)

taliesin: the "Observer Pattern" thing really provides some insight when pointers/pointed entities are classes (wich have destructors, and this is a very good place to detect when an object is being deleted!! .

Gorg: I will definately study your RCPtr! But I need some time to fully understand it in the whole! Also, it reminds that the "observed" doesn''t have to know the address of the observers. The only rule is that there can''t be observers when the observed is deleted (this also enforces thought-out design, because you have to determine where in your app your pointers should deliberately "forget" their information (=NULL))

Kylotan: Your tips will be very helpful! Some may sound obvious for some people, but hey, beginners feed themselves almost exclusively from obvious stuff! Learning to use things properly is an iterative make<->think process (my point is that this process should be as painless as possible; that would make a "friendly" language)

All your hints have being incredibly useful! If I get some more insight/results on this, I''ll post here.
Thanks!!

--
Fabio Reis Cecin
Amok Entertainment

Hmm, I just skimmed through all the replies, so this might have already been mentioned, but it seems like it would work: pointers to pointers. I''m currently having that same problem with my entity class right now, so I''ll use that as an example. The class has a pointer to an image, which can be the same for several entities, since there will most likely not be too many different kinds around at once, but when the entities go to delete their images, it causes errors, because they don''t know that another entity has deleted their image for them. If you used a double pointer (CSurface** in my case), you could use delete *surf; *surf = NULL; and then all others who had a pointer to *surf would be able to see that it was set to null. It would cause a bit of slowdown, but not too much.

That was pretty confusing... it makes sense to me, and I do think it would work. Hope you can tell what I''m getting at^^


-Deku-chan

DK Art (my site, which has little programming-related stuff on it, but you should go anyway^_^)

MFC uses a memory lick detection by overloading new and delete operators. This way its guaranteed that every new is mached with a delete. Some time ago i read a article on www.flipcode.com that showed how to implement this kind of system. I didn''t get into details of it but you can.

Also there are smart pointer library here http://www.boost.org/libs/smart_ptr/index.htm It was reviewed by many c++ profesionals so you can be pretty sure that implimentation is reliable.

Thanks Jonppe, this site you pointed rocks! And the free online book "C++ in action" looks like a very good read!

felix9x: memory leak is one problem I don't want to think about right now... but I'm sure it has already received great attention and there are lots of solutions out there (garbage collection being the "radical" ones).

DekuTree64: your "pointer-to-pointer" approach with Gorg's referenced-counted pointer inspired me to try a new thing. And it WORKS BEAUTIFULLY! But I'm having a hard time working with operator overloading, I tought if somebody could help me complete the code?

the solution's features/costs:
- pointed objects can be anything
- operator new can be left alone (but we can't escape working with a custom delete)
- our "aware" pointers are of a template class CCoolPointer, where T is the typename of the pointed object
- we can mix naked and "aware" pointers with NO interference whatsoever between them! (but the naked ones don't benefit from the protection)

and, perhaps the best of all:
- we can very easily make two compilations of our code: one that defines our pointers to cool pointers, and other that defines our pointers to raw pointers, without using lots of ugly evil macros!! place a #define USE_COOL_POINTERS on your project and your pointers suddenly become smarter for the next debugging session!

It requires the existance of a single "reference manager" entity, wich posesses a simple list of this struct:

struct tagRef
{
void* pointer
int count
}

(and please ignore the fact that I gave it a stupid name like "cool pointer")

how it works:

A CoolPointer is actually a pointer to an entry in the reference manager's list (a pointer to the "tagRef.pointer" member above, wich is the real pointer to the T instance).

Whenever the cool pointer gets assigned to a T*, we search for T* in the reference manager's list. If not found, it means that no other cool pointer is currently pointing to it, so we add it to the table and set it's count to 1.

If it's found, it means that other cool pointers are already pointing to it, so we increment the corresponding counter and set the cool pointer to point to the same entry. That way, for each pointed object, there's only ONE real pointer to it (entry in the manager's list), and all the cool pointers point to this unique pointer!

When a previously-assigned CoolPointer goes out of scope, or gets assigned to some other object, or gets assigned to NULL, it means that the cool pointer is "forgetting" about the object it's currently pointing to, so he decrements the count in the pointed object's entry in the reference manager's list. If the count gets to zero, the entry is removed from the list so it doesn't get full with entries to long-gone objects.

Now comes the "magic" part: because we modify operator delete, when anything on our system that has an adress (that is, anything!) gets deleted, we just check the reference manager's list to see if there is an entry matching the address of the object being deleted. if yes, we just set this entry to NULL.

Now, if any of the 6000 cool pointers in the application that were pointing to the deleted object is de-referenced, a NULL POINTER EXCEPTION will occur! A-ha, BUSTED!

Enough typing, let's go to the actual working cool code!

----- **** --------- **** --------- **** --------- **** ----
COOLPTR.CPP start
----- **** --------- **** --------- **** --------- **** ----

    /*    CoopPtr*/#include <map>#define		NOT_USED_REF		-666#define		REF_LIST_SIZE		512// reference recordstruct tagRef {			void	*poRef;	// the pointer that will be pointed at	int	nCount;	// count of cool pointers that point                        // to the void* up there. if                        // NOT_USED_REF, record is not used};// the cool reference managerclass CCoolReferenceManager {private:		// reference record list	tagRef		aRef[REF_LIST_SIZE];	int i;public:		// constructor - init list	CCoolReferenceManager() {		for (int i=0; i<REF_LIST_SIZE; i++) 			aRef[i].nCount = NOT_USED_REF;	};	// some cool pointer is telling us that it's        // referencing some object	// return address of indirect pointer for him	void** CoolPoint(void* poTarget) {		// look up poTarget in table to see if                //it already exists		for (i=0; i<REF_LIST_SIZE; i++)		{			if (aRef[i].poRef == poTarget)			{				// found: increase the count				aRef[i].nCount++;                                // ref. to the entry in table        			return & (aRef[i].poRef);		           	}		}		// not found: create new entry		for (i=0; i<REF_LIST_SIZE; i++)		if (aRef[i].nCount == NOT_USED_REF)		{			aRef[i].nCount = 1;			aRef[i].poRef = poTarget;			return & (aRef[i].poRef);		}		// FIXME: reaching here means "table is full"		return NULL; 	}	// some cool pointer is telling us that        // it's "forgetting" some object	// the argument passed is the pointer to the        // pointer of the object (it's a	// pointer to an entry in the table)	void CoolForget(void **poEntry) {				// decrement entry's counter. if zero,                // the entry can be disposed, as no more		// cool pointers are using it		if ( --(((tagRef*)poEntry)->nCount) == 0 )     	           ((tagRef*)poEntry)->nCount = NOT_USED_REF;	}	// called by delete operators just before an object        // at a specified address is deleted	void ObjectDeleted(void *poDeleted) {		// search the list. if you find it, set                // the pointer to NULL		for (i=0; i<REF_LIST_SIZE; i++)		{			if (aRef[i].poRef == poDeleted)				aRef[i].poRef = NULL;                }	}};// global reference managerCCoolReferenceManager		refman;// the cool pointertemplate <typename T> class CoolPointer {private:        // the pointer to the pointer	T**		ptr;		public:	// constructor: point to nothing	CoolPointer() {	ptr = NULL;	}	// destructor: "forget" pointed object	~CoolPointer() { Forget(); }	// forget previous pointed object	void Forget() {		if (ptr != NULL) {			refman.CoolForget((void**)ptr);			ptr = NULL;		}	}	// cool pointer assigned to T*	// FIXME: replace this with proper operator=	//	void Assign(T* newref)	{				Forget(); // forget previous pointed thing		if (newref == NULL)			ptr = NULL;		else                  	// point to other thing, letting                        // manager know that			ptr = ((T**)refman.CoolPoint(newref));	        }		// FIXME: make operator= for when CoolPointer<T>        // is assigned to a T*	// FIXME: make operator= for when CoolPointer<T>        // is assigned to another CoolPointer<T>!	// FIXME: make operator==	// FIXME: make operator*	// dereferencing CoolPointer<T> yelds T*, and        // it WILL BE NULL if the object was 	// deleted before!	T* operator->()		{		// we can now trap here two different                // pointer errors: if ptr is NULL, you're		// trying to dereference a NULL pointer.                // if ptr is not NULL but the pointer it                // points to is, it's the dreaded                // "dangling pointer use" error!		return ((T*)(*ptr));	}		const T* operator->() const	 // same	{				return ((T*)(*ptr));	}};// custom delete operator. this kind of sucks because we// can't just "append" code to existing delete operator.// and classes that define their own delete operators don't// work with cool pointers as well;//void operator delete(void* m) {	// let reference manager know	refman.ObjectDeleted(m);          // normal delete action I guess..	free(m);}// test type// I did this because I only managed to overload// operator->...struct test_t {		int	someint;};// ----- test ----int main() {	// create an int on the heap	test_t *n1 = new test_t;	n1->someint = 1;	// create a cool pointer to it	CoolPointer<test_t> pn1;	pn1.Assign(n1);	// point to the "1" instance	// delete the instance: no error yet...	delete n1;	// now let's use the cool pointer...	// BOOM! it's NULL! :-)// try to assign a value to a deleted object ... BUSTED!	pn1->someint = 999;			return 0;}    

----- **** --------- **** --------- **** --------- **** ----
COOLPTR.CPP end
----- **** --------- **** --------- **** --------- **** ----

To do:
- remove unnecessary C crap out of it, like #defines and fixed-size arrays. use STL.
- make CoolPointer operator overloads to make it very sweet to use.
- not being able to "append" code to existing operator delete surely takes some fancy-ness out of the solution...


So, what you think? I'm going to spend some time on it, if I find problems, or find a complete working solution, I'll post here.

--
Fabio Reis Cecin
Amok Entertainment

Edited by - fcecin on June 10, 2001 5:09:56 PM

Edited by - fcecin on June 10, 2001 5:11:40 PM

Edited by - fcecin on June 10, 2001 5:19:10 PM

Another note on the pointer to pointer idea ... this is what "handles" are in the Macintosh, and they are used so the memory manager can shuffle things around without you caring - this idea is fairly much obselete now with Mac OS X (and even the later incarnations of the classic memory manager). But it also has the side effect that if a pointer is invalidated (for example memory was purged, remembering 128k was the norm when this was diesgined ) then all references to it are also invalidated.

Oddly enough, this partial solution didn''t even occur to me when I was answering before - I guess I read about observers 6 months ago and the Mac memory manager a decade ago - so I''m glad someone else suggested it

--

Get a stripper on your desktop!