Remove any items from a list while iterating

Started by
15 comments, last by Alberth 4 years, 1 month ago

It's not especially elegant, but you could build a new list and copy any elements you don't want deleted to it. Then you can replace the original list with the new one. If you want an in-place solution, it's unfortunately beyond me to think of one that hasn't been mentioned yet.

-------R.I.P.-------

Selective Quote

~Too Late - Too Soon~

Advertisement

One solution is for the objects to mark themselves as "dead", perhaps using a flag or adding themselves (e.g. index or pointer) to a separate "dead list". The master loop can clear dead objects at a convenient time.

If your list only stores pointers remove the object and replace is with a null pointer, make sure your update function skips the null pointers. Once you are done updating remove all null pointers from the list or replace them with new instances if you need to add new ones.

Worked on titles: CMR:DiRT2, DiRT 3, DiRT: Showdown, GRID 2, theHunter, theHunter: Primal, Mad Max, Watch Dogs: Legion

Here's what I came up with. The hardest situation is breaking from the loop without doing any kind of cleanup on the iterator. I tried doing some smart pointer stuff, but I generally stay away from stuff like that:
List list;
        Link* link[3];
        ListIterator it;
        int i;
 
        //============================================
        // Test: Remove first element
        //============================================
 
        i=0;
 
        link[0] = list.AddLast(1);
        link[1] = list.AddLast(2);
        link[2] = list.AddLast(3);
 
        it = list.Begin();
 
        for (it = list.Begin(); it != list.End(); it++)
        {
                System::Print(it.link->value);
                if (link)
                {
                        link->Release();
                        link=NULL;
                }
        }
 
        //Check
        for (i=0; i<3; i++)
        {
                if (link) Debug::Assert(link->refcount==1);
        }
 
        list.Clear();
 
        //============================================
        // Test: Remove second element
        //============================================
 
        i=1;
 
        link[0] = list.AddLast(1);
        link[1] = list.AddLast(2);
        link[2] = list.AddLast(3);
 
        it = list.Begin();
 
        for (it = list.Begin(); it != list.End(); it++)
        {
                System::Print(it.link->value);
                if (link)
                {
                        link->Release();
                        link=NULL;
                }
        }
 
        it = ListIterator();
 
        //Check
        for (i=0; i<3; i++)
        {
                if (link) Debug::Assert(link->refcount==1);
        }
 
        list.Clear();
       
        //============================================
        // Test: Remove third element
        //============================================
 
        i=2;
 
        link[0] = list.AddLast(1);
        link[1] = list.AddLast(2);
        link[2] = list.AddLast(3);
 
        it = list.Begin();
 
        for (it = list.Begin(); it != list.End(); it++)
        {
                System::Print(it.link->value);
                if (link)
                {
                        link->Release();
                        link=NULL;
                }
        }
       
        //Check
        for (i=0; i<3; i++)
        {
                if (link) Debug::Assert(link->refcount==1);
        }
 
        list.Clear();
 
        //============================================
        // Test: Break during loop
        //============================================
 
        i=2;
 
        link[0] = list.AddLast(1);
        link[1] = list.AddLast(2);
        link[2] = list.AddLast(3);
 
        it = list.Begin();
 
        for (it = list.Begin(); it != list.End(); it++)
        {
                break;
        }
        it = ListIterator();
       
        //Check
        for (i=0; i<3; i++)
        {
                if (link) Debug::Assert(link->refcount==1);
        }
 
        list.Clear();
However, there are a lot of problems you can cause unless that code above isn't followed strictly. rip-off's suggestion was probably the best. Still, this strikes me as a really fundamental problem that ought to have a good solution.

10x Faster Performance for VR: www.ultraengine.com

You can erase items from a list as your iterating it, you just need to stash the next iterator before you do stuff to the current one.

That still wont cut it in this scenario, as the OP says"

It could also result in removing other objects from the list

That would mean that it's possible that the iterator obtained before doStuffWith(it); pointed to the item which got removed, which means that iterator is now invalid.
In fact if the current item was not removed but the next item was, then you must only advance the iterator at the end of the iteration, as per normal iteration.

The problem is that in the case being described, there is no way to know whether the current item it being deleted or the next item is being deleted. So you don't know whether to get the advanced iterator at the beginning of or the end of each iteration.

Hence it is not solvable without at least some information about what is deleted coming back to the code in this loop. If you don't know the current item or the next item is always still there at the end of each iteration, then you can't do it.
"In order to understand recursion, you must first understand recursion."
My website dedicated to sorting algorithms

You can erase items from a list as your iterating it, you just need to stash the next iterator before you do stuff to the current one.

That still wont cut it in this scenario, as the OP says"

It could also result in removing other objects from the list

That would mean that it's possible that the iterator obtained before doStuffWith(it); pointed to the item which got removed, which means that iterator is now invalid.
In fact if the current item was not removed but the next item was, then you must only advance the iterator at the end of the iteration, as per normal iteration.

The problem is that in the case being described, there is no way to know whether the current item it being deleted or the next item is being deleted. So you don't know whether to get the advanced iterator at the beginning of or the end of each iteration.

Hence it is not solvable without at least some information about what is deleted coming back to the code in this loop. If you don't know the current item or the next item is always still there at the end of each iteration, then you can't do it.

Why not move the delete back into the class that has the original list via a callback, this will allow you to know exactly which item to delete and what the iterator needs to do. Better is to mark that object as dead though and leave it in the list and then after update remove the values you don't need, as indicated before.

Generally you don't remove or insert objects into your containers (lists, queue and others that invalidate your iterator) whilst you are updating them, you do this as a post step to the update. Which means having a dead list and new list around, or marking dead ones in the list and having a new list, in the object that contains the actual container. Having objects being out by one frame is hardly noticeable on 30fps and even less so on higher frame rates.

Worked on titles: CMR:DiRT2, DiRT 3, DiRT: Showdown, GRID 2, theHunter, theHunter: Primal, Mad Max, Watch Dogs: Legion

@Josh Klint I think shared_ptr is a better way in c ++ 11, we can save the setting "dead flag" by calling std :: shared_ptr :: reset to set the pointer to null.

There is no real need for a separate "removal of dead" run with consecutively allocated items. You do need a “dead” flag so you can identify and skip the dead, but you can copy/move the still alive items towards the start of the list while performing the next iteration. If death is infrequent, you're mostly copying the items to the positions they are already, which is very cheap.

Deleting items immediately from the list is possible if you use an old-fashioned double linked list. In such a case, you can delete any item from a list except your own. (At least, until you surely know what ‘next’ is going to be.)

So instead of consecutive items in a list, each item has a next and previous pointer to its neighbours. Removing an item can simply be done with some pointer copying. Any item can remove itself from the list quite easily. Obviously, if a neighbouring item removes itself from the linked list, your next and/or previous pointer may change.

Thus, deleting an item is done in 3 stages. 1) Declare the entity in the item dead. It should remove all other relevant items from the list, except the current item. 2) When done, use the (possibly updated) next pointer to move to the (then) next item in the list. 3) Tell the previous item to remove itself from the list, and done.

Obviously, using pointers to move between list elements is likely not very cache-friendly, so this may do more harm than the simpler “dead” flag.

This topic is closed to new replies.

Advertisement