I can't have a single flag. Panel A might fire his own "close" event, but the listener of that event can't simply set IsOverlayVisible to false because he needs to know which other panels are using the Overlay.
I already have events in place, but that only means that all my spaghetti is inside the Mediator (which i call Controller).
Now, if you are using observer, when a Panel A is triggered to close, it observes the registered object to be triggered when that close event is performed. One of the registered object should be the overlay. This overlay itself should be the one that checks the counter whether it's ready to close (equals or less than 0) or not. This goes the same to other UI objects registered to Panel A that listen when close is triggered.
So you spread all your conditions to each of its own. Panel A only tells that it is about to close, the others listen to that information (regardless of who call it, in this case, it's Panel A, but these guys don't know), and then make their own decision on what to do based on that call. It's basically back to what Kylotan said.
Just an addition, you can pass the Panel A if you like so the registered objects know who make that call, in case you need to know who.