The basic background to this project is that I am trying to move my level generation code into a separate thread so that I can generate the next level in advance while the player is still on the previous level, or while the game is loading. I have a pretty fancy system now that allows my main program to send a variety of signals to the thread, including "quit" "start generating" "whats your status?" and "I am ready to receive the level when you are."

Right now, I have a loop that calls "status" repeatedly, waits for a reply, and if the reply is success, sets up to receive the level. However, at the end of the loop, one pesky value somehow got stuck in the pipe buffer during the process. The specific value is the number 0, used to signal the loop completing. However, I know I received at least one 0 already, which triggered the loop to stop, so I have no idea where the extra 0 came from.

I have been spending too much time trying to fix this problem, so if someone who has experience with this type of things sees some obvious mistake or at least knows some things I could try to diagnose the mistake, that would be very helpfull.

This is my code for the loop in the main program:

        assert not self.pipe[0].poll(), "Some leftover information got stuck"
        
        status=1
        while status:
            self.pipe[0].send("stat")
            status=self.pipe[0].recv()
            print status,"numstats=",numstats
        
        assert not self.pipe[0].poll(), "Some leftover information got stuck"

And this is my code for the other thread:

def generate(pipe1):
    pipe1.send("starting...")
    generating=False
    g=None
    level=None
    objects=None
    size=None
    roomsdone=0
    roomstodo=0
    running=True
    waitingfor=None
    
    
    while running:
        if pipe1.poll() or not generating:
            t=pipe1.recv()
            
            if t=="quit":
                running=False
            elif not generating:
                if waitingfor=="level":
                    level=t
                    print "level: ",level
                    waitingfor="size"
                elif waitingfor=="size":
                    size=t
                    print "size:",size
                    waitingfor="start"
                    generating=True
                    roomsdone=0
                    g=generator.Generator(size, level)
                    
                elif t=="gener":
                    waitingfor="level"
                elif t=="retu":
                    print "sending Grid",g
                    pipe1.send(g)
                    print "sending object",objects
                    pipe1.send(objects)
                    print "returned object"
                elif t=="stat":
                    pipe1.send(0)
                else:
                    
                    raise ValueError(str(t)+" is not a option I know of")
            else:
                if t=="stat":
                    pipe1.send(roomsdone+1) #0 is the quit code, so I add one so it can never be 0
                    
                elif t=="retu":
                    pipe1.send(None)
                else:
                    raise ValueError(str(t)+" is not an option I know of while generating")
        elif generating:
            if waitingfor=="start":
                g.start()
                waitingfor=None
                roomstodo=g.numrooms
            elif roomsdone<roomstodo:
                f=g.step()
                if f:
                    roomsdone+=1
                else:
                    objects=g.finish()
                    generating=False
                    roomsdone=0
            else:
                objects=g.finish()
                generating=False
                roomsdone=0

Some things I have tried:

• Counting how many times I called stat from both sides, they seem to be equal
• Sleeping for a second between calls to stat to check if calls somehow change order because of sending many things fast, no difference
• popping the extra 0 from the pipe, the code compiles, but I don't know where the 0 comes from
• checking in the middle of the loop if ever two things are added, the extra 0 seems to have been put in in the last iteration i.e the last else in the code, or the second to last else, depending on whether the maximal amount of rooms where built or not (usually, they aren't) or the call to stat after generating is false.
• The top line in the code checks that the pipe is empty before the loop, so the problem must be in this line.

I know people don't have time to read long pieces of code, so I shortened it a lot. I was afraid I might take out something important though.

I'm a little suspicious of the first condition inside the loop of your generator thread:

if pipe1.poll() or not generating:

I presume that pipe1.poll() returns true if there's something on the pipe to receive, and false otherwise. What happens if poll() returns false, but the level generation just finished the previous iteration and so generating is also false? Does pipe1.recv() return junk? Possibly the previous message? Because if t somehow gets set equal to "stat", you'll push a 0 onto the pipe, even though no one actually asked for the status. And then the main loop will legitimately ask for the status, and your generator thread will happily reply with a second 0.

It looks to me like you need the following branch structure:

if pipe1.poll():
    t=pipe1.recv()
    if t=="quit":
        running=False
    elif generating:
        #respond based on not started, waiting, or completed status
    else:
        #respond based on running status
elif generating:
    #normal generation code
else:
    #do nothing, maybe sleep for a bit

This way, you're guaranteed to have received some content from the pipe in the first block, and the other two at the bottom don't even attempt to call pipe1.recv(), since pipe1.poll() clearly returned false.

Edit: On further thought, I'd also recommend moving your elif generating branch into it's own independent if block:

if pipe1.poll():
    t=pipe1.recv()
    #respond
elif not generating:
    #do nothing, maybe sleep for a bit

if generating:
    #normal generation code

This way, your generator thread won't starve even if its constantly receiving requests on the pipe. For every one message it receives, it still makes progress on the generation. It's not the best structure, because now it might be a little slow to consume incoming messages on the pipe, but it's better than the possibility of getting drowned in messages and never actually generating the level. You can get fancier by receiving up to a fixed number of messages within an inner loop before taking a break from messages and doing some generation, but I don't think that level of sophistication is warranted in this case.