Jump to content

  • Log In with Google      Sign In   
  • Create Account


Python Twisted: need help on deferred


Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

  • You cannot reply to this topic
9 replies to this topic

#1 Herwin P   Members   -  Reputation: 634

Like
0Likes
Like

Posted 24 May 2013 - 12:00 AM

Hi everyone. I'm writing a Twisted based daemon for a simple text-based multiplayer game. The game logic goes like this. The client sends a string in a particular format (command.arguments (also separated by period)) to the server. The server receives that string, split it, and run the command along with the arguments. I already made it working.

 

Some of the commands called (login for example) has to get data from database. I'm using Twistar to make things fancy.  Here's where I found some difficulties. The function which calls the available commands returns the result of the called command. The problem is that I don't know how to make my login function to return the result of the query.

 

Simply, what I want : send strings (or list, or number, or whatever) gained from the database to the client.

What I get now : the server sends a deferred object or an empty string or list (I've tried some ways to send things to the client).

 

I know it sounds so newbie, but this is my first Twisted program, and I'm still learning, so please spare me.

 

Well, I think the scripts will make things clearer:

 

The server.py.

Spoiler

 

The protocols.py

Spoiler

 

The services.py

 

Spoiler

 

There's no complicated logic yet. I started writing this thing just three days ago.

 

The deferred does fire. It prints the "bla bla bla logged in" properly.

 

The client is just a simple client which sends 'login.Sky Warden.123'. The arguments will be separated by the '.' as well.

 

I think that's all. Maybe it's just me who don't fully understand deferreds, or it's just a logic hole. Or maybe both. XD

 

Thank you everyone. smile.png



Sponsor:

#2 hplus0603   Moderators   -  Reputation: 5185

Like
1Likes
Like

Posted 24 May 2013 - 10:20 AM

In general, for TCP servers, each client has a socket that the server sends responses back to.

In a general communications crossbar, you will pass along a "client" object to each handler of a command.

This handler will then call some "enqueue message" function on that "client" object.

The client object in turn is responsible for taking those enqueued messages and turning them into byte streams you send back out over the socket.


enum Bool { True, False, FileNotFound };

#3 Herwin P   Members   -  Reputation: 634

Like
0Likes
Like

Posted 24 May 2013 - 11:26 PM

In general, for TCP servers, each client has a socket that the server sends responses back to.

In a general communications crossbar, you will pass along a "client" object to each handler of a command.

This handler will then call some "enqueue message" function on that "client" object.

The client object in turn is responsible for taking those enqueued messages and turning them into byte streams you send back out over the socket.

Thanks. That makes sense. I was thinking to send the data over the socket via callback, but I'm stuck since the service is in a different class with the protocol.

 

Can you please give the example code to do this? It will help me a lot.



#4 hplus0603   Moderators   -  Reputation: 5185

Like
0Likes
Like

Posted 25 May 2013 - 10:36 AM

Where do you think the callback should come from?

Typically, you keep a queue of outgoing messages per client.

Once per network tick, you loop through all the clients, and bundle up all the messages for a client into a network packet, and send that.


enum Bool { True, False, FileNotFound };

#5 Herwin P   Members   -  Reputation: 634

Like
0Likes
Like

Posted 26 May 2013 - 12:14 AM

Where do you think the callback should come from?

Uhm... soon after the deferred is made?

 

 

Typically, you keep a queue of outgoing messages per client.

Once per network tick, you loop through all the clients, and bundle up all the messages for a client into a network packet, and send that.

 

So it's like we add new bytes of data to the queued message for each loop. It's like we append the new chunk of data to the a list of the message right? I think I understand that concept, but I guess I'm weak at coding it. I'm new to Twisted after all.

 

Can you please give me an example code? I'm not lazy or what, but that's the way I can understand things better. By learning the code.


Edited by Sky Warden, 26 May 2013 - 12:23 AM.


#6 hplus0603   Moderators   -  Reputation: 5185

Like
1Likes
Like

Posted 26 May 2013 - 11:11 AM


 

Can you please give me an example code? I'm not lazy or what, but that's the way I can understand things better. By learning the code.

 

Hmm. Actually, I think the problem is this: You don't want the deferred/callback to be scheduled by the thing that wants to send out the message. The thing that wants to send out the message should just enqueue it on the client-representing entity.

Then, there's a separate Deferred (probably a timer) that polls each client entity once every network tick, and bundles up all the enqueued messages and sends them out through the socket.

The interface for the game logic objects is simply "here's a command that came in, and here's a queue where you enqueue any particular responses you may have." No Deferreds, or Twisted, needed at all. This also makes the game logic much easier to unit test!


enum Bool { True, False, FileNotFound };

#7 Herwin P   Members   -  Reputation: 634

Like
0Likes
Like

Posted 26 May 2013 - 09:50 PM

Hmm. Actually, I think the problem is this: You don't want the deferred/callback to be scheduled by the thing that wants to send out the message. The thing that wants to send out the message should just enqueue it on the client-representing entity.

 

Kinda like that. Well, actually it's simpler. What I want is to make the login() function to return the data I get from the database. Just that. My problem is that I don't know how to do that. :lol:

 

The getLogin() needs to fetch data from the database, so it returns a deferred. Then I add a callback (checkLogin) to process the result when the deferred fires. Nah! The problem is here. The function which gets the result is the callback (the checkLogin), but the function which needs to return it is login(), the caller.

 

Sorry for the bold text. I just use that to mention that they're the important part. laugh.png

 

By the way, I've tried to follow your advice about a client object handler. Sounds cool, but I don't know how to get the address when a connection is made (I'm sure it's in the startProtocol function). I know how to do it in TCP, but not in UDP.

 

Random question: are you the creator of that pymmo module?



#8 hplus0603   Moderators   -  Reputation: 5185

Like
1Likes
Like

Posted 27 May 2013 - 10:32 AM

You can't directly make the login() function return the data. You have to instead defer a function that gets called with the result of the login. If I remember correctly, you use reactor.callLater() for this: http://twistedmatrix.com/documents/12.3.0/core/howto/defer.html

 

You might want to look at the imvu.task library, which makes this somewhat easier -- the code you write looks as if it's serial, but the use of coroutines makes it actually wait for the deferred operations. Off-hand, I don't remember whether this will work as-is with Twisted or not.

 

For getting the source address, in UDP, there is only one client socket, and the source address is available on each incoming packet; you then have to do the mapping from address to client yourself, whereas TCP does that for you.

 

And, yes, the 4-hour "pymmo" example was indeed something I did one late night :-)


enum Bool { True, False, FileNotFound };

#9 Herwin P   Members   -  Reputation: 634

Like
0Likes
Like

Posted 27 May 2013 - 07:55 PM

hplus0603, on 27 May 2013 - 23:25, said:
You can't directly make the login() function return the data. You have to instead defer a function that gets called with the result of the login. If I remember correctly, you use reactor.callLater() for this: http://twistedmatrix.com/documents/12.3.0/core/howto/defer.html


Which functions I need to defer? The caller of the login() function (the celcommand_login())? So I need to fetch the result of that function after it's ready? And if it needs more than x seconds it will return nothing?

Uhm... code please? If you fix my code I can learn it quickly. I know it sounds demanding and lazy, but I'm not. I'm very eager to learn, and the best way I can learn logic is by reading code. unsure.png

I really need to get this concept. I'm sure once I understand this problem I will be able to make an interesting progress.


I'm also trying to make another version (in case my logic is wrong in this version) with another logic flow. The difference is that I write the data to the socket via callbacks. The problem I get in this version is circular dependency. The DatagramProtocol class needs the Service class to call the login function in it, and the Service class needs the DatagramProtocol class to do the transport.write.

Oh my brain...
 

 

I also made the more ignorant version. Everything runs in one service. It's very very dirty, but kinda working. I still don't feel right about it though. I will only use it if the first version can't be helped.

 

hplus0603, on 27 May 2013 - 23:25, said:
For getting the source address, in UDP, there is only one client socket, and the source address is available on each incoming packet; you then have to do the mapping from address to client yourself, whereas TCP does that for you.

Oh, yes. I know that the datagram contains the address of the sender. I was wondering if we could get the address exactly when a connection is made, like in TCP. I forgot that UDP only has one socket. laugh.png

 

hplus0603, on 27 May 2013 - 23:25, said:
And, yes, the 4-hour "pymmo" example was indeed something I did one late night :-)


That thing is amazing, sir, and you said you did it while learning a new language. Your pymmo really needs more publication. blink.png

By the way, why do I have to connect to the Internet to use UDP? If I'm not connected to the Internet it will say "no such devices" or something similar.


Edited by Sky Warden, 28 May 2013 - 12:08 AM.


#10 Herwin P   Members   -  Reputation: 634

Like
0Likes
Like

Posted 29 May 2013 - 12:14 AM

Well, apparently I made the second version (the one which writes data to the socket via callback) working. I solved that circular dependency problem by subclassing the protocol class. The subclass of the protocol class is a client object which handles requests like what you suggested, so I think it's done for now. :lol:

 

Thanks for the advice. :)






Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.



PARTNERS