I've been stuck on this for over 12 hours! I've been googling and reading the Valve Networking article, reddit, gamedev topics, etc. I feel like it's time I'd rather just ask for help because I'm so confused :X

My game uses Crystal lang for the server, and Godot for the client. 

My server physics step (Default Move Speed is 200). There really isn't any "physics" stuff yet, but trying to just get the basics down to sync the x position (moving towards the right)

  property server_tick = 0.00

  def physics_loop
    delta = 0.0167 # 60 fps
    loop do
      players.each do |player_id, player_obj|
        if player_obj.right
          player_obj.position.x += (DEFAULT_MOVE_SPEED * delta)
        end

        # puts player_obj.position.x
      end

      @server_tick += delta
      sleep delta
    end
  end

Client side input sending (In Godot).  I've heard JSON is not a good choice for bidirectional communication, but honestly, just using it right now because it's keeping everything simple for me. I will most likely use something else later

var MSG_BUFFER_GAME = {}
var player_states = {right = 0}
var old_player_states = {right = 0}
var game_tick = 0
var last_message_sent_game = 0
var last_message_received_game = 0

func _input(event):
	
	if event.is_action_pressed("d"):
		player_states.right = 1
        
	if event.is_action_released("d"):
		player_states.right = 0
        
	if old_player_states.right != player_states.right:
		send_to_game_server({ right = player_states.right}, "MOVE")
		old_player_states.right = player_states.right

        
func send_to_game_server(message, cmd = "PING"):
	MSG_BUFFER_GAME.cmd = cmd
	MSG_BUFFER_GAME.message = message
	MSG_BUFFER_GAME.tick =  ("%.2f" % game_tick)
	
	#print("Sending.. %s" % to_json(MSG_BUFFER_GAME))
	TCP_SERVER_GAME.put_utf8_string(to_json(MSG_BUFFER_GAME))
	last_message_sent_game = OS.get_ticks_msec()
    
func _process(delta):
	var speed = 200
	var motion = Vector2()
	
	if player_states.right:
		motion = Vector2(1, 0)
		
	$player.global_position += motion * (speed * delta)
	
	game_tick += delta
    
func _on_tcp_timeout():  
    send_to_game_server("", "PING") # fake ping to sync game_tick from server
    
func MessageGameHandler(msg):
	last_message_received_game = OS.get_ticks_msec()
	
	var json = JSON.parse(msg).result
	var cmd = json.cmd
	var new_message = json.message
	if cmd == "PONG": # sync local game_tick with server's
		game_tick = new_message

The server's MOVE command just basically sets the player_obj.right to true or false.

The server's PING command just  sends the correct game_tick:

    when "PING"
      client.send server_tick, "PONG"
    end

The problem is, I don't know what to with this almighty game_tick value! Because when `right` is active, the x value on the server doesn't match the x value on my client (not synced). Here is a photo (server console on right). 

I've read in this reddit post that I need to get the difference in the game_ticks, and "adjust accordingly"? What does that mean?

Do I need to add the game_tick difference in my _process's delta on the client, or on the server's delta? (Assuming that's what that means). My mind just goes blank at this point , any help is appreciated!

How synchronized do you need this to be?

Most interactive games do not have synchronized processing.  The game server behaves as though it lives some time in the future, the game clients behave as though they live some time in the past, and each client is at some time that is independent from the rest.  When games require a time, often a game will call the start of the round a specific time (e.g. time 0) and all the events over the course of the game are based on time relative to that timer.  If the server gets a timestamp it doesn't think is right (either too far in the past or in the future), it will discard the message.

Even if your game has a great ping time, out in the real world you are unlikely to consistently be less than a frame, or about 16ms.  Even if you could send the messages in 15ms, you'd only have 1ms left for the game to do any work and hit 60 fps, the most common monitor refresh rate.  Round trip times of 50ms, 80ms, even 100ms are commonplace.  A 100 ms ping means waiting over six graphics frames before getting a response from a message.

If the games must be synchronized every frame you are unlikely to enjoy interactive framerates.  That works well for games that have low framerates, or games that simulate the game at a much slower pace.  Many of the early RTS games updated their game simulations at 4x per second (250ms between updates) so they had time to synchronize the networking stuff.

If you are building a game that doesn't require high communications speeds synchronized simulations can be accomplished by being extremely careful about what is sent, compressing data, the signal-to-noise ratio of what is sent versus the overhead of internet packets, and timing within the program itself.

If you do that, be prepared to add all kinds of animations and interpolation between frames to keep the player engaged.  Or write a game like chess or go where simulation steps can take a very long time.

33 minutes ago, frob said:

The game server behaves as though it lives some time in the future, the game clients behave as though they live some time in the past

Yea, I always think the game client is the future. I think that is where i'm struggling. Because I think "the client is sending input, so this will be something done in the future". This scrambles my brain

33 minutes ago, frob said:

How synchronized do you need this to be?

My game is similar to treasure arena / Chronicon, but not pixelated (but has poe/d2 like system with items, skills, etc). WASD Movement is essential, so it has to be somewhat sync'd for basic collisions to trigger on the server (a fireball entity being  moved across the game world, and triggering damage number updates)

I was looking at treasure arena's WebSocket frames, and they are doing something similar where they send just the player's movement vector whenever they change direction, with a timestamp. The problem is, I don't know what   to do when the server receives a "local" timestamp. And how that can play into effecting the delta to change game state? Do I just add the round trip / 2 to the delta value? Do I need to modify my physics loop on the server, and each player has their own latency property that acts as an accumulator to their x position? Or, does the client just need to adjust the delta based on the latency (or game_tick)?

It's really confusing for me, I appreciate your patience 

The way I think of it is, (in a typical authoritative server game), the game is taking place on the server. The client views are just some vague guesstimate of what is happening in that server game. If you are getting confused just forget everything except server time, everything else is just a bodge to give some kind of interactive experience on clients.

The client sends input to the server, and in a most basic version the client player does not even respond until that round trip has happened and the server moves the player. In action games, in order to make the client seem more responsive, client side prediction is used, and the client runs the same physics as the server and attempts to predict the position the player will move to. If this corresponds to what the server sends back, then all is well and good, if not, you do a correction on the client. In most cases the client side prediction will be correct unless there is interaction with another player.

Generally the other players will always be in the past (in respect to server time), and your view of your player will thus not correspond to the view of your player seen by other clients. This is why you often get things like being shot around corners etc. But the details of the relative times will depend on how you decide to implement it.

Glenn Fiedler has some great articles on how to do all this:

https://gafferongames.com/post/networked_physics_2004/

Also, don't use TCP:

https://gafferongames.com/post/udp_vs_tcp/

And don't send a single input in your packets to the server. Client should maintain a historical list of inputs (over e.g. a number of ticks). Each input packet should contain all the historical input as well as the current input, going back as far as the last acknowledgement (ACK) from the server.

So e.g.

Server: Player positions, ACK of input packet 5

Client: Input 6, 7, 8 , 9

Server: Player positions, ACK of input packet 7

Client: Input 8, 9, 10, 11, 12

etc

In this way the server can reconstruct the client moves in the case of a missed packet.

6 hours ago, Hodgman said:

whenever a message arrives from the server, the client can add the previously calculated difference to convert any server timestamps into local clock times.

Just for confirmation

• The client sends its local game_tick whenever a character's state is changed
• When the server receives this information, the server sends its game_tick value along with my game_tick value that was just sent 

Then, when the client receives that,  calculate the difference of those (for syncing), and add it to my game client's  delta to "advance time"?   If I got that right? I think I'm almost there, do I just need to add that difference to my "delta" variable in the  _process func?

53 minutes ago, hplus0603 said:

You can do the same thing on the client. If you communicate game steps in terms of step numbers, rather than times, then your client can be told by the server whether it's sending state "too early" or "too late," and the client will adjust its "start time" accordingly to send state for step numbers sooner, or later.

I think i'm getting it now. "if you communicate game steps in terms of step numbers, rather than times". In Godot, the _process is fixed I believe to the monitor's refresh rate, so I would need to create my own https://gafferongames.com/post/fix_your_timestep/ I'm guessing. Which should be relatively easy to convert to GDScript, then use that. 

Going to give this another shot today, I think i'm understanding it better and better, appreciate everyone's insights and patience as always