I have been wondering about this for a while now.
Lets say I had a matrix that held location, rotation and scale, however all I would need was location(X,Y,Z), 8 directions(Every 45 angle) and 1 constant scale(Always 1).
What would be the most optimal way to store the value? So that it would use the least amount of memory and be very small so that you could send millions of them over a data packet?
You can't send millions of anything in one packet.
Nor does the way you store a value necessarily relate to how you transmit a value.
One of 8 directions can be stored and transmitted in 3 bits. Scale is always 1 and therefore requires no storage or transmitted data. Location requires 3 numbers but the size of those depends on the precision of the values.
What kind of precision do you need for your position coordinates? What are your limits on data? I imagine for something like this, a “snapshot frame” could send the full precision of state (X, Y, Z, rotation), while intermediary frames use less precise or fewer bits for changes between them. If possible, I’d look into gzip compression, but I suspect it won’t have too much of an effect.
EDIT:
Some rough guesses. Assuming some reference frame that has the full precision of the `(X, Y, Z, rotation)`, and for each frame, we use halffloat precision for deltas for a total of 27 bits/entity, then several million entities will be about 54 Mbits/update. Or perhaps, you could use a variable number of bits and only send the components that need updating: `(ID, (component, value), ...)`…
8 minutes ago, Kylotan said:You can't send millions of anything in one packet.
Interesting. Why not?
8 minutes ago, fastcall22 said:What kind of precision do you need for your position coordinates?
Lets say the world ranges from -15 000 000 000 to +15 000 000 000 on all axis. No floating points.
12 minutes ago, Kylotan said:One of 8 directions can be stored and transmitted in 3 bits.
What would the code for this look like?
Lets say the idea is the have 1 billion 2D zombies update each frame and send the update to all locations in the most optimal way.
The minimum amount of data you can send is 1 bit. You can't put a million bits in one packet.
You could potentially send an update that concerns millions or billions of characters and it could compress down to 1 bit, if you're lucky. That's a different problem. It's also unrealistic.
Storing 8 directions in 3 bits is just a case of reserving 3 bits of whatever storage you're using for a given number. You can flip bits in C# with the | and & operators. I suspect this is largely a waste of time compared to more productive means of reducing the cost of what you send, however.
You are not going to be able to have 1 billion zombie updates per frame without either redefining the meaning of either (a) update, (b) frame, or (c) billion.
5 minutes ago, Kylotan said:You are not going to be able to have 1 billion zombie updates per frame without either redefining the meaning of either (a) update, (b) frame, or (c) billion.
Would there be a reasonable amount?
I mean particles can be in the millions while holding collision data. So to what limit could we push the amount of objects if we reduce the data they use?
" 65,542 bytes in a network pack" is what I got from a quick google search. What is the rough amount of 2D zombies I could update with this?
35 minutes ago, fastcall22 said:will be about 54 Mbits/update.
54 Mbits seem like a lot for what is needed.
Sorry if I am asking a lot of questions. I just cant understand why there are games that only update 16 characters and still send MB worth of data, while there are games updating 100 of characters for a few KB.
I assume it's because the later only send data it must and at very small precision. The question then is how? How do they reduce there data sizes?
19 minutes ago, Kylotan said:Storing 8 directions in 3 bits is just a case of reserving 3 bits of whatever storage you're using for a given number.
I assume it has something to do with this. I have always resorted to storing data in a smaller type if I needed less precision.
For example: int MyNumber = 100; sbyte SmallerData = MyNumber.
But that means sbyte is the smallest I know how to make a number. (-128 to 127).
Particles might number in the millions, but we don't try to send them across the network.
The amount of objects you can send is proportional to the amount of data each one needs. Sorry for such a flippant answer but there's no trick or magic here. You can send as much data as your network bandwidth allows (nothing much to do with packet size, incidentally) and the less data you need per entity, and the less frequently you want to update them, the more entities you can update.
To get transmission sizes down, you need to think in terms of information, not in terms of data. You're not trying to copy memory locations across the wire, you're trying to send whatever information is necessary so that the recipient can reconstruct the information you have locally. e.g. If I want to send the Bible or the Koran to another computer, that's hundreds of thousands of letters I need to transmit. But if that computer knows in advance that I will be sending either the Bible or the Koran, it can pre-store copies of those locally and I only have to transmit a single bit to tell it which one to use, as there are only 2 possible values of interest here.
Similarly, if I want to send a value that has 8 possible values - i.e. the directions we talked about - that's just 3 bits. I could pack 2 such directions into a single byte, and leave 2 bits spare. Or I could send 8 directions and pack them into 3 bytes (3 bytes at 8 bits per byte is 24 bits to play with). If you're not comfortable with bitpacking, maybe read this: https://gafferongames.com/post/reading_and_writing_packets/
6 minutes ago, Kylotan said:To get transmission sizes down, you need to think in terms of information, not in terms of data.You're not trying to copy memory locations across the wire, you're trying to send whatever information is necessary
I feel like a idiot for not realizing this. I would never have to update the scale as it is already there and never changed. Now it makes sense, I only need to update the differences.
I could limit the data send by simply limiting the possible differences.
So if I had a 3rd person game where characters can jump I wouldn't need to send: bool Jump = 0; Every update. Only when the character jumps would I ever have to alert the other PC.
19 minutes ago, Kylotan said:If you're not comfortable with bitpacking, maybe read this: https://gafferongames.com/post/reading_and_writing_packets/
I don't think I fully understand this yet. But a quick read of the link shows that it is more or less what I was thinking about. Thanks for all of the help.
Furthering what Kylotan said about only sending the information needed rather than all the data:
If you for some reason want a million zombies on a bunch of clients and a server, I'm guessing the zombies are AI. In which case, if you calculate the exact same numerical zombie AI simulation etc on each client, in theory you only need to send the things that might affect the zombie simulation (i.e. the real players and their inputs). This is a similarish idea to client side prediction, and afaik is useful for RTS games. Presumably you have to be very careful about your simulation to make sure it runs identical on all machines.
If you really do have lots (hoping not millions) of players, then usually each client doesn't need to know about them all, only the ones within visible range. An authoritative server might keep track of all the players, but it only needs to send a subset of these to each client. And of course in the other direction the input only comes from one player per client.
Some crash course about bit packing (i'm using hex numbers which makes it easy to understand):
int32 a = 0x01, b = 0x02, c = 0x55;
All numbers must be >=0 and <= 0xFF (255) - for that range we need 8 bits because 2^8 = 256 possibilities.
int32 compressed = a | (b<<8) | (c<<16);
We get 0x00550201 by bit shifting and OR to pack 3 numbers into one. (upper most 8 bits still unused - you could pack a 4th number)
Now unpack:
int32 A = compressed & 0xFF; // mask out the other numbers
int32 B = (compressed>>8) & 0xFF; // bit shift and mask
int32 C = (compressed>>16); // no mask necessary if there is no 4th number
What to do if our original numbers are larger, but we want to use only 8 bits?
int32 a = 0x00120004;
We could ignore less important bits:
int32 qA = a>>16; // 0x00000012
then after packing and unpacking you lost 16 bits, resulting in 0x00120000 but sometimes that's good enough. Edit: E.g. if you bin characters to smaller tiles of space, less bits become enough for the position relative to the tile, which is a form of delta compression.
2 hours ago, Scouting Ninja said:2 hours ago, Kylotan said:One of 8 directions can be stored and transmitted in 3 bits.
What would the code for this look like?
You would need to tell how you store your original data. In the worst case it's a switch with 8 cases.
