Spatial OS was supposed to revolutionize big-world MMOs. Huge numbers of players in the same world. Lots of hype in 2017-2018. Improbable claims to have raised US$500 million to build it.

Then, not much. Worlds Adrift and Mavericks went live and then went broke. Sominium Space has user counts in 1 digit. Nostos is supposed to come out this month, but it's apparently heavily sharded, with only a few players in each instance.

What went wrong? Doesn't work? Too slow? Costs too much? Doesn't scale? Having to run on Google servers is unacceptable?

Any big-world technologies coming along that really scale? (At least as good as Second Life, 50,000 users in one world, but without the lag of SL.)

It is now can be used in MMOGs building in Unreal and Unity.
Not sure what you mean https://improbable.io/

As I understand it, the basic Spatial OS idea works something like a modern multiprocessor cache. You have objects, which are accessible from any machine in the cluster. Local access is much faster than remote access, of course. So, when there are too many remote accesses to an object, the object is serialized, shipped over the network to where it is being accessed the most, and made live on a different machine. Objects that are only being read can have copies in multiple regions, but a change to any of the copies must invalidate all the read-only copies. (Yes, something like that is how shared memory multiprocessor caches work deep inside.)

For this to work, the game code probably has to be highly parallel, because you're going to block waiting for object moves over the network. So such blocks must not block the whole game loop, just one thread of it. Which requires highly parallel game and physics engines. Which, at last, do seem to exist.

This is a reasonable enough architecture, but does it really scale? They have demos with huge numbers of distant spaceships, but that's easy to do. As far as I can tell, Spatial OS maxed out with 1000 users on the ground in Mavericks in a one-time demo. They weren't all visible to each other, though. Could that approach handle Second Life, with 30,000 to 50,000 users in a shared world?

The Second Life approach, fixed sized regions that communicate at the edges only to move game objects and avatars, breaks down when too many users are in one region. The system only runs well with 20 or less users in a region, because the server has only one main thread.  It's often run with more users in a region than that, and it slows down badly. Second Life can't have crowds. Which is a lack. Everybody wants crowds.

10,000,000 users on a huge plain waving glow sticks at a distant DJ - probably not necessary. Fortnite did that in shards, and in one world it would be a lousy user experience anyway. (Although, Burning Man in VR. Maybe someday...) 1000 users in a big club or a busy shopping area or assaulting a D-Day beach is a reasonable goal for now. Could Spatial OS do that? Has anybody pushed it hard enough to find out?

Finding out if this approach scales is valuable. If it works, it's going to be like physics engines - soon, everyone will be doing it.

https://github.com/puniverse/galaxy

Typed up a reply but lost the text. This link may be of interest.

drainedman said:

https://github.com/puniverse/galaxy

That's an interesting substrate. As the note says, it's a lot like a cache coherency system.

One question is, if you have lots of concurrent accesses to this data structure, do they wait asynchronously, or does one blocked request lock the whole thing up until that request is satisfied. If you're using something like this for a game, you're going to need a very parallel game engine. Not just as many threads as CPUs, far more than that.

@undefined sorry I did answer all this but my post got deleted and I was too tired to answer again.

I worked with spatialos, as well as on site for a week. The tech was, despite the hype almost unusable and did not perform well. That was several years ago however.

The architecture from what I remember was clunky with lots of memory copying and I recall thinking it wouldn't scale well.

So instead as a proof of concept for my company I made my own single shard server based off open source in memory data grids. It worked a hell of a lot better and scaled on LAN and could do huge physics etc. The link I posted was most similar to what I did.

However, this simply wont work over internet. The delay and bandwidth issues would not be workable for any semi real time game. In fact the issues and solutions that are framed tend not to be the real hard issues. Making single shard server and load balancing, reads, writes, etc isnt that hard. Especially if you tightly design the game and data. But making it run as a general purpose MMO, that can be applied to any game without some massive compromises will never happen.

To answer your question. I believe there are no such issues reading data at high concurrency. I had no such issues regarding this. Regarding this kind of thing I feel its important to ask the right questions, and not get amazed by a solution thats not really targetting the problem domain. (below tech are also guilty of this problem)

Other technologies I have worked with include Cloudgine and HadeonOS. Also check out Dual universe, it uses caf. It will never work though because they misunderstand the problem of a global single shard server.

Assume all the servers for adjacent regions are in the same data center with 1Gb/s or better links. I realize that this is not going to work across links with significant delay.