O(1) What! What might bring it down to say 10's of ms? Looks like its some kind of optimizable wall that its 500 for everything.

Like with 10ms then online replication/backup — analogus to litestream for sqlite — but for in memory processes becomes feasible, no?

We're actually median under 500ms — ~320ms median — I just didn't want to piss of hacker news with over estimatation.

We have another set of optimizations that we believe can take us to ~200ms in the next few months but beyond that we're pretty much completely stuck.

Realistically other sandboxes will be able to get there before us because we've chosen to support so much of Linux/if you don't run an operating system or don't support custom snapshots that is much easier.

Insane. Does it possible to fork to another bare metal machine? Maybe multi region as fly io. If not, I bet you have huge disk sizes on your machines to store all the snapshots (you said, you store them and bill only for disk space).

So forking across multiple nodes in that speed is not possible — we run extremely beefy nodes in order to avoid moving VMs across nodes as much as possible.

We are researching systems of hot moving VMs across VMs but it would have very different performance characteristics.

Yeah, I see. Is it possible to get a corrupted state? Let’s say we had realtime database actively writing at that moment?

It is impossible.

Our tech is not decades old so there is a chance we've missed something but our layer management is atomic so I'd be shocked if you'd be able to corrupt state across forks/snapshots.