I see this argument all the time but its not the fork() that is the most expensive anymore, its the actual program initialization after the fork(). If your app is non trivial (lets say a websocket based chat server that needs to persist messages to a DB and use pubsub to sync them to other processes) it probably needs a connection to a database, a connection to a cache server like Redis or Memcached, etc, or perhaps a connection to some other backend service. Reinitializing these dependency connections from scratch in a new forked process for every single incoming websocket connection is expensive and slow.

On the other hand a Node.js or Go program can have a preestablished pool of keep alive connections to the backends already ready to go and reuse that connection pool for many hundreds or even thousands of concurrent websocket connections. You can approximate something like this with the fork() model by having a local daemon process that manages the connection pool and have your forked process talk to that local helper daemon when it needs a connection, but you are still going to pay a penalty for that compared with having a fully preestablished connection ready to use right there in the process already

Then consider posix_spawn(), since it pays that cost, but then we have to pay IPC as well:

fasthttp(go) can do around 80k/sec on my laptop; dash(C) can do almost 88k/sec on my laptop†. As soon as dash does IPC, it drops to around 51k/sec, and as soon as it needs a reply, we're down to 25k/sec††. I see no reason to believe fasthttp would do any faster.

That means I'm spending around 70% of my time in IPC -- something posix_spawn() would let me avoid (if my application were designed to do so). My same laptop will do 100k/sec posix_spawn() so I'd find this difficult to believe (1-4 msec per call) fork() or exec() is the bottleneck for any application with this architecture. Do you think posix_spawn() represents 70% of your costs? If our goal is to beat 51k/sec requests, sure, but NodeJS on my laptop (btw) gets 10k/sec, so if it's a contender, I'd say posix_spawn() is as well.

† https://github.com/geocar/dash/blob/master/README.techempowe...

†† https://github.com/geocar/dash/blob/master/README.md

Well, you can manage resources by shared memory and semaphores. That will make nearly all of the environment setting time go away. Better yet if you just serialize the processes and assign resources to a serial number.

What is just another way to make a NxM server, so maybe forget about it...

You can prefork. That is a tried and true method. And why do you need to create so many connections? Certainly you can architect the solution better than that.

8ms per fork means you can only accept 125 connections per second. (per core)

That means it's only viable for connections where the connection is very long lived and messages are very sparse (because context switches).

If you use websocket for short lived connections, you are doing something wrong.

--

Meta observation:

Maybe this is the bane of smartphone era and small screens, but the context of the discussion seems to disappear instantly.

Subject: websockets > forking processes > .. aaand the websocket context is lost and we are talking generally about forks in web applications with growing thread.

> If you use websocket for short lived connections, you are doing something wrong.

Pretty-much true, but I remember a funny story from Dropbox where their websocket service couldn't come back up after a crash because their normal users trying to re-open super long lived connections all at once was well-beyond the capacity of the system

I find it ironic that people will complain about milliseconds for a fork and talk about the time for context switches... and then serve their pages using an interpreted language that is an order of magnitude slower than it could be in a compiled language...

The comment immediately below the GP that claims that performance is relative literally says that you don't have to care about forking in Node.js.

Not defending node.js, but it is not interpreted but run with jit compilation.

I think the real issue now is not how fast you can accept and process a new connection, but how many you can have in parallel. If you're going to have a native os process for each, you'll soon run out of resources

Cores are cheap: I've got 200, and a lot of that cost is page faults.

It also means you should use posix_spawn instead of fork+exec since you can control when the page faults occur better.

I was going to mention CPU context switching as well, it's very expensive even when you don't account for the initial fork overhead but I guess it's definitely a better idea than CGI for HTTP given the long lived connection.

In an ideal scenario, you shouldn't have more active processes than you have CPU cores. As soon as that happens, context switching kicks in and performance degrades sharply.

Does this concern hold for containers per machine?

Yes I would think so because containers run on top of the OS.

If you can run multiple containerized apps side-by-side on the same machine at the same time on a single CPU core, then you can be sure that there is some kind of context switching happening.

Modern Operating Systems are good at minimizing the amount of context switching. If you run 4 CPU-intensive processes at the same time on a machine which has 4 CPU cores, then the OS will typically assign each process to a different CPU core (with minimal context switching). Then if you launch a 5th CPU-intensive process, then the OS will have no choice but to start doing context switching since it doesn't have any idle cores left.

On Linux, based on tests I did a couple of years ago with a multi-process WebSocket server, I can confirm that the penalty of context switching is proportional to the CPU usage of each process. So for example, if you have a very CPU-intensive process sharing a CPU core with a non-intensive process, then the penalty will be small, but if you have two intensive processes sharing the same core, the penalty will be high.

The initial fork is fast because of the copy on write memory semantic. You'll pay a price later.

Only 8 ms? That's slower than a ping.

You were right about performance being relative. And of course the trade off between ease of development, use and performance. At the end of the day, practical considerations are going to determine what is "expensive".

> Only 8 ms? That's slower than a ping.

Also depends what you're pinging. I'm in the UK, so everything in America is 30-80msec away anyway.

Related: when I worked on a server (on linux) that would spawn threads for new connections that were mostly short lived, I tried to use a thread pool instead. Hand crafted, with push and pop both O(1). It was still mostly slower than just spawning a new thread every time.

It's generally considered the right move to just spawn your own thread for things like connections instead of pooling threads - people tend to think they should throw everything into a threadpool but it's not actually encouraged to do that for stuff like web server connections, compiles, etc unless the task is short-lived. I wish more people knew that going in :) Some threadpool APIs actually ask you whether the job is going to take a while and if it is, they function more as a job limiter - not 500 active threads all competing for CPU - than a thread reuser.