Probably not a popular opinion but: I'm not really a fan of all the user-space async stuff that's become popular (IE: the nodejs/tornado style of single threaded apps with non-blocking IO).
Remember Windows 3.1 or old Mac OS with it's cooperative multitasking where you had to explicitly yield or risk freezing up the entire computer? It's basically the same thing reinvented with a nicer brand. I know I'm dating myself here, but it's really the same model. We came up with preemptive multitasking for a reason: because it's more robust and efficient.
Threads are dangerous, but basically you can avoid the danger for the most part if you avoid sharing objects across threads, and if you do that then with blocking IO your code flows linearly and you don't have to deal with callback hell.
> It's basically the same thing reinvented with a nicer brand.
Not at all. First, don't confuse preemptive scheduling with time-slice scheduling. All "true" user mode threads are preemptive (Erlang, Go, Quasar). Not all of them employ time-sliced preemption. In fact, when we implemented Quasar on the JVM we had time sharing but then took it out because it didn't gain us anything (other than increased implementation complexity). The reason is that fibers in interactive applications follow a certain pattern that always entails frequent blocking. You are right, though, that some types of computations do not fit well with the user-mode threading model -- long, CPU intensive computations. Those are best left for kernel threads that do time sharing well. Languages that don't give you access to kernel threads might implement time sharing for their lightweight threads in user mode (e.g. Erlang).
The parent was comparing the event model (e.g. callbacks everywhere) with a thread/process model; not comparing user-mode and kernel-mode threads. So I didn't follow your point.
Also, erlang doesn't use time slices, it counts "reductions" (essentially function calls, though I don't think it's always exactly one function call). It also penalizes processes more if they send to a mailbox that is already very large.
Erlang is considered truly pre-emptive, because (assuming you aren't writing your own functions in C or something) a function can't loop or use any operators or really do anything without potentially being pre-empted.
Go is generally considered partially pre-emptive, because it only pre-empts on a function call or during a memory allocation. You can write a simple loop that doesn't terminate, and it will never get pre-empted.
> Erlang is considered truly pre-emptive... Go is generally considered partially pre-emptive...
Well, in Quasar we started taking the "fully preemptive" route, but we saw that threads fall into two categories: those that block very often, and those that don't. Because the JVM, unlike Erlang and Go, gives you access to kernel threads, too, Quasar will simply warn you if you're using a fiber for a CPU-intensive operation that doesn't block often. Using reductions didn't work out so well because a "forcefully preempted" fiber still wants more CPU, which normally means it's doing something wrong.
Go actually does give you access to OS threads in the runtime package in the sense that you can easily reserve a thread for the current goroutine via http://golang.org/pkg/runtime/#LockOSThread and similarly can give it back up to the scheduler with runtime.UnlockOSThread()
Going partially pre-emptive is a reliability hit though. One errant process/thread/fiber can bring the system to a halt (or at least one hardware thread).
Erlang is designed for reliability, not CPU efficiency for CPU-intensive tasks.
You are wrong about Erlang. It's threads ("processes") are not preemptive and must explicitly yield for another thread to execute. Most of the time the VM handles it for you, every few hundred bytecode instructions it checks if another thread should run. But if you write a C extension, call sleep(10) then the Erlang VM will twiddle its thumbs for 10 seconds.
It's not a corner case -- it means any call to a C library needs to be carefully vetted so as to not cause an overly long computation or blocking because just a single one will totally ruin your concurrent experience. ALL VM:s using user mode threads/fibers suffer from that problem.
Real kernel threads doesn't have a lot of overhead and are almost always a much better choice.
Yes, and that's the definition of preemptive. There's nothing explicit about it if the VM handles it for you. What you're saying is that it cannot preempt kernel-thread blocking code.
> Real kernel threads doesn't have a lot of overhead and are almost always a much better choice.
Well, as usual, that depends. A kernel thread task-switch is 20-50us, while a fiber task-switch is a couple nanos at most; that's a 1-2 orders of magnitude difference.
That's just wrong. The definition of preemptive multitasking isn't that it works most of the time. A definition that lax would mean almost any cooperative threading system would be counted as "preemptive."
Also can you please find a reference that shows (Linux) kernel threads being 1-2 orders slower than user threads?
It's a little less dangerous when you only need to cooperate with yourself, but yes, it does mean you need to be very careful that you don't have any means for some requests to take an unexpectedly long amount of time.
In MacOS if you grabbed a window title bar for too long with the mouse then all your sockets would drop. Fun times.
> In MacOS if you grabbed a window title bar for too long with the mouse then all your sockets would drop. Fun times.
I wonder if the people developing windowing systems with client side decorations (e.g. Wayland + Qt) thought about this issue. Any information about that?
As far as I'm aware there's two approaches (often both used at once) in modern GUI systems that mitigate this. One is an explicit event loop, which can be locked up by user code but not normally by routine events, and the other approach is running the UI on its own thread. (Or, to put it another way, applications should perform any long-running operations not on the UI thread.) The second, particularly, would avoid the dropping-sockets problem, as network and drawing would very likely be on separate threads.
I don't want to apologize for the existing callback-oriented APIs, because I agree it's easy to create some pretty hard-to-follow code in them. But there's something to be said for using non-blocking I/O calls. Threads aren't free, and spinning up a lot of them can tax system resources and impede performance. Allowing a number of execution contexts to share a single thread, automatically relinquishing control when they initiate an operation whose result won't be available for a while, can be a more efficient approach in certain circumstances.
Given that, I do respect what Microsoft was trying to do with async/await in C#. I'm not convinced they were entirely successful, though. The sales pitch is that you can happily bang out asynchronous coroutines as if they were just sequential code, but the truth is that if you actually try to do that you're liable to run into some nasty bugs.
I'm somewhat more intrigued by what some functional languages have done with the dreaded M-word to attack the same problem. The sales pitch is similar, but it's rather easier to deliver on the promise when "Just code like you normally do" being directed at someone who's normally writing in a functional rather than imperative style.
Python has achieved pretty much the same callback hell avoidance, without the need for monads or a foray into functional programming (in an imperative language). Python does it using generators as preemption points and using syntactic sugar for generator composition (yield from).
Using generators is rather clever. It's a look on generators not as fancy iterators that generate values but as fancy iterators that generate code execution steps.
Does it avoid the "Oops I accidentally forked a bunch of stuff I needed to run sequentially off into parallel threads that will now execute in an indeterminate order" situation? That's probably the biggest problem I see with C#'s async/await. Lovely syntax alone isn't sufficient, you need lovely syntax that doesn't make it it easier to write buggy code with the syntax than without. I suspect, though, (but cannot prove) that the risk of callback tangling an inevitable consequence of trying to slap this kind of abstraction on top of imperative programming style.
The reason why I think success is more likely when functional languages try to do the same thing is the functional style itself: Explicitly passing your state around in arguments and return values rather than relying on side effects means that whenever you do need there to be a strict ordering among your callbacks, that will naturally express itself in the code.
That's not an uncommon opinion at all, and I share the same view as you. New languages/run times that leak their concurrency model to application code are ugly. I've sworn off Node, and all event driven run times, for this reason. Erlang is far superior in this regard. The VM has an amazing scheduler. Immutability and message passing are the icing on the cake that make writing performant and maintainable systems that scale a joy.
> Remember Windows 3.1 or old Mac OS with it's cooperative multitasking where you had to explicitly yield or risk freezing up the entire computer? It's basically the same thing reinvented with a nicer brand.
A thread deadlock can still freeze up your entire process if it held the wrong mutex. Multiple processes for hardware concurrency in a web server context makes way more sense. Want hardware concurrency? Just spin up multiple processes. You're still taking advantage of preemptive multitasking, while getting all the benefits of not worrying about it within a single process's context.
> We came up with preemptive multitasking for a reason: because it's more robust and efficient.
For coarse tasks, sure. Throw a few thousand OS threads at a problem and you've exhausted your entire 32-bit address space on stacks before doing anything useful. It will not perform well in a 64-bit address space, although at least it won't crash. Probably.
Traditionally the answer has been to turn the task into a state machine, callback spaghetti, or whatever else. I find these answers to be quite lacking vs e.g. .NET's user-space async stuff, which I can use alongside threads if I so desire (as I frequently do.)
> Threads are dangerous, but basically you can avoid the danger for the most part if you avoid sharing objects across threads
This defeats the entire point of using threads instead of processes.
I challenge you to name a single advantage that threads have over processes beyond ease of sharing data. I imagine that such an advantage exists, but I cannot think of one myself.
On modern OS's, don't threads generally start up quicker and take less memory than processes? My understanding was that was actually the original motivation for the invention of threads, making something lighter-weight than an OS process.
But I think it's true that programs generally share some data between threads; but best to remember to keep the shared data as minimal as possible, and/or limited to immutable data structures.
> On modern OS's, don't threads generally start up quicker and take less memory than processes?
Yes... but the thing is, that's largely as a function of sharing data!
The OS can skip image relocation for the executable and it's dynamic libraries by sharing the same data with the other threads of the process. Process metadata is shared, so no new process entries need be created.
Your program can skip the parsing of configuration files, command line arguments, and other initialization by sharing the same data already parsed read and initialized by the other threads of the process.
On some platforms, there's fork(), which lets you retain some of the advantages of shared data. Not all of them: For example, one of the tradeoffs of a compacting garbage collector is that the relocation of data can cause those pages to become no longer shared between the forked processes, even if the data on those pages was otherwise untouched and identical, because the compacting isn't shared between processes.
The general trick is to keep a pool of processes ready to handle work, much as efficient thread pools avoid spinning up new threads for every tasklet.
But every async platform I know of has a non-async way of handling long running tasks so you can background them. The async model is good for responding to things in a quick fashion, such as http calls to a simple API. If the process you want to accomplish takes a long time the standard operating procedure is to background it and send a callback for (possibly to another api) when it's completed.
And yes the threading model works, on one machine. But across machines or OS instances? For that you'll need some kind of mutex syncing and you're basically back in the same boat you were in with the methods mentioned above. So why not use a platform specifically tailored to multi-instance clusters and save yourself some hassle?
Whether or not you can scale across machines has nothing to do with the thread model you choose. It's just about whether or not your server is stateless across requests, but you can do that with both async and non-async servers.
I say "kinda" related because even TM is a lower-level thing than the author of this post is talking about. The treatment of the subject, however, is much more in-depth. Lambda the Ultimate discussion: http://lambda-the-ultimate.org/node/2990
There are videos linked in Section 4 of the paper.
It is still in the small program stage although Glitch as a C# framework is usable now (indeed, I've written an editor, UI, and so on, all very concurrent). As a framework, it is not really convenient without being a new language, but it could be interesting to do something like ReactJS using replay and rollback.
If you like Bret Victor-style demos, I'm working on a web essay that should be done sometime this summer.
You know, the embedded systems community has a heck of a lot of tools and formal logics for designing this stuff in the abstract. Lots of little DSLs and auto-checkers for thinking about processes and messages, that sort of thing. I'm not so sure about model -> code generation, I think that is mostly rolled by-hand.
For concurrency in the large, I wonder if there will be greater adoption of these sorts of engineering techniques.
I assume you are referring to the concurrency DSLs like Esterel? These are quite low level with a different emphasis; I'm not sure they would scale to larger non-embedded systems, and they like many of the niceties that programming for larger systems can afford.
"Languages and frameworks like Go, Akka and Erlang have come up now because they help solve the hard networking and concurrency problems that we need to build clusters."
What does Go offer for a cluster environment? As far as I can tell, it's still targeted at single machines (though perhaps with many cores).
If you're interested in concurrency, you owe it to yourself to spend some time working through this guide on ZeroMQ[1]. It could change the way you think about software architecture forever, because it makes it so straightforward to change your app into a multi-node, multi-language fabric of machines. It's also a fun way to get your feet wet with a new language, because every language under the sun is supported, and you can easily integrate the code in your new language with systems written in your old language.
Which is another reason it might change how you think about architecture. ZeroMQ makes it practical to bring in other new languages and libraries into your ecosystem.
I am not a hardware expert, but from what I understand, hardware architects didn't get around to supporting VMs and garbage collection as well as they might have. Systems continued to be optimized best for workloads that looked like scientific computing in Fortran from the 80's or desktop applications written in C++ from the 90's. Hardware architectures to support video games and media, on the other hand, seem to have taken huge strides over the same timeframe. Now, it seems like they are behind in terms of supporting distributed systems on multi-core machines. The kind of contortions required to detect and avoid problems like False Sharing indicate that today's hardware could be a good ways suboptimal for building these systems.
> Hardware architectures to support video games and media, on the other hand
I'll have to disagree about the video games bit. Today's consoles are a completely regular AMD x64 CPU together with a completely regular AMD GPU, one running a version of Windows, plus one that uses PowerPC.
Yesterday's were two with multi-core PowerPC and an AMD GPU, one running a version of Windows, plus one with that really weird PowerPC together with 7 tiny PowerPClets and an nVidia GPU that was famously awful to program for.
Going back even further, we have an Intel x86 together with an nVidia GPU, running a version of Windows, a PowerPC-based CPU and an ATI-developed GPU (minor note: the company, ArtX, that was doing the GPU got bought by ATI and their designs did end up in ATI GPUs, so I'm listing it as such), a Hitachi SuperH CPU together with a PowerVR GPU, running a version of Windows, and this weird design from SONY.
And going back a further iteration things get even weirder and so on and so forth.
I'll agree that the further we go, the more these systems are made from specialised components (hello Saturn) and don't resemble what you'd get from e.g. a Dell. But that is mostly because you needed to use specialised components to get really cutting-edge performance for those tasks. Today you can more or less stick in 4 x64 cores and forget about it.
Today you can more or less stick in 4 x64 cores and forget about it.
That only works for embarrassingly parallel tasks. Start requiring coordination, and today's hardware makes efficient parallelism plus concurrency hard.
The Mill CPU has an interesting approach to false sharing [0]- cache lines keep track of which bytes are valid for their core, so one core could be using one part of the line while another uses a different part, and they wouldn't try to communicate until/unless they actually tried to use the same bytes.
The first question to ask may well be "why does my problem need computation on a commodity cluster?". A massive multi-player spaceship battle may be best served by a HPC environment, using C++ and MPI. There's probably only a limited class of distributed problems that map nicely to a commodity cluster. Several worthy commentators have pointed out that even if the systems programming environment offered the same verbs for both, commodity network latency makes "distributed concurrency" very different from "in-box concurrency". To loosely borrow a term from physics, time delays can break the scaling symmetry of a renormalization group.
A systems architect with a sound background in mathematics may be able to reason out the cluster performance of a social algorithm operating on a Erdos-Renyi random graph. Plodders such as myself will try to prototype. I'll insist that a prototype has business value if only because otherwise I have no way of appearing to be busy.
For prototyping a distributed system on a commodity cluster, my personal preference would be Erlang. As rvirding commented with great insight in another post, Erlang has an OS feel to it. From my limited perspective, Erlang spares me the trouble of knowing Unix and networking (for example: I don't need to know what a TCP port is). Erlang gives me a minimal & consistent set of verbs, and that's all I need for prototyping.
I feel like the author of this piece doesn't really have personal knowledge about what he is talking about.
I use Go and I use Erlang. I've never used Akka.
Go is not in the same class as Erlang when it comes to Cloud Computing. In Erlang the state and behavior of your entire datacenter can be contained in Erlang itself. Erlang will take care of bringing up new processes when your jobs break. Erlang will handle messaging between processes executing on different machines.
Go doesn't do any of that out of the box. There are some interesting projects like Go Circuit, which sort of kinda want to emulate Erlang's OTP and bring your datacenter behavior into Go, but Go Circuit isn't being run in production by anyone I have heard of. Maybe its ready to go, but it looks more like a research project to me.
As such, Go has nothing to do with managing a cluster other than that it gives you some nice single node concurrency features. You will need to build or import everything else you need.
I can't think of anything else out there which is quite like Erlang/OTP. Maybe Julia has some tools that are similar but I don't know too much about it.
Erlang isn't as user friendly as Go, but it has a lot of stuff that can save you serious headaches if you think you will need to scale. Premature optimization is a waste of time ... blah blah etc., so maybe you don't need it for your project. Also Go on App Engine allegedly scales pretty well with very little programmer effort.
From those, only easy messaging actually helps creating clusters, and it can be implemented in a library without any problem.
The other may help the language being good, or help one get more throughput from a node, but offer no help in maintaining the consistency of a cluster.
I would also look outside of the JVM box and consider C#. It stays familiar, while having proper parallelism and concurrency constructs: async+await, TPL, TPL DataFlow, Parallel.
Seems pretty accurate. So this is probably a great time to mention my favorite new(ish) platform for handling concurrency in a polyglot way: http://vertx.io/
Coordination with the real world is and always has been the pain point with STM. It's fine if your STM universe lives entirely in a subset of your data on a single machine [1], or within a single database [2], or with a single decision-making unit [3].
Most languages have extremely poor support for separating interactions with the real world from interactions that occur entirely within one serializable context (a thread working with thread-local memory, for example). Can you safely replay arbitrary C, C++, etc.? No, because side-effecting code could run at any time and occur in any context. So that is one problem that has to be solved first.
Suppose you've solved that problem. Well, now for a distributed system, you need to make STM talk to STM. Make one module talk to another over a network, or a filesystem, or another database, or a client browser. Do you have STM working in the JavaScript running on client's machines? And even if you managed that feat, do you have end-to-end STM from your datastore to your client's actions?
Distributed STM that wasn't painful to use, either to write or in terms of performance, would be a sort of holy-grail of distributed computing. I don't think any language or toolchain is there yet.
[1] Haskell, Clojure, et al STM engines.
[2] SQL-compliant relational databases such as DB2, Oracle, SQL Server, as well as distributed databases like HyperDex that support true transactions.
[3] Paxos, Raft, and other decision-making algorithms only ever externally appear to be consistent, but are internally complex and might have an internal tug-of-war.
> Can you safely replay arbitrary C, C++, etc.? No, because side-effecting code could run at any time and occur in any context. So that is one problem that has to be solved first.
I spend my time solving this problem. You can do it with a programming model especially designed for it. Functional programming is not required, though can be convenient.
> Well, now for a distributed system, you need to make STM talk to STM. Make one module talk to another over a network, or a filesystem, or another database, or a client browser. Do you have STM working in the JavaScript running on client's machines? And even if you managed that feat, do you have end-to-end STM from your datastore to your client's actions?
STM is the wrong way of thinking about this problem, mainly because replay is not an intrinsic part of the paradigm (instead, users manage that themselves). Rather, go back further to Jefferson's virtual time/Time Warp system [1], which was designed specifically in the context of distributed systems.
> Can you safely replay arbitrary C, C++, etc.? No, because side-effecting code could run at any time and occur in any context. So that is one problem that has to be solved first.
With few restrictions, it's actually possible to replay arbitrary assembly. Namely, don't care about timing and no system calls that can't be replayed and you basically have what "checkpoint" does in gdb.
General-purpose STM has not only not taken off -- it's essentially dead, as no efficient enough implementations have been found. More restricted STM (like Clojure's refs) is used, but is still in its "experimental", or early-adoption phase.
Remember Windows 3.1 or old Mac OS with it's cooperative multitasking where you had to explicitly yield or risk freezing up the entire computer? It's basically the same thing reinvented with a nicer brand. I know I'm dating myself here, but it's really the same model. We came up with preemptive multitasking for a reason: because it's more robust and efficient.
Threads are dangerous, but basically you can avoid the danger for the most part if you avoid sharing objects across threads, and if you do that then with blocking IO your code flows linearly and you don't have to deal with callback hell.