(Not) using std::thread

This post is about std::thread but not about threads or multi-threading. This is not an introduction to C++ threads. I assume that you are already familiar with Standard Library components thread and async.

I encountered a couple of introductions to C++ threads that gave a code example similar to the one below:

void run_in_parallel(function<void()> f1, function<void()> f2)
{
  thread thr{f1}; // run f1 in a new thread
  f2();           // run f2 in this thread
  thr.join();     // wait until f1 is done
}

While it does give you a feel of what thread’s constructor does and how forking and joining works, I feel it does not stress enough how exception-unsafe this code is, and how unsafe using naked threads in general is. In this post I try to analyze this “safety” issues.

What if an exception is thrown? This question is (or at least should be) central to C++. In C++ nearly every function should be suspected of throwing an exception. The code have better be written in such a way that if any suspect function throws the program should remain in a well-defined, predictable state. What “well-behaved” and “suspect function” means is defined formally by exception safety guarantees. (If you want to learn more see for instance this article by Dave Abrahams.) Some people hate exception handling mechanism because of this additional responsibility, and resort to using return codes or similar solutions. But exceptions are fact in C++ and nearly any function can throw one. This also applies to functions f1 and f2 in our short example above.

So, what happens if function f1 throws an exception while being evaluated in another thread? While it is not mentioned often (at least in context of std::thread), each thread has its own call stack. Technically, the C++ Standard does not require any call stack. It only requires stack unwinding, and the only “entities” required in this stack are exception handlers (catch-clauses) and destructors of automatic objects. If the stack of the spawned thread is unwound and no matching exception handler found (this is the case when exception leaves f1), function std::terminate is called. This is similar to letting an exception out of function main. In a way, in our example, f1 for the spawned thread is like function main for the main thread.

What happens if function f2 throws an exception? Stack unwinding begins in the main thread. Joining instruction is skipped; the destructor of thr is called. As per the Standard, thr is joinable (it has neither be joined with nor detached from). Executing the destructor for a joinable thread results in calling std::terminate.

If you are well familiar with exception safety guarantees, you will observe that I was imprecise about one thing. Calling std::terminate is not necessarily exception-unsafe. Exception safety at minimum means that we leak no resources and we do not leave the program in an unstable state. std::terminate does not leave the program in an unstable state: it does not leave the program running at all. Regarding resource leaks, terminating the program might handle the release of some resources, and leaking resources like program’s free-store memory is irrelevant after the program is terminated. But calling std::terminate if either f1 or f2 throws gives little comfort here.

Why these restrictions?

Why does an unhandled exception in the child thread cause the call to std::terminate? There are other languages that choose to silently stop the the child thread but let the other threads continue running. In C++ philosophy, no exception is silently ignored. Every exception must be explicitly handled. If it is not possible to do it in any better way, the last way of handling it is std::terminate. And note that an exception can in fact be meaningfully handled by std::terminate to some extent (see here).

Why does the destructor of a joinable thread have to call std::terminate? After all, the destructor could join with the child thread, or it could detach from the child thread, or it could cancel the thread. In short, you cannot join in the destructor as this would result in unexpected (not indicated explicitly in the code) program freeze in case f2 throws.

try {
  if_we_join::thread thr{f1};
  
  if (skip_f2()) {         // suppose this is true
    throw runtime_error();
  }
  
  f2();
}
catch (std::exception const&) {
  // will not get here soon: f1 is still running
}

You cannot detach as it would risk the situation where main thread leaves the scope which the child thread was launched in, and the child thread keeps running and keeps references to the scope that is already gone.

try {
  Resource r{};            // a resource in local scope

  auto closure = [&] {     // the closure takes reference to 'r'
    do_somethig_for_1_minute();
    use(r);
  };

  if_we_detach::thread thr{closure};

  throw runtime_error();
}
catch (std::exception const&) {
  // at this time f1 is still using 'r' which is gone
}

The rationale for not joining and not detaching is explained in more detail in N2802. Why can’t destructor of thread cancel the child thread? Cancellation as in POSIX Threads (see here) does not work well with C++ resource management techniques: destructors and RAII. If a thread is cancelled no destructors of automatic objects are called; or at least, it is up to the implementation if they are called or not. This would cause too much resource leaks. Therefore, it is not possible to cancel a thread in C++. There is a similar mechanism though: thread interruption. Interruption is coöperative: to-be-cancelled thread must acknowledge the interruption. If interrupted, a special exception is thrown that unwinds child thread’s stack until it reaches the outermost scope. However, interruption mechanism is not available in C++11 either. It has been considered for C++11 but ultimately rejected for funny reasons, which deserve a separate post.

Note that the problem of std::thread’s destructor calling std::terminate is not really related to exceptions. It can be triggered when we simply forget to call either join or detach:

{
  thread thr{f1};
  // lots of code ...

  if (condition()) return;
  // lots of code ...

  thr.join();
  return;
}

If we have a long function with multiple return statements we may forget to join/detach on every scope exit. Note that the situation is similar to manually cleaning up a resource, except that here we have two ways of cleaning up the resource: either join or detach, and it is impossible to pick one automatically.

So, given all these safety issues withstd::threads, what are they good for? To answer this question, we have to digress a bit.

Low-level primitives

The following example is often used in job interviews (for C++ programmers):

// why is this unsafe?
Animal * animal = makeAnimal(params);
use(animal);
delete animal; 

In fact, using naked operator delete in the code is practically always a programming error. Programmers are told to use higher-level tools like std::unique_ptr. But given that we already have such higher-level tools in the Standard Library, is raw operator delete needed for anything? The answer is “yes”: it is needed to build higher-level tools, like std::unique_ptr.

The answer is somewhat similar for std::thread. It is a low-level abstraction intended to map 1-to-1 with operating system threads. We have a standard portable component that represents threads with no (or minimum) run-time overhead. Using this low-level tool, we can build as many high-level exception-save tools as we want.

If you want your thread class to join in the destructor, just write your thread class that manages in a RAII-like manner an std::thread inside, that does exactly what you want:

{
  JoiningThread thr{f1}; // run f1 in a new thread
  f2();                  // run f2 in this thread
}                        // wait until f1 is done

Do you want your thread class to detach in destructor? Write another thread class. You do not want to add a new class for every possible meaningful behavior in the destructor? Make your type configurable:

{
  SafeThread<Join> thr{f1};
  f2();                  
}

or:

{
  SafeThread thr{f1, join};
  f2();                  
}

This solution has been proposed for boost::thread (see here). Note that if you are using boost::thread, a reasonable alternative behavior in such thread wrapper is to first interrupt the child thread and then join with it. Similarly to operator delete, a reasonable advice for programmers is to always use such wrappers.

High-level multi-threading abstractions

So, what good high-level abstractions does the Standard Library offer for multi-threading? Practically none. There is one: function std::async. It is designed to solve one problem: safely launch a job in a new thread. It handles neatly situations when exception is thrown in the child thread (from our f1); it does not terminate when you forget to join/detach (it joins implicitly). In this post we do not have room to analyze it in more detail.

However, you may get surprised if you expect too much of it. For instance, our example above could be written like this:

void run_in_parallel(function<void()> f1, function<void()> f2)
{
  async(f1); // start as though in a new thread?
  f2();      // execute in main thread
}

You may be surprised to find out that f1 and f2 are not executed in parallel: the execution of f2 will not start until f1 is finished! See this paper by Herb Sutter for explanation. Also, Bartosz Milewski describes some invalid expectations of async in his article Async Tasks in C++11: Not Quite There Yet.

What about thread pools? What about non-blocking concurrent queues? “parallel algorithms”? Not in C++11. Why? The answer is fairly short. The C++ Committee lacked time to standardize them. At this point many people usually express their frustration and disappointment. I do not find it interesting or productive or even fair. What C++11 offers is solid foundations for concurrency: memory model, atomic operations and the concept of threads and locks. If I gave you the impression that std::thread is generally useless, this is not the case: it is as useful as a low-level primitive can be. It gives you the potential to build all sorts of higher-level multi-threading tools. My goal was only to show how to use it cautiously. This can be summarized as “do not use naked threads in the program: use RAII-like wrappers instead.”

The next revision of the standard is very likely to provide lots of concurrent and parallel computation abstractions. If you look at this year’s proposals in the committee mailing, lots of them address the subject.