(Not) using std::thread

This post is about std::thread but not about threads or multi-threading. This is not an introduction to C++ threads. I assume that you are already familiar with Standard Library components thread and async.

I encountered a couple of introductions to C++ threads that gave a code example similar to the one below:

void run_in_parallel(function<void()> f1, function<void()> f2)
{
  thread thr{f1}; // run f1 in a new thread
  f2();           // run f2 in this thread
  thr.join();     // wait until f1 is done
}

While it does give you a feel of what thread’s constructor does and how forking and joining works, I feel it does not stress enough how exception-unsafe this code is, and how unsafe using naked threads in general is. In this post I try to analyze this “safety” issues.

What if an exception is thrown? This question is (or at least should be) central to C++. In C++ nearly every function should be suspected of throwing an exception. The code have better be written in such a way that if any suspect function throws the program should remain in a well-defined, predictable state. What “well-behaved” and “suspect function” means is defined formally by exception safety guarantees. (If you want to learn more see for instance this article by Dave Abrahams.) Some people hate exception handling mechanism because of this additional responsibility, and resort to using return codes or similar solutions. But exceptions are fact in C++ and nearly any function can throw one. This also applies to functions f1 and f2 in our short example above.

So, what happens if function f1 throws an exception while being evaluated in another thread? While it is not mentioned often (at least in context of std::thread), each thread has its own call stack. Technically, the C++ Standard does not require any call stack. It only requires stack unwinding, and the only “entities” required in this stack are exception handlers (catch-clauses) and destructors of automatic objects. If the stack of the spawned thread is unwound and no matching exception handler found (this is the case when exception leaves f1), function std::terminate is called. This is similar to letting an exception out of function main. In a way, in our example, f1 for the spawned thread is like function main for the main thread.

What happens if function f2 throws an exception? Stack unwinding begins in the main thread. Joining instruction is skipped; the destructor of thr is called. As per the Standard, thr is joinable (it has neither be joined with nor detached from). Executing the destructor for a joinable thread results in calling std::terminate.

If you are well familiar with exception safety guarantees, you will observe that I was imprecise about one thing. Calling std::terminate is not necessarily exception-unsafe. Exception safety at minimum means that we leak no resources and we do not leave the program in an unstable state. std::terminate does not leave the program in an unstable state: it does not leave the program running at all. Regarding resource leaks, terminating the program might handle the release of some resources, and leaking resources like program’s free-store memory is irrelevant after the program is terminated. But calling std::terminate if either f1 or f2 throws gives little comfort here.

Why these restrictions?

Why does an unhandled exception in the child thread cause the call to std::terminate? There are other languages that choose to silently stop the the child thread but let the other threads continue running. In C++ philosophy, no exception is silently ignored. Every exception must be explicitly handled. If it is not possible to do it in any better way, the last way of handling it is std::terminate. And note that an exception can in fact be meaningfully handled by std::terminate to some extent (see here).

Why does the destructor of a joinable thread have to call std::terminate? After all, the destructor could join with the child thread, or it could detach from the child thread, or it could cancel the thread. In short, you cannot join in the destructor as this would result in unexpected (not indicated explicitly in the code) program freeze in case f2 throws.

try {
  if_we_join::thread thr{f1};
  
  if (skip_f2()) {         // suppose this is true
    throw runtime_error();
  }
  
  f2();
}
catch (std::exception const&) {
  // will not get here soon: f1 is still running
}

You cannot detach as it would risk the situation where main thread leaves the scope which the child thread was launched in, and the child thread keeps running and keeps references to the scope that is already gone.

try {
  Resource r{};            // a resource in local scope

  auto closure = [&] {     // the closure takes reference to 'r'
    do_somethig_for_1_minute();
    use(r);
  };

  if_we_detach::thread thr{closure};

  throw runtime_error();
}
catch (std::exception const&) {
  // at this time f1 is still using 'r' which is gone
}

The rationale for not joining and not detaching is explained in more detail in N2802. Why can’t destructor of thread cancel the child thread? Cancellation as in POSIX Threads (see here) does not work well with C++ resource management techniques: destructors and RAII. If a thread is cancelled no destructors of automatic objects are called; or at least, it is up to the implementation if they are called or not. This would cause too much resource leaks. Therefore, it is not possible to cancel a thread in C++. There is a similar mechanism though: thread interruption. Interruption is coöperative: to-be-cancelled thread must acknowledge the interruption. If interrupted, a special exception is thrown that unwinds child thread’s stack until it reaches the outermost scope. However, interruption mechanism is not available in C++11 either. It has been considered for C++11 but ultimately rejected for funny reasons, which deserve a separate post.

Note that the problem of std::thread’s destructor calling std::terminate is not really related to exceptions. It can be triggered when we simply forget to call either join or detach:

{
  thread thr{f1};
  // lots of code ...

  if (condition()) return;
  // lots of code ...

  thr.join();
  return;
}

If we have a long function with multiple return statements we may forget to join/detach on every scope exit. Note that the situation is similar to manually cleaning up a resource, except that here we have two ways of cleaning up the resource: either join or detach, and it is impossible to pick one automatically.

So, given all these safety issues withstd::threads, what are they good for? To answer this question, we have to digress a bit.

Low-level primitives

The following example is often used in job interviews (for C++ programmers):

// why is this unsafe?
Animal * animal = makeAnimal(params);
use(animal);
delete animal; 

In fact, using naked operator delete in the code is practically always a programming error. Programmers are told to use higher-level tools like std::unique_ptr. But given that we already have such higher-level tools in the Standard Library, is raw operator delete needed for anything? The answer is “yes”: it is needed to build higher-level tools, like std::unique_ptr.

The answer is somewhat similar for std::thread. It is a low-level abstraction intended to map 1-to-1 with operating system threads. We have a standard portable component that represents threads with no (or minimum) run-time overhead. Using this low-level tool, we can build as many high-level exception-save tools as we want.

If you want your thread class to join in the destructor, just write your thread class that manages in a RAII-like manner an std::thread inside, that does exactly what you want:

{
  JoiningThread thr{f1}; // run f1 in a new thread
  f2();                  // run f2 in this thread
}                        // wait until f1 is done

Do you want your thread class to detach in destructor? Write another thread class. You do not want to add a new class for every possible meaningful behavior in the destructor? Make your type configurable:

{
  SafeThread<Join> thr{f1};
  f2();                  
}

or:

{
  SafeThread thr{f1, join};
  f2();                  
}

This solution has been proposed for boost::thread (see here). Note that if you are using boost::thread, a reasonable alternative behavior in such thread wrapper is to first interrupt the child thread and then join with it. Similarly to operator delete, a reasonable advice for programmers is to always use such wrappers.

High-level multi-threading abstractions

So, what good high-level abstractions does the Standard Library offer for multi-threading? Practically none. There is one: function std::async. It is designed to solve one problem: safely launch a job in a new thread. It handles neatly situations when exception is thrown in the child thread (from our f1); it does not terminate when you forget to join/detach (it joins implicitly). In this post we do not have room to analyze it in more detail.

However, you may get surprised if you expect too much of it. For instance, our example above could be written like this:

void run_in_parallel(function<void()> f1, function<void()> f2)
{
  async(f1); // start as though in a new thread?
  f2();      // execute in main thread
}

You may be surprised to find out that f1 and f2 are not executed in parallel: the execution of f2 will not start until f1 is finished! See this paper by Herb Sutter for explanation. Also, Bartosz Milewski describes some invalid expectations of async in his article Async Tasks in C++11: Not Quite There Yet.

What about thread pools? What about non-blocking concurrent queues? “parallel algorithms”? Not in C++11. Why? The answer is fairly short. The C++ Committee lacked time to standardize them. At this point many people usually express their frustration and disappointment. I do not find it interesting or productive or even fair. What C++11 offers is solid foundations for concurrency: memory model, atomic operations and the concept of threads and locks. If I gave you the impression that std::thread is generally useless, this is not the case: it is as useful as a low-level primitive can be. It gives you the potential to build all sorts of higher-level multi-threading tools. My goal was only to show how to use it cautiously. This can be summarized as “do not use naked threads in the program: use RAII-like wrappers instead.”

The next revision of the standard is very likely to provide lots of concurrent and parallel computation abstractions. If you look at this year’s proposals in the committee mailing, lots of them address the subject.

About these ads
This entry was posted in programming and tagged , , , . Bookmark the permalink.

29 Responses to (Not) using std::thread

  1. I think you could have a catch block inside f1 and put whatever is thrown, into an std::exception_ptr object and after that make use of the promise/future path to ‘return’ that object to the run_in_parallel function which handles it or rethrows it, as necessary.

  2. Catherine says:

    “… is not possible to use std::threads safely”

    I think that’s a very misleading statement.

    As if you couldn’t join() on throw with RAII (when you say “we may forget to join/detach” we’re not C developer)
    As if std::current_exception couldn’t be used to transport exceptions from one thread to another.

    • @Catherine: Thanks for your feedback.
      “… is not possible to use std::threads safely” indeed implies that there is no way to use std::thread safely. It is not the case and it was not my intention. My message was “you shouldn’t use naked threads, but use them in wrapper classes”. I rephrased the unfortunate sentence.

      “As if you couldn’t join() on throw with RAII” — you are right: you could and you should. I thought I spent a significant portion of the post conveying exactly this message. Well, I will put name “RAII” explicitly, so that it is more visible.

      “As if std::current_exception couldn’t be used to transport exceptions from one thread to another.” — Yes, you could do that, but it would be unwise, given that std::async already does all of this for you. This is another mistake of mine, that I went on describing the drawbacks of async but not its positive aspects. Now I added a line in the post to make this more clear.

      My goal was to highlight the dangers of threads (and of async). I assumed the readers already know the two things and why they are good.

      Thanks for your valuable input.

  3. ceretullis says:

    Great article. Really highlights the potential dangers of std::thread (and boost::thread for that matter).

  4. Alexander says:

    C++ neophyte here, apologies for a basic question. What’s wrong with the “job interview” C++ example? I get using the merits of unique_ptr since it automatically manages deleting and prevents double-deleting, but I don’t see the problem in the sample code.

    • If function use for some reason throws an exception, subsequent lines (including delete) are skipped, and therefore you leak memory. This is an example of a broader category of errors and programming problems: it is virtually impossible to manually control the usage of resources in a big or even small program. You need to be familiar with RAII idiom that automates basic resource management. If you say you are new to C++, allow me to make this recommendation. Learning RAII is a must for C++ programmers. unique_ptr is just one example of the idiom.

    • Marek says:

      Also you need to read and remember documentation for the make() function to know who is responsible for deleting the returned animal. Because the make() method could return one from its pool or pointer to static variable inside function.

  5. Why don’t you just use std::thread with a lambda that calls f1 but catches the exception?

    • Hi Guillermo.
      My goal was to highlight the issues with using threads rather than suggest solutions. The readers will likely come up with the solutions, probably better than mine. Yours is one example. To catch the exception in the child thread — good idea; but how to handle it? Remember that with lambda you will be handling it still in the child thread. Do you want the main thread to be informed? Note that the Standard component: function async, already addresses this issue of f1 throwing an exception.

  6. Joel Lamotte says:

    Hi Andrzej, after reading this article I took a look at one use of std::thread I did recently. Basically it looks like this:

    class Engine
    {
    public:
         Engine() 
         { 
             // some initialization code
             m_thread = std::move( std::thread( &main_loop ) );
         }
    
         ~Engine()
         {
              m_exit_flag = true; // for synchronization
              m_thread.join();
         }
    
    private:
         std::thread m_thread;
         atomic m_exit_flag; 
    
         void main_loop();
    };
    
    

    It is not clear to me if this way is safe or not compared to the examples you have given. I observe that the destructor will be called only if the constructor have been fully executed, which means I am assured that joining will be called if construction went well. However I’m not sure if there isn’t hidden concerns I should be aware of?
    One potential problem I see is the call to join if the thread is finished, I’m not sure how to fix this safely though.

    • Hi Joel. Thank you for the question.

      I believe that your example does implement a good practice “use thread only in a wrapper and join or detach in destructor.” People who use boost::thread typically call sequence:

      m_thread.interrupt();
      m_thread.join();
      

      And this way no shared flag to indicate the request to stop the other thread (m_exit_flag) is necessary anymore.

      Note that your constructor only works because launching of the thread is the last instruction in the constructor body. If your constructor was changed to:

      Engine::Engine() 
      { 
        // some initialization code
        m_thread = std::move( std::thread( &main_loop ) );
        aFunctionThatMayThrow();
      }
      

      And if this function threw, you would be dead: since the constructor of Engine is not finished, the destructor of Engine must not be called, but the child thread has been launched and std::thread’s destructor will still be called (because std::thread’s destructor did not throw).

      For a general rule, it might be better to separate the wrapping of thread out of your class Engine, and only use the thread wrapper inside:

      class Engine
      {
      public:
           Engine() : m_thread{ (someInitialization(), &main_loop) }
           { 
           }
      
           ~Engine()
           {
                m_exit_flag = true; // for synchronization
           }
      
      private:
           JoiningThread m_thread;
           atomic m_exit_flag; 
      
           void main_loop();
      };
      

      The expression (someInitialization(), &main_loop) is a hack to perform the other initialization before thread’s constructor.

      The fact that your main thread joins with the child thread that has finished executing is not wrong: in fact it is necessary. A child thread that finished executing his task is still joinable and still needs to be joined with or detached from in order to avoid termination. So, your destructor is fine. No fix needed IMHO.

      One problem I see is the implicitly generated move assignment. (I think it is generated for your class, although I am not sure). If it is used, it will cause the (move) assignment to the joinable m_thread. Since move assignment is conceptually equivalent to destruction followed by move construction, the same termination rules apply: move-assigning to a joinable thread causes the call to std::terminate. I recommend either explicitly deleting the move assignment of Engine or carefully implementing it to avoid assignment to a joinable thread. But it may not be easy due to the shared m_exit_flag.

      • Joel Lamotte says:

        Thanks for all these recommandations, it’s very helpful. Indeed to make the code more future proof (not become wrong if I add some code after the thread spawn later) I will use a wrapper as you suggested.
        For the move, I misread my implementation, in fact it’s:

        
        auto main_loop_impl = [](){ /* .. */ };
        m_thread = std::thread( std::move(main_loop) ); 
        

        So I didn’t move the thread, only a lambda-expression generated object to avoid some copies. Not even sure it’s necessary.
        The Engine type itself isn’t copyable or movable so I shouldn’t have a problem, I think.

        Thanks again, very helpful!

        • auto main_loop_impl = [](){ /* .. */ };
          m_thread = std::thread( std::move(main_loop) ); 
          

          “So I didn’t move the thread, only a lambda-expression generated object to avoid some copies. Not even sure it’s necessary.”

          Well, you do move-assign a thread in this line (even though you do not use std::move). However, in this case this is save because the thread you assign to has been default-constructed and is therefore not joinable.

  7. atch says:

    Hi Andrzej
    Nice article. Really enjoyed it. I suppose you wanted to talk just about std::thread, but it seems a good idea to let people know what tools there are already for concurrent/parallel programming. I use (and love) Intel’s TBB.

  8. Hi Andrzej,

    Can I ask you another question, not related to std::thread, but to multi-threading and C++ exceptions.

    I am one of the several authors of MySQL++, C++ interface to MySQL server, based very much on STL. In early years of 21st century there were lot’s of unexplained crashes stemming from C++ exceptions used in the environment with many concurrent threads each running some complicated code. I saw lot’s of projects, based on C++ using exceptions, discarded due to this problem. Things would run smoothly under small load, several threads, but the same code in many threads lead to crashes with totally unexplainable stack traces.

    Just wanted to ask you if this problem still persists in some compilers. Thank you very much in advance.

    • Hi Sinisa,
      If I understood you correctly, your observation is about the “quality of implementation” of the C++ Standard in some/most C++ compilers. Right?

      If so, I may no be able to answer your question competently. Looking at GCC bug database, there are some in “open” state that mention EH and multi-threading (52604, 32344), but this is just one compiler out of many. I could not say what is the status of EH support in MT environments across all compilers.

      Crashes in big programs can be also caused by too complicated program flow that the programmers loose control over. EH mechanism introduces complicated control paths difficult to follow unless you are using safe idioms, like RAII. Threads and mutexes are practically not comprehend-able in big applications. Race conditions’ negative effects may not occur in medium sized applications, but only reveal in big programs — and not always. Perhaps the combination of EH and multi-threading reaches the limit of program complexity beyond which you cannot reason about program correctness and cannot even notice the UB you have introduced? There is not much help here. Threads are simply too low-level a mechanism for big applications.

      Note that languages like Java do not help in that case either: they may get you no-crash guarantee, but cannot offer correctness guarantee. in this sense a crash is a safety feature.

      I am afraid I cannot help you answer your question. Sorry.

  9. Hi Andrzej,

    You answered it more then you may think.

    Thank you very much !!!

  10. Vasilis Daras says:

    I don’t get how std::thread::join and std::thread::detach are safe to be called from a destructor, the C++ specification states that they may throw.

    • Sinisa Milivojevic says:

      I have not tried calling those methods in a destructor, but if EH is handled in destructor (and why not ???) then I do not see why it would not be possible. You could do so much with good design AND implementation and very little without those.

      • Vasilis Daras says:

        There are two ways of handling an exception in a destructor. The first one is to catch the exception and do nothing, thus destruction will fail silently and this is rarely in an application’s best interests. The other one is to propagate the exception or throw another one but this isn’t safe, since the destructor may be called during stack unwinding that occurred by another exception and this means that the application will be terminated.

  11. Hi Vasilis. This is an interesting question. It is based on a well established “best practise” that one should not call functions that could potentially throw in ones destructors.

    However, note that the standard carefully specifies the conditions under which these functions are allowed to throw. For detach():

    • if the thread is not valid.
    • if the thread is not joinable.

    For join():

    • if deadlock is detected or this->get_id() == std::this_thread::get_id(),
    • if the thread is not valid,
    • if the thread is not joinable.

    These situations all describe programmer’s errors: joining/detaching an already joined thread, joining/detaching a null-thread or a non-existent thread, joining with ourselves, deadlocking.

    Programming errors are something you do not encounter, when you do not have bugs in your program, and if you do, you have a more serious problem than handling exceptions or not-throwing from destructors.

    • Vasilis Daras says:

      Thank you very much for your insightful answer! This means that managing carefully ones threads makes it safe to call join or detach in a destructor.

      I’m still not very sure if it would be principled to do that, since a new C++ specification could add some new exceptions that would render such an implementation non-portable among compiler versions.

      • Vasilis, I feel I have to add something to my reply. I focused on defending my solution, but I somehow dismissed your concern, which I sympathize with. Every C++ programmer should at least feel discomfort when calling a potentially throwing function from a destructor.

        I note that the situation is very special with class std::thread: it has been designed in such a way that it cannot be used “safely” (under some definition of safety), without risking serious consequences. It is very fragile. The problem is really not with calling std::terminate, but with the fact that spanning and then joining with the thread does not fully fit into RAII design.

        Class wrappers proposed in this post, make the situation “safer” (however you define “safe”), but as you note, it is still not fully comfortable. On the other hand, I believe that there exists no perfect solution, and I choose these wrappers because they are the best thing I can come up with. Given the, perhaps incorrect, initial conditions that we have to use threads in RAII manner.

        One could be tempted with implementing a try-catch block in the destructor, but it doesn’t look like a solution — how do you handle the failure to join with a thread that the OS does not recognize.

        Your observation really got me thinking. Perhaps using RAII for threads is not the best choice. One conclusion I get from this, is that rather than playing with naked (or partially clothed) threads, one should use a parallel task library. There are some available already, and C++ is going to provide one in the future.

        For dessert, consider the following example (not related to threads):

        double Sqrt(double d) noexcept(false) // can throw
        {
          if (d < 0) throw exception{};
          return std::sqrt(d);
        }
        
        double AbsSqrt(double d) 
        {
          return Sqrt(std::fabs(d));
        }
        

        Is function AbsSqrt safe to be called from a destructor?

        • Vasilis Daras says:

          By reading the implementation of both functions I can say that it is safe to call AbsSqrt in a destructor because I know that it can’t throw. But, if I didn’t have access to the implementation, I wouldn’t call it from a destructor because the function’s signature doesn’t state that it doesn’t throw.

  12. Dimitri Spukolski says:

    I don’t really understand the intention of this article.
    Apart from std::thread, what would your general exception handling be?
    There are (I think) three options:
    * std::terminate by not handling it
    * catching the exception within the thread itself
    * Handling the exception in an other thread (i.e. std::exception_ptr)

    Am I missing a further possibility, which is not possible?
    I understand, that you don’t like the decision to be std::terminate the default one.
    It’s good, that you point out the fact that you have to take care of the objects within the surrounding scope as you might run out of it, but this is due to the fact, that you have to do your memory management in C++ – preferred by RAII – yourself.

    What I really like in your article, is the part, where you point on the fact, that although async is used, there is no parallel execution. I didn’t know that by now. That’s an argument to use std::thread instead ;)

    • Hi Dimitri. I am glad you liked the post. My goal was this. I often see short examples by differrent people showing how to use std::thread. But it surprises me that hardly anyone mentions that you cannot use threads in so simple a way in real programs. They never mention the consequences of not catching the exception in the child thread (and this is unfortunate, because programmers will asume they would be ignored (as in Java)). They do not mention consequences of destroying a joinable thread. The goal was to draw attention of the readers to what happens in these suituations. I do not have an emotional attitude to how these things work. I am simply describing things that I have never seen described. It is not that I dislike the decision to std::terminate. My goal is to indicate that such decision has been made — and I am convinced that some programmers do not know that.

      • Dimitri Spukolski says:

        I’m still amazed, that the async-example isn’t run asynchronously, as the destructor of the future returned by async blocks.

        Thx again

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s