Deducing your intentions

The language feature in C++17 known as class template argument deduction was intended to supersede factory functions like make_pair, make_tuple, make_optional, as described in p0091r2. This goal has not been fully achieved and we may still need to stick to make_ functions. In this post we will briefly describe what class template argument deduction is, and why it works differently than what people often expect.

The original problem was that people had to write code like this:

f(std::pair<int, std::vector::iterator>(0, v.begin()));

Being required to spell out the types of 0 and v.begin() is inconvenient, so the library came with the convenience function:

f(std::make_pair(0, v.begin()));

This works because while template parameters of class templates (prior to C++17) could not be deduced from constructor arguments, they can be deduced for function templates from function arguments. This is an improvement, although sometimes you want to go with the former construct: when you want to adjust the types of the arguments:

g(std::pair<short, std::string>(0, "literal"));

Next improvement, in C++17, was to actually allow deduction of class template parameters from arguments passed in construction, so that the workaround with make_pair should not be needed:

// C++17
f(std::pair(0, v.begin()));

And it works for make_pair, and most of the time it also works for other factories, like make_tuple, except when it does not.

Consider the following example:

auto o = std::make_optional(int{});
  // o is optional<int>

auto p = std::make_optional(Threshold{});
  // p is optional<Threshold>

auto q = std::make_optional(std::optional<int>{});
  // q is optional<optional<int>>

Function make_optional just wraps the argument into a std::optional. This is obvious for the first two cases with o and p. Regarding the third option someone might have doubts, expecting that nested optionals would collapse into one, but that would not be a good idea: sometimes we want them to nest. For instance optional<int> could model a “threshold”, where no-value means threshold is infinity; whereas optional<optional<int>> could mean a “threshold that may not be known”. Actually, Threshold in the second example could be an alias for
optional<int>. Also, consider a general case inside a template:

template <typename T>
void f(T v)
  auto o = std::make_optional(v);

We want the assertion to hold regardless of what type the template is instantiated with.

Now, let’s try to replace make_optional in our examples with class template argument deduction:

std::optional o (int{});
  // o is optional<int>

This works as expected.

std::optional q (std::optional<int>{});
  // q is optional<int> !

This works different than make_optional! This is because the class template argument deduction logic rather than working uniformly, tries to guess what your intention is differently in different context. This logic assumes that in line 2 below:

std::optional o (1); // intention: wrap
std::optional q (o); // intention: copy

our intention was more likely to make a copy. In some cases this guess works, but it does not work when our intention is always to wrap inside an make_optional. Now, this case:

std::optional p (Threshold{});
  // p may be optional<Threshold>
  // p may be Threshold

We do not know if the type of p is optional<Threshold> or Threshold, because we do not know if type Threshold is an instance of std::optional or not. This means that inside a template, when we want to be sure that we are always wrapping a T into an optional<T>, we have to use make_optional and cannot rely on class template argument deduction. This is one of these cases where compiler can deduce something else than what you expect. We have a similar situation with make_tuple:

std::tuple t (1);           // tuple<int>
std::tuple u (t);           // tuple<int>
std::tuple v (Threshold{}); // ???

This problem occurs because we have two contradictory expectations of a deduction like this. One is that it should wrap, the other is that it should make an exact same type:

std::optional o (1); // intention: wrap
std::optional q (o); // intention: copy

These expectations are contradictory in the cases like line 2 above.

std::optional has a second, similar problem. Consider this case:

optional<int> o = 1;
assert (o != nullopt);

This works as expected because T can be converted to optional<T>. The following also works as expected:

optional<int> a = 1;
optional<long> b = a;
assert (b != nullopt);

This works because optional<U> can be converted to optional<T> whenever U is convertible to T, a valueless optional<U> being converted into a valueless optional<T>. This is intuitive. But consider this case:

optional<int> a {};
optional<optional<int>> b = a;

assert (b == nullopt); // correct?

Should the initialization in line 2 be treated as a conversion from T to optional<T>? (In that case the assertion will fire.) Or should it be treated as a conversion from optional<U> to optional<T>? (In that case the assertion will pass.) The compiler will try to guess what we intended, but it will likely guess wrong. Note that this problem will be more difficult to spot if the code looks like this:

Threshold a {};
optional<Threshold> b = a;

assert (b == nullopt); // correct?

Or like this:

template <typename T>
void f(T v)
  optional<T> o = v;
  assert (o == nullopt); // correct?

For this reason, in generic context (and even in non-generic context, when you cannot be sure about the properties of the type), in order to avoid bugs resulting from ambiguity described above, you cannot rely on the “intelligent” logic in opitonal’s constructors. You had better state your intentions directly:

template <typename T>
void f(T v)
  std::optional<T> o {std::in_place, v};
  assert (o != nullopt); // correct

The bottom line of this post is: intelligent deductions usually work as expected, but sometimes they do something else.

This entry was posted in programming and tagged , , . Bookmark the permalink.

14 Responses to Deducing your intentions

  1. Igor says:

    Nice article, great job! Example based learning is the best way 😉

    I found few bugs in code examples – sometimes variables names are switched or incorrectly doubled, like below:

    Threshold o {};
    optional o = p;

    assert (o == nullopt); // correct?

    Probably first one should be p 😉

  2. Ying says:

    typo in “first to cases” should be “two”

  3. Argothiel says:

    My intuition tells me, that you shouldn’t be nesting optionals anyway. The problem is not that the compiler cannot deduce your intention — it’s that you’re using the same term (optional threshold) in two different meanings. If you want Threshold to have three different value types, you should probably use variant or something similar:

    using GeneralThreshold =
      std::variant<Threshold, InfiniteT, UnknownT>;
    • This advice is good. I would probably go with providing a separate class Threshold. But it does not protect you from the situations where a nested optional is created by a template, which adds an optional on some T, and it does not know if the T is optional itself:

      template <Callable F>
      auto f(int num, F f) -> optional<invoke_result_t<F>> 
        optional<invoke_result_t<F>> result {};
        if (ok(num))
           result = f();
        assert(bool(ok(num)) == bool(result));
        return result;

      And will the author be always aware that this function template for some instantiations creates a nested optional?

      • Argothiel says:

        That’s a valid point. For now I lean towards a view that all nested optionals are evil. So in your case this function should be invalid for invoke_result_t(F) == optional.

        Maybe it would be good to allow a programmer that he doesn’t want optionals to be nested ever?

        Anyway, to even think of that, one has to be aware of pitfalls mentioned by you. So — thank you for your post!

  4. Carsten D. says:

    I wanted to write a similar article as a guest post on Jonathans fluent C++ webpage but was to busy in the last couple of month. Nice job explaining the issue here. optional is indeed a bitch…

    • I do not think the problem is specific to optional. Consider a vector:

      std::vector v {1}; // wrap
      std::vector w {v}; // copy

      This can also give counter-intuitive results inside templates. Or even this:

      template <typename T, typename U>
      auto transform(vector<T> const& vt) -> vector<U>
        vector<U> result {vt.begin(), vt.end()};
        assert(vt.size() == result.size());
        return result;

      Does the expectation expressed with the assertion hold regardless of types T and U?

  5. magras says:

    A few more typos:

    > … what people often expect.
    > *fi*
    > The original problem …

    > This is one *one* of these cases where compiler can deduce something else than what you expect.

  6. A L says:

    Cool sideburns! I love your picture Andrzej

  7. Great post Andrzej! I think this “issue” goes only as far as the difficulties of using auto. The rule of the thumb would say: When in doubt, just use explicit types and do not rely on deduction. I guess we will just have to find a great balance in the usage, to avoid any misunderstanding.

  8. Pingback: All Major C++17 Features You Should Know - My Blog

  9. Pingback: All Major C++17 Features You Should Know – DevStream

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.