Last time we were able to run some tasks asynchronously. Things worked great and it was pretty straightforward. However real life scenarios are not as simple as the ones I used in the previous post. For instance networking environment can be quite hostile – the server you are connecting to may go down for any reason, the connection might get dropped at any time, etc. Any of this condition will typically result in an exception which, if unhandled, will crash your process and bring down your application. C++ async is no different – you can easily check it for yourself by running this code:

pplx::task_from_result()
    .then([]()
    { 
        throw std::exception("test exception"); }
    ).get();

(the “for C# Developers” part – Note that in the .NET Framework 4 UnobservedTaskExceptions would terminate the application. It was later changed in .NET Framework 4.5 where UnobservedTaskExceptions no longer terminate applications (it is still recommended to observe and handle exceptions though). The behavior in C++ async with Casablanca is more in line with the .NET Framework 4 – any unobserved exception will eventually lead to a crash).
You might think that the way to handle this exception is just to wrap this call in a try…catch block. This would work if you blocked (e.g. used .get()) since you would be executing the code synchronously. However if you start a task, it will run on a different thread and the exception will be thrown on this new thread so, not where you are trying to catch it. As a result your app would still crash. The idea is that you have to observe exceptions not where tasks are started but where they are completed (i.e. where you call .get() or .wait()). Using a continuation for exception handling seems like a great choice because continuations run only after the previous task has completed. So, let’s build on the previous code snippet and add a continuation that handles the exception. It would look like this (I am still using .get() at the very end but it is only to prevent the main thread from exiting and terminating the other thread):

pplx::task_from_result()
    .then([]()
    { 
        throw std::exception("test exception"); }
    )
    .then([](pplx::task<void> previous_task)
    {
        try
        {
            previous_task.get();
        }
        catch (const std::exception& e)
        {
            std::cout << e.what() << std::endl;
        }
    }).get();

One very important thing to notice is that the continuation I added takes pplx::task<void> as the parameter. This is a so called “task based continuation” and is different from continuations we have seen so far which took the value returned by the previous task (or did not take any parameter if the previous task was void). The continuations we had worked with before were “value based continuations” (we will come back to value based continuations in the context of exception handling shortly). With task based continuations you don’t receive the result from the previous task but the task itself. Now you are in the business of retrieving the result yielded by this task. As we know from the previous post the way to get the result returned by a task is to call .get() or .wait(). Since exceptions are in a sense also results of executing a task if the task threw calling .get()/.wati() will result in rethrowing this exception. We can then catch it and handle and thus make the exception “observed” so the process will no longer crash. When I first came across this pattern it puzzled me a bit. I thought ‘.get() is blocking and I use async to avoid blocking so isn’t it a contradiction?’. But then I realized that we are already in a continuation so the task has already been completed and .get() is no longer blocking – it merely allows to get the result of the previous task (be it a value or an exception).
Coming back to value based continuations – let’s see what would happen if we added a value based continuation after the continuation that throws but before the continuation that handles this exception – just like this:

pplx::task_from_result()
    .then([]()
    { 
        throw std::exception("test exception"); }
    )
    .then([]()
    {
        std::cout << “calculating The Answer…” << std::endl;
        return 42;
    })
    .then([](pplx::task<int> previous_task)
    {
        try
        {
            std::cout << previous_task.get() << std::endl;
        }
        catch (const std::exception& e)
        {
            std::cout << e.what() << std::endl;
        }
    }).get();

(One thing to notice – since the continuation we inserted now returns int (or actually pplx::task<int> – there are some pretty clever C++ tricks used to allow returning just a value or (just throwing an exception) even though the .then() function ultimately returns a pplx::task<T> or pplx::task<void>) the task valued continuation now has to take a parameter of pplx::task<int> type instead of pplx::task<void> type). If you run the above code the result will be exactly as from the previous example. Why? When a task throws an exception all value based continuations are skipped until a task based continuation is encountered which will be invoked and will have a chance to handle the exception. This is a big and a very important difference between task based and value based continuations. This also makes a lot of sense – something bad happened and in value based continuations you have no way of knowing that it did or what it was since you have no access to the exception. There is also nothing to pass if the previous task would return something were there not for the exception. As a result executing value based continuations if nothing has happened would be plainly wrong.
If you have played a little bit with Casablanca or have seen some more advanced code that is using Casablanca you might have come across the pplx::task_from_exception() function. You might have been wondering why it is needed if you can just throw an exception. Typically tasks are executed on multiple threads and it is very common that an exception thrown in one thread is being observed on a different thread. As a result it is impossible to just unwind the stack when trying to find an exception handler. Rather, the exception has to be saved (which will make the task faulted) and then is re-thrown when the user calls .get() or .wait() to get the result. If you use the .then() function all this happens behind the scenes – you throw an exception from a continuation and the .then() function will catch it and turn into a faulted task which will be passed to the next available task based continuation. However consider the following function:

pplx::task<int> exception_test(bool should_throw)
{
    if (should_throw)
    {
        throw std::exception("bogus exception");
    }

    return pplx::task_from_result<int>(42);
}

If you pass true it will throw an exception, otherwise it will return a value. Note that I cannot just return 42; here because the return type of the function is pplx::task<int> and not int and there is no Casablanca magic involved which could turn my 42 into a task. Therefore I have to use pplx::task_from_result<int>() to return a completed task with the result. Now, let’s try to build a task based continuation that observes the exception we throw – something like this:

exception_test(true)
    .then([](pplx::task<int> previous_task)
    {
        try
        {
            previous_task.get();
        }
        catch (const std::exception& e)
        {
            std::cout << "exception: " << e.what() << std::endl;
        }
    }).get();

If you run this code it will crash. The reason is simple – we just synchronously throw from the exception_test function and no one is handling this exception. Note that we are not able to handle this exception in the continuation since it is never invoked – because there was no handler the exception crashed the application before it got to the .then(). To fix this the exception_test function needs to be modified as follows:

pplx::task<int> exception_test(bool should_throw)
{
    if (should_throw)
    {
        return pplx::task_from_exception<int>(std::exception("bogus exception"));
    }

    return pplx::task_from_result<int>(42);
}

Now instead of throwing an exception we return a faulted task. This task is then passed to our task based continuation which can now handle the exception.

That’s it for today. Next time we will look at cancellation.

Now that we know a bit about lambda functions in C++ we finally can take a look at C++ async programming with the C++ Rest SDK (a.k.a. cpprestsdk a.k.a. Casablanca). C++ Rest SDK is cross platform but thanks to the NuGet support for native packages using it with Visual Studio is extremely easy. If you would like to use it on non-Windows platforms in majority of cases you will have to compile the code yourself (note to Raspbery PI fans – I tried compiling it on my Raspberry Pi but unfortunately compilation failed due to a gcc internal compiler error). With Visual Studio you can create a new Win32 Console Application, right click on the project in the Solution Explorer and click the “Manage NuGet Packages” option. In the Manage NuGet Packages window type “cppresdk” in the in the search box and install the C++ REST SDK: This is it – you are now ready to write asynchronous programs in C++. If you prefer, you can also install the package from the Package Manager Console with the Install-Package cpprestsdk command. Once the C++ Rest SDK is installed we can start playing with it. Let’s start with something really simple like displaying a textual progress bar. We will create a task that will loop 10 times. In each iteration it will sleep for a short period of time and then write an asterisk to the console. To show that it works asynchronously we will have a similar loop in the main thread where we similarly will wait and write a different character to the console. I expect the characters to be mixed which would prove that the progress task is actually running on a different thread. Here is the code:

#include "stdafx.h"
#include "pplx\pplxtasks.h"
#include <iostream>

int main()
{
    pplx::task<void>([]()
    {
        for (int i = 0; i < 10; i++)
        {
            pplx::wait(200);
            std::cout << "*";
        };

        std::cout << std::endl << "Completed" << std::endl;
    });

    for (int i = 0; i < 10; i++)
    {
        pplx::wait(300);
        std::cout << "#";
    }

    std::cout << std::endl;

    return 0;
}

And here is the result I got:

*#**#*#**#*#**#*
Completed
####
Press any key to continue . . .

From the output it appears that everything went as planned. Let’s take a closer look at what is really happening in the program. pplx::task<void> creates a new task that is scheduled to be run on a different thread. Once the task is created the loop writing # characters is being executed. In the meantime the scheduled task is being picked up and executed on a different thread. Note that the loop in the main thread will take longer to execute that the loop in the task. What would happen if the main thread did not live long enough – e.g. we would not have the loop in the main thread that runs longer than the task? You can easily check this by decreasing the timeout but basically the main thread would exit and would terminate all other threads – including the one the task is running on so the task would be killed. You can, however, block a thread and wait until a task is completed by using the .get() or the .wait() method. In general you want to avoid blocking threads but sometimes it can be helpful. (Note that this is in contrast to the managed world (e.g. C#) where the expectation is that apps using async features are async inside out and blocking oftentimes leads to bad things like deadlocks. The built-in support for async like async/await keywords and exception handling in async methods help tremendously meet this expectation.) Here is a new version of the program from above which is now using the .get() method to block execution of the main funcion until the task completes:

int main()
{
    auto task = pplx::task<void>([]()
    {
        for (int i = 0; i < 10; i++)
        {
            pplx::wait(200);
            std::cout << "*";
        };

        std::cout << std::endl << "Completed" << std::endl;
    });

    std::cout << "waiting for task to complete" << std::endl;
    task.get();
    std::cout << "task completed" << std::endl;

    return 0;
}

The program should output this:

waiting for task to complete
**********
Completed
task completed
Press any key to continue . . .

The output shows that .get() did the job – once the .get() method was reached the main thread was blocked waiting for the task to complete and then when the task finished the main thread was unblocked. This is great but can we take it to the next level – for instance – can we return a value from the task? This is actually quite easy – to do that you just need to return the value from the task. In our case we can return how much time (in nanoseconds) our task took to execute. We will use types from the std::chrono namespace so don’t forget to add #include <chrono> – I am leaving includes out for brevity.

int main()
{
    auto task = pplx::task<std::chrono::nanoseconds>([]()
    {
        auto start = std::chrono::steady_clock::now();

        for (int i = 0; i < 10; i++)
        {
            pplx::wait(200);
            std::cout << "*";
        };
        
        std::cout << std::endl << "Completed" << std::endl;

        return std::chrono::steady_clock::now() - start;
    });

    std::cout << "waiting for task to complete" << std::endl;
    auto task_duration = task.get();
    std::cout << "task completed in " << task_duration.count() << " ns." << std::endl;

    return 0;
}

If you look close at the code you will notice that I modified the type of the task – instead of being pplx::task<void> it is now pplx::task<std::chrono::nanoseconds> – and modified the lambda body so that it now returns a value. As a result the .get() method is no longer void – it now returns a value of the std::chrono::nanoseconds type which is actually the value we returned from our task. For completness this is what I got on my screen when I ran this program:

waiting for task to complete
**********
Completed
task completed in 2003210000 ns.
Press any key to continue . . .

While being able to run a task asynchronously is powerful oftentimes you would want to run another task that runs after a task has completed and that uses the result from the previous task. Both tasks should run asynchronously and should not require blocking the main thread to pass the result from one task to the other task. For instance you are working with a service that returns a list of ids and names in one request but also can return details for a given id in a different request. If you want to get details for a given name you would need to first send a request to get ids for names and then send another request to get the details for the id. From the perspective of the main thread you just want to say: “give me details for this name (and I don’t care how you do it)”. This can be achieved with task chaining. You chain tasks using the .then() method. In the simplest form it just takes the value returned by the previous task as the parameter. For example, in our case, if we wanted to get the result in milliseconds and not nanoseconds we could write a continuation that does the conversion (yes, there is no real benefit of doing such a simple conversion in a continuation especially that it isn’t an asynchronous operation and can easily be done in the first continuation or in the main thread but imagine you need to connect to a service that does the conversion for you) like this:

int main()
{
    auto task = pplx::task<std::chrono::nanoseconds>([]()
    {
        auto start = std::chrono::steady_clock::now();

        for (int i = 0; i < 10; i++)
        {
            pplx::wait(200);
            std::cout << "*";
        };

        std::cout << std::endl << "Completed" << std::endl;

        return std::chrono::steady_clock::now() - start;
    })
    .then([](std::chrono::nanoseconds duration)
    {
        std::cout << "task duration in ns: " << duration.count() << std::endl;
        return duration.count() / 1000000;
    });

    std::cout << "waiting for task to complete" << std::endl;
    auto task_duration = task.get();
    std::cout << "task completed in " << task_duration << " ms." << std::endl;

    return 0;
}

Running the program results in the following output:

waiting for task to complete
**********
Completed
task duration in ns: 2004372000
task completed in 2004 ms.
Press any key to continue . . .

That’s pretty much it for today. Next time we will look at different types of continuations, exception handling and possibly cancellation.

Warning: Some examples in this post belong to the “don’t do this at home” category.

In the first part of C++ Async Development (not only for) for C# Developers we looked at lambda functions in C++. They are cool but as you probably can guess there are some “gotchas” you need to be aware of to avoid undefined behavior and crashes. Like many problems in C++ these are related to object lifetimes where a variable is used after it has already been destroyed. For instance what do you think will be printed (if anything) if you run the following code:

std::function<int()> get_func()
{
    auto x = 42;

    return [&x]() { return x; };
}

int main()
{
    std::cout << get_func()() << std::endl;
    return 0;
}

?

On my box it prints -858993460 but it could write anything as well as could it crash. I believe a crash would have actually been better because I would not have carried on with a random result. The reason why it happened is simple. The variable x was captured by reference meaning the code inside lambda referred to the actual local variable. The lambda was returned from the get_func() function extending the lifetime of the closure. However the local variable x went out of scope when we left the get_func() so the reference captured in the closure no longer referenced x but rather it referenced a random location in memory that has already been reclaimed and potentially reused. A variation of this problem can also be encountered when working with classes. Take a look at this code:

class widget
{
public:
    std::function<int()> get_func()
    {
        return [this]() { return m_x; };
    }

private:
    int m_x = 42;
};

std::function<int()> get_func()
{
    return widget{}.get_func();
}

int main()
{
    std::cout << get_func()() << std::endl;
    return 0;
}

It basically has the same flaw as the previous example. As you probably remember from the post about lambdas the this pointer is always captured by value. So, the lambda captures a copy of the this pointer to be able to access class members but by the time the lambda is actually invoked the object is gone so the pointer is God knows what…
Now let’s try a little game. Find one difference between this class:

class widget
{
public:
    std::function<int()> get_func()
    {
        return [=]() { return m_x; };
    }

private:
    int m_x = 42;
};

and the class from the previous example. If you have not found it yet look at how we capture variables. Before we captured the this pointer explicitly but now we don’t do it. So, we should good, right? Not so fast. If you look at the lambda body you will see we are still accessing the m_x variable. This is a class variable and class variables cannot be accessed without having an instance of the class the variable is a member of. So, what happened is that when we captured variables implicitly the compiler captured the this pointer because we used a class variable in the lambda body. Since the this pointer is always captured by value the new class has the same problem as the previous one. To make things even more interesting the result would be exactly the same if we captured variables implicitly by reference (e.g. if we used [&] to capture variables). This is because the compiler notices that we use class variables in our lambda and therefore the this pointer needs to be captured. However the this pointers is always captured by value (notice that [&this] won’t compile) so even though we used [&] to request capturing variables by reference the this pointer was still captured by reference.
If you think that the scenarios shown above rarely happen in real life you might be right. However the “real life” is a bit different when doing things asynchronously. In this case lambdas represent tasks that are to be run sometime in the future. In general you have little or no control over how or when these tasks will be run (there are some advanced settings you can pass when scheduling a task which may give you some control – I have not investigated them yet). Basically, when a thread becomes available a scheduled task will be run on this thread. This means that the task (i.e. the lambda) will run completely independently on the code that scheduled it and there is no guarantees that the variables captured when creating a task will be around when the task actually runs. As a result you can encounter scenarios similar to the ones I described above quite often. So, what to do then? Over the past few months I developed a few rules and patterns which are helpful in these situations.

Do not capture implicitly
This is probably a little controversial but I like to capture variables explicitly by listing them in the capture list. This prevents from introducing a bug where I capture the this pointer implicitly and use it even though I don’t have a guarantee that it will be valid when the lambda is invoked. This rule has unfortunately one drawback – sometimes after I refactor the code I forget to remove from the capture list a variable that is no longer used in the lambda. In general, if you understand when not to use the this pointer in your lambda you should be fine capturing implicitly.

Try to capture variables by value if possible/feasible (if you can’t guarantee the lifetime)
If you look at the first case we capture the local variable x by reference but it is not necessary. Had we captured it by value we would have avoided all the problems. For instance we could do this:

std::function<int()> get_func()
{
    auto x = 42;

    return [x]() { return x; };
}

Sometimes you may want to capture by reference only to avoid copying. It is a valid reason and it is fine as long as you can guarantee that the variable captured in the closure outlives the closure itself. Otherwise it is better to be slower and correct instead of faster and return invalid results or crash.
The same applies to capturing class variables. If you can capture them by value – do it. In the second case we could change the code as follows:

class widget
{
public:
    std::function<int()> get_func()
    {
        auto x = m_x;
        return [x]() { return x; };
    }

private:
    int m_x = 42;
};

Now instead of capturing the this pointer we copy the value of the class variable to a local variable and we capture it by value.
This works in simple cases where we don’t modify the variable value in the lambda and don’t expect the variable to be modified between it was captured and the lambda was invoked. Oftentimes this is however not enough – we want to be able to modify the value of the variable captured in the closure and be able to observe the new value. What do we do in these cases?

Use (smart)pointers
The problems outlined above were caused by automatic variables whose lifetimes are tied to their scope. Once they go out of scope they are destroyed and must no longer be used. However, if we create the variable on the heap we will control the lifetime of this variable. Since this is no longer 1998 we will not use raw pointers but smart pointers. Actually, there is even more important reason to not use raw pointers – since we allocated memory we should release it. In simple cases it may not be difficult. You just delete the variable in the lambda and you are done with it. More often than not it is not that simple – you may want to capture the same pointer in multiple closures, someone else may still want to use it after the lambda was invoked, you may want to invoke the lambda multiple times etc. In these cases the simplistic approach won’t work. Even if originally your code worked correctly it is very easy to inadvertently introduce issues like these later during maintenance. Using smart pointers makes life much easier and safer. So, yeah, we will use smart pointers. For local variables it is as easy as:

std::function<int()> get_func()
{
    auto x = std::make_shared<int>(42);

    return [x]() { return *x; };
}

Notice that we create a shared_ptr variable and capture it by value. When a shared_ptr is passed by value its internal ref count is incremented. This prevents from deleting the value the pointer points to. When the pointer goes out of scope the ref count is decremented. Once the ref count reaches 0 the pointer deletes the variable it tracks. In our case we create a closure (the ref count is incremented) and return it to the caller. The caller may extend its lifetime (e.g. by returning it to its caller) or can just invoke it and let it go out of scope. One way or another the closure will eventually go out of scope and when this happens the ref count will be decremented and – if no one else is using the shared_ptr – the memory will be released. One thing worth noting is how using shared_ptrs plays nicely with the previous rule about capturing variables by value.
We can also use shared_ptrs to capture the current instance of a class as a substitute for capturing the this pointer. It’s however a bit more involved than it was in case of variables. First, we cannot just do shared_ptr(this). Doing this could lead to creating multiple shared_ptr instances for a single object (e.g. when a function creating a shared_ptr this way would be called multiple times) which would lead to invoking the destructor of object tracked by the shared_ptr multiple times which not only creates undefined behavior but can also lead to dangling pointers. (Note that this is not specific to the this pointer but applies to any raw pointer). The way around it is to derive the class from the enable_shared_from_this<T> type. Unfortunately classes derived from enable_shared_from_this<T> are not very safe and using them incorrectly may put you into the realm of undefined behavior very quickly. The biggest issue is that to be able to create a shared_ptr from the this pointer for a class derived from enable_shared_from_this<T> the instance must already be owned by a shared_ptr. The easiest way to enforce this is probably by not allowing the user to create instances of the class directly by making all the constructors (including the ones created by the compiler) private and exposing a static factory method that creates a new instance of the class and returns a shared_ptr owning the newly created instance. This is how code capturing a shared_ptr tracking this would look like:

class widget : public std::enable_shared_from_this<widget>
{
public:

    static std::shared_ptr<widget> create()
    {
        return std::shared_ptr<widget>(new widget);
    }

    std::function<int()> get_func()
    {
        auto w = shared_from_this();

        return [w]() { return w->m_x; };
    }

private:

    widget() 
    { }

    widget(const widget&) = delete;

    int m_x = 42;
};

std::function<int()> get_func()
{
    return widget::create()->get_func();
}

int main()
{
    std::cout << get_func()() << std::endl;
    return 0;
}

A word of precaution. One general thing you need to aware of when using shared_ptrs is circular dependencies. If you create a circular dependency between two (or more) shared_ptrs their ref counts will never reach 0 because each shared_ptr is always being referenced by another shared_ptr. As a result the instances the shared_ptrs point to will never be released causing a memory leak. If you cannot avoid a dependency like this you can replace one of the shared_ptrs with a weak_ptr which will break the cycle. weak_ptrs similarly to shared_ptrs can be passed by value so they are easy to use with lambda functions. If you have a weak_ptr and you need a shared_ptr you just call the weak_ptr::lock() function. Note that you need to check the value returned by the weak_ptr::lock() function since it will return an empty shared_ptr if the instance managed by shared_ptr was deleted because all the strong references were removed in the meantime.

This post was quite long and heavy but knowing this stuff will be very helpful later when we dive in the actual async programming which I think we will start looking into in the next part.