C++20 - Coroutines

Coroutines, introduced in C++20, bring a powerful mechanism for handling asynchronous operations, generators and cooperative multitasking. Unlike conventional functions, which execute sequentially from start to return, coroutines are functions that can suspend and resume execution at specific points, effectively allowing you to pause execution, yield results, and return control to the caller without fully exiting the function.

1. Introduction

At their core, coroutines empower cooperative multitasking by enabling a function to be suspended and later resumed without losing its state. This is in contrast to preemptive multitasking—such as that implemented with threads—where the operating system arbitrarily decides when to suspend a task and switch context. Coroutines, by contrast, give the programmer explicit control over when and where to suspend or resume execution, making them a lighter and more efficient alternative for certain asynchronous programming patterns.

When a coroutine is called, it doesn’t run to completion immediately. Instead, it returns a special object, typically referred to as a coroutine handle, which the caller can use to resume or finalize the coroutine’s execution. This behavior introduces a level of flexibility and control that traditional functions simply do not possess.

Coroutines can be applied across a wide range of use cases, including:

• Asynchronous I/O:

  Efficiently suspending operations until data becomes available from a network or file system, avoiding costly thread blocking.

• Event-driven systems:

  Enabling highly responsive applications by reacting to incoming events without halting the system, while simultaneously maintaining clarity in control flow.

• Lazy data generation:

  Producing data incrementally, without needing to allocate or compute the entire sequence at once, thus optimizing memory and computation resources.

2. Objective

In this article, we will implement a simple generator-like coroutine that produces a sequence of integers, yielding values incrementally as they are requested. Consider the following example:

int main() {

    // Create the coroutine to count from 1 to 5
    Generator counter = simple_counter(1, 5);  

    // Iterate through the yielded values
    while (counter.next()) {

        // Output each yielded value
        std::cout << counter.get() << " ";  
    }

    std::cout << std::endl;
    return 0;
}

The expected output for this program would be:

1 2 3 4 5

The primary coroutine function we aim to implement is a generator that yields consecutive integers between a given start and end value:

// Coroutine function that yields consecutive numbers
Generator simple_counter(int start, int end) {
    for (int i = start; i <= end; ++i) {
        // yield the current value
        co_yield i;
    }
}

In the code above, co_yield is the key operator introduced by C++20 coroutines. When encountered, it pauses the execution of the coroutine and returns a value to the caller. The coroutine can then be resumed from this point, maintaining the state of local variables.

Let’s break down the essential components involved in making this coroutine functional and efficient in C++20.

3. Formal Definition

In C++20, coroutines are implemented under the hood as a finite state machine (FSM)—a widely-recognized computational model where the system resides in one of a finite number of states at any given time and transitions between states occur based on events. For coroutines, these transitions involve saving the current state of execution and potentially performing actions such as yielding a value or suspending execution. This FSM model is particularly well-suited for coroutines, as they are designed to pause, yield results, and resume at precise points during their execution.

A coroutine’s state persists across suspension points, enabling the function to resume from exactly where it was last paused upon the next invocation. Control over these state transitions in C++20 coroutines is achieved through three key operations:

• co_yield:

  The co_yield operator suspends the coroutine while yielding a value back to the caller. It also allows a coroutine to generate a sequence of values over multiple suspensions, maintaining its state between yields. This makes it highly suitable for generator-style coroutines, where values are produced incrementally.

• co_return:

  The co_return statement signals the completion of the coroutine, returning a final value. Once a coroutine encounters co_return, it cannot be resumed, and the coroutine’s lifetime ends.

• co_await:

  The co_await operator suspends the coroutine, handing control back to the caller, until a specific condition—typically represented as an asynchronous event—is met. When the awaited condition is fulfilled, the coroutine resumes execution at the point where it was suspended. The type of object that is awaited must conform to the Awaitable concept, which means it must provide the following methods:

  1. await_ready(): Determines if the operation is ready to proceed or if it needs to suspend.
  2. await_suspend(): Suspends the coroutine and potentially schedules it to be resumed later.

• await_resume(): Returns the result of the awaited operation when the coroutine is resumed.

3.1. State Transitions in Coroutines

The execution of a coroutine is driven by state transitions, which are managed by the co_await, co_yield, and co_return operations. These transitions occur between the following states:

1. Initial State: When a coroutine is first called, it enters the initial state. At this point, the function promise_type::initial_suspend() decides whether the coroutine should immediately suspend or continue execution. This gives the developer control over whether the coroutine begins running immediately or waits to be explicitly resumed.

2. Running State:
   Once execution begins, the coroutine proceeds normally until it encounters a suspension point, such as a co_await or co_yield. Upon reaching these points, it returns control to the caller and moves into the suspended state.

3. Suspended State:
   While suspended, the coroutine’s execution context—including its local variables and call stack—is preserved. At any point, the coroutine can be resumed, at which point it continues execution from the last suspension point, progressing toward its next suspension or completion.

4. Final State:
   After the coroutine encounters a co_return, it enters the final state, indicating the completion of its execution. The function promise_type::final_suspend() ensures that no further actions can be taken on the coroutine once it has reached this point, allowing for proper cleanup and termination.

These operations and state transitions make it easier to manage asynchronous workflows, allowing for clear and efficient state management while maintaining a straightforward control flow. By combining these with custom promise_type objects and awaitable objects, C++20 coroutines offer a powerful abstraction for managing both synchronous and asynchronous tasks.

3.2. Lifecycle of a Coroutine

The lifecycle of a coroutine is intricately managed by these state transitions. The following diagram captures the transitions and operations:

stateDiagram-v2
    %% Starting point
    [*] --> InitialState
    InitialState: Initial State

    %% Transition to Running or Suspended State based on initial_suspend()
    InitialState --> RunningState: coroutine called
    InitialState --> SuspendedState: initial_suspend()

    %% Running State
    RunningState: Running State
    RunningState --> SuspendedState: co_await / co_yield
    RunningState --> FinalState: co_return

    %% Suspended State
    SuspendedState: Suspended State
    SuspendedState --> RunningState: resume()
    SuspendedState --> FinalState: co_return / Scope End

    %% Final State
    FinalState: Final State
    FinalState: final_suspend()

By explicitly controlling state transitions, C++20 coroutines provide a structured, efficient mechanism for managing asynchronous workflows. Unlike traditional multithreading, where the operating system handles preemptive task switching, coroutines give the programmer direct control over suspensions and resumptions, leading to a more predictable and performant system. Moreover, the use of promise_type and awaitable objects allows developers to customize the behavior of coroutines, providing fine-grained control over both synchronous and asynchronous tasks.

4. Implementing the Coroutine

In order to employ the co_yield operator within a function, the return type must conform to the coroutine concept as defined by the C++20 standard. When the compiler detects the presence of keywords like co_yield, co_await, or co_return, it automatically flags the function as a coroutine, which fundamentally changes its behavior.

Instead of following the conventional function model of entering, executing, and exiting, coroutines return an intermediate object (e.g. Generator) that encapsulates their state and enables suspension and resumption of execution. This returned object serves as the interface through which the coroutine interacts with the calling environment.

In our example, the coroutine will return an object of type Generator, which will function as a generator of integers. To fulfill the coroutine’s requirements and the compiler’s expectations, we need to define a corresponding generator class, Generator, that adheres to the coroutine concept. At the core of this concept lies the promise_type, a structure that mediates between the caller and the coroutine, orchestrating the coroutine’s lifecycle, including suspension, resumption, and finalization.

Below is the skeletal structure of our generator Generator, which we will expand to implement the necessary coroutine mechanisms:

class Generator {
    // Required by the compiler to control the coroutine
    struct promise_type;  

    // constructor
    Generator(std::coroutine_handle<promise_type> handle);   

private: 
    // Handle to control coroutine execution
    std::coroutine_handle<promise_type> coroutine_handle_; 
}

4.1. The Heart of Coroutines: promise_type

The promise_type structure is the core component in C++20 coroutines, responsible for managing the coroutine’s internal state and interacting with the calling code. Whenever a coroutine is invoked, the compiler generates an instance of the promise_type. The promise_type interface defines how the coroutine behaves at each phase of its execution, including initial suspension, resumption, final suspension, and error handling.

The promise_type must provide the following methods to manage coroutine state transitions:

struct promise_type {
    // Returns the object that the coroutine will hand back to the caller
    Generator get_return_object();

    // Suspend execution immediately after starting
    std::suspend_always initial_suspend();

    // Suspend execution after the coroutine is done
    std::suspend_always final_suspend() noexcept;

    // Handle any exceptions that occur within the coroutine
    void unhandled_exception();
};

Key methods that must be implemented in promise_type include:

• 1. Generator get_return_object():
     This method returns the object that will represent the coroutine’s execution state to the caller. It is called as soon as the coroutine is instantiated. In the case of a generator, it returns a handle to the coroutine itself, allowing the caller to resume or query the coroutine’s state.

  The typical implementation for this method is as follows:

    Generator Generator::promise_type::get_return_object()  
    {
        // The coroutine_handle manages the lifetime and resumption of the coroutine
        return Generator{std::coroutine_handle<promise_type>::from_promise(*this)};  
    }
  

  Here, the from_promise() function creates a coroutine handle that ties the promise_type to the coroutine object, providing the caller with full control over the coroutine’s lifecycle.

• std::suspend_always initial_suspend():
  This method determines whether the coroutine should suspend execution immediately upon creation. The return type must satisfy the awaitable concept, which determines how the coroutine behaves at suspension points. Returning std::suspend_always ensures that the coroutine will suspend after being invoked, giving the caller control over when to resume execution.

  Example implementation:

    std::suspend_always Generator::promise_type::initial_suspend()  
    {  
        // The coroutine starts in a suspended state and must be resumed explicitly
        return {};  
    }
  

  In this case, we return std::suspend_always, indicating that the coroutine suspends right after its invocation, awaiting an explicit resume action.

• std::suspend_always final_suspend():
  This method is called when the coroutine reaches its end, either through a co_return or because the function runs to completion. It determines whether the coroutine should suspend before final cleanup or exit immediately. The typical pattern is to use std::suspend_always to ensure that the caller has an opportunity to finalize any resources tied to the coroutine handle.

  Example:

    std::suspend_always Generator::promise_type::final_suspend()  
    {  
        // After the coroutine is done, it suspends one last time before being destroyed
        return {};  
    }
  

  This final suspension is a key point in the coroutine’s lifecycle, giving the caller a chance to perform any necessary cleanup or synchronizing actions.

• void unhandled_exception()

  This method defines the coroutine’s behavior when an exception is thrown inside the coroutine body and not explicitly handled. The most common implementation involves terminating the program, but more advanced handlers could log the error or propagate the exception to the caller.

  Example implementation:

    void Generator::promise_type::unhandled_exception()  
    {  
        // Terminate the program or just log an error and freeze the coroutine
        std::terminate();  
    }
  

  In this basic implementation, std::terminate() is called to halt the program when an unhandled exception occurs within the coroutine. More advanced coroutine systems may instead propagate exceptions back to the caller.

4.2. The Awaitable Concept

The awaitable concept in C++ defines a set of rules for objects that can be used in conjunction with co_await. These objects determine whether a coroutine should suspend, what happens when it suspends, and how the coroutine resumes. In the context of coroutines, both std::suspend_always and std::suspend_never are common awaitables that provide predefined behavior for suspension.

• std::suspend_always:
  This is an awaitable that always suspends the coroutine. It is used when the coroutine should explicitly suspend, allowing the caller to determine when to resume it.

    struct std::suspend_always {
        bool await_ready() const noexcept { return false; }
        void await_suspend(std::coroutine_handle<>) const noexcept {}
        void await_resume() const noexcept {}
    };
  

  • await_ready(): Returns false, indicating that the coroutine should suspend.
  • await_suspend(): Performs the actual suspension.
  • await_resume(): Defines what happens when the coroutine resumes.

• std::suspend_never:
  This awaitable ensures that the coroutine never suspends at a particular suspension point, allowing it to continue execution without interruption.

    struct std::suspend_never {
        bool await_ready() const noexcept { return true; }
        void await_suspend(std::coroutine_handle<>) const noexcept {}
        void await_resume() const noexcept {}
    };
  

  • await_ready(): Returns true, meaning the coroutine does not suspend.
  • await_suspend() and await_resume() are no-ops, as no suspension occurs.

4.3. Coroutine Operators

In C++20, coroutines utilize specialized operators (co_await, co_yield, and co_return) to manage control flow, yielding values, awaiting asynchronous events, or signaling coroutine completion. For a coroutine to function correctly, it must overload and implement at least one of these operators. The choice of the operator depends on the desired behavior—whether the coroutine should suspend while awaiting, yield intermediate results, or complete execution and return a value.

4.3.1. The Operator co_yield

The co_yield operator facilitates the suspension of the coroutine, returning control to the caller while also passing back a value. To overload this operator, the promise_type structure must implement at least one of two forms of yield_value: awaitable yield_value(T) or awaitable yield_void().

struct promise_type {
    Generator get_return_object();
    std::suspend_always initial_suspend();
    std::suspend_always final_suspend() noexcept;
    void unhandled_exception();

    // Overload for yielding an integer value (e.g., `co_yield 5;`)
    std::suspend_always yield_value(int val);

    // Overload for yielding a string value (e.g., `co_yield "Hello";`)
    std::suspend_always yield_value(std::string val);

    // Yielding control without passing a value (e.g., `co_yield;`)
    std::suspend_always yield_void();
};

In the context of a generator design like Generator, the yield_value function is used to store or update the coroutine’s internal state (current_value), which the user can later retrieve. However, coroutines are much more powerful and can solve more complex control flow problems than simple generators.

• awaitable yield_value(T):

  This method is invoked when co_yield is used in the coroutine. It suspends the coroutine and yields the value T to the caller. At this suspension point, the state of the coroutine is preserved, allowing it to resume at a later time. The caller can retrieve the yielded value after resumption.

  Example implementation:

    std::suspend_always promise_type::yield_value(int value) {
        value_ = value;  // Store the yielded value in the internal state
        return {};       // Suspend the coroutine
    }
  

• awaitable yield_void():

  This method is called when the coroutine is suspended without yielding a specific value to the caller. It can be used when the coroutine needs to pause execution but doesn’t produce an output. This operator is less common in generator-style coroutines but can be useful in control flow scenarios.

  Example:

    std::suspend_always promise_type::yield_void() {
        // Suspend the coroutine without yielding a value
        return {};
    }
  

Hence for our Generator example, we could define

class Generator  
{  
 public:
    struct promise_type  
    { 
     public:  
        Generator get_return_object();  
        std::suspend_always initial_suspend();  
        std::suspend_always final_suspend() noexcept;   
        void unhandled_exception();  
    
        // Overload co_yield, 
        std::suspend_always yield_value(int val);
     public:
        friend class Generator;
        int value_{ 0 };  
    };

    // type alias to reduce code
    using handle_type = std::coroutine_handle<promise_type>;  

private:
    handle_type coroutine_handle_;
}

4.3.2. The Operator co_return

The co_return operator signals the end of a coroutine and allows the coroutine to return a value to the caller. The promise_type must implement return_value(T) or return_void() to overload this operator.

Example:

struct promise_type {
    Generator get_return_object();
    std::suspend_always initial_suspend();
    std::suspend_always final_suspend() noexcept;
    void unhandled_exception();

    // Return an integer value upon coroutine completion (e.g., `co_return 42;`)
    std::suspend_always return_value(int val);

    // Return a string upon coroutine completion (e.g., `co_return "Done";`)
    std::suspend_always return_value(std::string val);

    // Signal coroutine completion without returning a value (e.g., `co_return;`)
    std::suspend_always return_void();
};

In the Generator example, we could overload the return_value and return_void operators to handle final results or simply to indicate that the coroutine has completed its execution.

• awaitable return_value(T):

  This method is called when co_return is used in the coroutine, allowing the coroutine to return a final value to the caller. It suspends the coroutine, signals completion, and passes the return value back to the caller.

    std::suspend_always Generator::promise_type::return_value(int val) {
        value_ = val;  // Store the final returned value
        return {};     // Suspend the coroutine before finalization
    }
  

  In this example, the return_value method stores the value passed to co_return in the value_ field of the promise_type. The coroutine is then suspended one last time, allowing the caller to retrieve the value before the coroutine is destroyed.

  Usage in a coroutine:

    Generator simple_counter(int start, int end) {
        for (int i = start; i <= end; ++i) {
            co_yield i;
        }
        // Return a final result after the loop completes
        co_return end + 1;  
    }
  

• awaitable return_void():

  This method is invoked when co_return is used without a value, simply marking the coroutine as complete and suspending it for final cleanup. This is useful when the coroutine’s task is done, and no final value needs to be returned. After calling return_void(), the coroutine transitions to its final suspended state and is then cleaned up.

    std::suspend_always promise_type::yield_void() {
        // Suspend the coroutine without yielding a value
        return {};
    }
  

  In this case, the coroutine finishes without yielding or returning any specific value. The final suspension allows the coroutine’s resources to be cleaned up in an orderly fashion.

  Usage in a coroutine:

    Generator simple_counter(int start, int end) {
        for (int i = start; i <= end; ++i) {
            co_yield i;
        }
        // End the coroutine without returning a value
        co_return;
    }
  

4.4. Managing the Coroutine Handle

In the above implementation, we leverage std::coroutine_handle to manage the lifecycle of the coroutine. This handle acts as a lightweight wrapper that directly interfaces with the coroutine, allowing control over its execution. The handle is essential for interacting with the coroutine at various stages—suspension, resumption, finalization, and cleanup.

Here is a more detailed explanation of the key functions of std::coroutine_handle utilized in our Generator example:

1. void destroy():
   The destroy() method is used to explicitly destroy the coroutine. It ensures that once the coroutine has completed its execution, its resources are properly cleaned up, preventing memory leaks. In our Generator class, we call destroy() within the destructor, ensuring that if the coroutine is still alive, it gets terminated and cleaned up before the Generator object goes out of scope.

   Example usage in the destructor:

    Generator::~Generator() {
        // Clean up coroutine resources
        if (coroutine_handle_) coroutine_handle_.destroy();  
    }
   

2. void resume():
   The resume() method allows the coroutine to proceed from its last suspension point. This is used within the next() function, which resumes the coroutine and attempts to move it to the next co_yield or co_return. By invoking resume(), the coroutine’s execution is continued until it either reaches another suspension point or completes execution.

3. bool done():
   The done() method checks whether the coroutine has reached its final state and is no longer resumable. It returns true if the coroutine has completed execution. This is important for ensuring that the coroutine is not resumed after it has finished. In the next() function, done() is checked both before and after calling resume() to ensure that the coroutine is still active.

4. promise_type& promise():
   The promise() method provides access to the promise_type instance associated with the coroutine. This is crucial because the promise_type holds the internal state of the coroutine, including the value that was last yielded or returned. The promise() method is used in the get() function to retrieve the current value yielded by the coroutine.

4.5. Retrieving Values in the Generator

To allow interaction between the Generator generator and the caller, we need two essential methods:

• bool next(): This method continues the execution of the coroutine until it suspends again, either due to a co_yield or co_return. It checks if the coroutine is done, resumes execution, and returns whether the coroutine has yielded a new value. If the coroutine has completed execution, it returns false.

  Full implementation:

    bool Generator::next() const  
    {  
       if (!coroutine_handle_ || coroutine_handle_.done()) return false;  
  
       // Resume the coroutine  
       coroutine_handle_.resume();  
  
       // Check again if it's done after resuming  
       return !coroutine_handle_.done();
    }
  

  Before resuming the coroutine, we first ensure that the coroutine handle is properly initialized and that the coroutine has not already completed its execution. This is accomplished by checking if the handle is valid and if the coroutine is not in the done state.

  Once confirmed, we can safely invoke the resume() method to move the coroutine forward to its next suspension point. The use of co_yield within the coroutine may return a value and suspend execution at multiple points before the function fully completes. After calling resume(), it is essential to check again whether the coroutine has reached its final state by invoking done().

  If the resumption of the coroutine leads to its termination without yielding any additional values, we return false, indicating that the coroutine has completed its task. This prevents further attempts to resume a completed coroutine.

  However, if we employ co_return at the end of the coroutine, and it results in a final value being returned, we would return true, as this indicates that the coroutine has successfully yielded a value before reaching its conclusion. In this case, the caller can retrieve the final value using the get() method.

• int get(): This method retrieves the current value yielded by the coroutine. It accesses the promise_type through the coroutine handle to fetch the stored value. This method should be called after next() returns true, indicating that the coroutine has successfully yielded a new value. Full implementation:

    int Generator::get() const  
    {  
        return coroutine_handle_.promise().value_;  
    }
  

  Here, the method accesses the promise_type instance via the coroutine_handle_, fetching the value_ field, which stores the value yielded during the last co_yield or co_return.