A Coroutine Library in C#

In this post I’m going to introduce an implementation of coroutines written in C#, the code I’m divulging is the result of my initial foraging into this unfamiliar (to me) concept so please bear this in mind if it appears a little preliminary.

The coroutine library provides a mechanism for two or more methods to transfer control to one another in such a way that when a method resumes execution, it does so at the position from which it previously passed control to another method.

The C# language includes support for writing iterator methods which use the yield return operation to return control to the caller in such a way that when re-invoked at a later time, execution resumes at the statement following the yield return. This mechanism is the basis for my implementation of coroutines.

The approach I’ve used is to define an abstract base class which provides a means to call user written iterator methods (henceforth referred to as coroutines) so that they can return the necessary information to enable another iterator method to be invoked. The base class thus does the invocation of the coroutines on our behalf and  masks some involved “plumbing” code which is necessary to make the coroutine mechanism easy to use.

Because coroutines are a set of mutually cooperating methods it is natural to design the coroutine library in such a way that we define the coroutines as methods which are all members of  a class. This class derives from an underlying base class that does the housekeeping necessary for tracking each coroutine’s state. The abstract base class is named Cooperative and all the user need do is derive a class from this and in that derived class implement a set of coroutines.

Here is an example of simple class that leverage Cooperative and defines two coroutines, this will help you see how coroutines actually look before we explore how the underlying implementation is coded:

public class KeyboardCoroutines : Cooperative
{
    private Queue<ConsoleKeyInfo> key_queue = new Queue<ConsoleKeyInfo>();

    public override void BeginProcessing(object Arg)
    {
        StartByActivating(ProduceFromKeyboard,Arg);
    }

    private IEnumerator<Activation> ProduceFromKeyboard(object Arg)
    {
        ConsoleKeyInfo info = Console.ReadKey(true);

        while (info.Key != ConsoleKey.Escape)
        {
            while (key_queue.Count < 10 && info.Key != ConsoleKey.Escape)
            {
                key_queue.Enqueue(info);
                info = Console.ReadKey(true);
            }
            
            if (info.Key == ConsoleKey.Escape)
                yield return Activate(ConsumeFromQueue,1);
            else
            {
                yield return Activate(ConsumeFromQueue, 2);
                key_queue.Enqueue(info);
            }

            Debug.WriteLine("ProduceFromKeyboard sees a result of: " + Result.ToString());
        }
    }

    private IEnumerator<Activation> ConsumeFromQueue(object Arg)
    {
        ConsoleKeyInfo key = key_queue.Dequeue();

        while (key.Key != ConsoleKey.Escape)
        {
            while (key_queue.Count > 0 && key.Key != ConsoleKey.Escape)
            {
                Console.Write(key.KeyChar);
                key = key_queue.Dequeue();
            }

            if (key.Key == ConsoleKey.Escape)
                yield return Activate(ProduceFromKeyboard,3);
            else
            {
                Console.Write(key.KeyChar);
                yield return Activate(ProduceFromKeyboard,4);
            }
            Debug.WriteLine("ConsumeFromQueue sees a result of: " + Result.ToString());
        }
    }
}

You’ll notice right away that a coroutine has a return type of IEnumerable<Activation> and a coroutine passes control to some other coroutine by executing a yield return expression which is a function call to Activate in the base class. The base class therefore enumerates a coroutine and uses the value returned by each iteration to select and call the enumerator associated with the next coroutine to execute. Each coroutine’s execution is temporarily suspended at the point it yields and resumes at the next statement when some other coroutine passes control back to it.

In my next post on this subject I’ll show you the base class implementation and explore some alternative ways to expose this coroutine mechanism.

 

 

 

 

Nested IEnumerables

I’ve been exploring the subject of coroutines recently and I’ll be writing more about this in a separate post in the near future. Designing an implementation of coroutines for C# requires taking full advantage of C#’s support for yield return and IEnumerable<T>.

A spinoff from this exploratory work has been a practical mechanism for supporting yield return for both sequence values and complete sequences – a capability sadly absent from the C# language.

Conceptually here’s some pseudo-code that conveys this idea:

        public IEnumerable<string> FirstSequence()
        {
            yield return "1";
            yield return "2";
            yield sequence SecondSequence();
            yield return "7";
            yield return "8";
        }

        public IEnumerable<string> SecondSequence()
        {
            yield return "3";
            yield return "4";
            yield return "5";
        }

The yield sequence keywords are of course fictitious but convey the requirement nicely – namely that when enumerating values from FirstSequence() the value present within SecondSequence() are automatically enumerated and returned – as if they’d been yielded directly from within FirstSequence().

The current C# language (Version 5) does not permit such constructs and one must code the following in order to get the desired effect:

        public IEnumerable<string> FirstSequence()
        {
            yield return "1";
            yield return "2";
            foreach (string V in SecondSequence())
               yield return V;
            yield return "7";
            yield return "8";
        }

        public IEnumerable<string> SecondSequence()
        {
            yield return "3";
            yield return "4";
            yield return "5";
        }

It seems – to me at least – that the implementation of yield return is unduly limited, mainly because there seems to be no significant reason why the C# compiler cannot transform: yield sequence S; into: foreach (var V in S) yield return V; since the latter is fully supported and provides the desired semantics and the transformation seems straightforward.

We can overcome this limitation and approach the elegance and simplicity of our imagined yield sequence by adopting a similar design to that adopted for implementing coroutines which I’ll discuss in a future post. Namely we create an object which manages the enumeration for us – a sort of enumeration proxy – this iterator object can then provide the processing required to make everything work. We can’t transform the code into another form (as the C# compiler does when it encounters the yield keyword) but we can invisibly enumerate embedded sequences by creating a stack of enumerators enabling us to suspend enumeration of one sequence and begin enumeration of the embedded sequence.

Once all elements have been enumerated from the embedded sequence we can pop the stack and resume enumeration using the previous enumerator thus continuing with the original sequence, this technique will be explored along with some real code in a future post.

 

A Superlean Inter-Thread Queue

Download the complete sample solution – scroll to bottom of post.

There are times when a design calls for an ability to pass information from one thread to another within an application. The Actor design pattern hinges upon such a capability as do other bespoke architectures in which dedicated threads play a central role. In an asynchronous design information passes between threads by queuing requests to a thread pool, the operating system internally schedules the processing of queued work by selecting some arbitrary thread within the thread pool and causing that thread to invoke a callback that you supply directly or indirectly like when using async/await.

Continue reading

Leveraging TaskCompletionSource

The TaskCompletionSource class is the means by which we can create and “manage” a Task ourselves.  By “manage” I mean implement the underlying mechanism that embodies the progression of an asynchronous activity including its completion or possible termination due to exceptions.

Continue reading

Pointers in C#

C# allows you to declare raw pointers to memory, such pointers can only be declared and manipulated if the project in which they appear has the “Allow unsafe code” option enabled and the code block has the “unsafe” keyword specified.

Continue reading