Nested IEnumerables

I’ve been exploring the subject of coroutines recently and I’ll be writing more about this in a separate post in the near future. Designing an implementation of coroutines for C# requires taking full advantage of C#’s support for yield return and IEnumerable<T>.

A spinoff from this exploratory work has been a practical mechanism for supporting yield return for both sequence values and complete sequences – a capability sadly absent from the C# language.

Conceptually here’s some pseudo-code that conveys this idea:

        public IEnumerable<string> FirstSequence()
        {
            yield return "1";
            yield return "2";
            yield sequence SecondSequence();
            yield return "7";
            yield return "8";
        }

        public IEnumerable<string> SecondSequence()
        {
            yield return "3";
            yield return "4";
            yield return "5";
        }

The yield sequence keywords are of course fictitious but convey the requirement nicely – namely that when enumerating values from FirstSequence() the value present within SecondSequence() are automatically enumerated and returned – as if they’d been yielded directly from within FirstSequence().

The current C# language (Version 5) does not permit such constructs and one must code the following in order to get the desired effect:

        public IEnumerable<string> FirstSequence()
        {
            yield return "1";
            yield return "2";
            foreach (string V in SecondSequence())
               yield return V;
            yield return "7";
            yield return "8";
        }

        public IEnumerable<string> SecondSequence()
        {
            yield return "3";
            yield return "4";
            yield return "5";
        }

It seems – to me at least – that the implementation of yield return is unduly limited, mainly because there seems to be no significant reason why the C# compiler cannot transform: yield sequence S; into: foreach (var V in S) yield return V; since the latter is fully supported and provides the desired semantics and the transformation seems straightforward.

We can overcome this limitation and approach the elegance and simplicity of our imagined yield sequence by adopting a similar design to that adopted for implementing coroutines which I’ll discuss in a future post. Namely we create an object which manages the enumeration for us – a sort of enumeration proxy – this iterator object can then provide the processing required to make everything work. We can’t transform the code into another form (as the C# compiler does when it encounters the yield keyword) but we can invisibly enumerate embedded sequences by creating a stack of enumerators enabling us to suspend enumeration of one sequence and begin enumeration of the embedded sequence.

Once all elements have been enumerated from the embedded sequence we can pop the stack and resume enumeration using the previous enumerator thus continuing with the original sequence, this technique will be explored along with some real code in a future post.