Unfortunately futures aren’t included in the Parallel Pattern Library or the Asynchronous Agents Library in the Visual Studio 2010 CTP. But I’d like to show a couple of ways to build a simple future class with the CTP.

A synopsis for a simple future

Here’s a simple template class synopsis for a future that allows you to construct a future with a functor and then retrieve it’s value asynchronously:

   template
   <class T>
   class future{
   public:
       template
       <class Functor>
       future(Functor fn);
       ~future();
       T GetValue();
   };

Here’s a simple example of creating a future with a lambda and retrieving its value:

   future<int> f([](){
       return 2;
   });
   int result = f.GetValue();
   assert(result == 2);

The scenario above isn’t super exciting, and it isn’t really asynchronous but it’s functional and in practice a lot of concurrency can be realized by chaining and combining futures together into expressions and waiting on the final result.

Implementing it step-by-step

Let’s take a look at one possible way of implementing this template class.  There are in fact other strategies which can be more efficient if you’re building lots of very fine grained futures, but this is very easy to write and in practice for relatively large expressions is probably quite fine.  I’ll mention the overheads and tradeoffs as I go along.

The first thing we’ll need in our future class is a place to store that value and a way to compute it asynchronously. From the CTP, I’m going to use both the task constructs in <ppl.h> and the message blocks in <agents.h>. To do this let’s create a couple private member variables starting with an overwrite_buffer:

   overwrite_buffer<T> value_;

The overwrite_buffer is a template class defined in agents.h, called a “message block” that allows a single value to be “sent” to it or stored, and copies of the most recent value are extracted upon request typically using a helper function called ‘”receive”.  We will use this to store our result. It’s also important to note that ideally this would be a single assignment variable, but that isn’t included in the CTP (so I’m not using it here).

The next thing we need is a way to compute the value asynchronously and to do this I’m going to use a task_group from the PPL.

   task_group tg_;

task_group is a class defined in <ppl.h> and is used to schedule tasks, wait for them all to complete or cancel any outstanding tasks assigned to it.  In the CTP the task_group is actually thread-safe so I can add tasks to it, call wait or cancel from any thread and get well defined behavior.

Now let’s implement the constructor which will schedule a task to run the functor passed in and then place it’s result into our overwrite_buffer.  We schedule a task with the task_group::run template member function and assign it to the overwrite_buffer with the template method send defined in agents.h, note that here in the lambda I’m capturing fn & this by value and copying them into the lambda:

   template
   <class Functor>
   future(Functor fn){
       tg_.run([fn,this](){
           send(this->value_,fn());
       });
   }

I mentioned I would call out overheads and we’ve just created one. The overload for task_group::run that we just used will have a heap allocation because if you notice the ‘fn’ object is created on the stack and would be destructed if once the method exits, so we need to allocate a copy on the heap and delete it after it’s been executed (or canceled, or faulted).  There is an overloaded task_group::run method that takes a reference to a task_handle<T> that can be used which can avoid this particular overhead, but it will rely on the user to manage the lifetime of that task and ensure it is still valid whenever it is executed.

Next we need to implement GetValue to return the result.  To do this we’re going to use the template method receive, which is the analog of send.  When receive is used on a ‘message block’ it will actually block and wait for that value to be computed.  One of the important things about using the agents APIs to do this is that receive will generate a blocking notification to the Concurrency Runtime and the runtime will use this blocking notification to run additional work if it’s been scheduled.  Here’s how GetValue is implemented, it’s incredibly straightforward:

   T GetValue(){
       return receive(value_);
   };

The last thing we need to do is implement the destructor. Here I’m just calling task_group::wait to ensure that our task is cleaned up.  You may also want to call task_group::cancel if you don’t need that task to run to completion.

   ~future(){
       tg_.wait();
   };

Putting it all together

Here’s the complete program, that was a lot of typing for not very much code:

#include <agents.h>
template
<class T>
class future{
    overwrite_buffer<T> value_;
    task_group tg_;
public:
    template
    <class Functor>
    future(Functor fn){
        tg_.run([fn,this](){
            send(this->value_,fn());
        });
    }
    ~future(){
        tg_.wait();
    };
    T GetValue(){
        return receive(value_);
    };
};
void main(){
    future<int> f([](){
        return 2;
    });
    int result = f.GetValue();
    assert(result == 2);
}

Wrapping it up

I’m still not doing anything very interesting here and I could always do something like compute the fibonacci sequence the slow way. But in practice there are many more uses for a future, like applying a filter to an image or various forms of i/o where there is a combination of CPU work and perhaps a significant amount of latency.

As I alluded there are other implementation strategies here, for example the transform message block in <agents.h> could be used instead of a task_group and a series of message blocks could be combined to implement richer functionality like that described in the C++0x specification, most notably a timer and choice (wait for one of n inputs to be sent) or join would likely be required to ease implementation.

and here’s that recursive Fibonacci sequence, it’s late and I can’t resist:

   int fib(int n){
       if(n < 2)
           return n;
       future<int> n1([n](){return fib(n-1);});
       future<int> n2([n](){return fib(n-2);});
       return n1.GetValue() + n2.GetValue();
   }

-Rick

There will be several more talks from the Parallel Computing Platform team at PDC this week, our architect Niklas will be doing a deep dive on the Concurrency Runtime and discussing extensibility, Daniel Moth will be talking soon about the Task Parallel Library and PLINQ and there’s several Symposium sessions lined up.

While I’m here at the PDC I’ll post on my personal blog summarizing experiences and I’ll also try to get a few more examples posted either here or on our team blog which demonstrate the functionality included in the CTP.  That’s it for now though, it’s been a long day. 

-Rick

I’ll be honest and state that I haven’t read this book yet, but it is on my to read list.  Given that I can’t comment yet on the book, but I can share some traits I find beautiful in code.

Readable, it should be very clear what the code is doing from the names of the objects, their instances and any functions or methods.  This can be done with or without language support, and in a very personal sense, I will tend to avoid language features that make code unreadable.

I also really admire code that is decisive some folks like to call this declarative, but I think it can be more.  I dislike excessive branching mostly because I am very cautious and tend to get worried about testing a method with a lot of branching in it.

I admire stateless functions and objects I tend to find it fascinating when I see code that enables useful scenarios that is built out of stateless objects.  This also tends to lead to rather simple code which is also quite nice, because when I’ve long forgotten what code does if it’s readable and straightforward I can usually pick it back up again soon.

I fully expect your mileage to vary because appreciation of beauty is so personal.  I’ll also chime back in after I’ve read the book.

~Rick

Thinking about this on the plane

Last week I was in sunny & warm California talking to some customers and I picked up Game Programming Gems 6 from the library.  One of the most interesting articles to me was an article on using a variant of the Levenshtein string matching algorithm to help find close matches in library routines for maps and other developer / designer created content.

Loved the article, hated the code

Let’s be really, really clear about this. I loved this article; I’m really a geek about this sort of thing.  Unfortunately when I read the code example, like Harrison Ford in now two movies I can only say… "I have a bad feeling about this"

Here it is:

// This is an example of a template function that can be used to find
// a closest match in a std map, assuming the map uses strings as
// the primary index. Note that we must search through every string
// in the map to find the closest match, so it is a very
// slow function: O(n), which is linear time. It’s handy for
// debugging during development, but should not be used in
// release builds.
template<class _InIt>
inline _InIt closestMatch(_InIt _First, _InIt _Last, char const *val, float limit)
{
   float maxVal = limit;
   _InIt _max = _Last;
   for (; _First != _Last; ++_First){
     float newVal = stringMatch(_First->first.c_str(), val);
     if (newVal > maxVal)
    {
        maxVal = newVal; _max = _First;
     } 
   } 
   return (_max);
}

The first thing I noticed about this template function is that it the naming convention seemed a bit inconsistent.

Then I noticed something more problematic,  this function looks awfully familiar.

In fact it looks almost identical to std::max_element or std::accumulate depending on the mood your in; even the parameters are the same.  Only rather than reusing the existing STL function it really looks like accumulate was copied and pasted into the sample and then edited.  Here’s max_element from my VS installation, I hope it’s clear how similar this code is:

// TEMPLATE FUNCTION max_element WITH PRED
template<class _FwdIt, class _Pr>
inline _FwdIt _Max_element(_FwdIt _First, _FwdIt _Last, _Pr _Pred)
{ 
   // find largest element, using _Pred
   _FwdIt _Found = _First;
   if (_First != _Last)
      for (; ++_First != _Last; )
         if (_DEBUG_LT_PRED(_Pred, *_Found, *_First))
            _Found = _First; 
   return (_Found);
}

Let’s clean this up

I’d also like to take a moment to share where I’d like to take this, but showing my hand there’s an extraneous function call here and I think that we can get a lot closer to code that looks like this:

template<class InputIterator>
inline InputIterator newClosestMatch( InputIterator begin,
                                                             InputIterator end, 
                                                             std::string val, float limit
                                                             )
{ 
       return std::max_element(begin,end,[&](…){….});
}

But first we have to unravel it a bit.  Perhaps it isn’t so hard…  We need to note that we’re using the stringMatch function as our comparison function.  The key piece to unravel is the call to stringMatch to retrieve the comparison number and that this can be re-expressed in a comparison function as:

      stringMatch(leftString, val) < stringMatch(rightString,val)

The next important change is that we need to refactor slightly to work with the collection element (in this case it’s a hash_map) directly as follows:

typedef std::pair<std::string,int> StrPair;

template<class InputIterator>
inline InputIterator newClosestMatch( InputIterator begin,
                                                             InputIterator end, 
                                                             std::string val, float limit
                                                             ) {  
   return std::max_element( begin, end, 
          [&](const StrPair& left, const StrPair& right){  
             return ( stringMatch(left.first.c_str(),val.c_str()) < 
                          stringMatch(right.first.c_str(),val.c_str()) ); 
    };
}

Oh and by the way we’re done now and we basically have reduced several lines of user written code with declarative code that uses a single lambda to compute the expression.

I dunno about you, but this makes me very happy.

How does IE use 4-cores?

Well, if you remember that IE let’s you have multiple home pages in multiple tabs.  I use this feature to track the news sites I read and my web-mail, and it’s definitely using so here’s what it looks like on a quad core:

You should be able to see that it’s taking about 7.5 seconds to launch IE then the CPU to settle down.  More importantly, there’s a period of approximately 2 seconds between the 3.5 &  5.5 second tickmarks where the CPU utilization is at > 70%.

What about 8-cores?

I happen to be fortunate enough to run this same scenario on a dual-socket quad core and so I collected a trace on that as well.  Keep in mind that the clock speeds of these 2 are completely different so comparing benchmarks between them is quite inadvisable, but I’m going to do it anyways :).

You can see in the capture below that on the 8-way the time has been reduced to 5.5 seconds… it’s still using > 70% of the CPU but now it’s 70% of  8 cores instead of 4…

And that period of high CPU utilization (between 8.5 seconds & 9 seconds on this box) is now only about a second long.

Hey it didn’t get twice as fast, why is this good?

I could very easily have shown a paint.net application getting twice as fast as it did on my quad-core, but that’s actually not very interesting.  The point here is that we took a look at an existing application that was easy to find use that was already multi-threaded and was designed to take advantage of inherent latencies in I/O and showed that some decent speedups were available to be taken advantage of with a multi-core system.   This is really just an opportunistic performance win and real users,  using real applications will see some noticeable improvements without installing developer tools or running gadzillions of applications at once (and don’t knock that – I think I have about 40 windows open on my desktop right now).

Perhaps in a future post, I’ll post a couple ideas of how to take even better advantage of multiple cores using something like the Task Parallel Library or PLINQ, but for now that’s it.

-Rick