Tag Archives: concurrency

WCF Web API– Handling requests asynchronously

This is the ninth post on a series about the Preview 4 of WCF Web API. The previous posts were:

In this post, I show how to handle HTTP requests asynchronously.

.NET’s Asynchronous Programming Model (APM)

The first version of the .NET platform introduced the Asynchronous Programming Model (APM) to represent asynchronous operations. It is characterized by the following:

  • Each asynchronous operation is divided into two methods, the Begin method and the End method;
  • The Begin method receives the operation’s inputs and two extra parameters – an AsyncCallback and an object state. This method must return an IAsyncResult, representing the asynchronous operation.
  • The End method receives the above IAsyncResult and returns the operation result, waiting if it is not yet available

The APM model is used in several places of the .NET platform. Some examples are stream reading and writing; and asynchronous delegate invocation.

Currently, WCF also uses the APM model both on the client side and on the service side:

  • On the client-side, the generated proxies can expose each WCF operation as a Begin/End method pair.
  • On the service-side, the implementing class may expose a Begin/End method pair that will be called asynchronously by the WCF runtime.

The Preview 4 release of WCF Web API also supports this model. The following code excerpt shows an example of decomposing an Web API operation into a Begin and End methods

[OperationContract(AsyncPattern = true)]
[WebGet(UriTemplate = "image?uri={uri}")]
private IAsyncResult BeginGetImage(HttpRequestMessage req, string uri,
                                                 AsyncCallback ac, object state)

private HttpResponseMessage EndGetImage(IAsyncResult ar)

The usage of the asynchronous model is defined by the AsyncPattern property assigned with true on the OperationContract attribute.

More details about the AsyncPattern in “classical” WCF can be found in the following post series: WCF and the AsyncPattern property.

The Task-based model

The version 4.0 of .NET introduced a new asynchronous model based on the Task<T> class and TPL library. A Task<T> represents an asynchronous operation that returns a value of type T. On a first look, this class may seem similar to IAsyncResult, however there are significant differences:

  • No need to call an End operation to retrieve the result. Instead, this value can be obtained from the task itself.
  • Better support for cancellation, through cancellation tokens.
  • Better support for exception handling.
  • Native support for asynchronous operation composition, via the ContinueWith and the ContinueWhenAll/ContinueWhenAny methods.

The announced C#async support is also based on the Task<T> model and will provide a language integrated way of building and composing asynchronous operations.

In the .NET framework, this new Task<T> model is slowly replacing the APM model. For instance:

  • The new .NET’s HTTP client (HttpClient class) provides a set of XxxAsync methods, returning Task<HttpResponseMessage>, for performing asynchronous HTTP requests.
  • The WCF Web API message handlers, briefly described in the sixth post of this series, also use the new Task<T> model.

Using both asynchronous models

Unfortunately, the Preview 4 of WCF Web API doesn’t yet support the Task<T> model for the operation implementation. However, it is not hard to implement a APM based interface using Task<T>, as described in the following reference: TPL and Traditional .NET Asynchronous Programming. The key fact is that the Task<T> class also implements the IAsyncResult interface.

An example

To exemplify these concepts, the following example shows a service for image transcoding. It has one operation, that handles GET requests on the following URI template “image?uri={uri}”, and performs the following actions:

  • GETs the representation of the resource (an image) identified by the {uri}
  • Tries to build a bitmap with this resource’s representation
  • Encodes and returns the bitmap using the JPEG format

Obtaining the response to the GET request and all the associated representation bytes may take a significant time. In order to avoid keeping a thread blocked during this time, this operation’s implementation will be asynchronous.

private HttpResponseMessage _response;
private HttpResponseException _exception;

[OperationContract(AsyncPattern = true)]
[WebGet(UriTemplate = "image?uri={uri}")]
private IAsyncResult BeginGetImage(string uri, AsyncCallback ac, object state)
    var task = Task.Factory.Iterate(GetImageAndConvertItToJpeg(uri), state);
    task.ContinueWith(t =>
         Trace("Main task completed, calling callback");
         if (ac != null) ac(task);
        }, TaskContinuationOptions.ExecuteSynchronously);
     return task;

private HttpResponseMessage EndGetImage(IAsyncResult ar)
    var task = ar as Task;
    if (task == null || task.IsFaulted)
        throw new HttpResponseException(HttpStatusCode.InternalServerError);
    if (_exception != null) throw _exception;
    return _response;

The previous code excerpt presents the BeginGetImage and EndGetImage methods.

Regarding the Begin method:

  • The operation core processing is started on line 9, using a iterator based technique described in the following section, and is represented by a task.
  • On line 10, a continuation is added to the above task, so that the WCF callback is called when the task completes. This will trigger the call, by the WCF runtime, of the EndGetImage method.
  • On line 15, the task created on line 9 is returned. Notice that the operation’s return type is IAsyncResult. However, as stated in the beginning of this post, the Task class implements IAsyncResult interface. This is the key aspect to translate from the Task model into the APM model. The other aspect is the use of the state parameter when creating the task.

Regarding the End method:

  • On line 21, the IAsyncResult is casted to a task. Notice that, according to the APM, the IAsyncResult passed into the End method is the one returned by the Begin method.
  • Finally, the _exception and _response instance fields are consulted to produce an exception or response. These fields are assigned by the asynchronous process started on line 9 of the Begin method, as described in the following section.

Notice that the above code is rather generic. It could be used to implement other operations, just by changing line 9.

Using iterators to define asynchronous behavior

The typical way of defining asynchronous behavior with the Task<T> model is by chaining tasks with actions, producing other tasks. This breaks the code into multiple methods/anonymous methods/lambdas, reducing the readability.

In this post, we use an iterator based technique proposed by J. Richter for the APM and adapted to the Task<T> model by the ParallelExtensionExtras. The following code excerpt shows the asynchronous processing of the GetImageAndConvertItToJpeg method, implemented as a C# iterator.


private IEnumerable<Task> GetImageAndConvertItToJpeg(string uri)
    using (var client = new HttpClient
            MaxResponseContentBufferSize = 1024*1024
        uri = Uri.UnescapeDataString(uri);
        Task<HttpResponseMessage> readTask = null;
            readTask = client.GetAsync(uri);
        catch (Exception)
            _exception = new HttpResponseException(HttpStatusCode.NotFound);
            yield break;
        Trace("yielding for HTTP response...");
        yield return readTask;

        Trace("...resuming after HTTP response completed");
        if (readTask.IsFaulted || readTask.Result.StatusCode != HttpStatusCode.OK)
            _exception = new HttpResponseException(HttpStatusCode.NotFound);
            yield break;
        var httpContent = readTask.Result.Content;
        using (var ms = new MemoryStream())
            var copyTask = httpContent.ContentReadStream.CopyStreamToStreamAsync(ms);
            Trace("yielding for stream copy...");
            yield return copyTask;

            Trace("...resuming after copy completed");
            if (copyTask.IsFaulted)
                _exception = new HttpResponseException(HttpStatusCode.NotFound);
                yield break;
            Bitmap bitmap = null;
                bitmap = new Bitmap(ms);
            catch (Exception)
                _exception = new HttpResponseException(HttpStatusCode.NotFound);
                yield break;
            var output = new MemoryStream();
            bitmap.Save(output, ImageFormat.Jpeg);
            output.Seek(0, SeekOrigin.Begin);
            var content = new StreamContent(output);
            content.Headers.ContentType = new MediaTypeHeaderValue("image/jpeg");
            _response = new HttpResponseMessage
                    Content = content


This code performs the following:

  • An HttpClient is created to retrieve the upstream image. If any exception occurs while creating and starting the request, then the _exception instance field is assigned with an HttpResponseException and the iterator stops (yield break). If not, the result of the client.GetAsync(uri) statement is a Task<HttpResponseMessage>, representing the HTTP request. This task is then returned by the iterator, without blocking the hosting thread.
  • The iterator is resumed only when this Task<HttpResponseMessage> completes. Then it checks for errors, and if everything is ok, creates a task to copy the HttpResponseMessage’s stream into a memory stream. This is done to ensure that all the image’s content are in memory when the bitmap is created. If the bitmap was created directly from the  HttpResponseMessage’s stream, then the Bitmapconstructor would block the hosting thread while waiting for all the bytes (and we don’t want to block threads!).The stream copy uses the CopyStreamToStreamAsync extension method, also available on the ParallelExtensionExtras. The task returned by this method is then returned by the iterator, so that the stream copy is performed asynchronously.
  • The iterator is resumed again when the stream copy completes. After some error checking, the Bitmap is finally created and saved into another memory stream used in the response message. Notice that this can be done synchronously, since it doesn’t require any I/O.

Unfortunately, the above code is slightly cluttered by the error handling and by the fact that a yield statement cannot be inside a try-catch block.

The chaining of the tasks returned by the above iterator is done by the ParallelExtensionExtras Iterate method, that wraps the overall resulting asynchronous processing in a Task (remember line 9 of the second code excerpt).

This chaining is performed by the following:

  • When initially called, the Iterate method schedules a task, using StartNew, that will call the iterator to retrieve the first task. It also creates a TaskCompletionSource to represent the  overall task.
  • The first task retrieved from the iterator (representing the HTTP request) is then chained with another call to the iterator, using a ContinueWith.
  • The second task retrieved from the iterator (representing the stream copy) is also chained with another call to the iterator.
  • The final call to the iterator doesn’t return any element (MoveNext returns false), so Iterate signals the TaskCompletionSource that all the work is completed. This results in the call of the WCF callback.

The following diagram presents a sequence diagram, with the interaction between the different components.


The following color scheme is used:

  • Red – WCF runtime components;
  • Green – user code (Iterator represents the enumerable returned by the GetImageAndConvertItToJpeg method);
  • Light blue – TPL scheduler;
  • Dark blue – Iterate extension method (from the ParallelExtensionExtras)

The tracing output one of execution is (the first number is the managed thread id)

3: BeginGet…

6: yielding for HTTP response…

9: …resuming after HTTP response completed

9: yielding for stream copy…

6: …resuming after copy completed

6: Main task completed, calling callback

8: …EndGet

Conclusions and final remarks

  • Currently, WCF Web API only allows asynchronous operation implementations exposing the APM model.
  • However, it is not difficult to use a Task<T> based model with an APM interface, using the fact that Task implements IAsyncResult.
  • While the future C# 5 asynchronous support is not available, the use of iterators and the ParallelExtensionExtras’s Iterate method provides for a similar experience.
  • In this post I was not concerned with measuring the scalability improvements due to the use of asynchronous operation implementation. Namely, asynchronous processing may be disadvantageous if the blocking time is very small. My main goal here was the how and not the why of using asynchronous processing.
  • The ParallelExtensionExtras library is full of interesting utilities, of which the Iterate and the CopyStreamToStreamAsync are just examples.

Finally, feedback is greatly welcomed.

Java Memory Model

Some resources about the Java Memory Model:

WCF asynchronous processing and IIS hosting

Recommending the post “WCF Request Throttling and Server Scalability“.

It contains important information regarding WCF’s asynchronous processing when hosting on IIS. For example, I was unaware that

So in the release of .Net 3.0 and 3.5, WCF implemented synchronous versions of HTTP module and handler instead of asynchronous ones

meaning that even with asynchronous service implementation (asyncpattern=true), a thread will be blocked for the duration of the request’s processing. This thread will be the ASP.NET’s thread from the CLR ThreadPool.

Since I do most of my tests and experiences using self-hosting, I didn’t have noticed this.

However, the SP1 of .NET 3.5 adds support for asynchronous HTTP modules and handlers.

WCF and the AsyncPattern property (part 3)

This is the third in a series of posts about the AsyncPattern in WCF.

The previous two posts described the purpose and usage of this property on the service side. The present post addresses the client side.

AsyncPattern at the client side

The AsyncPattern=true is used at the client side to expose an asynchronous programmatic interface to a service operation. Similarly to what happens at the service side, this property defines the local communication pattern between the service’s client code and the WCF runtime (at the client side).

Consider the following WSDL excerpt defining a portType with two request-response operations

   1: <wsdl:portType name="IService">
   3:   <wsdl:operation name="Operation1">
   4:     <wsdl:input wsaw:Action="..." message="..." /> 
   5:     <wsdl:output wsaw:Action="..." message="..." /> 
   6:   </wsdl:operation>
   8:   <wsdl:operation name="Operation2">
   9:     <wsdl:input wsaw:Action="..." message="..." /> 
  10:     <wsdl:output wsaw:Action="..." message="..." /> 
  11:   </wsdl:operation>
  13: </wsdl:portType>

By default, the interface generated by svcutil.exe has a synchronous interface

   1: public interface IService {
   3:     [System.ServiceModel.OperationContractAttribute(Action=".../Operation1", ReplyAction=".../Operation1Response")]
   4:     ReplyData Operation1(RequestData rd);
   6:     [System.ServiceModel.OperationContractAttribute(Action="http://tempuri.org/IService/Operation2", ReplyAction="http://tempuri.org/IService/Operation2Response")]
   7:     ReplyData Operation2(RequestData rd);
   9: }

However, using the /async switch, the svcutil.exe can also generate an interface with synchronous and asynchronous methods.

   1: public interface IService {
   3:     // Operation 1 synchronous method   
   4:     [System.ServiceModel.OperationContractAttribute(Action=".../Operation1", ReplyAction=".../Operation1Response")]
   5:     ReplyData Operation1(RequestData rd);
   7:     // Operation 1 asynchronous method pair         
   8:     [System.ServiceModel.OperationContractAttribute(AsyncPattern=true, Action=".../Operation1", ReplyAction=".../Operation1Response")]
   9:     System.IAsyncResult BeginOperation1(RequestData rd, System.AsyncCallback callback, object asyncState);        
  10:     ReplyData EndOperation1(System.IAsyncResult result);
  12:     // Operation 2 synchronous method   
  13:     [System.ServiceModel.OperationContractAttribute(Action=".../Operation2", ReplyAction=".../Operation2Response")]
  14:     ReplyData Operation2(AsyncPattern.Client.AsyncClient.RequestData rd);
  16:     // Operation 2 asynchronous method pair         
  17:     [System.ServiceModel.OperationContractAttribute(AsyncPattern=true, Action=".../Operation2", ReplyAction=".../Operation2Response")]
  18:     System.IAsyncResult BeginOperation2(RequestData rd, System.AsyncCallback callback, object asyncState);        
  19:     ReplyData EndOperation2(System.IAsyncResult result);
  21: }

Note that:

  • This setting does not change the communication pattern between the client and the service.
  • Using AsyncPattern=true at the client side is not in any way related to using AsyncPattern=true at the service side (the service doesn’t even have to implemented over WCF!)

AsyncPattern=true versus BeginInvoke

As stated here, using the delegate’s BeginInvoke method is not a way to achieve the same effect as AsyncPattern=true.


  • If the delegate target is a client channel instance (returned by a ChannelFactory), then the call will block (as described in a previous post).


  • If the delegate target is a “client proxy” (ClientBase derived class generated by svcutil.exe), then the call will not block. However a thread will remain blocked until the reply is received (the thread from the thread pool used to call the asynchronous method). This does not happens when using AsyncPattern=true, since the WCF supports full asynchronous requests.
    The blocking of this thread may not be significant on a smart client scenario. However, this “thread consumption” may be problematic on a middle-tier scenario, where a significant amount of requests may be pending at a given moment in time.

WCF and the AsyncPattern property (part 2)

This is the second in a series of posts about the AsyncPattern in WCF.

The previous post described the consequences of setting AsyncPattern = true on an OperationContractAttribute.

However, a question still remains: when and how to use AsyncPattern = true on the service-side?


  • The WCF documentation states that

Use an asynchronous approach in a service operation implementation if the operation service implementation makes a blocking call, such as doing I/O work. When you are in an asynchronous operation implementation, try to call asynchronous operations and methods to extend the asynchronous call path as far as possible

  • Similarly, Kenny Wolf (from the WCF/WF group) states that

If you have an operation that is blocking (accessing SQL, Channels, etc) then you should use AsyncPattern=true.  That way you’ll free up whatever thread we’re using to call your operation from

      and also that

If you aren’t doing something that’s “natively async”, then you shouldn’t be using AsyncPattern=true. That is, you shouldn’t just create a thread just for the sake of performing “background work” as part of an asynchronous operation

My conjecture is that asynchronous operation implementation should be used when there are conditions to release the calling thread, without requiring the creation of a new thread, and the time needed for the operation completion is significant. This happens when the operation performs “lengthy” IO requests. If the IO interface has an asynchronous interface, then the asynchronous operation call can be “linked” to the asynchronous IO call (see diagram below). Note that a service request is just a special kind of an IO request and that WCF also provides an asynchronous interface for clients.


The following sequence diagram shows the “linkage” between the asynchronous operation implementation and an asynchronous IO request.


  1. The WCF dispatcher calls the BeginOperation method, which in turn starts the asynchronous IO request (BeginIO) and returns immediately,. This sequence is done on an IO thread, which is “released” after BeginOperation returns.
  2. When the IO request is completed, the Operation Callback is called on an IO thread (not necessarily the same from the above sequence). This callback:
    1. Calls EndIO to finish the IO request and collects the IO result.
    2. Calls the WCF provided callback (passed as a parameter to BeginOperation).
    3. The WCF callback then calls EndOperation to finish the operation and collect the operation result.

Notice that the operation’s execution is divided into two sequences (two threads):

  • The “left-to-right” sequence processes the request, running on the IO thread associated with the service request.
  • The “right-to-left” sequence produces the reply, running on the IO thread associated with the IO request completion.

Namely there is no blocked thread between the start of the operation and the completion of the IO. This property is important in scenarios were there can be a significant amount of operations waiting for “lengthy” IO requests (e.g. middle-tier services).

WCF and the AsyncPattern property (part 1)

The OperationContractAttribute is used, in WCF, to define operations on a service contract. This attribute contains the AsyncPattern property that can be used to implement or use services asynchronously.

AsyncPattern at the service side

Consider the following excerpt

   1: [ServiceContract]
   2: interface IService {
   3:     [OperationContract]
   4:     ReplyData Operation1(RequestData rd);
   6:     [OperationContract(AsyncPattern = true)]
   7:     IAsyncResult BeginOperation2(RequestData rd, AsyncCallback ac, object obj);
   8:     ReplyData EndOperation2(IAsyncResult ar);
   9: }
  • The first [OperationContract], which has AsyncPattern set to false (the default), defines one request-response operation called ‘Operation1’. This operation is implemented synchronously by one method: Operation1. Synchronously means that the WCF dispatcher will call the Operation1 method, passing the RequestData. This call will block during the execution of the operation. Finally, the method returns the ReplyData that will be used in the response sent to the client.


  • The second [OperationContract], which has AsyncPattern set to true, also defines one request-response operation called ‘Operation2’. However, this operation is implemented asynchronously by two methods: BeginOperation2 and EndOperation2. Asynchronously (see Asynchronous Programming Overview) means that:
    • The WCF dispatcher will call the BeginOperation2 method, passing the RequestData, an AsyncCallback and some state information.
    • This method should start the execution of the operation and return an IAsyncResult immediately.
    • When the execution of the operation is completed, then the AsyncCallback should be called.
    • This AsyncCallback, implemented by the WCF runtime, will then call EndOperation2 to collect the ReplyData that will be used in the response sent to the client.

Note that, when using AsyncPattern set to true, there is no longer a 1-to-1 relation between the service contract methods and the service operations. Namely, the BeginOperation2 and EndOperation2 methods are both associated to a single operation named ‘Operation2’.

Also, from the outside of the service, both operations (‘Operation1’ and ‘Operation2’) are seen as request-response operations. This is visible in the generated WSDL description

   1: <wsdl:portType name="IService">
   3:   <wsdl:operation name="Operation1">
   4:     <wsdl:input wsaw:Action="..." message="..." /> 
   5:     <wsdl:output wsaw:Action="..." message="..." /> 
   6:   </wsdl:operation>
   8:   <wsdl:operation name="Operation2">
   9:     <wsdl:input wsaw:Action="..." message="..." /> 
  10:     <wsdl:output wsaw:Action="..." message="..." /> 
  11:   </wsdl:operation>
  13: </wsdl:portType>

So, concluding, the AsyncPattern is an implementation detail of the service: it defines the communication pattern between the WCF runtime and the service implementation but not the communication pattern between the service client and the service.

Asynchronous delegate invocation of a WCF client channel

In an older post, Nicholas Allen stated that an asynchronous delegate invocation on a WCF client channel object will block until the response is received from a service.

The only given explanation was:

The problem is that BeginInvoke knows about and only works with specific types of proxy objects, which do not include the proxy objects generated by ChannelFactory

In a related post, the blog’s author stated that

why does BeginInvoke break when all the TP [Transparent Proxy]and RP [Real Proxy] plumbing is in place? This question is much trickier than it seems to be, after a bit of research on the default implementation of TP and RP mechanism used by .NET remoting using both  .NET reflector and the rotor 2.0 implementation of CLR, I finally figure out that for the current implementation of TP and RP mechanism, it only supports asynchronous call when the default RemotingProxy is in place, since the ServiceChannelProxy is WCF’s own implementation, it gets ignored by the BeginInvoke mechanism, and the BeginInvoke call against WCF’s proxies will be performed synchronously.

This explanation was not completely clear to me, however it provided some clues to the reasons why the asynchronous call could block.

So, my next steps were to grab the SSCLI source code and analyze how BeginInvoke is implemented. My observations follow:

  1. When a compiler defines a delegate class, the BeginInvoke method does not contain any implementation. Instead, this method is annotated with the MethodImplAttribute attribute, with the MethodCodeType field set to MethodCodeType.Runtime. This means that the BeginInvoke implementation will be given by the runtime.
  2. In SSCLI, the class COMDelegate (defined in the comdelegate.h and comdelegate.cpp files) represents the native methods of the Delegate class. In this C++ class, the GetInvokeMethodStub method is used to get the stubs for each of the synchronous and asynchronous invoke methods. For the BeginInvoke method, this stub is returned by the TheAsyncDelegateStub function (defined in remoting.h).
  3. The stub returned by the TheAsyncDelegateStub method is created by the CTPMethodTable::EmitSetupFrameCode method (defined in remotingx86.h). This last method emits x86 assembly code that includes a call to the CTPMethodTable::OnCall method (defined in remoting.cpp).
  4. The CTPMethodTable::OnCall, among other things, tests if the target of the delegate is a transparent proxy.
  5. If it IS NOT a transparent proxy:
    1. the OnCall method calls the RemotingProxy.Invoke static method, which  performs the expected ThreadPool.QueueUserWorkItem call.
  6. However, if it IS a transparent proxy:
    1. the OnCall method obtains the real proxy behind the transparent proxy, and calls the RealProxy.PrivateInvoke method.
    2. This method ends up calling the virtual Invoke method of ServiceChannelFactory (the type of  real proxy behind the transparent proxy returned by a ChannelFactory<T>). This method checks if the called channels’ method is synchronous, and if so completes the call synchronously, despite the call being made via a delegate’s BeginInvoke.

So, in conclusion:

  1. A delegates’s BeginInvoke method call is handled by the remoting infrastructure. Aparently, this so to optimize the asynchronous remoting calls, avoiding extra context switchs (see this post by Chris Brumme).
  2. If the target of the delegate is not a transparent proxy, then a regular ThreadPool.QueueUserWorkItem is performed.
  3. If the target is a transparent proxy, then the BeginInvoke call is processed by the real proxy. If the delegate’s method is synchronous, then the WCF’s real proxy performs a synchronous service call.

Note, however, that an asynchronous delegate call on the synchronous method of the “client proxy” generated by svcutil.exe will not block waiting for the service’s response.  This is so because this “client proxy” is a regular class and not a transparent or real proxy, so the “normal” BeginInvoke behavior applies.

This does not mean that using asynchronous delegate calls on the synchronous “client proxy” methods is recommended. The appropriate method is to use the /async switch of the svcutil.exe tool to generate a service interface and a “client proxy” with asynchronous methods, as described in the original Nicholas Allen’s post.