Tag Archives: HttpClient

Better timeout handling with HttpClient

The problem

If you often use HttpClient to call REST APIs or to transfer files, you may have been annoyed by the way this class handles request timeout. There are two major issues with timeout handling in HttpClient:

  • The timeout is defined at the HttpClient level and applies to all requests made with this HttpClient; it would be more convenient to be able to specify a timeout individually for each request.
  • The exception thrown when the timeout is elapsed doesn’t let you determine the cause of the error. When a timeout occurs, you’d expect to get a TimeoutException, right? Well, surprise, it throws a TaskCanceledException! So, there’s no way to tell from the exception if the request was actually canceled, or if a timeout occurred.

Fortunately, thanks to HttpClient‘s flexibility, it’s quite easy to make up for this design flaw.

So we’re going to implement a workaround for these two issues. Let’s recap what we want:

  • the ability to specify timeout on a per-request basis
  • to receive a TimeoutException rather than a TaskCanceledException when a timeout occurs.

Specifying the timeout on a per-request basis

Let’s see how we can associate a timeout value to a request. The HttpRequestMessage class has a Properties property, which is a dictionary in which we can put whatever we need. We’re going to use this to store the timeout for a request, and to make things easier, we’ll create extension methods to access the value in a strongly-typed fashion:

public static class HttpRequestExtensions
{
    private static string TimeoutPropertyKey = "RequestTimeout";

    public static void SetTimeout(
        this HttpRequestMessage request,
        TimeSpan? timeout)
    {
        if (request == null)
            throw new ArgumentNullException(nameof(request));

        request.Properties[TimeoutPropertyKey] = timeout;
    }

    public static TimeSpan? GetTimeout(this HttpRequestMessage request)
    {
        if (request == null)
            throw new ArgumentNullException(nameof(request));

        if (request.Properties.TryGetValue(
                TimeoutPropertyKey,
                out var value)
            && value is TimeSpan timeout)
            return timeout;
        return null;
    }
}

Nothing fancy here, the timeout is an optional value of type TimeSpan. We can now associate a timeout value with a request, but of course, at this point there’s no code that makes use of the value…

HTTP handler

The HttpClient uses a pipeline architecture: each request is sent through a chain of handlers (of type HttpMessageHandler), and the response is passed back through these handlers in reverse order. This article explains this in greater detail if you want to know more. We’re going to insert our own handler into the pipeline, which will be in charge of handling timeouts.

Our handler is going to inherit DelegatingHandler, a type of handler designed to be chained to another handler. To implement a handler, we need to override the SendAsync method. A minimal implementation would look like this:

class TimeoutHandler : DelegatingHandler
{
    protected async override Task<HttpResponseMessage> SendAsync(
        HttpRequestMessage request,
        CancellationToken cancellationToken)
    {
        return await base.SendAsync(request, finalCancellationToken);
    }
}

The call to base.SendAsync just passes the request to the next handler. Which means that at this point, our handler does absolutely nothing useful, but we’re going to augment it gradually.

Taking into account the timeout for a request

First, let’s add a DefaultTimeout property to our handler; it will be used for requests that don’t have their timeout explicitly set:

public TimeSpan DefaultTimeout { get; set; } = TimeSpan.FromSeconds(100);

The default value of 100 seconds is the same as that of HttpClient.Timeout.

To actually implement the timeout, we’re going to get the timeout value for the request (or DefaultTimeout if none is defined), create a CancellationToken that will be canceled after the timeout duration, and pass this CancellationToken to the next handler: this way, the request will be canceled after the timout is elapsed (this is actually what HttpClient does internally, except that it uses the same timeout for all requests).

To create a CancellationToken whose cancellation we can control, we need a CancellationTokenSource, which we’re going to create based on the request’s timeout:

private CancellationTokenSource GetCancellationTokenSource(
    HttpRequestMessage request,
    CancellationToken cancellationToken)
{
    var timeout = request.GetTimeout() ?? DefaultTimeout;
    if (timeout == Timeout.InfiniteTimeSpan)
    {
        // No need to create a CTS if there's no timeout
        return null;
    }
    else
    {
        var cts = CancellationTokenSource
            .CreateLinkedTokenSource(cancellationToken);
        cts.CancelAfter(timeout);
        return cts;
    }
}

Two points of interest here:

  • If the request’s timeout is infinite, we don’t create a CancellationTokenSource; it would never be canceled, so we save a useless allocation.
  • If not, we create a CancellationTokenSource that will be canceled after the timeout is elapsed (CancelAfter). Note that this CTS is linked to the CancellationToken we receive as a parameter in SendAsync: this way, it will be canceled either when the timeout expires, or when the CancellationToken parameter will itself be canceled. You can get more details on linked cancellation tokens in this article.

Finally, let’s change the SendAsync method to use the CancellationTokenSource we created:

protected async override Task<HttpResponseMessage> SendAsync(
    HttpRequestMessage request,
    CancellationToken cancellationToken)
{
    using (var cts = GetCancellationTokenSource(request, cancellationToken))
    {
        return await base.SendAsync(
            request,
            cts?.Token ?? cancellationToken);
    }
}

We get the CTS and pass its token to base.SendAsync. Note that we use cts?.Token, because GetCancellationTokenSource can return null; if that happens, we use the cancellationToken parameter directly.

At this point, we have a handler that lets us specify a different timeout for each request. But we still get a TaskCanceledException when a timeout occurs… Well, this is going to be easy to fix!

Throwing the correct exception

All we need to do is catch the TaskCanceledException (or rather its base class, OperationCanceledException), and check if the cancellationToken parameter is canceled: if it is, the cancellation was caused by the caller, so we let it bubble up normally; if not, this means the cancellation was caused by the timeout, so we throw a TimeoutException. Here’s the final SendAsync method:

protected async override Task<HttpResponseMessage> SendAsync(
    HttpRequestMessage request,
    CancellationToken cancellationToken)
{
    using (var cts = GetCancellationTokenSource(request, cancellationToken))
    {
        try
        {
            return await base.SendAsync(
                request,
                cts?.Token ?? cancellationToken);
        }
        catch(OperationCanceledException)
            when (!cancellationToken.IsCancellationRequested)
        {
            throw new TimeoutException();
        }
    }
}

Note that we use an exception filter : this way we don’t actually catch the OperationException when we want to let it propagate, and we avoid unnecessarily unwinding the stack.

Our handler is done, now let’s see how to use it.

Using the handler

When creating an HttpClient, it’s possible to specify the first handler of the pipeline. If none is specified, an HttpClientHandler is used; this handler sends requests directly to the network. To use our new TimeoutHandler, we’re going to create it, attach an HttpClientHandler as its next handler, and pass it to the HttpClient:

var handler = new TimeoutHandler
{
    InnerHandler = new HttpClientHandler()
};

using (var client = new HttpClient(handler))
{
    client.Timeout = Timeout.InfiniteTimeSpan;
    ...
}

Note that we need to disable the HttpClient‘s timeout by setting it to an infinite value, otherwise the default behavior will interfere with our handler.

Now let’s try to send a request with a timeout of 5 seconds to a server that takes to long to respond:

var request = new HttpRequestMessage(HttpMethod.Get, "http://foo/");
request.SetTimeout(TimeSpan.FromSeconds(5));
var response = await client.SendAsync(request);

If the server doesn’t respond within 5 seconds, we get a TimeoutException instead of a TaskCanceledException, so things seem to be working as expected.

Let’s now check that cancellation still works correctly. To do this, we pass a CancellationToken that will be cancelled after 2 seconds (i.e. before the timeout expires):

var request = new HttpRequestMessage(HttpMethod.Get, "http://foo/");
request.SetTimeout(TimeSpan.FromSeconds(5));
var cts = new CancellationTokenSource(TimeSpan.FromSeconds(2));
var response = await client.SendAsync(request, cts.Token);

This time, we receive a TaskCanceledException, as expected.

By implementing our own HTTP handler, we were able to solve the initial problem and have a smarter timeout handling.

The full code for this article is available here.

Fun with the HttpClient pipeline

A few years ago, Microsoft introduced the HttpClient class as a modern alternative to HttpWebRequest to make web requests from .NET apps. Not only is this new API much easier to use, cleaner, and asynchronous by design, it’s also easily extensible.

You might have noticed that HttpClient has a constructor that accepts a HttpMessageHandler. What is this handler? It’s an object that accepts a request (HttpRequestMessage) and returns a response (HttpResponseMessage); how it does that is entirely dependent on the implementation. By default, HttpClient uses HttpClientHandler, a handler which sends a request to a server over the network and returns the server’s response. The other built-in handler implementation is an abstract class named DelegatingHandler, and is the one I want to talk about.

The pipeline

DelegatingHandler is a handler that is designed to be chained with another handler, effectively forming a pipeline through which requests and responses will pass, as shown on this diagram:

HttpClient pipeline diagram

(Image from the official ASP.NET website)

Each handler has a chance to examine and/or modify the request before passing it to the next handler in the chain, and to examine and/or modify the response it receives from the next handler. Typically, the last handler in the pipeline is the HttpClientHandler, which communicates directly with the network.

The handler chain can be setup like this:

var pipeline = new MyHandler1()
{
    InnerHandler = new MyHandler2()
    {
        InnerHandler = new HttpClientHandler()
    }
};
var client = new HttpClient(pipeline);

But if you prefer fluent interfaces, you can easily create an extension method to do it like this:

var pipeline = new HttpClientHandler()
    .DecorateWith(new MyHandler2())
    .DecorateWith(new MyHandler1());
var client = new HttpClient(pipeline);

All this might seem a little abstract at this point, but this pipeline architecture enables plenty of interesting scenarios. See, HTTP message handlers can be used to add custom behavior to how requests and responses are processed. I’ll give a few examples.

Side note: I’m presenting this feature from a client-side perspective (since I primarily make client apps), but the same HTTP message handlers are also used on the server-side in ASP.NET Web API.

Unit testing

The first use case that comes to mind, and the first I ever used, is unit testing. If you’re testing a class that makes online payments over HTTP, you don’t want it to actually send requests to the real server… you just want to ensure that the requests it sends are correct, and that it reacts correctly to specific responses. An easy solution to this problem is to create a "stub" handler, and inject it into your class to use instead of HttpClientHandler. Here’s a simple implementation:

class StubHandler : HttpMessageHandler
{
    // Responses to return
    private readonly Queue<HttpResponseMessage> _responses =
        new Queue<System.Net.Http.HttpResponseMessage>();

    // Requests that were sent via the handler
    private readonly List<HttpRequestMessage> _requests =
        new List<System.Net.Http.HttpRequestMessage>();

    protected override Task<HttpResponseMessage> SendAsync(
        HttpRequestMessage request,
        CancellationToken cancellationToken)
    {
        if (_responses.Count == 0)
            throw new InvalidOperationException("No response configured");

        _requests.Add(request);
        var response = _responses.Dequeue();
        return Task.FromResult(response);
    }

    public void QueueResponse(HttpResponseMessage response) =>
        _responses.Enqueue(response);

    public IEnumerable<HttpRequestMessage> GetRequests() =>
        _requests;
}

This class lets you record the requests that are sent via the handler and specify the responses that should be returned. For instance, you could write a test like this:

// Arrange
var handler = new StubHandler();
handler.EnqueueResponse(new HttpResponseMessage(HttpStatusCode.Unauthorized));
var processor = new PaymentProcessor(handler);

// Act
var paymentResult = await processor.ProcessPayment(new Payment());

// Assert
Assert.AreEqual(PaymentStatus.Failed, paymentResult.Status);

Of course, rather than creating a stub manually, you could use a mocking framework to generate a fake handler for you. The fact that the SendAsync method is protected makes it a little harder than it should be, but you can easily work around the issue by making a subclass that exposes a public virtual method, and mock that instead:

public abstract class MockableMessageHandler : HttpMessageHandler
{
    protected override sealed Task<HttpResponseMessage> SendAsync(
        HttpRequestMessage request,
        CancellationToken cancellationToken)
    {
        return DoSendAsync(request);
    }

    public abstract Task<HttpResponseMessage> DoSendAsync(HttpRequestMessage request);
}

Usage example with FakeItEasy:

// Arrange
var handler = A.Fake<MockableMessageHandler>();
A.CallTo(() => handler.DoSendAsync(A<HttpRequestMessage>._))
    .Returns(new HttpResponseMessage(HttpStatusCode.Unauthorized));
var processor = new PaymentProcessor(handler);
...

Logging

Logging sent requests and received responses can help diagnose issues. This can easily be done with a custom delegating handler:

public class LoggingHandler : DelegatingHandler
{
    private readonly ILogger _logger;

    public LoggingHandler(ILogger logger)
    {
        _logger = logger;
    }

    protected override async Task<HttpResponseMessage> SendAsync(
        HttpRequestMessage request,
        CancellationToken cancellationToken)
    {
        _logger.Trace($"Request: {request}");
        try
        {
            // base.SendAsync calls the inner handler
            var response = await base.SendAsync(request, cancellationToken);
            _logger.Trace($"Response: {response}");
            return response;
        }
        catch (Exception ex)
        {
            _logger.Error($"Failed to get response: {ex}");
            throw;
        }
    }
}

Retrying failed requests

Another interesting use case for HTTP message handlers is to automatically retry failed requests. For instance, the server you’re talking to might be temporarily unavailable (503), or it could be throttling your requests (429), or maybe you lost Internet access. Handling the retry for these cases at the application level is a pain, because it can happen virtually in any part of your code. Having this logic at the lowest possible level and implemented in a way that is completely transparent to the callers can make things much easier.

Here’s a possible implementation of a retry handler:

public class RetryHandler : DelegatingHandler
{
    protected override async Task<HttpResponseMessage> SendAsync(
        HttpRequestMessage request,
        CancellationToken cancellationToken)
    {
        while (true)
        {
            try
            {
                // base.SendAsync calls the inner handler
                var response = await base.SendAsync(request, cancellationToken);

                if (response.StatusCode == HttpStatusCode.ServiceUnavailable)
                {
                    // 503 Service Unavailable
                    // Wait a bit and try again later
                    await Task.Delay(5000, cancellationToken);
                    continue;
                }

                if (response.StatusCode == (HttpStatusCode)429)
                {
                    // 429 Too many requests
                    // Wait a bit and try again later
                    await Task.Delay(1000, cancellationToken);
                    continue;
                }

                // Not something we can retry, return the response as is
                return response;
            }
            catch (Exception ex) when(IsNetworkError(ex))
            {
                // Network error
                // Wait a bit and try again later
                await Task.Delay(2000, cancellationToken);
                continue;
            }
        }
    }

    private static bool IsNetworkError(Exception ex)
    {
        // Check if it's a network error
        if (ex is SocketException)
            return true;
        if (ex.InnerException != null)
            return IsNetworkError(ex.InnerException);
        return false;
    }
}

Note that it’s a pretty naive and simplistic implementation; for use in production code, you will probably want to add exponential backoff, take the Retry-After header into account to decide how long you have to wait, or be more subtle in how you check if an exception indicates a connection issue. Also, note that in its current state, this handler will retry forever until it succeeds; make sure to pass a cancellation token so that you can stop retrying if necessary.

Other use cases

I can’t give examples for every possible scenario, but here are a few other possible use cases for HTTP message handlers:

  • Custom cookie handling (I actually did that to work around a bug in CookieContainer)
  • Custom authentication (also something I did to implement OAuth2 Bearer authentication)
  • Using the X-HTTP-Method-Override header to pass proxies that forbid certain HTTP methods (see Scott Hanselman’s article for details)
  • Custom encryption or encoding
  • Caching

As you can see, there’s a whole world of possibilities! If you have other ideas, let me know in the comments!

Uploading data with HttpClient using a "push" model

If you have used the HttpWebRequest class to upload data, you know that it uses a “push” model. What I mean is that you call the GetRequestStream method, which opens the connection if necessary, sends the headers, and returns a stream on which you can write directly.

.NET 4.5 introduced the HttpClient class as a new way to communicate over HTTP. It actually relies on HttpWebRequest under the hood, but offers a more convenient and fully asynchronous API. HttpClient uses a different approach when it comes to uploading data: instead of writing manually to the request stream, you set the Content property of the HttpRequestMessage to an instance of a class derived from HttpContent. You can also pass the content directly to the PostAsync or PutAsync methods.

The .NET Framework provides a few built-in implementations of HttpContent, here are some of the most commonly used:

  • ByteArrayContent: represents in-memory raw binary content
  • StringContent: represents text in a specific encoding (this is a specialization of ByteArrayContent)
  • StreamContent: represents raw binary content in the form of a Stream

For instance, here’s how you would upload the content of a file:

async Task UploadFileAsync(Uri uri, string filename)
{
    using (var stream = File.OpenRead(filename))
    {
        var client = new HttpClient();
        var response = await client.PostAsync(uri, new StreamContent(stream));
        response.EnsureSuccessStatusCode();
    }
}

As you may have noticed, nowhere in this code do we write to the request stream explicitly: the content is pulled from the source stream.

This “pull” model is fine most of the time, but it has a drawback: it requires that the data to upload already exists in a form that can be sent directly to the server. This is not always practical, because sometimes you want to generate the request content “on the fly”. For instance, if you want to send an object serialized as JSON, with the “pull” approach you first need to serialize it in memory as a string or MemoryStream, then assign that to the request’s content:

async Task UploadJsonObjectAsync<T>(Uri uri, T data)
{
    var client = new HttpClient();
    string json = JsonConvert.SerializeObject(data);
    var response = await client.PostAsync(uri, new StringContent(json));
    response.EnsureSuccessStatusCode();
}

This is fine for small objects, but obviously not optimal for large object graphs…

So, how could we reverse this pull model to a push model? Well, it’s actually pretty simple: all you have to do is to create a class that inherits HttpContent, and override the SerializeToStreamAsync method to write to the request stream directly. Actually, I intended to blog about my own implementation, but then I did some research, and it turns out that Microsoft has already done the work: the Web API 2 Client library provides a PushStreamContent class that does exactly that. Basically, you just pass a delegate that defines what to do with the request stream. Here’s how it works:

async Task UploadJsonObjectAsync<T>(Uri uri, T data)
{
    var client = new HttpClient();
    var content = new PushStreamContent((stream, httpContent, transportContext) =>
    {
        var serializer = new JsonSerializer();
        using (var writer = new StreamWriter(stream))
        {
            serializer.Serialize(writer, data);
        }
    });
    var response = await client.PostAsync(uri, content);
    response.EnsureSuccessStatusCode();
}

Note that the PushStreamContent class also provides a constructor overload that accepts an asynchronous delegate, if you want to write to the stream asynchronously.

Actually, for this specific use case, the Web API 2 Client library provides a less convoluted approach: the ObjectContent class. You just pass it the object to send and a MediaTypeFormatter, and it takes care of serializing the object to the request stream:

async Task UploadJsonObjectAsync<T>(Uri uri, T data)
{
    var client = new HttpClient();
    var content = new ObjectContent<T>(data, new JsonMediaTypeFormatter());
    var response = await client.PostAsync(uri, content);
    response.EnsureSuccessStatusCode();
}

By default, the JsonMediaTypeFormatter class uses Json.NET as its JSON serializer, but there is an option to use DataContractJsonSerializer instead.

Note that if you need to read an object from the response content, this is even easier: just use the ReadAsAsync<T> extension method (also in the Web API 2 Client library). So as you can see, HttpClient makes it very easy to consume REST APIs.