Tag Archives: nunit

Async unit tests with NUnit

Recently, my team and I started writing unit tests on an application that uses a lot of async code. We used NUnit (2.6) because we were already familiar with it, but we had never tried it on async code yet.

Let’s assume the system under test is this very interesting Calculator class:

    public class Calculator
    {
        public async Task<int> AddAsync(int x, int y)
        {
            // simulate long calculation
            await Task.Delay(100).ConfigureAwait(false);
            // the answer to life, the universe and everything.
            return 42;
        }
    }

(Hint: this code has a bug… 42 isn’t always the answer. This came to me as a shock!)

And here’s a unit test for the AddAsync method:

        [Test]
        public async void AddAsync_Returns_The_Sum_Of_X_And_Y()
        {
            var calculator = new Calculator();
            int result = await calculator.AddAsync(1, 1);
            Assert.AreEqual(2, result);
        }

async void vs. async Task

Even before trying to run this test, I thought to myself: This isn’t gonna work! an async void method will return immediately on the first await, so NUnit will think the test is complete before the assertion is executed, and the test will always pass even if the assertion fails. So I changed the method signature to async Task instead, thinking myself very clever for having avoided this trap…

        [Test]
        public async Task AddAsync_Returns_The_Sum_Of_X_And_Y()

As expected, the test failed, confirming that NUnit knew how to handle async tests. I fixed the Calculator class, and stopped thinking about it. Until one day, I noticed that my colleague was writing test methods with async void. So I started to explain to him why it couldn’t work, and tried to demonstrate it by introducing an assertion that would fail… and to my surprise, the test failed, proving that I was wrong. Mind blown!

Having an inquisitive mind, I immediately started to investigate… My first idea was to check the current SynchronizationContext, and indeed I saw that NUnit had changed it to an instance of NUnit.Framework.AsyncSynchronizationContext. This class maintains a queue of all the continuations that are posted to it. After the async void test method has returned (i.e., the first time a not-yet-completed task is awaited), NUnit calls the WaitForPendingOperationsToComplete method, which executes all the continuations in the queue, until the queue is empty. Only then is the test considered complete.

So, the moral of the story is: you can write async void unit tests in NUnit 2.6. It also works for delegates passed to Assert.Throws, which can have an async modified. Now, just because you can doesn’t mean you should. Not all test frameworks seem to have the same support for this. The next version of NUnit (3.0, still in alpha) will not support async void tests.

So, unless you plan on staying with NUnit 2.6.4 forever, it’s probably better to always use async Task in your unit tests.

Easy unit testing of null argument validation

When unit testing a method, one of the things to test is argument validation : for instance, ensure that the method throws a ArgumentNullException when a null argument is passed for a parameter that isn’t allowed to be null. Writing this kind of test is very easy, but it’s also a tedious and repetitive task, especially if the method has many parameters… So I wrote a method that automates part of this task: it tries to pass null for each of the specified arguments, and asserts that the method throws an ArgumentNullException. Here’s an example that tests a FullOuterJoin extension method:

[Test]
public void FullOuterJoin_Throws_If_Argument_Null()
{
    var left = Enumerable.Empty<int>();
    var right = Enumerable.Empty<int>();
    TestHelper.AssertThrowsWhenArgumentNull(
        () => left.FullOuterJoin(right, x => x, y => y, (k, x, y) => 0, 0, 0, null),
        "left", "right", "leftKeySelector", "rightKeySelector", "resultSelector");
}

The first parameter is a lambda expression that represents how to call the method. In this lambda, you should only pass valid arguments. The following parameters are the names of the parameters that are not allowed to be null. For each of the specified names, AssertThrowsWhenArgumentNull will replace the corresponding argument with null in the provided lambda, compile and invoke the lambda, and assert that the method throws a ArgumentNullException.

Using this method, instead of writing a test for each of the arguments that are not allowed to be null, you only need one test.

Here’s the code for the TestHelper.AssertThrowsWhenArgumentNull method (you can also find it on Gist):

using System;
using System.Linq;
using System.Linq.Expressions;
using NUnit.Framework;

namespace MyLibrary.Tests
{
    static class TestHelper
    {
        public static void AssertThrowsWhenArgumentNull(Expression<TestDelegate> expr, params string[] paramNames)
        {
            var realCall = expr.Body as MethodCallExpression;
            if (realCall == null)
                throw new ArgumentException("Expression body is not a method call", "expr");

            var realArgs = realCall.Arguments;
            var paramIndexes = realCall.Method.GetParameters()
                .Select((p, i) => new { p, i })
                .ToDictionary(x => x.p.Name, x => x.i);
            var paramTypes = realCall.Method.GetParameters()
                .ToDictionary(p => p.Name, p => p.ParameterType);
            
            

            foreach (var paramName in paramNames)
            {
                var args = realArgs.ToArray();
                args[paramIndexes[paramName]] = Expression.Constant(null, paramTypes[paramName]);
                var call = Expression.Call(realCall.Method, args);
                var lambda = Expression.Lambda<TestDelegate>(call);
                var action = lambda.Compile();
                var ex = Assert.Throws<ArgumentNullException>(action, "Expected ArgumentNullException for parameter '{0}', but none was thrown.", paramName);
                Assert.AreEqual(paramName, ex.ParamName);
            }
        }

    }
}

Note that it is written for NUnit, but can easily be adapted to other unit test frameworks.

I used this method in my Linq.Extras library, which provides many additional extension methods for working with sequences and collections (including the FullOuterJoin method mentioned above).