Tag Archives: unit testing

What’s new in FakeItEasy 3.0.0?

FakeItEasy is a popular mocking framework for .NET, with an very intuitive and easy-to-use API. For about one year, I’ve been a maintainer of FakeItEasy, along with Adam Ralph and Blair Conrad. It’s been a real pleasure working with them and I had a lot of fun!

Today I’m glad to announce that we’re releasing FakeItEasy 3.0.0, which supports .NET Core and introduces a few useful features.

Let’s see what’s new!

.NET Core support

In addition to .NET 4+, FakeItEasy now supports .NET Standard 1.6, so you can use it in .NET Core projects.

Note that due to limitations in .NET Standard 1.x, there are some minor differences with the .NET 4 version:

  • Fakes are not binary serializable;
  • Self-initializing fakes are not supported (i.e. fakeService = A.Fake<IService>(options => options.Wrapping(realService).RecordedBy(recorder)).

Huge thanks to the people who made .NET Core support possible:

  • Jonathon Rossi, who maintains Castle.Core. FakeItEasy relies heavily on Castle.Core, so it couldn’t have supported .NET Core if Castle.Core didn’t.
  • Jeremy Meng from Microsoft, who did most of the heavy lifting to make both FakeItEasy 3.0.0 and Castle.Core 4.0.0 work on .NET Core.

Analyzer

VB.NET support

The FakeItEasy analyzer, which warns you of incorrect usage of the library, now supports VB.NET as well as C#.

New or improved features

Better syntax for configuring successive calls to the same member

When you configure calls on a fake, it creates rules that are "stacked" on each other, which means you can override a previously configured rule. Combined with the ability to specify the number of times a rule must apply, this lets you say things like "return 42 twice, then throw an exception". Until now, to do that you had to configure the calls in reverse order, which wasn’t very intuitive and meant you had to repeat the call specification:

A.CallTo(() => foo.Bar()).Throws(new Exception("oops"));
A.CallTo(() => foo.Bar()).Returns(42).Twice();

FakeItEasy 3.0.0 introduces a new syntax to make this easier:

A.CallTo(() => foo.Bar()).Returns(42).Twice()
    .Then.Throws(new Exception("oops"));

Note that if you don’t specify how many times the rule must apply, it will apply forever until explicitly overridden. Hence, you can only use Then after Once(), Twice() or NumberOfTimes(...).

This is a breaking change at the API level, as the shape of the configuration interfaces has changed, but unless you manipulate those interfaces explicitly, you shouldn’t be affected.

Automatic support for cancellation

When a method accepts a CancellationToken, it should usually throw an exception when it’s called with a token that is already canceled. Previously this behavior had to be configured manually. In FakeItEasy 3.0.0, fake methods will now throw an OperationCanceledException by default when called with a canceled token. Asynchronous methods will return a canceled task.

This is technically a breaking change, but most users are unlikely to be affected.

Throw asynchronously

FakeItEasy lets you configure a method to throw an exception with Throws. But for async methods, there are actually two ways to "throw":

  • throw an exception synchronously, before actually returning a task (this is what Throws does)
  • return a failed task (which had to be done manually until now)

In some cases the difference can be important to the caller, if it doesn’t directly await the async method. FakeItEasy 3.0.0 introduces a ThrowsAsync method to configure a method to return a failed task:

A.CallTo(() => foo.BarAsync()).ThrowsAsync(new Exception("foo"));

Configure property setters on unnatural fakes

Unnatural fakes (i.e. Fake<T>) now have a CallsToSet method, which does the same as A.CallToSet on natural fakes:

var fake = new Fake<IFoo>();
fake.CallsToSet(foo => foo.Bar).To(0).Throws(new Exception("The value of Bar can't be 0"));

Better API for specifying additional attributes

The syntax to specify additional attributes on fakes was a bit unwieldy; you had to create a collection of CustomAttributeBuilders, which themselves had to be created by specifying the constructor and argument values. The WithAdditionalAttributes method has been retired in FakeItEasy 3.0.0 and replaced with a simpler WithAttributes that accepts expressions:

var foo = A.Fake<IFoo>(x => x.WithAttributes(() => new FooAttribute()));

This is a breaking change.

Other notable changes

Deprecation of self-initializing fakes

Self-initializing fakes are a feature that lets you record the results of calls to a real object, and replay them on a fake object. This feature was used by very few people, and didn’t seem to be a good fit in the core FakeItEasy library, so it will be removed in a future version. We’re considering providing the same functionality as a separate package.

Bug fixes

  • Faking a type multiple times and applying different attributes to the fakes now correctly generates different fake types. (#436)
  • All non-void members attempt to return a Dummy by default, even after being reconfigured by Invokes or DoesNothing (#830)

The full list of changes for this release is available in the release notes.

Other contributors to this release include:

A big thanks to them!

Automatically inject fakes in test fixture with FakeItEasy

Today I’d like to share a nice feature I discovered recently in FakeItEasy.

When you write unit tests for a class that takes dependencies, you typically need to create fake/mock dependencies and manually inject them into the SUT (System Under Test), or use a DI container to register the fake dependencies and construct the SUT. This is a bit tedious, and a few months ago I came up with an auto-mocking Unity extension to make it easier. Now I just realized that FakeItEasy offers an even better solution: just declare the dependencies and SUT as fields or properties in your test fixture, and call Fake.InitializeFixture on the fixture to initialize them. Here’s how it looks:

    public class BlogManagerTests
    {
        [Fake] public IBlogPostRepository BlogPostRepository { get; set; }
        [Fake] public ISocialNetworkNotifier SocialNetworkNotifier { get; set; }
        [Fake] public ITimeService TimeService { get; set; }

        [UnderTest] public BlogManager BlogManager { get; set; }

        public BlogManagerTests()
        {
            Fake.InitializeFixture(this);
        }

        [Fact]
        public void NewPost_should_add_blog_post_to_repository()
        {
            var post = A.Dummy<BlogPost>();

            BlogManager.NewPost(post);

            A.CallTo(() => BlogPostRepository.Add(post)).MustHaveHappened();
        }

        [Fact]
        public void PublishPost_should_update_post_in_repository_and_publish_link_to_social_networks()
        {
            var publishDate = DateTimeOffset.Now;
            A.CallTo(() => TimeService.Now).Returns(publishDate);

            var post = A.Dummy<BlogPost>();

            BlogManager.PublishPost(post);

            Assert.Equal(BlogPostStatus.Published, post.Status);
            Assert.Equal(publishDate, post.PublishDate);

            A.CallTo(() => BlogPostRepository.Update(post)).MustHaveHappened();
            A.CallTo(() => SocialNetworkNotifier.PublishLink(post)).MustHaveHappened();
        }
    }

The SUT is declared as a property marked with the [UnderTest] attribute. Each dependency that you need to manipulate is declared as a property marked with the [Fake] attribute. Fake.InitializeFixture then initializes the SUT, creating fake dependencies on the fly, and assigns those dependencies to the corresponding properties.

I really like how clean the tests look with this technique; the boilerplate code is reduced to a minimum, all you have to do is configure the dependencies where necessary and get on with your tests.

Two remarks about the code above:

  • You can use private fields instead of public properties for the fakes and SUT, but since the fields are never explicitly set in code, it raises a compiler warning (CS0649), so I prefer to use properties.
  • The tests in my example use xUnit, so I put the call to Fake.InitializeFixture in the fixture constructor, but if you use another test framework like NUnit or MSTests, you would typically put it in the setup method.

Also, note that there are limits to the scenarios supported by this approach:

  • only constructor injection is supported, not property injection (i.e. the dependencies must be constructor parameters of the SUT)
  • named dependencies are not supported; only the type is taken into account, so you can’t have multiple distinct dependencies with the same type
  • dependency collections are not supported (i.e. if your class receives a collection of dependencies, e.g. IFooService[])

Create an auto-mocking container with Unity and FakeItEasy

Unit testing can be tedious sometimes, especially when testing classes that have complex dependencies. Fortunately, some tools make it somewhat easier. I’ve been using FakeItEasy a lot recently; it’s a very easy to use mocking framework for .NET. It has a very lean and simple API based on generics and lambda expressions, and is a real pleasure to work with. It came as a breath of fresh air compared to the old RhinoMocks I had been using before.

But as nice as FakeItEasy is, the process of registering all the fake dependencies for the class you are testing is still a bit tedious. Wouldn’t it be nice if the IoC container could automatically create the fakes on demand ? So this code:

var container = new UnityContainer();

// Setup dependencies
var fooProvider = A.Fake<IFooProvider>();
container.RegisterInstance(fooProvider);
var barService = A.Fake<IBarService>();
container.RegisterInstance(barService);
var bazManager = A.Fake<IBazManager>();
container.RegisterInstance(bazManager);

var sut = container.Resolve<SystemUnderTest>();

Could be reduced to this:

var container = new UnityContainer();

// This will cause the container to provide fakes for all dependencies
container.AddNewExtension<AutoFakeExtension>();

var sut = container.Resolve<SystemUnderTest>();

Well, it’s actually pretty easy to do with Unity. Unity is usually not considered the “cool kid” in the small world of IoC containers, but it’s well supported, easy to use, and extensible. I came up with the following extension to enable the above scenario:

public class AutoFakeExtension : UnityContainerExtension
{
    protected override void Initialize()
    {
        Context.Strategies.AddNew<AutoFakeBuilderStrategy>(UnityBuildStage.PreCreation);
    }
    
    private class AutoFakeBuilderStrategy : BuilderStrategy
    {
        private static readonly MethodInfo _fakeGenericDefinition;
    
        static AutoFakeBuilderStrategy()
        {
            _fakeGenericDefinition = typeof(A).GetMethod("Fake", Type.EmptyTypes);
        }
        
        public override void PreBuildUp(IBuilderContext context)
        {
            if (context.Existing == null)
            {
                var type = context.BuildKey.Type;
                if (type.IsInterface || type.IsAbstract)
                {
                    var fakeMethod = _fakeGenericDefinition.MakeGenericMethod(type);
                    var fake = fakeMethod.Invoke(null, new object[0]);
                    context.PersistentPolicies.Set<ILifetimePolicy>(new ContainerControlledLifetimeManager(), context.BuildKey);
                    context.Existing = fake;
                    context.BuildComplete = true;
                }
            }
            base.PreBuildUp(context);
        }
    }
}

A few comments on this code:

  • The logic is a bit crude (it generates fakes only for interfaces and abstract classes), but can easily be adjusted if necessary.
  • The ugly reflection hack is due to the fact that FakeItEasy doesn’t have an non-generic overload of A.Fake that accepts a Type as a parameter. Well, nobody’s perfect…
  • The lifetime is set to “container controlled”, because if you need to configure method calls, you will need to access the same instance that is injected into the system under test:
var fooProvider = container.Resolve<IFooProvider>();
A.CallTo(() => fooProvider.GetFoo(42)).Returns(new Foo { Id = 42, Name = “test” });

Note that if you register a dependency explicitly, it will take precedence and no fake will be created. So you can use this extension and still be able to manually specify a dependency:

var container = new UnityContainer();
container.AddNewExtension<AutoFakeExtension>();
container.RegisterType<IFooProvider, TestFooProvider>();
var sut = container.Resolve<SystemUnderTest>();

Of course, this extension could easily be modified to use a different mocking framework. I guess the same principle could be applied to other IoC containers as well, as long as they have suitable extension points.

What about AutoFixture?

Before you ask: yes, I know about AutoFixture. It’s a pretty good library, and I actually tried to use it as well, with some success. The resulting code is very similar to the examples above. The main reason why I didn’t keep using it is that AutoFixture is not really an IoC container (though it does some of the things an IoC container does), and I prefer to use the same IoC mechanism in my unit tests and in the actual application. Also, I’m not very comfortable with the way it handles properties when creating an instance of the SUT. By default, it sets all public writable properties to dummy instances of their type; this is fine if those properties are used for dependency injection, but IMO it doesn’t make sense for other properties. I know I can suppress this behavior for specific properties, but I have to do it manually on a case by case basis; it doesn’t take IoC container specific attributes like [Dependency] into account. So eventually I found it easier to use my custom Unity extension.

Async unit tests with NUnit

Recently, my team and I started writing unit tests on an application that uses a lot of async code. We used NUnit (2.6) because we were already familiar with it, but we had never tried it on async code yet.

Let’s assume the system under test is this very interesting Calculator class:

    public class Calculator
    {
        public async Task<int> AddAsync(int x, int y)
        {
            // simulate long calculation
            await Task.Delay(100).ConfigureAwait(false);
            // the answer to life, the universe and everything.
            return 42;
        }
    }

(Hint: this code has a bug… 42 isn’t always the answer. This came to me as a shock!)

And here’s a unit test for the AddAsync method:

        [Test]
        public async void AddAsync_Returns_The_Sum_Of_X_And_Y()
        {
            var calculator = new Calculator();
            int result = await calculator.AddAsync(1, 1);
            Assert.AreEqual(2, result);
        }

async void vs. async Task

Even before trying to run this test, I thought to myself: This isn’t gonna work! an async void method will return immediately on the first await, so NUnit will think the test is complete before the assertion is executed, and the test will always pass even if the assertion fails. So I changed the method signature to async Task instead, thinking myself very clever for having avoided this trap…

        [Test]
        public async Task AddAsync_Returns_The_Sum_Of_X_And_Y()

As expected, the test failed, confirming that NUnit knew how to handle async tests. I fixed the Calculator class, and stopped thinking about it. Until one day, I noticed that my colleague was writing test methods with async void. So I started to explain to him why it couldn’t work, and tried to demonstrate it by introducing an assertion that would fail… and to my surprise, the test failed, proving that I was wrong. Mind blown!

Having an inquisitive mind, I immediately started to investigate… My first idea was to check the current SynchronizationContext, and indeed I saw that NUnit had changed it to an instance of NUnit.Framework.AsyncSynchronizationContext. This class maintains a queue of all the continuations that are posted to it. After the async void test method has returned (i.e., the first time a not-yet-completed task is awaited), NUnit calls the WaitForPendingOperationsToComplete method, which executes all the continuations in the queue, until the queue is empty. Only then is the test considered complete.

So, the moral of the story is: you can write async void unit tests in NUnit 2.6. It also works for delegates passed to Assert.Throws, which can have an async modified. Now, just because you can doesn’t mean you should. Not all test frameworks seem to have the same support for this. The next version of NUnit (3.0, still in alpha) will not support async void tests.

So, unless you plan on staying with NUnit 2.6.4 forever, it’s probably better to always use async Task in your unit tests.

Easy unit testing of null argument validation

When unit testing a method, one of the things to test is argument validation : for instance, ensure that the method throws a ArgumentNullException when a null argument is passed for a parameter that isn’t allowed to be null. Writing this kind of test is very easy, but it’s also a tedious and repetitive task, especially if the method has many parameters… So I wrote a method that automates part of this task: it tries to pass null for each of the specified arguments, and asserts that the method throws an ArgumentNullException. Here’s an example that tests a FullOuterJoin extension method:

[Test]
public void FullOuterJoin_Throws_If_Argument_Null()
{
    var left = Enumerable.Empty<int>();
    var right = Enumerable.Empty<int>();
    TestHelper.AssertThrowsWhenArgumentNull(
        () => left.FullOuterJoin(right, x => x, y => y, (k, x, y) => 0, 0, 0, null),
        "left", "right", "leftKeySelector", "rightKeySelector", "resultSelector");
}

The first parameter is a lambda expression that represents how to call the method. In this lambda, you should only pass valid arguments. The following parameters are the names of the parameters that are not allowed to be null. For each of the specified names, AssertThrowsWhenArgumentNull will replace the corresponding argument with null in the provided lambda, compile and invoke the lambda, and assert that the method throws a ArgumentNullException.

Using this method, instead of writing a test for each of the arguments that are not allowed to be null, you only need one test.

Here’s the code for the TestHelper.AssertThrowsWhenArgumentNull method (you can also find it on Gist):

using System;
using System.Linq;
using System.Linq.Expressions;
using NUnit.Framework;

namespace MyLibrary.Tests
{
    static class TestHelper
    {
        public static void AssertThrowsWhenArgumentNull(Expression<TestDelegate> expr, params string[] paramNames)
        {
            var realCall = expr.Body as MethodCallExpression;
            if (realCall == null)
                throw new ArgumentException("Expression body is not a method call", "expr");

            var realArgs = realCall.Arguments;
            var paramIndexes = realCall.Method.GetParameters()
                .Select((p, i) => new { p, i })
                .ToDictionary(x => x.p.Name, x => x.i);
            var paramTypes = realCall.Method.GetParameters()
                .ToDictionary(p => p.Name, p => p.ParameterType);
            
            

            foreach (var paramName in paramNames)
            {
                var args = realArgs.ToArray();
                args[paramIndexes[paramName]] = Expression.Constant(null, paramTypes[paramName]);
                var call = Expression.Call(realCall.Method, args);
                var lambda = Expression.Lambda<TestDelegate>(call);
                var action = lambda.Compile();
                var ex = Assert.Throws<ArgumentNullException>(action, "Expected ArgumentNullException for parameter '{0}', but none was thrown.", paramName);
                Assert.AreEqual(paramName, ex.ParamName);
            }
        }

    }
}

Note that it is written for NUnit, but can easily be adapted to other unit test frameworks.

I used this method in my Linq.Extras library, which provides many additional extension methods for working with sequences and collections (including the FullOuterJoin method mentioned above).