Tag Archives: .net core

Hosting an ASP.NET Core 2 application on a Raspberry Pi

As you probably know, .NET Core runs on many platforms: Windows, macOS, and many UNIX/Linux variants, whether on x86/x64 architectures or on ARM. This enables a wide range of interesting scenarios… For instance, is a very small machine like a Raspberry Pi, which its low performance ARM processor and small amount of RAM (1 GB on my RPi 2 Model B), enough to host an ASP.NET Core web app? Yes it is! At least as long as you don’t expect it to handle a very heavy load. So let’s see in practice how to deploy an expose an ASP.NET Core web app on a Raspberry Pi.

Creating the app

Let’s start from a basic ASP.NET Core 2.0 MVC app template:

dotnet new mvc

You don’t even need to open the project for now, just compile it as is and publish it for the Raspberry Pi:

dotnet publish -c Release -r linux-arm

Prerequisites

We’re going to use a Raspberry Pi running Raspbian, the official Linux distro for Raspberry Pi, which is based on Debian. To run a .NET Core 2.0 app, you’ll need version Jessie or higher (I used Raspbian Stretch Lite). Update: as Tomasz mentioned in the comments, you also need a Raspberry Pi 2 or more recent, with an ARMv7 processor; The first RPi has an ARMv6 processor and cannot run .NET Core.

Even though the app is self-contained and doesn’t require .NET Core to be installed on the RPi, you will still need a few low-level dependencies; they are listed here. You can install them using apt-get:

sudo apt-get update
sudo apt-get install curl libunwind8 gettext apt-transport-https

Deploy and run the application

Copy all files from the bin\Release\netcoreapp2.0\linux-arm\publish directory to the Raspberry Pi, and make the binary executable (replace MyWebApp with the name of your app):

chmod 755 ./MyWebApp

Run the app:

./MyWebApp

If nothing went wrong, the app should start listening on port 5000. But since it listens only on localhost, it’s only accessible from the Raspberry Pi itself…

Exposing the app on the network

There are several ways to fix that. The easiest is to set the ASPNETCORE_URLS environment variable to a value like http://*:5000/, in order to listen on all addresses. But if you intend to expose the app on the Internet, it might not be a good idea: the Kestrel server used by ASP.NET Core isn’t designed to be exposed directly to the outside world, and isn’t well protected against attacks. It is strongly recommended to put it behind a reverse proxy, such as nginx. Let’s see how to do that.

First, you need to install nginx if it’s not already there, using this command:

sudo apt-get install nginx

And start it like this:

sudo service nginx start

Now you need to configure it so that requests arriving to port 80 are passed to your app on port 5000. To do that, open the /etc/nginx/sites-available/default file in your favorite editor (I use vim because my RPi has no graphical environment). The default configuration defines only one server, listening on port 80. Under this server, look for the section starting with location /: this is the configuration for the root path on this server. Replace it with the following configuration:

location / {
        proxy_pass http://localhost:5000/;
        proxy_http_version 1.1;
        proxy_set_header Connection keep-alive;
}

Be careful to include the final slash in the destination URL.

This configuration is intentionnally minimal, we’ll expand it a bit later.

Once you’re done editing the file, tell nginx to reload its configuration:

sudo nginx -s reload

From your PC, try to access the app on the Raspberry Pi by entering its IP address in your browser. If you did everything right, you should see the familiar home page from the ASP.NET Core app template!

Note that you’ll need to be patient: the first time the home page is loaded, its Razor view is compiled, which can take a while on the RPi’s low-end hardware. ASP.NET Core 2.0 doesn’t support precompilation of Razor views for self-contained apps; this is fixed in 2.1, which is currently in preview. So for now you have 3 options:

  • be patient and endure the delay on first page load
  • migrate to ASP.NET Core 2.1 preview, as explained here
  • make a non self-contained deployment, which requires .NET Core to be installed on the RPi

For this article, I chose the first options to keep things simple.

Proxy headers

At this point, we could just leave the app alone and call it a day. However, if your app is going to evolve into something more useful, there are a few things that aren’t going to work correctly in the current state. The problem is that the app isn’t aware that it’s behind a reverse proxy; as far as it knows, it’s only listening to requests on localhost on port 5000. Which means it cannot know:

  • the actual client IP (requests seem to come from localhost)
  • the protocol scheme used by the client
  • the actual host name specified by the client

For the app to know these things, it has to be told by the reverse proxy. Let’s change the nginx configuration so that it adds a few headers to incoming requests. These headers are not standard, but they’re widely used by proxy servers.

    proxy_set_header X-Forwarded-For    $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Host   $http_host;
    proxy_set_header X-Forwarded-Proto  http;

X-Forwarded-For contains the client IP address, and optionally the addresses of proxies along the way. X-Forwarded-Host contains the host name initially specified by the client, and X-Forwarded-Proto contains the original protocol scheme (hard-coded to http here since HTTPS is not configured).

(Don’t forget to reload the nginx configuration)

We also need to change the ASP.NET Core app so that it takes these headers into account. This can be done easily using the ForwardedHeaders middleware; add this code at the start of the Startup.Configure method:

app.UseForwardedHeaders(new ForwardedHeadersOptions
{
    ForwardedHeaders = ForwardedHeaders.All
});

In case you’re wondering what a middleware is, this article might help!

This middleware will read the X-Forwarded-* headers from incoming requests, and use them to modify:

  • the Host and Scheme of the request
  • the Connection.RemoteIpAddress, which contains the client IP.

This way, the app will behave as if the request was received directly from the client.

Expose the app on a specific path

Our app is now accessible at the URL http://<ip-address>/, i.e. at the root of the server. But if we want to host several applications on the Raspberry Pi, it’s going to be a problem… We could put each app on a different port, but it’s not very convenient. It would be better to have each app on its own path, e.g. with URLs like http://<ip-address>/MyWebApp/.

It’s pretty easy to do with nginx. Edit the nginx configuration again, and replace location / with location /MyWebApp/ (note the final slash, it’s important). Reload the configuration, and try to access the app at its new URL… The home page loads, but the CSS and JS scripts don’t: error 404. In addition, links to other pages are now incorrect, and point to http://<ip-address>/Something instead of http://<ip-address>/MyWebApp/Something. What’s going on?

The reason is quite simple: the app isn’t aware that it’s not served from the root of the server, and generates all its links as if it were… To fix this, we can ask nginx to pass yet another header to our app:

proxy_set_header X-Forwarded-Path   /MyWebApp;

Note that this X-Forwarded-Path header is even less standard than the other ones, since I just made it up… So of course, there’s no built-in ASP.NET Core middleware that can handle it, and we’ll need to do it ourselves. Fortunately it’s pretty easy: we just need to use the path specified in the header as the path base. In the Startup.Configure method, add this after the UseForwardHeaders statement:

// Patch path base with forwarded path
app.Use(async (context, next) =>
{
    var forwardedPath = context.Request.Headers["X-Forwarded-Path"].FirstOrDefault();
    if (!string.IsNullOrEmpty(forwardedPath))
    {
        context.Request.PathBase = forwardedPath;
    }

    await next();
});

Redeploy and restart the app, reload the nginx configuration, and try again: now it works!

Run the app as a service

If we want our app to be always running, restarting it manually every time it crashes or when the Raspberry Pi reboots isn’t going to be sustainable… What we want is to run it as a service, so that it starts when the system starts, and is automatically restarted if it stops working. To do this, we’ll take advantage of systemd, which manages services in most Linux distros, including Raspbian.

To create a systemd service, create a MyWebApp.service file in the /lib/systemd/system/ directory, with the following content:

[Unit]
Description=My ASP.NET Core Web App
After=nginx.service

[Service]
Type=simple
User=pi
WorkingDirectory=/home/pi/apps/MyWebApp
ExecStart=/home/pi/apps/MyWebApp/MyWebApp
Restart=always

[Install]
WantedBy=multi-user.target

(replace the name and paths to match your app of course)

Enable the service like this:

sudo systemctl enable MyWebApp

And start it like this (new services aren’t started automatically):

sudo systemctl start MyWebApp

And that’s it, your app is now monitored by systemd, which will take care of starting or restarting it as needed.

Conclusion

As you can see, running an ASP.NET Core 2.0 app on a Raspberry Pi is not only possible, but reasonably easy too; you just need a bit of fiddling with headers and reverse proxy settings. You won’t host the next Facebook or StackOverflow on your RPi, but it’s fine for small utility applications. Just give free rein to your imagination!

Understanding the ASP.NET Core middleware pipeline

Middlewhat?

The ASP.NET Core architecture features a system of middleware, which are pieces of code that handle requests and responses. Middleware are chained to each other to form a pipeline. Incoming requests are passed through the pipeline, where each middleware has a chance to do something with them before passing them to the next middleware. Outgoing responses are also passed through the pipeline, in reverse order. If this sounds very abstract, the following schema from the official ASP.NET Core documentation should help you understand:

Middleware pipeline

Middleware can do all sort of things, such as handling authentication, errors, static files, etc… MVC in ASP.NET Core is also implemented as a middleware.

Configuring the pipeline

You typically configure the ASP.NET pipeline in the Configure method of your Startup class, by calling Use* methods on the IApplicationBuilder. Here’s an example straight from the docs:

public void Configure(IApplicationBuilder app)
{
    app.UseExceptionHandler("/Home/Error");
    app.UseStaticFiles();
    app.UseAuthentication();
    app.UseMvcWithDefaultRoute();
}

Each Use* method adds a middleware to the pipeline. The order in which they’re added determines the order in which requests will traverse them. So an incoming request will first traverse the exception handler middleware, then the static files middleware, then the authentication middleware, and will eventually be handled by the MVC middleware.

The Use* methods in this example are actually just "shortcuts" to make it easier to build the pipeline. Behind the scenes, they all end up using (directly or indirectly) these low-level primitives: Use and Run. Both add a middleware to the pipeline, the difference is that Run adds a terminal middleware, i.e. a middleware that is the last in the pipeline.

A basic pipeline with no branches

Let’s look at a simple example, using only the Use and Run primitives:

public void Configure(IApplicationBuilder app)
{
    // Middleware A
    app.Use(async (context, next) =>
    {
        Console.WriteLine("A (before)");
        await next();
        Console.WriteLine("A (after)");
    });

    // Middleware B
    app.Use(async (context, next) =>
    {
        Console.WriteLine("B (before)");
        await next();
        Console.WriteLine("B (after)");
    });

    // Middleware C (terminal)
    app.Run(async context =>
    {
        Console.WriteLine("C");
        await context.Response.WriteAsync("Hello world");
    });
}

Here, each middleware is defined inline as an anonymous method; they could also be defined as full-blown classes, but for this example I picked the more concise option. Non-terminal middleware take two arguments: the HttpContext and a delegate to call the next middleware. Terminal middleware only take the HttpContext. Here we have two middleware A and B that just log to the console, and a terminal middleware C which writes the response. Here’s the console output when we send a request to our app:

A (before)
B (before)
C
B (after)
A (after)

We can see that each middleware was traversed in the order in which it was added, then traversed again in reverse order. The pipeline can be represented like this:

Basic pipeline

Short-circuiting middleware

A middleware doesn’t necessarily have to call the next middleware. For instance, if the static files middleware can handle a request, it doesn’t need to pass it down to the rest of the pipeline, it can respond immediately. This behavior is called short-circuiting the pipeline.

In the previous example, if we comment out the call to next() in middleware B, we get the following output:

A (before)
B (before)
B (after)
A (after)

As you can see, middleware C is never invoked. The pipeline now looks like this:

Short-circuited pipeline

Branching the pipeline

In the previous examples, there was only one "branch" in the pipeline: the middleware coming after A was always B, and the middleware coming after B was always C. But it doesn’t have to be that way. You might want a given request to be processed by a completely different pipeline, based on the path or anything else.

There are two types of branches: branches that rejoin the main pipeline, and branches that don’t.

Making a non-rejoining branch

This can be done using the Map or MapWhen method. Map lets you specify a branch based on the request path. MapWhen gives you more control: you can specify a predicate on the HttpContext to decide whether to branch or not. Let’s look at a simple example using Map:

public void Configure(IApplicationBuilder app)
{
    app.Use(async (context, next) =>
    {
        Console.WriteLine("A (before)");
        await next();
        Console.WriteLine("A (after)");
    });

    app.Map(
        new PathString("/foo"),
        a => a.Use(async (context, next) =>
        {
            Console.WriteLine("B (before)");
            await next();
            Console.WriteLine("B (after)");
        }));

    app.Run(async context =>
    {
        Console.WriteLine("C");
        await context.Response.WriteAsync("Hello world");
    });
}

The first argument for Map is a PathString representing the path prefix of the request. The second argument is a delegate that configures the branch’s pipeline (the a parameter represents the IApplicationBuilder for the branch). The branch defined by the delegate will process the request if its path starts with the specified path prefix.

For a request that doesn’t start with /foo, this code produces the following output:

A (before)
C
A (after)

Middleware B is not invoked, since it’s in the branch and the request doesn’t match the prefix for the branch. But for a request that does start with /foo, we get the following output:

A (before)
B (before)
B (after)
A (after)

Note that this request returns a 404 (Not found) response: this is because the B middleware calls next(), but there’s no next middleware, so it falls back to returning a 404 response. To solve this, we could use Run instead of Use, or just not call next().

The pipeline defined by this code can be represented as follows:

Non-rejoining branch

(I omited the response arrows for clarity)

As you can see, the branch with middleware B doesn’t rejoin the main pipeline, so middleware C isn’t called.

Making a rejoining branch

You can make a branch that rejoins the main pipeline by using the UseWhen method. This method accepts a predicate on the HttpContext to decide whether to branch or not. The branch will rejoin the main pipeline where it left it. Here’s an example similar to the previous one, but with a rejoining branch:

public void Configure(IApplicationBuilder app)
{
    app.Use(async (context, next) =>
    {
        Console.WriteLine("A (before)");
        await next();
        Console.WriteLine("A (after)");
    });

    app.UseWhen(
        context => context.Request.Path.StartsWithSegments(new PathString("/foo")),
        a => a.Use(async (context, next) =>
        {
            Console.WriteLine("B (before)");
            await next();
            Console.WriteLine("B (after)");
        }));

    app.Run(async context =>
    {
        Console.WriteLine("C");
        await context.Response.WriteAsync("Hello world");
    });
}

For a request that doesn’t start with /foo, this code produces the same output as the previous example:

A (before)
C
A (after)

Again, middleware B is not invoked, since it’s in the branch and the request doesn’t match the predicate for the branch. But for a request that does start with /foo, we get the following output:

A (before)
B (before)
C
B (after)
A (after)

We can see that the request passes trough the branch (middleware B), then goes back to the main pipeline, ending with middleware C. This pipeline can be represented like this:

Rejoining branch

Note that there is no Use method that accepts a PathString to specify the path prefix. I’m not sure why it’s not included, but it would be easy to write, using UseWhen:

public static IApplicationBuilder Use(this IApplicationBuilder builder, PathString pathMatch, Action<IApplicationBuilder> configuration)
{
    return builder.UseWhen(
        context => context.Request.Path.StartsWithSegments(new PathString("/foo")),
        configuration);
}

Conclusion

As you can see, the idea behind the middleware pipeline is quite simple, but it’s very powerful. Most of the features baked in ASP.NET Core (authentication, static files, caching, MVC, etc) are implemented as middleware. And of course, it’s easy to write your own!

Common MSBuild properties and items with Directory.Build.props

To be honest, I never really liked MSBuild until recently. The project files generated by Visual Studio were a mess, most of their content was redundant, you had to unload the projects to edit them, it was poorly documented… But with the advent of .NET Core and the new "SDK-style" projects, it’s become much, much better.

MSBuild 15 introduced a pretty cool feature: implicit imports (I don’t know if it’s the official name, but I’ll use it anyway). Basically, you can create a file named Directory.Build.props anywhere in your repo, and it will be automatically imported by any project under the directory containing this file. This makes it very easy to share common properties and items across projects. This feature is described in details in this documentation page.

For instance, if you want to share some metadata across multiple projects, just write a Directory.Build.props file in the parent directory of your projects:

<Project>

  <PropertyGroup>
    <Version>1.2.3</Version>
    <Authors>John Doe</Authors>
  </PropertyGroup>

</Project>

You can also do more interesting things like enabling and configuring StyleCop for all your projects:

<Project>

  <PropertyGroup>
    <!-- Common ruleset shared by all projects -->
    <CodeAnalysisRuleset>$(MSBuildThisFileDirectory)MyRules.ruleset</CodeAnalysisRuleset>
  </PropertyGroup>

  <ItemGroup>
    <!-- Add reference to StyleCop analyzers to all projects  -->
    <PackageReference Include="StyleCop.Analyzers" Version="1.0.2" />
    
    <!-- Common StyleCop configuration -->
    <AdditionalFiles Include="$(MSBuildThisFileDirectory)stylecop.json" />
  </ItemGroup>

</Project>

Note that the $(MSBuildThisFileDirectory) variable refers to the directory containing the current MSBuild file. Another useful variable is $(MSBuildProjectDirectory), which refers to the directory containing the project being built.

MSBuild looks for the Directory.Build.props file starting from the project directory and going up until it finds a matching file, then it stops looking. In some cases you might want to define some properties for all projects in your repo, and add some more properties in a subdirectory. To do this, the "inner" Directory.Build.props file will need to explicitly import the "outer" one:

  • (rootDir)/Directory.build.props:
<Project>

  <!-- Properties common to all projects -->
  <!-- ... -->
  
</Project>
  • (rootDir)/tests/Directory.build.props:
<Project>

  <!-- Import parent Directory.build.props -->
  <Import Project="../Directory.Build.props" />

  <!-- Properties common to all test projects -->
  <!-- ... -->
  
</Project>

The documentation mentions another approach, using the GetPathOfFileAbove function, but it didn’t seem to work when I tried… Anyway, I think using a relative path is easier to get right.

Using implicit imports brings the following benefits:

  • smaller project files, since common properties and items can be factored to common properties files.
  • single point of truth: if all projects reference the same package, the version to reference is defined in a single place; no more inconsistencies!

It also has a drawback: Visual Studio doesn’t care about where a property or item comes from, so if you change a property or a package reference from the IDE (using the project properties pages or NuGet Package Manager), it will be changed in the project file itself, rather than the Directory.Build.props file. The way I see it, it’s not a major issue, because I got into the habit of editing the projects manually rather than using the IDE features, but it might be annoying for some people.

If you want a real-world example of this technique in action, have a look at the FakeItEasy repository, where we use multiple Directory.Build.props files to keep the project files nice and clean.

Note that you can also create a Directory.Build.targets file, following the same principles, to define common build targets.

Linq performance improvements in .NET Core

By now, you’re probably aware that Microsoft released an open-source and cross-platform version of the .NET platform: .NET Core. This means you can now build and run .NET apps on Linux or macOS. This is pretty cool in itself, but it doesn’t end there: .NET Core also brings a lot of improvements to the Base Class Library.

For instance, Linq has been made faster in .NET Core. I made a little benchmark to compare the performance of some common Linq methods, and the results are quite impressive:


The full code for the benchmark can be found here. As with all microbenchmarks, it has to be taken with a grain of salt, but it gives an idea of the improvements.

Some lines in this table are quite surprising. How can Select run 5000 times almost instantly? First, we have to keep in mind that most Linq operators are lazy: they don’t actually do anything until you enumerate the result, so doing something like array.Select(i => i * i) executes in constant time (it just returns a lazy sequence, without consuming the items in array). This is why I included a call to Count() in my benchmark, to make sure the result is enumerated.

Despite this, it runs 5000 times in 413µs… This is possible due to an optimization in the .NET Core implementation of Select and Count. A useful property of Select is that it produces a sequence with the same number of items as the source sequence. In .NET Core, Select takes advantage of this. If the source is an ICollection<T> or an array, it returns a custom enumerable object that keeps track of the number of items. Count can then just retrieve this value and return it, which produces a result in constant time. The full .NET Framework implementation, on the other hand, naively enumerates the sequence produced by Select, which takes much longer.

It’s interesting to note that in this situation, .NET Core will not execute the projection specified in Select, so it’s a breaking change compared to the desktop framework for code that was relying on side effects of this projection. This has been identified as an issue which has already been fixed on the master branch, so the next release of .NET Core will execute the projection on each item.

OrderBy followed by Count() also runs almost instantly… did Microsoft invent a O(1) sorting algorithm? Unfortunately, no… The explanation is the same as for Select: since OrderBy preserves the item count, the information is recorded so that it can be used by Count, and there is no need to actually sort the input sequence.

OK, so these cases were pretty obvious improvements (which will be rolled back anyway, as mentioned above). What about the SelectAndToArray case? In this test, I call ToArray() on the result of Select, to make sure that the projection is actually performed on each item of the source sequence: no cheating this time. Still, the .NET Core version is 68% faster than the full .NET Framework version. The reason has to do with allocations: since the .NET Core implementation knows how many items are in the result of Select, it can directly allocate an array of the correct size. In the .NET Framework, this information is not available, so it starts with a small array, copies items into it until it’s full, then allocates a larger array, copies the previous array into it, copies the next items from the sequence until the array is full, and so on. This causes a lot of allocations and copies, hence the degraded performance. A few years ago, I suggested an optimized version of ToList and ToArray, where you had to specify the size. The .NET Core implementation basically does the same thing, except that you don’t have to pass the size manually, since it’s passed along the Linq method chain.

Where and WhereAndToArray are both about 8% faster on .NET Core 1.1. Looking at the code (.NET 4.6.2, .NET Core), I can’t see any obvious difference that could explain the better performance, so I suspect it’s mostly due to improvements in the runtime. In this case, ToArray doesn’t know the length of the input sequence, since there is no way to predict how many items Where will yield, so it can’t use the same optimization as with Select and has to build the array the slow way.

We already discussed OrderBy + Count(), which wasn’t a fair comparison since the .NET Core implementation didn’t actually sort the sequence. The OrderByAndToArray case is more interesting, because the sort can’t be skipped. And in this case, the .NET Core implementation is slightly slower than the .NET 4.6.2 one. I’m not sure why this is; again, the implementation is very similar, although there has been a bit of refactoring in .NET Core.

So, on the whole, Linq seems generally faster in .NET Core than in .NET 4.6.2, which is very good news. Of course, I only benchmarked a limited numbers of scenarios, but it shows the .NET Core team is working hard to optimize everything they can.