Understanding Azure Bot State

When you chat with a bot built with Azure Bot Service, you will have to maintain user state. Example, the options that the user chose last time, or the usernames of all users in a conversation.

Conversation State Types

Azure Bot Service has 3 different states:

User State
This state is related to the user on a specific channel regardless how many conversation they have. So, if the user is chatting with a bot from Facebook channel and connected to the same bot in another conversation , any values saved in user state will be visible in both chats. This is useful in scenarios where you want to save an info about the user like the username or preference that will be used in all conversation for this user.

Conversation State
This state is scoped to the conversation only but not to each user. If you have 3 users chatting with the bot, any data saved in conversation state will be shared across all users. This is used in cases where the data saved is not related to a specific user such as statistics. If the user connected to another session with the bot, the info saved her will not be visible in the other conversation

Private Conversation State
Here, the framework manages the state for each user in each conversation. Imagine a scenario where the user is submitting a ticket. You want the ticket info to be scoped to this conversation only because they may be submitting another ticket with another conversation.

The state storage saves each data in a key/value store. The keys used for each state type is as below:

  • User Sate: {Activity.ChannelId}/users/{Activity.From.Id}#YourPropertyName
  • Conversation state: {Activity.ChannelId}/conversations/{Activity.Conversation.Id}#YourPropertyName
  • Private conversation: {Activity.ChannelId}/conversations/{Activity.Conversation.Id}/users/{Activity.From.Id}#YourPropertyName

Conversation Storage

By default, the state variables are stored in memory. But you can change that from the startup.cs. In the example below, I have configured it to use Azure Table storage to store the state, this makes it persistent. You can create your own state by inheriting from IStorage

Next, you configure which state types you want to use, in this example, I use the conversation state. To use the user state, you have to create an instance of UserState and add it as a singleton just like conversation state.

public void ConfigureServices(IServiceCollection services)
        {
          //IStorage dataStore = new MemoryStorage(); 
          IStorage dataStore = new Microsoft.Bot.Builder.Azure.AzureBlobStorage("connection string", storageContainer);

          var conversationState = new ConversationState(dataStore);
            services.AddSingleton(conversationState);
        }

State property accessors

https://docs.microsoft.com/en-us/azure/bot-service/v4sdk/media/bot-builder-state.png?view=azure-bot-service-4.0

Now, its time to use the state in our bot. To do that, you have to use a property accessor which allows you to read and update data to the state storage. Note that calling the GetAsync and SetAsync on the state accessor doesn’t actually save the state data in the storage system. You have to call the SaveChangesAsync as you can see from the sequence diagram above.

To understand more about state property accessors and how to use it to read and update data into bot state, go through this link

Advertisements

Create Azure Bot Service from Scratch using Empty Asp.Net Core

To create a new Azure Bot Service Project, you can use the easy way by utilising the Azure Bot Service SDK Templates which you can download here. You will also need the Bot Service SDK which is a list of nuget packages. Current version is 4

Once you downloaded the bot template, installed it, you should find a new project template in visual studio. You can choose a project name, click Ok and you will end up with an echo bot that repeats what you say.

Visual Studio project

But this isn’t fun. In this post, I will create an empty asp.net project and add the needed boilerplate to make it a bot. Lets get started

Open Visual Studio (I am using VS 2019 Preview 2). CLick New Project, select Asp.net Core Web Application, name is HelloAzureBot and Choose Empty as the project template. Then click Ok

Adding reference to Bot Service SDK

To enable bot service in our asp.net core web app, we will need to add reference to the Bot SDK packages. We need the following packages:

  • Microsoft.Bot.Builder
  • Microsoft.Bot.Builder.Integration.AspNet.Core
  • Microsoft.Bot.Configuration
  • Microsoft.Bot.Connector
  • Microsoft.Bot.Schema

You can add it via NuGet Package manager Console, Manage NuGet Packages dialog or you just edit the project file and add the following to it:


<PackageReference Include="Microsoft.Bot.Builder" Version="4.2.2" /&gt;
<PackageReference Include="Microsoft.AspNetCore.Razor.Design" Version="2.2.0" PrivateAssets="All" /&gt;
<PackageReference Include="Microsoft.Bot.Builder.Integration.AspNet.Core" Version="4.2.2" /&gt;
<PackageReference Include="Microsoft.Bot.Configuration" Version="4.2.2" /&gt;
<PackageReference Include="Microsoft.Bot.Connector" Version="4.2.2" /&gt;
<PackageReference Include="Microsoft.Bot.Schema" Version="4.2.2" /&gt;

Your project file should look like this now:

Injecting Bot Service in Asp.Net Core

Now, lets inject the Bot Service into asp.net core middleware. This will basically create an endpoint that will accept post requests sent to /api/messages .
Open startup.cs file and in Configure method, add the following line. Don’t forget to remove the default middleware that print hello world.

app.UseBotFramework();

Now, our solution is ready to create a new Bot.
Create a new class and name it EchoBot. Make it inherits from IBot interface and implement the OnTurnAsync. This is the method that get called when its your bot’s turn to speak to the user.

    public class EchoBot : IBot
    {
        public async Task OnTurnAsync(ITurnContext turnContext, CancellationToken cancellationToken = default(CancellationToken))
        {
            await turnContext.SendActivityAsync($"Hi, You said: {turnContext.Activity.Text}");
        }
    }

This is a very simple bot that does nothing but echo what you say. To send a message to the user, we use the TurnContext.SendActivtyAsync. Note that activity is any thing (message, voice, card ..) going between the user and the bot.

Tell Bot service about your Bot

Now, your project is ready except that it does not know about your Bot.We injected the Bot Service in the asp.net core middleware but we didn’t tell it about the echo bot. Its the same process as configuring MVC in Configure method but we have to also inject it a service in the ConfigureServices method. Now open startup.cs file again and edit the ConfigureServices method to match the following:

        public void ConfigureServices(IServiceCollection services)
        {
            services.AddBot<EchoBot&gt;();
        }

Testing the Echo Bot

The bot is ready now to accept chat requests. Press F5 to run your project. It will open the browser with a new windows that tells you that no page can be found. This is normal as we didn’t inject any middleware to handle the root URL. In the SDK template, it uses static HTML file under wwwroot that has an introduction to the service. But this is not mandatory.

To test the bot, we will need a channel to talk to it. Microsoft released the bot emulator for that purpose. It allows you to connect and chat with your bot without deploying it to Azure Bot Service. The current emulator version is 4. But for some reason, it just does not work with me for local bots. It needs the bot to be deployed to Azure or at least create an Azure Bot Service and configure the Bot URL to be your local bot. You will expose the bot to Azure using ngrok. Another option is to just use Emulator V3 which you can easily provide the Bot URL which takes the format http://localhost:PORT/api/messages.

Bot Emulator

Download the Bot Emulator V3 from here and install it then run it. On the top, add the URL for your Bot in the format mentioned before and hit enter. Start playing around and enter any message and you should receive the reply with: Hi, You said: YOUR_MESSAGE.

You will notice that you will immediately receive the messages: “Hi, You said: and Hi, You said:” just after you entered the Bot URL. This is because the OnTurnAsync method is called when the user joins and when the bot joins or when you send a message. To filter, you have to add the following to your OnTurnAsync method

        public async Task OnTurnAsync(ITurnContext turnContext, CancellationToken cancellationToken = default(CancellationToken))
        {
            if (turnContext.Activity.Type == ActivityTypes.Message)
            {
                await turnContext.SendActivityAsync($"Hi, You said: {turnContext.Activity.Text}");
            }
        }

You can check if the activity type is ActivityTypes.ConversationUpdate and then check the turnContext.Activity.MembersAdded property and use that to send a message to welcome the new members. ex: John has joined the conversation.

Handling Errors

When your bot encounters an issue or unhandled exception, you can catch that exception through an error handler. You can configure that in the ConfigureServices method when you call the AddBot method. The Startup.cs should be as below. We utilise the OnTurnError handler to send a message to the user informing him/her that something went wrong and log the exception.

 public class Startup
    {
        public ILoggerFactory LoggerFactory { get; }

        public Startup( ILoggerFactory loggerFactory)
        {
            this.LoggerFactory = loggerFactory;
        }

        // This method gets called by the runtime. Use this method to add services to the container.
        // For more information on how to configure your application, visit https://go.microsoft.com/fwlink/?LinkID=398940
        public void ConfigureServices(IServiceCollection services)
        {
            services.AddBot<EchoBot&gt;(options =&gt;
            {
                // Creates a logger for the application to use.
                ILogger logger = this.LoggerFactory.CreateLogger<EchoBot&gt;();

                // Catches any errors that occur during a conversation turn and logs them.
                options.OnTurnError = async (context, exception) =&gt;
                {
                    logger.LogError($"Exception caught : {exception}");
                    await context.SendActivityAsync("Sorry, it looks like something went wrong.");
                };
            });
        }

        // This method gets called by the runtime. Use this method to configure the HTTP request pipeline.
        public void Configure(IApplicationBuilder app, IHostingEnvironment env)
        {
            if (env.IsDevelopment())
            {
                app.UseDeveloperExceptionPage();
            }

            app.UseBotFramework();
        }
    }

.Bot File

The emulator that comes with SDK V4 uses a .bot file that describes the bot, the services that the bot uses and the URL where it runs.
A typical .bot file contains the following:

{
  "name": "HelloAzureBot",
  "services": [
    {
      "type": "endpoint",
      "name": "development",
      "endpoint": "https://localhost:44331/api/messages",
      "appId": "",
      "appPassword": "",
      "id": "1"
    }
  ],
  "padlock": "",
  "version": "2.0"
}

The content of the file is self explanatory but the most important details are the name and endpoint. This is what tells the Emulator how to reach your bot.
If the bot is deployed on Azure Bot Service, you will need to provide the appId and appPassword

That will be the end of our post. In the next post, we will talk about the state management. If you didn’t read the previous post, please do.

Azure Bot Service Overview

Today, I started learning Azure BOT Service, I will summarise each section into a blog post.

What is BOT?

A BOT is a modern way of user interaction. Instead of user clicking buttons and typing data in text fields, the user will be chatting or talking to an application that will understand the human language and respond accordingly.
This of course increase the human interaction with your site or application and also decrease the need for employees to handle user requests.
Imagine a user who wants to know about car insurance quote, instead of calling the insurance company, they can chat with the bot or even call it and ask them about the prices and the bot can respond as if it was a human or delegate the call to a human if it is not able to assist. This can save a lot for the company.

How BOT works?

Users interact with the BOT through a channel. A channel can be DirectLine like the case when you chat directly with the BOT from web site. It can be Facebook Messenger, Twitter, or any other popular social media platform.
The chat between users and BOT can be a text messages, a card that display graphic content to the user or a voice message or even an interactive form that ask the user to enter some information like SSN or Policy Number.
BOT can also use AI services such as Azure Congnitive Services to make it smarter.

https://haithamshaddad.files.wordpress.com/2019/02/84064-1xmuakcv_xkoszd4odvrika.png

BOT HTTP Communication

The communication between the BOT and Users/Channels take place through HTTP Post requests. When a user joins a conversation, an HTTP message is posted to the BOT service and the BOT just acknowledge the message with 200 OK response. Then, the BOT joins the conversation and again another HTTP Post message is sent to the channel and the channel replies back with a 200 Ok. These messages are called ConversationUpdate messages.
All these messages between the BOT and the channel are called activities. During the ConversationUpdate activity, you can check the members added or removed.
Once the connection is established, the messages send from the channel to the BOT are sent as an HTTP Post requests and the BOT replies back with a different HTTP Post not via response to the original HTTP Post request. So, as you can see from the image below, the Channel sent a Hi message, the BOT just echoed the message but it can send a different message. Then, the channel confirmed with a 200 Ok to the BOT and finally the BOT replies back to the original hi message with 200 Ok.
With the current version (4) it may not be very clear about the HTTP post messages but if you tried with the older version (3), it was very clear and when you create a new project you can clearly see an API controller called MessagesController and you can access it via /api/messages.

activity diagram
https://docs.microsoft.com/en-us/azure/bot-service/v4sdk/media/bot-builder-activity.png

Defining Rounds

Humans talk one at a time, one talk and the others listen (At least this is what should happen šŸ™‚ ).

activity processing stack
https://docs.microsoft.com/en-us/azure/bot-service/v4sdk/media/bot-builder-activity-processing-stack.png

The important thing to note in the above image is the TurnContext
which holds information about whose turn and what data was sent (Typically JSON) and the middleware which holds some components that execute in order to do something with the activity received. After all middleware components finish execution, it passes the control to the Turn Handler which will execute the application logic that will decide the response that will be sent to the channel/user.
The middleware is executed on the incoming activity received from the channel and also on the outbound activity sent from the bot to the channel itself.

Azure Functions anatomy – Part 2 – What is inside a Function App?

In the previous post (Azure Functions 2.0 Anatomy- Part 1), we saw the structure of a function app. In this part, I will explain the different components of an Azure Function.

Components

Every Azure Function 2.0 consists of the following components:

  • Trigger
  • Bindings (one of which is the trigger). Binding can be input or output
  • Value Converter/Binder
  • Listener

A binding can be a Blob file, a queue message, an event from event hub or event grid or any custom binding developed by anyone. One of these bindings can initiate the function call and in this case it is called a Trigger.
A binding can also be Input or Output. Input binding delivers data to the function and output binding is used by the function to write data.

Now, assume we have the following function and its configuration

        [FunctionName("SampleFunction")]
        public static void Run([QueueTrigger("myqueue-items", Connection = "AzureWebJobsStorage")]string myQueueItem, [Blob("sample-output")]Stream output, TraceWriter log)
        {
            log.Info($"C# Queue trigger function processed: {myQueueItem}");
        }

This function has 2 binding parameters. myQueueItem which is both an input binding and a trigger, and output which is an output binding.

How Azure Functions call this method and convert the queue message to a string and the stream to a Blob file?

Binding, Listener, Value Provider and Converter

When the azure function app starts, it scans all functions that exist in the app directory like we saw in the previous post, and for each function it reads the input, output and trigger from the function config JSON file.

The scanning process uses a descriptor provider which creates the input binding, output binding, trigger and an invoker

Once all functions are loaded, it tried to load the types for each function that was developed outside the portal. It uses the scriptFile and EntryPoint as you can see below. The entry point in the previous code snippet is FunctionApp1.Function1.Run method that exist in the assembly FunctionApp1.dll

{
  "generatedBy": "Microsoft.NET.Sdk.Functions-1.0.14",
  "configurationSource": "attributes",
  "bindings": [
    {
      "type": "timerTrigger",
      "schedule": "0 */5 * * * *",
      "useMonitor": true,
      "runOnStartup": false,
      "name": "myTimer"
    }
  ],
  "disabled": false,
  "scriptFile": "../bin/FunctionApp1.dll",
  "entryPoint": "FunctionApp1.Function2.Run"
}

Each function app can either have a DLL function or a function created by portal. But not both.

Now, lets see what are the other components inside a binding:

  • Listener (IListener): This one has a method named StartAsync, which basically start listening for an event to trigger the function. For Azure Blob trigger, it scans the logs for new or modified files. For Queue trigger, it check for new messages in the specified queue.
  • Value Provider (IValueProvider): Once the listener has the event that should trigger the function, it uses the value provider to fill all input parameters. If the trigger was a blog trigger, then the value will be the file, the metadata, the name and content of the file itself. For a Queue, it will be content of the message and so on and so forth.
  • Value Converter (IAsyncConverter): This will be used to convert the values found in the value provider to all the input parameters configured for the function. In the previous example, StorageQueueMessageToStringConverter will be used to convert the CloudQueueMessage instance to the string parameter named myQueueItem. The same will be used to write data to output parameters, the framework will choose the best converter that matches the Output parameter type (Stream) and the type of variable that will be used to write to stream, ex: String or any Stream Writer.
  • Executor (ITriggerExecutor): Now we know that the function should be triggered, we have the data from input binding and we bound all input parameters, now we need to call the function body. This takes place using an implementation of ITriggerExecutor which can use reflection to call the function or in a very special case, it may get executed as a web hook like the case with HTTP trigger.

In the next post, I will dig more into Azure Function App to explore the extensions and host.

Azure Functions 2.0 anatomy – Part 1

In these series of posts, I will explain the internals of Azure Functions and how it works. By the end of it, you should be able to understand what happens once an Azure Function is deployed till the moment it is triggered and invocation completes.

Is it a Web App?

Yes, every Azure Function App that uses .Net as a Language runs as as.net core web app. When the app starts, it loads all your functions and proxies and set up the routes and listeners for each function.

Web App Structure

Just like any other Azure Web App, it consists of the following folders:

  • data: This folder contains another folder named “functions” which has another folder called extensions. The extensions folder has a list of all extensions installed to your Functions App. Each file has a reference to a nuget package that has the extension implementation.
  • Log Files
  • site: it has the wwwroot which hosts the asp.net core web app and all the functions
Data -> Functions folder structure

But for Azure Functions App, the following folders are added:

.nuget

Contains all nuget packages installed when you install a new extension. A reference for each extension installed exist in a file under: data\functions\extensions. The sample extension file below specify the package :Microsoft.Azure.WebJobs.Extensions.ServiceBus

In Azure Functions 1.0, it was running on .Net 4.6 and the runtime already included all the supported triggers and bindings. But, with version 2.0, the runtime only includes http and timer binding and all the other bindings can be developed and installed separately using extensions. This allows you to develop your own extensions and upload it without the need to wait for a new version of Azure Functions.

{
  "Id": "d3a33dd0-3941-4acf-a303-72773995901d",
  "Status": 1,
  "StartTime": "2019-01-02T02:49:23.5568341+00:00",
  "EndTime": "2019-01-02T02:50:50.1029788+00:00",
  "Error": null,
  "Properties": {
    "id": "Microsoft.Azure.WebJobs.Extensions.ServiceBus",
    "version": "3.0.0"
  }
}

ASP.NET

This folder contains all data protection keys used in the web app

To have a look at this folder structure and explore it yourself, use the following URL: https://%5BFUNCTION_APP_NAME%5D.scm.azurewebsites.net/DebugConsole

wwwroot folder structure

wwwroot folder structure

As you can see, there are 3 folders. Each folder represent a single function. All DLLs for each function exist in the bin folder.

The host.json file by default only has the version number for the function

{
  "version": "2.0"
}

Inside each function, there is a function.json file that has the configuration for this function.

{
  "generatedBy": "Microsoft.NET.Sdk.Functions-1.0.14",
  "configurationSource": "attributes",
  "bindings": [
    {
      "type": "timerTrigger",
      "schedule": "0 */5 * * * *",
      "useMonitor": true,
      "runOnStartup": false,
      "name": "myTimer"
    }
  ],
  "disabled": false,
  "scriptFile": "../bin/FunctionApp1.dll",
  "entryPoint": "FunctionApp1.Function2.Run"
}

The Azure Functions run-time use this file to load the metadata for each function during application startup. The most important properties are the scriptFile which points to the DLL that has the code and the entryPoint that has the exact c# method name that will be executed when the function triggers.
The configurationSource specifies where to read the function configuration such as connection strings for the blob storage or service bus. attributes means it will be retrieved from code.
The binding key lists all bindings for this function. For this sample, it is a timer and it is the trigger, so it has a single binding. For others, it can be bindings that represents input, output and a trigger.

In the next post, I will explain the internals of each function, and how the bindings and triggers works.

How Azure Functions Blob Trigger works

Introduction

Azure functions enable you to quickly build a piece of functionality that can be triggered by an external system such as Azure Blob Storage, Storage Queue, CosmoDB, Event Hub and the list goes on. In this post, I will explain how Azure Blob Trigger works.

Sample Function

If you created a new Azure Function using Visual Studio, you will end up with the following code:

 public static class Function1
    {
        [FunctionName("Function1")]
        public static void Run([BlobTrigger("samples-workitems/{name}", Connection = "")]Stream myBlob, string name, TraceWriter log)
        {
            log.Info($"C# Blob trigger function Processed blob\n Name:{name} \n Size: {myBlob.Length} Bytes");
        }
    }

The secret lies in the attribute [BlobTrigger]. It accepts the path to the storage container which Azure Web Jobs will monitor. As you know, Azure Functions is based on Azure Web Jobs, so the same triggers are used.

How Azure Web Job knows about a file that is being added, updated or removed to a container?

Run the sample app you just created in visual studio, it should work on the local storage emulator. Now, you should be able to invoke the function by creating a blob container called samples-workitems in your development storage account and you should end up with the following structure.

blob container

You can see the container “samples-workitems” which Azure Web Jobs will monitor and invoke the function whenever a file is changed there. But there is also another container named “azure-webjobs-hosts” which is the secret for how Azure monitor the files in that container.

If you opened that container, you will find a folder called “blobreceipts” which has a folder for the function name.

Inside the folder with the function name, there are some other folders with strange names like below

etags

These strange names are the ETag of each blob file added, edited or removed. So, when you change a file in any container, Web Jobs will create a new folder here with the new version ETag for that file and inside this folder you will find the same structure of the file being edited. In our case, there should be a folder called samples-workitems and inside this folder the file that was modified.

When a new file is added, Azure will check ifĀ  its ETag exist in theĀ azure-webjobs-hosts folder and if not, then it will call the Azure Function. This way it will prevent duplicate calls for the same file. This pattern is called blob receipt.

Note that this process depends on Azure Blob Storage Logging which can take up to 10 minutes to write to the containerĀ azure-webjobs-hosts to improve performance. If you need your function to trigger faster then consider using Storage Queue trigger instead.

 

Download Attachments in Single Page App and Asp.Net Core

Introduction

If you have a SPA built with any JavaScript framework and it has an attachment feature, you must hit the part where you need to allow the user to download an attachment and the App is authenticating users using Tokens.

The problem

With normal forms authentication based on cookies the browser will simply send the authentication cookie with each request to your web server without you doing anything. If you have a link that will allow the user to download a file from your server, the browse will automatically send the authentication cookie when the user clicks the link. This makes it so easy for you. But if you are using token based authentication, it is your responsibility to send a token for each request sent to the server via Ajax by using the Authorization header.

Unfortunately, you cannot control the headers sent to the server if the user is opening a link in a new browser window and the user will end up with unauthorized request.

The solution

Download the file using Ajax Request

In this solution, you have to request the endpoint that downloads the file using Ajax Request which will include the authorization header and then get all the file content in a variable and push the content to the user. This works fine if the file size is very small. But imagine the case when you are downloading a 500MB file. This is not going to work since the file is stored in a JavaScript variable before the download takes place.

Make the API that download the attachment anonymous

If the endpoint that downloads the file doesn’t require authentication then we are good. But now the file will be available for every one to download. So, we have to find a way to secure the file even when the endpoint is anonymous.

If you have some experience with Azure Storage, you may have heard of Azure Storage Shared Access Signature. The idea is simple. When the user requests a file, generate a token, save it to a temporary storageĀ  and append it to the URL of the download file endpoint. When the user clicks the link, the endpoint will be called and the token will be validated against the temporary storage and if it matches then send the file contents. This way we will be sure that the link was generated by the application to that user. Still, if the link was shared to another user, he will be able to download the file. But this is another issue that we can worry about later.

Implementation

We will create a new asp.net core site with an endpoint to download files but I will not create a SPA in this article. That will be left for the reader. I will test the idea though using Postman.

Open Visual Studio, Create a new project of type “Asp.NET Core Web Application” then Choose “API” in the next dialog. You can still choose “Web Application (Model-View-Controller)”. I will leave authentication to the default “No Authentication”.

Right Click on the Controllers folder and choose “New Controller”, choose “API Controlller – Empty” and name it AttachmentsController. You should end up with the following

[Produces("application/json")]
[Route("api/Attachments")]
 //[Authorize]
public class AttachmentsController : Controller
{
}

Notice that I have commented the [Authorize] attribute since I didn’t setup authentication in this demo. In real life scenario, you will setup authentication and authorization using Token based Authentication.

Create a folder named Services and then create a new interface called ISecureUrlGenerator. The content should look like the following:

   public interface ISecureUrlGenerator
    {
        string GenerateSecureAttachmentUrl(string id, string url);
        bool ValidateUrl(string url, string id);
        bool ValidateToken(string id, string token);
    }

Now, add class to implement the previous interface

using Microsoft.Extensions.Caching.Memory;
using System;
using System.Collections.Generic;

namespace SecureAttachmentsDownload.Services
{
    public class SecureUrlGenerator : ISecureUrlGenerator
    {
        private readonly IMemoryCache memoryCache;

        public SecureUrlGenerator(IMemoryCache memoryCache)
        {
            this.memoryCache = memoryCache ?? throw new ArgumentNullException(nameof(memoryCache));
        }

        public string GenerateSecureAttachmentUrl(string id, string url)
        {
            var token = Guid.NewGuid().ToString().ToLower();
            StoreToken(id, token);
            var separator = url.Contains("?") ? "&" : "?";
            return $"{url}{separator}token={token}";
        }

        public bool ValidateToken(string id, string token)
        {
            var tokens = memoryCache.Get(id);
            if (tokens != null && tokens.Contains(token))
                return true;

            return false;
        }

        public bool ValidateUrl(string url, string id)
        {
            var uri = new Uri(url);
            var queryStringParams = uri.Query.Split("&");
            foreach (var param in queryStringParams)
            {
                var values = param.Split("=");
                if (values[0].ToLower() == "token")
                {
                    return ValidateToken(id, values[1]);
                }
            }

            return false;
        }

        private bool IsTokenValid(string id, string token)
        {
            var tokens = memoryCache.Get(id);
            if (tokens != null && tokens.Contains(token))
                return true;

            return false;
        }

        private void StoreToken(string id, string token)
        {
            var tokens = memoryCache.Get(id);
            if (tokens == null)
                tokens = new List();
            tokens.Add(token);

            memoryCache.Set(id, tokens);
        }
    }
}

In this implementation, I am storing the tokens in asp.net core memory cache. To enable this feature, you have to add the caching service in Starup.cs file

public void ConfigureServices(IServiceCollection services)
        {
            services.AddMemoryCache();
            services.AddMvc();
        }

You can replace the memory cache with a database if you want the tokens to be permanent and in this case you have to add an expiration date.

Before we utilize the secure URL genrator, we need a class to hold the attachments metadata since the user will request the list of attachments first and then download it.
Create a folder called Models and put the following class in it.

namespace SecureAttachmentsDownload.Models
{
    public class AttachmentMetadata
    {
        public int Id { get; set; }

        public string DownloadUrl { get; set; }

        public string Name { get; set; }

        public int FileSize { get; set; }
    }
}

Now, lets get to the part where we utilise our secure URL generator.
The flow will be as below:

  1. The user requests endpoint to return a list of attachments to be displayed to the user. Here, the DownloadUrl will have the token already. This will be secured by tokens
  2. The SPA will display this list to the user as links or buttons that the user can click to download the file. The href for the anchor tag will be the DownloadURl property
  3. The user will click the link to download the attachment
  4. The AttachmentController will be called and the endpoint will validate the token and return the file or else a 401

Open the AttachmentsController file and add the following 2 action methods

  using System.Collections.Generic;
using System.IO;
using System.Linq;
using Microsoft.AspNetCore.Authorization;
using Microsoft.AspNetCore.Hosting;
using Microsoft.AspNetCore.Mvc;
using SecureAttachmentsDownload.Models;
using SecureAttachmentsDownload.Services;

namespace SecureAttachmentsDownload.Controllers
{
    [Produces("application/json")]
    [Route("api/Attachments")]
    //[Authorize]
    public class AttachmentsController : Controller
    {
        private readonly ISecureUrlGenerator _secureUrlGenerator;
        private readonly IHostingEnvironment _hostingEnvironment;

        private readonly List Attachments = new List()
            {
                new AttachmentMetadata
                {
                    Id = 1,
                    Name = "bitcoin.pdf",
                    ContentType = "application/pdf",
                    FileSize = 1024
                },
                  new AttachmentMetadata
                {
                    Id = 2,
                    Name = "report 1.pdf",
                    FileSize = 3024
                },
                  new AttachmentMetadata
                {
                    Id = 3,
                    Name = "report 2.pdf",
                    FileSize = 2024
                }
            };

        public AttachmentsController(ISecureUrlGenerator secureUrlGenerator, IHostingEnvironment hostingEnvironment)
        {
            _secureUrlGenerator = secureUrlGenerator;
            _hostingEnvironment = hostingEnvironment;
        }

        [HttpGet]
        [Route("")]
        public IActionResult Get()
        {
            foreach (var attachment in Attachments)
            {
                var url = Url.Action(nameof(AttachmentsController.Get), "Attachments", new { attachment.Id }, Url.ActionContext.HttpContext.Request.Scheme);
                attachment.DownloadUrl = _secureUrlGenerator.GenerateSecureAttachmentUrl(attachment.Id.ToString(), url);
            }

            return Ok(Attachments);
        }

        [HttpGet]
        [Route("{id}")]
        [AllowAnonymous]
        public IActionResult Get(int id, string token)
        {
            if (!_secureUrlGenerator.ValidateToken(id.ToString(), token))
                return Forbid();

            var attachment = Attachments.FirstOrDefault(a => a.Id == id);
            if (attachment == null)
                return NotFound();

            var stream = new FileStream($"{_hostingEnvironment.WebRootPath}\\Files\\{attachment.Name}", FileMode.Open);

            return File(stream, attachment.ContentType);
        }
    }
}

Now run the application and open the URL /api/Attachments. You will get the following excepttion:

InvalidOperationException: Unable to resolve service for type ‘SecureAttachmentsDownload.Services.ISecureUrlGenerator’ while attempting to activate ‘SecureAttachmentsDownload.Controllers.AttachmentsController’.

To fix it, open the startup.cs file and add the following line to the ConfigureServices method

  public void ConfigureServices(IServiceCollection services)
        {
            services.AddMemoryCache();
            services.AddScoped ();
            services.AddMvc();
        }

Now open the URL api/attachments again and you should see the following JSON response

[
{
"id": 1,
"downloadUrl": "http://localhost:53098/api/Attachments/1?token=b78763c2-0109-4c12-b771-5f5cc5d19017",
"name": "bitcoin.pdf",
"fileSize": 1024,
"contentType": "application/pdf"
},
{
"id": 2,
"downloadUrl": "http://localhost:53098/api/Attachments/2?token=12497a4a-8f08-44ba-b9f6-914c4b484cc5",
"name": "report 1.pdf",
"fileSize": 3024,
"contentType": null
},
{
"id": 3,
"downloadUrl": "http://localhost:53098/api/Attachments/3?token=8647bb52-e47f-4580-8149-0b1d238ab0e2",
"name": "report 2.pdf",
"fileSize": 2024,
"contentType": null
}
]

As you can see, the downloadUrl property has the absolute URL for the file and the `token` query string parameter is appended. If you open the first link in a new browser window, the Action Get(id) will be called and the token will be bound to the parameter token.
In my implementation, I have put some files in a folder called Files under the wwwroot folder. But in actual projects, you may retrieve the files from a Database, FTP or any Document Management System.

If you want to make sure that it is really working, just try to change any character in the token query string and you should get a forbid response from the server. In this example you will get an exception: InvalidOperationException: No authenticationScheme was specified, and there was no DefaultForbidScheme found.
This is because I didn’t configure the authentication middleware.

You can find the source code for this article on GitHub.

This implementation has as flaw. The list of attachments are returned with the download URL and the tokens are saved in memory. If the user didn’t click the link but after sometime, the tokens may have been already expired. So, either you save the tokens in a DB or you before clicking the link, fire an Ajax request to an endpoint that gets the metadata for a single attachment. This way, the downloadUrl will be always fresh and working.

If you have any questions or suggestions, please leave a comment below.