This is the fifth part of Building Microservice Applications with Azure Container Apps and Dapr. The topics we’ll cover are:
- Tutorial for building Microservice Applications with Azure Container Apps and Dapr – Part 1
- Deploy backend API Microservice to Azure Container Apps – Part 2
- Communication between Microservices in Azure Container Apps – Part 3
- Dapr Integration with Azure Container Apps – Part 4
- Azure Container Apps State Store With Dapr State Management API – (This Post)
- Azure Container Apps Async Communication with Dapr Pub/Sub API – Part 6
- Azure Container Apps with Dapr Bindings Building Block – Part 7
- Azure Container Apps Monitoring and Observability with Application Insights – Part 8
- Continuous Deployment for Azure Container Apps using GitHub Actions – Part 9
- Use Bicep to Deploy Dapr Microservices Apps to Azure Container Apps – Part 10
- Azure Container Apps Auto Scaling with KEDA – Part 11
- Azure Container Apps Volume Mounts using Azure Files – Part 12
- Integrate Health probes in Azure Container Apps – Part 13
Azure Container Apps State Store With Dapr State Management API
In this post, we will switch the in-memory store of tasks and use a key/value persistent store (Azure Cosmos DB) by using the Dapr State Management Building Block, we will see how we can store the data in Azure Cosmos DB without installing any Cosmos DB SDK or write specific code to integrate our Backend API with Azure Cosmos DB.
Moreover, we will use Redis to store tasks when we are running the application locally, you will see that we can switch between different stores without any code changes, thanks to the Dapr pluggable state stores feature! It is a matter of adding new Dapr Component files and the underlying store will be changed. This page shows the supported state stores in Dapr.
The source code for this tutorial is available on GitHub. You can check the demo application too.
Overview of Dapr State Management API
Dapr’s state management API to save, read, and query key/value pairs in the supported state stores. To try this out and without doing any code changes or installing any NuGet packages we can directly invoke the State Management API and store the data on Redis locally, if you remember from the previous post once we initialized Dapr in a local development environment, it installed Redis container instance locally, so we can use Redis locally to store and retrieve state. If you navigate to the path “<UserProfile>\.dapr\components” you find a file named “statestore.yaml”. Inside this file, you will see the properties needed to access the local Redis instance. The state store template component file structure can be found on this link.
To try out the State Management APIs, run the Backend API from VS Code by running the below command or using the Run and Debug tasks we have created in the previous post.
1 |
~\TasksTracker.ContainerApps\TasksTracker.TasksManager.Backend.Api> dapr run --app-id tasksmanager-backend-api --app-port 7088 --dapr-http-port 3500 --app-ssl dotnet run |
Now from any rest client, invoke the below POST request to the endpoint http://localhost:3500/v1.0/state/statestore
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 |
POST /v1.0/state/statestore HTTP/1.1 Host: localhost:3500 Content-Type: application/json Content-Length: 682 [ { "key": "Book1", "value": { "title": "Parallel and High Performance Computing", "author": "Robert Robey", "genere": "Technical" } }, { "key": "Book2", "value": { "title": "Software Engineering Best Practices", "author": "Capers Jones", "genere": "Technical" } }, { "key": "Book3", "value": { "title": "The Unstoppable Mindset", "author": "Jessica Marks", "genere": "Self Improvment", "formats":["kindle", "audiobook", "papercover"] } } ] |
What we have done here is simple, the value “statestore” in the endpoint should match the “name” value in the component file statestore.yaml.
We have sent a request to store 3 entries of books, you can put any JSON representation in the value property.
To see the results visually, you can install a VS Code extension to connect to Redis DB and see the results, in our case, I’m using the extension named “Redis Xplorer“, after you connect to Redis locally, you should see the 3 entries similar to the below image. Notice how each entry key is prefixed by the Dapr App Id, in our case it is “tasksmanager-backend-api”. More about key prefix strategy later in the post.
To get the value of a key, you need to issue a GET request to the endpoint http://localhost:3500/v1.0/state/statestore/{YourKey} and you should receive the value of the key store, in our case if we do a GET http://localhost:3500/v1.0/state/statestore/Book3 the results will be the below object
1 2 3 4 5 6 7 8 9 10 |
{ "formats": [ "kindle", "audiobook", "papercover" ], "title": "The Unstoppable Mindset", "author": "Jessica Marks", "genere": "Self Improvment" } |
Use Dapr Client SDK for State Store Management
Now we will introduce a change on the Backend API and create a new service named “TasksStoreManager.cs” which will implement the interface “ITasksManager.cs” to start storing tasks data on the persistence store. Locally we will start testing with Redis, then we are going to change the state store to use Azure Cosmos DB.
Step 1: Add Dapr Client SDK to the Backend API
Similar to what we have done in the Frontend Web App, we need to use Dapr Client SDK to manage the state store, to do so, open the file named “TasksTracker.TasksManager.Backend.Api.csproj” and Install the NuGet package “Dapr.AspNetCore” below
1 2 3 4 |
<ItemGroup> <PackageReference Include="Dapr.AspNetCore" Version="1.8.0" /> <!-- Other packages are removed for brevity --> </ItemGroup> |
Step 2: Create a new concrete implementation to manage task persistence.
As you recall from the previous post, we were storing the tasks in memory, now we need to store them in Redis and later on Azure Cosmos DB. To do so add a new file named “TasksStoreManager.cs” under the folder named “Services” and this file will implement the interface “ITasksManager”. Copy & Paste the code below:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 |
namespace TasksTracker.TasksManager.Backend.Api.Services { public class TasksStoreManager : ITasksManager { private static string STORE_NAME = "statestore"; private readonly DaprClient _daprClient; private readonly IConfiguration _config; private readonly ILogger<TasksStoreManager> _logger; public TasksStoreManager(DaprClient daprClient, IConfiguration config, ILogger<TasksStoreManager> logger) { _daprClient = daprClient; _config = config; _logger = logger; } public async Task<Guid> CreateNewTask(string taskName, string createdBy, string assignedTo, DateTime dueDate) { var taskModel = new TaskModel() { TaskId = Guid.NewGuid(), TaskName = taskName, TaskCreatedBy = createdBy, TaskCreatedOn = DateTime.UtcNow, TaskDueDate = dueDate, TaskAssignedTo = assignedTo, }; _logger.LogInformation("Save a new task with name: '{0}' to state store", taskModel.TaskName); await _daprClient.SaveStateAsync<TaskModel>(STORE_NAME, taskModel.TaskId.ToString(), taskModel); return taskModel.TaskId; } public async Task<bool> DeleteTask(Guid taskId) { _logger.LogInformation("Delete task with Id: '{0}'", taskId); await _daprClient.DeleteStateAsync(STORE_NAME, taskId.ToString()); return true; } public async Task<TaskModel?> GetTaskById(Guid taskId) { _logger.LogInformation("Getting task with Id: '{0}'", taskId); var taskModel = await _daprClient.GetStateAsync<TaskModel>(STORE_NAME, taskId.ToString()); return taskModel; } public async Task<List<TaskModel>> GetTasksByCreator(string createdBy) { //Currently, the query API for Cosmos DB is not working when deploying it to Azure Container Apps, this is an open //issue and prodcut team is wokring on it. Details of the issue is here: https://github.com/microsoft/azure-container-apps/issues/155 //Due to this issue, we will query directly the cosmos db to list tasks per created by user. var query = "{" + "\"filter\": {" + "\"EQ\": { \"taskCreatedBy\": \"" + createdBy + "\" }" + "}}"; var queryResponse = await _daprClient.QueryStateAsync<TaskModel>(STORE_NAME, query); var tasksList = queryResponse.Results.Select(q => q.Data).OrderByDescending(o=>o.TaskCreatedOn); return tasksList.ToList(); //Workaround: Query cosmos DB directly //_logger.LogInformation("Query tasks created by: '{0}'", createdBy); //var result = await QueryCosmosDb(createdBy); //return result; } public async Task<bool> MarkTaskCompleted(Guid taskId) { _logger.LogInformation("Mark task with Id: '{0}' as completed", taskId); var taskModel = await _daprClient.GetStateAsync<TaskModel>(STORE_NAME, taskId.ToString()); if (taskModel != null) { taskModel.IsCompleted = true; await _daprClient.SaveStateAsync<TaskModel>(STORE_NAME, taskModel.TaskId.ToString(), taskModel); return true; } return false; } public async Task<bool> UpdateTask(Guid taskId, string taskName, string assignedTo, DateTime dueDate) { _logger.LogInformation("Update task with Id: '{0}'", taskId); var taskModel = await _daprClient.GetStateAsync<TaskModel>(STORE_NAME, taskId.ToString()); var currentAssignee = taskModel.TaskAssignedTo; if (taskModel != null) { taskModel.TaskName = taskName; taskModel.TaskAssignedTo = assignedTo; taskModel.TaskDueDate = dueDate; await _daprClient.SaveStateAsync<TaskModel>(STORE_NAME, taskModel.TaskId.ToString(), taskModel); return true; } return false; } public async Task<List<TaskModel>> GetTasksByTime(DateTime waterMark) { throw new NotImplementedException(); } public async Task MarkOverdueTasks(List<TaskModel> overDueTasksList) { throw new NotImplementedException(); } } } |
What we have implemented here is simple, we have injected the “DaprClient” into the new service and DaprClient has a set of methods to support CRUD operations. Notice how we are using the state store named “statestore” which should match the name in the component file.
Note: The query API will not work against the local Redis store as you need to install RediSearch locally on your machine. It will work locally once we switch to Azure Cosmos DB.
Step 3: Register the new service and DaprClient
Now we need to register the new service named “TasksStoreManager” and RedisClient when the Backend API app starts up, to do so open the file “Program.cs” and register both as the below. Do not forget to comment out the registration of the “FakeTasksManager” service as we don’t want to store tasks in memory anymore.
1 2 3 4 5 6 7 8 9 10 11 |
var builder = WebApplication.CreateBuilder(args); // Add services to the container. builder.Services.AddSingleton<DaprClient>(_ => new DaprClientBuilder().Build()); //builder.Services.AddSingleton<ITasksManager, FakeTasksManager>(); builder.Services.AddSingleton<ITasksManager, TasksStoreManager>(); //Code removed for brevity |
Now you are ready to run both applications and debug them, you can store new tasks, update them, delete existing tasks and mark them as completed, the data should be stored on your local Redis instance. (Rember query API will not work locally)
Use Azure Cosmos DB with Dapr State Store Management API
Step 1: Cosmos DB Resources
Now we will create an Azure Cosmos DB account, Database, and a new container that will store our tasks, you can use the below PowerShell script to create the Cosmos DB resources on the same resource group we used in the previous post. You need to change the variable name of the $COSMOS_DB_ACCOUNT to a unique name as this name is already used under my account.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 |
$COSMOS_DB_ACCOUNT="taskstracker-state-store" ` $COSMOS_DB_DBNAME="tasksmanagerdb" ` $COSMOS_DB_CONTAINER="taskscollection" ## Check if Cosmos account name already exists az cosmosdb check-name-exists ` --name $COSMOS_DB_ACCOUNT ## returns false ## Create a Cosmos account for SQL API az cosmosdb create ` --name $COSMOS_DB_ACCOUNT ` --resource-group $RESOURCE_GROUP ## Create a SQL API database az cosmosdb sql database create ` --account-name $COSMOS_DB_ACCOUNT ` --resource-group $RESOURCE_GROUP ` --name $COSMOS_DB_DBNAME ## Create a SQL API container az cosmosdb sql container create ` --account-name $COSMOS_DB_ACCOUNT ` --resource-group $RESOURCE_GROUP ` --database-name $COSMOS_DB_DBNAME ` --name $COSMOS_DB_CONTAINER ` --partition-key-path "/id" ` --throughput 400 |
Once the scrips are completed we need to get the “primaryMasterKey” of the CosmosDB account, to do this you can use the below PowerShell script. Copy the value of “primaryMasterKey” as we will use it in the next step.
1 2 3 4 |
## List Azure CosmosDB keys az cosmosdb keys list ` --name $COSMOS_DB_ACCOUNT ` --resource-group $RESOURCE_GROUP |
Step 2: Create a Component file for State Store Management
Dapr uses a modular design where functionality is delivered as a component. Each component has an interface definition. All of the components are pluggable so that you can swap out one component with the same interface for another
Components are configured at design-time with a YAML file which is stored in either a components/local folder within your solution, or globally in the .dapr folder created when invoking dapr init . These YAML files adhere to the generic Dapr component schema, but each is specific to the component specification.
It is important to understand that the component spec values, particularly the spec “metadata”, can change between components of the same component type As a result, it is strongly recommended to review a component’s specs, paying particular attention to the sample payloads for requests to set the metadata used to interact with the component.
I’m borrowing the diagram below from Dapr official documentation which shows some examples of the components for each component type, we are now looking at the State Stores components, specifically the Azure Cosmos DB
To add the component file state store add a new folder named “components” under the directory “TasksTracker.ContainerApps” and add a new yaml file named “dapr-statestore-cosmos.yaml”. Paste the code below
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
apiVersion: dapr.io/v1alpha1 kind: Component metadata: name: statestore spec: type: state.azure.cosmosdb version: v1 metadata: - name: url value: https://taskstracker-state-store.documents.azure.com:443/ - name: masterKey value: "<value>" - name: database value: tasksmanagerdb - name: collection value: taskscollection scopes: - tasksmanager-backend-api |
Note that we used the name “statestore” which should match the name of statestore we’ve used in the “TaskStoreManager.cs” file. As well we have set the metadata key/value to allow us to connect to Azure Cosmos DB. You need to replace the “masterKey” value with your Cosmos Account key as well as update the other properties. For full metadata specs, you can check this page.
Note about The “Scopes” property: By default, all Dapr-enabled container apps within the same environment will load the full set of deployed components. By adding scopes to a component, you tell the Dapr sidecars for each respective container app which components to load at runtime. Using scopes is recommended for production workloads.
In our case, we have set the scopes to read “tasksmanager-backend-api” as this will be the application that needs access to Azure Cosmos DB State Store.
Now you should be ready to launch both applications and start doing CRUD operations from the Frontend Web App including querying the store, all your data will be stored in Cosmos DB Database you just provisioned, it should look like the below:
Key Prefix Strategies
When you look at the key stored per entry and for example “tasksmanager-backend-api||aa3eb856-8309-4e68-93af-119be0d400e8”, you will notice that the key is prefixed with the Dapr application App Id responsible to store this entry, in our case, it will be “tasksmanager-backend-api”, there might be some scenarios which you need to have another service to access the same data store (not recommended as each service should be responsible about its own data store) then you can change the default strategy, this can be done by adding the below meta tag to the component file, for example, if we need to set the value of the prefix to a constant value such as “TaskId” we can do the following:
1 2 3 4 |
spec: metadata: - name: keyPrefix - value: TaskId |
If we need to totally omit the key prefix so it is accessed across multiple Dapr applications, we can set the value to none.
Deploy the Backend API and Frontend Web App Projects to Azure Container Apps
Now we are ready to deploy all local changes from this post and previous Post to Azure Container Apps, but we need to do one addition before deploying, we have to create a component file for Azure Cosmos DB which is meeting the specs defined by Azure Container Apps. This is how it is documented by Product Group.
Step 1: Create a Component file matching Azure Container Apps Specs
I prefer to separate the component files that will be used when deploying to Azure Container Apps from the ones which we will use when running our application locally (Dapr self-hosted). So go ahead and create a new folder named “aca-components” under the directory “TasksTracker.ContainerApps”
Add a new file named “containerapps-statestore-cosmos.yaml” and paste the code below
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
componentType: state.azure.cosmosdb version: v1 metadata: - name: url value: https://taskstracker-state-store.documents.azure.com:443/ - name: masterkey secretRef: cosmoskey - name: database value: tasksmanagerdb - name: collection value: taskscollection secrets: - name: cosmoskey value: "<key value here>" scopes: - tasksmanager-backend-api |
Two important things to notice here, we didn’t specify the component name “statestore” when we created this component file, we are going to specify it once we add this dapr component to Azure Apps Environment via CLI.
The second thing we are not setting the Cosmos DB secret directly here, this will not be secure, we are using “secretRef” which will allow us to set the actual masterKey after we deploy this component to the Azure Container Environment. So this key will not be stored in the source code by mistake.
Step 2: Build Frontend Web App and Backend API App images and push them to ACR
As we have done previously we need to build and deploy both app images to ACR so they are ready to be deployed to Azure Container Apps, to do so, continue using the same PowerShell console and paste the code below (Make sure you are on directory “TasksTracker.ContainerApps”):
1 2 3 |
az acr build --registry $ACR_NAME --image "tasksmanager/$BACKEND_API_NAME" --file 'TasksTracker.TasksManager.Backend.Api/Dockerfile' . az acr build --registry $ACR_NAME --image "tasksmanager/$FRONTEND_WEBAPP_NAME" --file 'TasksTracker.WebPortal.Frontend.Ui/Dockerfile' . |
Step 3: Add Cosmos DB Dapr State Store to Azure Container Apps Environment
We need to run the command below to add the yaml file “.\aca-components\containerapps-statestore-cosmos.yaml” to Azure Container Apps Environment, to do so run the below PowerShell command:
1 2 3 4 |
az containerapp env dapr-component set ` --name $ENVIRONMENT --resource-group $RESOURCE_GROUP ` --dapr-component-name statestore ` --yaml '.\aca-components\containerapps-statestore-cosmos.yaml' |
Notice that we set the component name “statestore” when we added it Container Apps Environment.
Once the command completes and from the Azure Portal, navigate to your Container Apps Environment, select “Dapr Components”, then click on “statestore” component, and provide your Azure Cosmos DB masterKey in the Secrets text box value for secret “cosmoskey” and click “Edit” button. It will be similar to the below image
Step 4: Enable Dapr for the Frontend Web App and Backend API Azure Container Apps
Until this moment Dapr was not enabled on the Azure Container Apps we have provisioned, you can check this via Azure Portal or CLI and it will look like the below image.
What we need to do now is to enable Dapr for bot Container Apps, to do so, run the below command in the PowerShell console:
1 2 3 4 5 6 7 8 9 |
az containerapp dapr enable --name $BACKEND_API_NAME ` --resource-group $RESOURCE_GROUP ` --dapr-app-id $BACKEND_API_NAME ` --dapr-app-port 80 az containerapp dapr enable --name $FRONTEND_WEBAPP_NAME ` --resource-group $RESOURCE_GROUP ` --dapr-app-id $FRONTEND_WEBAPP_NAME ` --dapr-app-port 80 |
Step 5: Deploy new revisions of the Frontend Web App and Backend API to Azure Container Apps
The last thing we need to do here is to update both container apps and deploy the new images from ACR, to do so we need to run the below command. I’ve used a revision-suffix property so it will append to the revision name and gives you better visibility on which revision you are looking at.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
## Update Frontend web app container app and create a new revision az containerapp update ` --name $FRONTEND_WEBAPP_NAME ` --resource-group $RESOURCE_GROUP ` --revision-suffix v20220812 ` --cpu 0.25 --memory 0.5Gi ` --min-replicas 1 ` --max-replicas 2 ## Update Backend API App container app and create a new revision az containerapp update ` --name $BACKEND_API_NAME ` --resource-group $RESOURCE_GROUP ` --revision-suffix v20220812-1 ` --cpu 0.25 --memory 0.5Gi ` --min-replicas 1 ` --max-replicas 2 |
With this final step, we should be able to access the Frontend Web App, call the backend API app using Dapr sidecar, and store tasks to Azure Cosmos DB.
In the next post, I will introduce the Dapr Pub/Sub Building block which we will publish messages to Azure Service Bus when a task is saved, and a new background service will process those incoming messages and send an email to the task assignee. Stay tuned for the next post and happy coding 🙂
Leave a Reply