Azure CI CD Pipeline with Docker in Web API

Description

In our prior blogs I have written about how to do CI CD for a web Razor Application for reference click here, Now we are going see how to do the CI CD of a Web API with Docker Containerization in Azure Devops

Prerequisite

ü Azure Sql Database (I have written blog here about how to create Azure Sql Db)

ü Little knowledge of Docker

ü Little knowledge of Web API

ü C#

ü Azure Subscription

ü VS 2022

Our Target

Now, our target is we are going to create a Web API using .Net 6.0 with Entity Framework; then make the project into containerization then make CI CD and deploy into Azure Container Registry (ACR) Linux system

Basics of Docker

Docker is a tool designed to make it easier for developers to develop, ship, and run applications by using containers. Containers allow devs to package an application with all of its requirements and configurations, such as libraries and other dependencies and deploy it as a single package. By doing so, developers can rest easy knowing that the software will work properly on other systems besides the one they used for writing the code.

First install Docker on our local dev machine, use link to download and install

After installed docker, make the Linux containers on our Docker using the steps

Step 1 Create Web API project and move into Azure Artifact

1. From start menu, open visual studio then click New Project

2. Then type Blank solution on the search bar then scroll down little bit select Blank Solution and click Next

3. The solution name I did mark it as DemoDockerWebAPI and mention the project path and click Create

4. Next, we are going to add Web Api project to our solution by Right click on the Solution file from Solution Explorer then type Restful api then from the result select ASP.NET Core Web API Click Next Mention the project name as ApiDemo then mention project location to the same solution folder and click Next

5. Choose the framework as .NET 6.0 Uncheck HTTPS also at this moment Im not going to include docker so, just leave it as Uncheck and click Create

6. Now, the solution file is created, Rest API project is also created, Next we are going to create the project in Azure Devops, this I have written here so, please refer it to create new project and move the current project to Azure Devops repository

7. Now, our project is looks like below structure

8. Right click on ApiDemo and click Manage Nuget Packages from the Nuget pack window click on Browse then on the search bar type EntityFrameworkCore and press enter, form the result Install below packages with latest versions

a. Microsoft.EntityFrameworkCore.Design

b. Microsoft.EntityFrameworkCore.SqlServer

c. Microsoft.EntityFrameworkCore.Tools

Next we are going to design a table in azure sql database, for info refer my another blog from that Sql DB, created new table called Invoice below is the table structure

CREATE TABLE [dbo].[Invoice](

[invoiceId] [int] IDENTITY(1,1) NOT NULL,

[entityName] [varchar](100) NOT NULL,

[invoiceDate] [datetime] NOT NULL,

[dueAmount] [money] NOT NULL,

[addDate] [datetime] NOT NULL,

CONSTRAINT [PK_Invoice] PRIMARY KEY CLUSTERED

(

[invoiceId] ASC

)WITH (STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, OPTIMIZE_FOR_SEQUENTIAL_KEY = OFF) ON [PRIMARY]

) ON [PRIMARY]

ALTER TABLE [dbo].[Invoice] ADD CONSTRAINT [DF_Invoice_addDate] DEFAULT (getdate()) FOR [addDate]

9. Next, on VS 2022, open appsettings.json file to configure the connection string of our Azure Sql Dbase

10. Next, we are going to create Models using scaffold method so, below is the command

a. Scaffold-DbContext "Data Source={YOUR SERVER NAME OR IP };Initial Catalog={YOUR AZURE SQL DB}; Persist Security Info=False;User ID={YOUR DB USER ID};Password={YOUR DB PASSWORD} ;MultipleActiveResultSets=False;Encrypt=True;TrustServerCertificate=True;Connection Timeout=30;" Microsoft.EntityFrameworkCore.SqlServer -OutputDir Models

b. After filled up your db details, execute this command in Package Manager Explorer

c. After execution, we could see the Models folder in Solution explorer

11. Next, Right click on Controllers folder from solution explorer window and click New Controller, from the scaffold window click API then click API Controller with actions , using Entity Framework and click Add button then select Model calss as Invoice, select the our DataContext file which is under Models folder then make the controller name as InvoicesController then click Add button, it would take some time to create the controller so, wait for some seconds.

12. Now, the controller has been created, next we are going to run this application and see the result of our end points , when I try to run the POST operation with below data

but, we are receiving error as “System.InvalidOperationException: Unable to resolve service for type 'ApiDemo.Models.TrainDriverContext' while attempting to activate 'ApiDemo.Controllers.InvoicesController'.” so, strange, why and how this error happens,

a. The error is from DataContext .cs file, I have commented the OnConfiguring method then

b. Did move the connection into Program.cs file like below

13. Then run the application, now Im able to process my data using POST verbose using below json data

{

"invoiceId": 0,

"entityName": "Kings International Plywood",

"invoiceDate": "2022-11-08",

"dueAmount": 15000,

"addDate": "2022-11-08T11:09:53.909Z"

}

and click on Execute button and result is 200 SUCCESS

So, from the step1 we, knew how to create Web API using Entity Framework and access them

Step 2 Create Dockerfile and Docker Compose

Next, we are going to implement docker related changes.

Usually we do run our project with IIS express from Dev Laptop’s right so, here after we are going to use Docker, the reason why Docker is because, its very helpful for packing and deploying our software’s to avoid manual mistakes.

Lets think practically, we have to publish our API in IIS then we will move the Dependency files first then config files then DLL then Batch files right, so, in this order if we missed one config file then what will happen, the entire project will not work and we have to find what cause this error and move that config alone to arreset the error, correct!!

When we use Docker, it make our project(s) into a image(IO) format with all dependencies, DLL, Views, configs then this image is going to publish in server so, we avoided manual work and not missing any single file.

Hope this explanation is enough, for more informations, we could refer here

1. Right click on APIDemo from solution explorer click Add then click Docker Support

2. From the Docker File options choose Linux since, we are going use launch our api in Linux environment

3. Once, this step is completed then immediately VS is start preparing the images on our local Docker, now, open our Docker in laptop we could see the Container with our project name

4. The dockerfile has lot of commands, we will step through little later, now we are going to add Container Orchestrator so, right click on API Demo cick New then click Container Orchestrator Support

from the popup window select Docker Compose and click Ok

then from the OS selection choose Linux and click Ok

5. Now, we do have new project in Solution explorer called Docker-Compose like below, which means here after we are not going to run our application via IIS Express rather its Docker

6. So, the docker file commands are going use the related packages from mcr.microsoft.com/dotnet/aspnet:6.0

a. Next move it into app folder

b. Then it find the csproj file from original location to build location

c. Then it start build the project from the new location(not the actual source)

d. When its build then start make publish the files into /app/publish location

7. Lets run the project now, and make sure are we able to get data from Invoice controller, well its working for us

Step 3 how to create Azure Container Registry

1. Open portal.azure.com from your web browser and login with your credentials, then after from the home page search bar type Container registries then at the bottom select the service called Container registries then click + Create

a. Type Registry name as DockerCICDDemo

b. Location West Europe

c. Resource group as RG-Docker

d. Sku as Basic

e. Click Review + Create at the end of the tabs click Create button

f. This will take some time, once its completed then click on Go to resource

g. Then Access Key from the left pane there click Admin User Enabled since, this is the keys which is going to access in ACR from next step

Step 4 How to create Build CI from Azure Devops for Container

1. Before this step, Push all the docker, controller, models from git to Azure Repository

2. Then after visit dev.azure.com, within the organization choose the project then click Pipelines and click Create pipeline

3. From the next menu click Classic Editor

4. Choose source as Azure Repos Git then automatically the project, Repository, Branch are selected then click Continue

5. From the template window choose Docker Container and click Apply

6. Next window is like below, here we are going to setup which to build as image and where from the repo take, from below steps

a. Change the name of pipeline into DockerDemoCI from DockerCICDDemo-Docker Contain…

b. Agent Pool as Azure Pipelines

c. Agent Specification as ubuntu-20.04

d. Click Agent Job 1, change the display name as Dev-Agent job 1

e. Agent pool as Azure Pipelines

f. Agent Specification as ubuntu-20.04

g. Click on Build an Image

h. Container registry type as Azure Container Registry

i. Authorize subscription with Azure Service Authorization

j. On the Azure Container Registry section choose our ACR created in above steps as DockerCICDDemo

k. By Default the docker file section is wrong so, we have to choose the exact location of docker file by clicking the browse button

l. Click Push an image , keep Container Registry Type as Azure Container Registry

m. Azure Subscription by Service subscription

n. Choose Azure Container Registry as DockerCICDDemo

o. Go to Triggers tab and tick Enable Continuous integration then click Save & Queue

p. To all other places Keep it by default, then click Save & Queue which will start build our project.

7. From the upcoming page click Dev-Agent job1, will show the logs

8.  Unfortunately we received an error at step 6/20 as COPY failed: file not found in build context or excluded by .dockerignore: stat ApiDemo/ApiDemo.csproj: file does not exist

9.  It says that the Container is not able to find our CSPROJ file, which is the core file has all references of our project so, go back to our VS2022 project and open Dockerfile

a.   Made a change into COPY ["ApiDemo.csproj", "ApiDemo/"] from COPY ["ApiDemo/ApiDemo.csproj", "ApiDemo/"]

10. Then after PUSHED the code change and CI is automatically taken to build since, we activated it in Trigger section but, again we received error at step 10 so, we proud that moved 4 steps ahead . Complete error message is “CSC : error CS5001: Program does not contain a static 'Main' method suitable for an entry point [/src/ApiDemo/ApiDemo.csproj]”

a.   To resolve this error, again we have to add new change in Docker file like below , the change I made is added another COPY . .(highlighted in yellow) after this change push it then this time the BUILD is success and all are fine

#See https://aka.ms/containerfastmode to understand how Visual Studio uses this Dockerfile to build your images for faster debugging.

FROM mcr.microsoft.com/dotnet/aspnet:6.0 AS base

WORKDIR /app

EXPOSE 80

FROM mcr.microsoft.com/dotnet/sdk:6.0 AS build

WORKDIR /src

COPY ["ApiDemo.csproj", "ApiDemo/"]

RUN dotnet restore "ApiDemo/ApiDemo.csproj"

COPY . .

WORKDIR "/src/ApiDemo"

COPY . .

RUN dotnet build "ApiDemo.csproj" -c Release -o /app/build

FROM build AS publish

RUN dotnet publish "ApiDemo.csproj" -c Release -o /app/publish /p:UseAppHost=false

FROM base AS final

WORKDIR /app

COPY --from=publish /app/publish .

ENTRYPOINT ["dotnet", "ApiDemo.dll"]

Step 5 Create Azure App Service with Docker Container

Here we are going to create the App Service to launch our API with Docker Container

1. Go to portal.azure.com and login

2. From Home page, Click App Services then click + Create

a. From the configuration tabs Resource group as RG-Docker

b. App name as DockerCiCdDemoApp

c. Publish Docker Container

d. Operating System Linux

e. Region West Europe

f. Customized App Service Plan as AppServicePlan-Linux(f1)

g. Pricing plan Click Change Size from the popup select Dev/Test and select F1 for free of cost then click Apply

h. Click Next: Docker

i. On Docker tab Options as Single Container

j. Image Source as Azure Container Registry

k. Registry as DockerCICDDemo

l. Then Image is automatically comes front since, this is the only image available under the registry DockerCICDDemo

m. Click Next and click Yes in Application Insights under Monitoring

i. Create new Application insight as AppInsightsDockerDemo

ii. Region as West Europe

n. Click Review + Create then click Create

3. Now, we have Azure Devops CI, ACR(Azure Container Registry), App Service for our API are ready. Next we are going to create Release CD pipeline

Step 6 How to create Azure Release Pipeline CD

With this release Pipeline we are going to make the successfully build projects to move into server to publish.

1. Go back to azure devops, Click Releases under Pipelines then click New Pipleline

2. From the next window choose Azure App Service deployment and click Apply

3. Rename Stage 1 into Dev

4. Click Add an artifact, from the popup project is by default got select then choose the build name DockDemoCI and click Add

5. Click on Flash symbol like below screenshot, there Enable Continuous deployment trigger this will make the deployment happens when the build CI is succeeded .

6. Click 1 job 1 task from Dev it

7. On the Dev Azure Subscription as Service Subscription

a. App type as Web App for containers(Linux)

b. App service name as DockerCiCdDemoApp

c. Repository as dockercicddemo, this we can copy, paste from Azure portal under Azure Contrainer Services under Repositories (these we have created in earlier steps, FYR)

d. Registry or namespace as dockercicddemo.azurecr.io this is also we have created earlier, please refer below screenshot

e. Change New release pipeline into CD DockerCD-Demo

f. Click Run on agent

g. Agent pool as Azure Pipelines

h. Agent specification ubuntu-20.04

i. All others are by default

8. Click on Deploy azure app service

a. Connection type as Azure Resource Manager

b. All others are ok just keep as it is, click Save

9. For the first time we do manual start the service since, now we didn’t made any new code change and push into devops so, CI wont run automatically because of this CD wont start automatically so, we are going to manually make it start Create release button from the top

10. From the next screen Dev Pipeline flash and click Create button then a small div opens and says that Release 1 has been created , click on that, on the next screen Mouse hover on Dev, Click on Deploy