Automation is Key to Getting the Most Out of the Cloud
The cloud offers many benefits to companies that move their infrastructure there or develop applications using cloud native services, but automation is the key to making the cloud efficient and cost-effective. We have to get into the “way back” machine for a second to return to the days of on-premise servers, storage, and network gear to get a full grasp of how important automation is. In the old days, IT was typically siloed into four or five teams that all had expertise on different IT components: server team, storage team, network team, application team, and security team. When a basis move, add or change needed to be made the entire team was not only consulted but also the played a role in making the change happen. This made basic changes time consuming and fairly complicated to orchestrate. Additionally, changes were typically done manually because the siloed groups almost never created cross-functional tools or scripts because of the siloed organizational structure. This drove up cost and slowed down the pace of innovation.
Ok, now let’s jump forward into modern times. The public cloud offers a new way to manage change for IT organizations and it is forcing IT professionals to evolve from being builders to becoming orchestrators. In the public cloud, IT professionals can now perform cross-functional changes using one common and ubiquitous console that has access to servers, storage, network, applications, and security. But that’s not the best part, you can now use a common scripting and API interface to perform all of the orchestration and you don’t need multiple subject matter experts with compartmentalized product knowledge to participate in it. All of the cloud services that you use can be treated as application code which allows for direct programming and automation. Moves, adds, and changes are now developed in code and executed in seconds with a simple command. This accelerates an organizations ability to change and allows them to innovate rapidly.
Let’s explore how you can do this with Azure services, I think you will find it enlightening.
Azure provides a set of PowerShell cmdlets (AzureRM) that use the Azure Resource Manager model for the management, provisioning or updating of Azure resources. However, when executing these scripts, the user is required to log into Azure from PowerShell using the cmdlet Connect-AzureRmAccount which provides authentication and authorization.
If you plan to manage Azure PowerShell, the scripts should be executed under an Azure Active Directory (AAD) Service Principal, rather than using your own credentials.
Azure Service Principal
An Azure service principal is a security identity used by user-created apps, services, and automation tools to access specific Azure resources. Think of it as a ‘user identity’ (username and password or certificate) with a specific role, and tightly controlled permissions. A service principal should only need to do specific things, unlike a general user identity. It improves security if you only grant it the minimum permissions level needed to perform its management tasks.
The following example creates a self-signed certificate, associates the certificate to an Azure Active Directory (AAD) Service Principal and connects to Azure from PowerShell, the AAD Service Principal provides the authentication and authorization.
1. Use Connect-AzureRMAccount to log into Azure from PowerShell using an account that has contributor rights.
2. Execute the following PowerShell commands to create a Self-Signed Certificate, the certificate will be stored in the local (server) Windows Certificate Store.’
3. Execute the following PowerShell command to create an Azure Service Principal.
Technical Note #1
The PowerShell command in Step #2 creates the Self-Signed Certificate along with the Key Value ($KeyValue) of the certificate, the $KeyValue is used as the -CertValue parameter when the Azure Service Principal is created in Step #3 and assigned the Display Name infrastructure.
The Azure Service Principal is created in Azure Active Directory in App registrations.
4. Execute the following PowerShell command to assign Contributor rights to the Azure Service Principal.
5. Execute the following PowerShell commands to retrieve the Azure Tenant Id and the Application Id (the Azure Service Principal) created in Step #3 and the Thumbprint of the Self-Signed Certificate created in Step #2.
Technical Note #2
The Tenant Id ($TenantId) is the Id associated with your Azure tenant, the Application Id ($ApplicationId) is the Azure Active Directory Service Principal that was created in Step #3 and the Thumbprint ($Thumbprint) was created in Step #2.
Connecting to Azure using a Service Principal
6. Using the variables $TenantId, $ApplicationId and $Thumbprint from Step #5 as input parameters, execute the following PowerShell command (Connect-AzureRmAccount).
The Connect-AzureRmAccount PowerShell cmdlet successfully executed and displays the Azure Context information. The context information is the Active Directory account (Account, i.e. Service Principal), Subscription Name (SubscriptionName), Subscription Id, Active Directory tenant (Tenant Id) and Azure Environment (Environment).
Take it One Step Further
Now that the Service Principal has been created, let’s take it one step further and automate an Azure PowerShell script using a CICD environment.
The CICD environment will be Jenkins which provides an automation server for Continuous Integration and Continuous Delivery.
Azure Network Infrastructure
An Azure Virtual Network has been provisioned which contains dev, stage and prod subnets, Azure Network Security groups are associated with each subnet.
The Jenkins server (xprovisioning) is a Windows 2016 Datacenter server, jenkins-2.138.2 and the AzureRM PowerShell cmdlets have been installed on the server.
A Jenkins project titled Create-Update Network Security Group is a Freestyle project which is configured to execute PowerShell scripts.
The Jenkins project will leverage Jenkins Credentials Binding plugin, this allows you to configure and inject credentials or text as environment variables which are stored as secrets in Jenkins.
The Jenkins project Build will execute an Azure PowerShell script Infrastructure_NSG_CreateUpdate.ps1
The PowerShell script will be configured to read the Jenkins environment variables (secret text), connect to Azure from PowerShell using a Service Principal, execute a publicly available Microsoft script (GitHub) that updates an Azure Network Security Group and logs out of Azure.
The Jenkins Dashboard provides access to all the Jenkins projects that have been set up for the environment, click on the Create-Update Network Security Group project.
Jenkins Project Dashboard
The Jenkins Project Dashboard allows a user to modify, build or delete the project. The following is the output of the successfully executed Jenkins project Create-Update Network Security Group.
Putting It All Together
An Azure service principal was created with a Self-Signed Certificate, this provides authentication and authorization for connecting to Azure.
A Jenkins Freestyle project was created that executes the PowerShell script Infrastructure_NSG_CreateUpdate.ps1. The project also leveraged the Jenkins Credential Bindings Plug-in which allowed the Tenant Id ($TenantId) and Application Id ($ApplicationId) created in Step #3 and the Thumbprint ($Thumbprint) created in Step #2 to be stored as secret text.
The PowerShell script Infrastructure_NSG_CreateUpdate.ps1 calls a PowerShell script CreateUpdateNSGFromCsv.ps1, this script updates the inbound security rules for the Azure Network Security Group prod-nsg which is associated with the prod-subnet.
A network security group contains security rules that allow or deny inbound network traffic to, or outbound network traffic from, several types of Azure resources. You can enable network security group flow logs to analyze network traffic to and from resources that have an associated network security group.
Network Security Group – prod-nsg
Before the execution of the PowerShell script, the network security group contained one custom inbound security rule; Priority 100, Name Allow_RDP port 3389. This rule allowed Remote Desktop Protocol (RDP) access to virtual servers in the prod-subnet.
After the execution of the PowerShell script, additional inbound security rules were added, and the priority of the inbound security rule Allow_RDP was changed to Priority 200.
The additional inbound security rules allow inbound web traffic for ports 80 (http), 443 (https) and inbound SQL Server traffic for port 1443.