Practical PowerShell Scripting for DevOps - Part 4
Practical Scripting Challenges and Solutions for DevOps with PowerShell
Table of Contents
Hi, I’m back with you with Part 4 of this series. On this part, I will show you how to check if a particular Git Reference belongs to a certain folder and also has a valid Semantic Versioning format. This type of script is a utility function that helps to fill gaps in particular automation scenario especially in deployment pipelines. Another common term used for these scripts is Glue Scripts. Glue in this context means filling a gap in your toolchain with a custom script and achieve your goals.
As DevOps engineers, we like to use complete and mature CI/CD tools. During my career I had used various CI/CD tools such as Jenkins, TeamCity, Azure DevOps, Octopus Deploy and Github Actions. Although all of these tools has rich repository of official and vendor provided tasks/actions almost all of them has common features to allow you run your own custom scripts. When pre-defined tasks comes short or you can’t necessarily find something that does the trick out of the box, then you have go back to your biggest strength, writing your own custom glue script(s) to fill in the gaps.
Most of these CI/CD tools comes with a concept called predefined variables. These are system generated environment variables that are passed to your runners/workers that allows you to tap into certain platform or repository related metadata depending on the unique pipeline run. Today we will be looking at some sample GitHub Reference strings and find out if they are under a certain folder/path and also has valid Semantic Versioning format.
Semantic Versioning, or in short SemVer format, is a artifact versioning system that is widely accepted that allows engineers who write and maintain software in distinct versions. It is so widely used you’ve most likely came across this format in one shape or form.
Let’s say you are releasing a product or a software service for the first time, your typical version number for this particular release can be 1.0.0 then let say your next release with some features add can be 1.1.0. Let say you’ve detected some defects and want to push some patches to second release, then you can name your new version as 1.1.1. Next release would be 1.2.0 and so on.
Going from 1.x.x to 2.x.x denotes a major version upgrade, of then denotes changes that are MAJOR and to point out that you are doing changes which contains good amount of change in the software. This is the simplest explanation of Semantic versioning and I can’t possibly explain how Semantic Versioning works fully but I urge you to check out this page to fully understand it and google some more to learn about it.
https://semver.org/
I’m going to give examples from GitHub and Azure DevOps but I’m almost certain that all CI/CD tool that works with Git repositories would have something similar.
Check out the pages for each product for predefined/environment variables available to you in their pipelines contexts.
Azure DevOps: https://docs.microsoft.com/en-us/azure/devops/pipelines/build/variables?view=azure-devops&tabs=yaml
GitHub Actions https://docs.github.com/en/actions/learn-github-actions/environment-variables
To simulate that we have one of these variable available to us in our script context, we will start of with a sample input array.
$refsInputArray = @("refs/heads/releases/1.0.0",
"refs/heads/releases/1.a.0",
"refs/heads/releases/1.2.0.5.6.9",
"refs/heads/release/1.1.1",
"refs/heads/releases/1.0.1",
"refs/heads/releases/2.4.3",
"refs/heads/main",
"refs/heads/develop")
As you can see, we don’t get to choose the format we will receive our String Inputs, hence we need to know how to get certain parts of interest in a complex string like this and then perform our checks.
Without further due, lets get in to our problem statement and requirements in a real use case DevOps scenario.
Problem
Hey fellow DevOps Engineer! We have a pipeline for an API service we have, and we have a delivery pipeline that allows us to deploy our code to our serverless hosting platform from a particular branch of our choosing.
Although we give our developers flexibility to deploy to all environments from any feature branch they choose to deploy from, we would like to protect our Production environment. We have strict branch protection rules around releases/* branches and only wish to be able to deploy from these branches to our Production workers. We also use Semantic versioning to version our releases and cut our release branches with the version number of that particular release.
We’ve look into our CI/CD tool’s out of the box and marketplace tasks, but couldn’t find anything particular that can achieve what we are looking for, hence we need your help.
We would like you to write us a PowerShell script to perform a simple test to verify if the Git Branch we are running the pipeline from is actually adheres to our release versioning rules, this would save us from deploying the wrong branch to the Production environment.
1- We would like you to first check if the branch we are deploying from is under releases/ folder.
2- We also like to check if the branch name actually adheres to Semantic Versioning format.
What Success Looks like for this Challenge
Normally, each pipeline run would have singular input for a Git Reference hence the script would normally finish execution with a proper exit code.
Tip: exit 0 means successful execution and exit 1 means an execution with an unhandled exception or in other words a failure.
But because we are doing a mock challenge, in this example, I want you to start your script with the sample input array I’ve provided above and then simply return an appropriate message to the terminal stating if the input string fulfills the requirements or not.
Successful Run:
Some Pointers For Writing Your Own Solution
So let’s breakdown the challenge a little bit. First of all we need to identify the folder of the branch.
In order to do this, we need the portion of the substring between the last two / characters. There may be multiple way of doing this so try to figure out how to achieve this first, once you have that portion then you can actually perform a string comparison to check if it’s equal to releases
If your input passed the first requirement, then basically you need to capture the substring after the last / character. Once you have that then you can’t simply do a string comparison as semantic version number can be multiple things, we want to check if it adheres to a pattern.
PowerShell being a scripting language has operators to make your life easy and allow you to achieve this in various ways, you can devise a PowerShell flavored solution. The most bulletproof method is to use something called regular expressions
At the bottom of the page I’ve provided above on semantic versioning they also provide regular expression to check if a certain string adheres to semantic versioning.
I’m using the second one in section https://semver.org/#is-there-a-suggested-regular-expression-regex-to-check-a-semver-string
^(0|[1-9]\d*)\.(0|[1-9]\d*)\.(0|[1-9]\d*)(?:-((?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*)(?:\.(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*))*))?(?:\+([0-9a-zA-Z-]+(?:\.[0-9a-zA-Z-]+)*))?$
You can also script you own logic to check is this particular string you have adheres to the format by looking at it’s components and if it has numerical values separated by dots in between. However, learning regular expression will help you in various string grouping and filtering scenarios in your cloud and DevOps career so I really urge you to learn and try to understand what they are and how they are used.
Event most seasoned engineers are struggling with regular expressions and so don’t be discouraged if they appear to you as too complex at first. Even a small bit of knowledge goes a long way.
The Main Learnings From the Challenge
The main learnings from this challenge are:
1- PowerShell comparison operators
2- Regular Expressions
3- PowerShell String operator split to divide a certain string into substrings
4- PowerShell Arrays and how to target a particular Array Index
Links:
Matching operators
Tasks You Need To Perform In Your Script
So to give another breakdown of what you need to do in your script,
1- Loop through the input array to check each input string individually
2- Figure out a way to isolate the substring between the last two / characters check if it’s equal to releases
3- Figure out a way to isolate the substring after the last / character and check if it has a valid Semantic Versioning format (I recommend using a regular expression/regex pattern to perform this as it would also familiarise yourself with regex)
4- Print out relevant log messages to terminal that allows you to clearly see the input being evaluated and on failure, what requirement it fails and if satisfies both conditions a relevant success message. (Normally you would either fail or proceed with the pipeline runs according to the result but just printing a message is fine for this mock challenge)
Final Tips
From this point below you will see my sample solution. Depending on your level and learning style, either take a look before or after you gave it your shot.
My Sample Solution
Files for my Sample Solution
Test-SemVerWithRegex.ps1
$refsInputArray = @("refs/heads/releases/1.0.0",
"refs/heads/releases/1.a.0",
"refs/heads/releases/1.2.0.5.6.9",
"refs/heads/release/1.1.1",
"refs/heads/releases/1.0.1",
"refs/heads/releases/2.4.3",
"refs/heads/main",
"refs/heads/develop")
#https://semver.org/#is-there-a-suggested-regular-expression-regex-to-check-a-semver-string
$semVerRegex = '^(0|[1-9]\d*)\.(0|[1-9]\d*)\.(0|[1-9]\d*)(?:-((?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*)(?:\.(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*))*))?(?:\+([0-9a-zA-Z-]+(?:\.[0-9a-zA-Z-]+)*))?$'
$refsInputArray | ForEach-Object {
$refParts = ($PSItem -split '/')
if ($refParts[-2] -eq 'releases') {
if ("$($refParts[-1])" -match $semVerRegex) {
Write-Host "Reference $($PSItem) is in releases folder and has a correct semver format"
}
else {
Write-Error "Reference $($PSItem) is in releases folder but has an incorrect semver format"
}
}
else {
Write-Error "Reference $($PSItem) is not in releases folder"
}
}
This is my solution and as I’ve stated during this part, there are multiple way of structuring your script and achieving the asked outputs. Only logic you are writing matters here, you can choose to have one complex if statement, you can use one complex conditional or opt-in for a switch statement for some more challenge, as long as you are fulfilling the challenge requirements you’ve accomplished the learning goals of this challenge.
Sample Runs
As our script is deterministic, it should always return the same result with the given input.
Conclusion
No matter, the purpose, the fundamental and primitive data types such as strings and arrays are things you will come across in various context no matter what environment you work in. Having a good understanding about these concepts are always beneficial and will allow you to increase your speed in devising solutions for complex problems.
Strings in particular plays a crucial role in DevOps scripting, Infrastructure as Code and pipeline automation as we usually parameterize certain templates and provide string based parameter inputs to those templates to achieve scalable automation solutions. Parametrizing templates and promoting along the software development lifecycle environments with one consistent template with different parameters will give you confidence in going to Production as you will be sure that you are deploying and testing the same set of resources in your lower environments.
This is a mock challenge to give you a medium to learn the skills to be able to work with string and arrays with Powershell, but as you continue along you will see they will come in handy in all type of different situations. I hope I was able to help you learn something new about PowerShell today, and I’ll see you on the next one.
