So you (like myself) have been undergoing a perilous voyage through the landscape of cloud-native and container technologies. Mind boggling isn’t it? Maybe you’ve explored the world of ingress controllers with NGINX, or monitoring and observability with Prometheus and Grafana. You might have even become comfortable enough to deploy your own application as a Kubernetes deployment, and hook it up to a service and ingress for public access. Maybe you’ve done a couple of or none of these things, it doesn’t matter. You are here to get an answer to a question that has left you quaking in your boots for far too long:
“What on earth is CI/CD?”
My mind floats back to the conferences and presentations where CI/CD tools and their endless features and properties were eluded to as if they were as clear as the air we breathe! In those instances, I was left needing some more context. After hours trawling through documentation, before slowly decrypting this exciting, yet daunting segment within the behemoth of a landscape that the CNCF provides, I now feel prepared to extend my hand out to you, fellow voyager, to bring you some clarity on the subject, once and for all.
Trying to make sense of the CNCF Landscape
What even is CI/CD?
Let’s cut to the chase. CI/CD stands for “Continuous Integration & Continuous Delivery”. Pretty obvious right? No? Good, because in reality, CI/CD is a very ambiguous topic, holding different meaning to different people. So, let’s break it down.
Continuous integration - signed, sealed
To assist you through with understanding this concept, I find that it is helpful to imagine the quality control process that might take place in a factory. Similar to what is seen in this video, CI involves all the steps taken before the phone is approved for shipping, checking for all minor imperfections that might exist.
Wouldn't be nice if we could replace this guy with a robot? 🤔
Let’s Git things going
These days, when an application is written, it is typically stored in a place that makes accessing the source code easy. Git hosting platforms such as GitHub, make the life of developers easier, as it turns programming into a team sport. Want your friends to look over the code you’ve written? Create a pull request and let them critique your changes. Do you think that Jane might be the perfect person to solve a bug that has appeared? She can checkout your branch and make the necessary changes, before pushing it for you to see the changes she made.
Partner with experts to verify, maintain and optimize your Kubernetes production environment
Let the machines do repetitive tasks 🦾
The good news is, the benefits of Git certainly don’t stop there. If the code that’s being written can be accessed by any human, why can’t we leverage the machines that run our applications, to perform some of this reviewing and oversight for us? Provided we give the machine some instructions, this is where Continuous Integration comes into its own. With the help of this handy little concept, we can use existing and even create new software tools to perform all of the mundane checks you can think of.
Does it even build? 👷♀️
At a basic level, you can compare CI tooling to tools like spellcheckers in word processors. There are CI tools out there that can not only check for syntax and formatting errors in your code, but also other issues, such as finding unused code, inefficient functions, potential security vulnerabilities and more. A good example of this is golangci-lint, which provides an extensive list of golang “linting” tools, that can all be executed against your code in a CI Pipeline.
So you can see a tool like golangci-lint in action
Hold on: What’s a pipeline? 🚇
Sorry, I was getting a bit carried away! A pipeline is simply the word used to describe a list of steps or ‘jobs’ that are performed against any new changes to an application, before it is brought to production. This term tends to differ slightly from tool to tool (e.g. in Github Actions, it is referred to as a ‘workflow’), but for the most part, they are all the same thing. In the case of this blog post, we have two stages to our pipeline; the CI stage, and the CD stage 😃.
Does it do what it says on the Tin? 🥫
This is where continuous integration gets personal. Believe it or not, it’s good practice these days to write code that tests your code 🤯. Yes, you heard me.
Development and quality assurance (QA) teams write such tests, with the intention that they can be executed as steps in the CI stage to test that each part of the application is still working, even after any new features have been added. Think about it. How annoying would it be if you added a small visual change to your application, only to find out that it caused something to break elsewhere, after pushing it to production 🤦♀️.
Unit tests - an example
I thought I’d make a more practical example of what one of these mysterious tests might look like, and how they relate to the application being written by the developer. Close your eyes and imagine you are on a tropical beach in the Caribbean, sipping a Mojito and writing an application in Go 🍹… Okay maybe that is a bit far fetched, but alas; You have written the function that is displayed below:
package main
func Sum(x int, y int) int {
return x + y
}
func main() {
Sum(5, 5)
}You think to yourself, “my function called Sum() seems to be working perfectly now… but what if things get more complicated when I start writing the functions that calculate the Boolean Pythagorean Triples problem… I know right? How frightening! To save yourself, you create a new file called tests.go and write a ‘Unit Test’ function called TestSum() that serves one sole purpose; Testing that my Sum() function still works:
package main
import "testing"
func TestSum(t *testing.T) {
total := Sum(5, 5)
if total != 10 {
t.Errorf("Sum was incorrect, got: %d, want: %d.", total, 10)
}
}When does it run and where can I see it?
So things are looking great! We have our tests ready to go and now we can ensure that all changes to our applications are assured for their quality, before they get thrown in front of our large audience of users.
I wouldn’t be surprised if at this point you still had a couple of questions. How do we specify when these tests are going to run? Where can I see the status of these tests? I don’t need another dashboard surely 😩. Wouldn’t it be nice if you could open a web browser, go to a GitHub pull request and see a big green tick ✅ or a red cross ❌ against it, to signify that the tests passed and the changes can be trusted? This is what CI services such as Jenkins and Travis offer, by offering the ability to link themselves directly to Github. Features like this are what will make CI your new best friend.
Why test manually when you have a friend like Travis?
Wrap it up! 📦
As a final step in our CI pipeline, we might want to neatly package our new features for production, by performing a trusty docker build and docker push.
Continuous delivery - delivered
The idea of CD involves the automation of the processes that ‘delivers’ any changes to the production environment at a click of a button, with as little interruption to the client side as possible, but preferably none. Continuing on from our factory analogy, we’ve checked our product for any abnormalities and prepared it for delivery. There seems to be no reason why we can’t open the flood gates and deliver the fruits of our labours to the world. So how are we going to make the the process of sending it out to our customers seamless? Time to explore.
An example of a CD tool is Tekton - Here's what the UI looks like 🙂
Don’t sweat it - Let the machine roll it out to production! 🚀
If we can use machines to take care of the mundane and repetitive CI tasks, doesn’t it make sense for us to do the same for the delivery? If you instruct it to do so, a process running in a pod (or on a regular server / vm for that matter) can pick up the latest and greatest changes to our manifests and docker images, and patch them onto any number of production systems we might have running out in the wild. By adding CD jobs inside your pipeline, engineering time is saved and sweaty palms are avoided, removing the possibility of those all too familiar human failures that lead to the ever dreaded, production outage 🚨.
Why can’t I just do it?
For anyone who has spent time in the world of software, they will be all too familiar with how possible it is to make catastrophic blunders when they’re not paying attention. Some of my fondest memories of exploring the Linux world for the first time were those moments where I was showing my friend how cool it all was, only to get a bit too excited and rm -rf / the whole filesystem. It was bad enough taking my whole home server down back then, but at least I had no real user base that cared. Best to make sure that the same thing doesn’t happen to our production Kubernetes cluster, right 😕?
CD can help avoid the risk of engineers getting trigger happy on production systems.
If the instructions are written on a page, they can go through CI!
So you decide to put your trust in machines for deploying your changes, and you write an elegant script or config file for your CD tool to execute, in the language of your choice. Could it get any better? Well, what is even more excellent news, is that now you’ve written the ‘machine’s guide to delivery’, not only can other team members review it, but you can take advantage of your brand new CI pipeline to ensure everything is squeaky clean and ready to go. Thanks again, trusty CI robots 🙂.
To be Safe - Put it in staging first.
Another advantage of writing a CD job for the delivery process, is that you can add some detours for the sake of extra reassurance and verification processes. Maybe there is a bug that, by this stage, has been stealthy enough to avoid being flagged up in the CI process? Maybe your boss just doesn’t believe that you managed to get their 20 year old Fortran application running in a stateless Docker container? Well, this is where a staging environment would have you covered. Simply add a step inside your CD pipeline that takes your newly built fortran image and rolls it to a like for like clone of your target environment, but only available to those internal to your organization. The CD process can then be paused, allowing time for any manual checks to be performed before rolling out to production. Behind the likes of a ‘Google Sign-In’ page, and away from unwanted eyes, your manager can login and look for himself, before eating his own words and letting you know that you’re good to go. “Resume the pipeline, let’s release this thing to the world”.
With tools like Argo, you can give yourself a big shiny "Approve" button to press, allowing you to decide exactly when to send it out to production!
But wait, there’s more!
So that’s it. That is the magical world behind the covers of the words CI/CD. Hopefully you are feeling more confident, because the fun definitely does not stop there. You’ve opened a can of worms into a whole new world of tools that will make your life easier, and I’m sure you are itching to get started. If that’s the case, I would encourage you to head over to a blog post by fellow team member Luke Addison, who takes a look into what makes CI/CD “Kubernetes Native”, and how such tools can be leveraged to manage a fleet of walking, talking, Kubernetes Clusters.
When our experts are your experts, you can make the most of Kubernetes
Cover photo by Ivan Bandura on Unsplash
Machine Identity Security Summit 2024
Help us forge a new era of cybersecurity
☕ We're spilling all the machine identiTEA Oct. 1-3, but these insights are too valuable to just toss in the harbor! Browse the agenda and register now.