Understanding DevOps Artifacts and How to Use Them
To move quickly in large development projects, developers know to use immutable artifacts so the build progresses exactly the same through testing, pre-production and production. But what is an artifact in DevOps? Essentially, it’s any file or set of files produced as part of the development process – these could be binaries, libraries, containers, or even documentation. An immutable package gives you the most confidence that the package that was tested is what will be deployed to production.
It is the job of the build pipeline to create these immutable artifacts, with multiple types of artifacts available depending on what purpose it serves. Read on as we provide a detailed answer to the common question “what is a build artifact?”
What are artifacts in DevOps
- Deployment
- Library
- Bundle
- Pipeline
Types of DevOps Artifacts
DevOps artifacts serve as the critical checkpoints of the development pipeline, embodying the code’s journey from ideation to production. They are essentially the byproducts of the build process, ranging from compiled binaries to libraries required for software deployment. Each artifact is a snapshot in time, representing a stable and version-controlled state of the application.
As we dive deeper into the types of DevOps artifacts, we will unpack their complexity to illuminate their pivotal function in a robust DevOps strategy.
Deployment Artifacts
Deployment artifacts typically take the form of a released executable for public consumption. This includes deb, rpm, exes, tar.gz (single binaries), msi, img, or any other form of release. Lately, a popular destination for such an artifact is Github Releases. Other package management apps like pip, npm, apt, chocolatey, or nix can also be a target for deployment artifacts.
Arguably, the most popular deployment artifact in the industry at the moment is the lowly container image.
Library Artifacts
Library Artifacts are very similar to deployment artifacts. They can even end up in some of the same artifact repositories. Library artifact destinations include; maven, npm, pip, or nuget. One key element that differentiates a library artifact from a deployment artifact is that a library’s purpose is for use in other development efforts.
If you are developing some application and a version of a library randomly starts acting differently there can be very real downstream issues. Because of this, of utmost importance is that the library artifact is immutable and permanently versioned. That is, we should not be able to overwrite a particular deployed version of the library once the library artifact has been published.
Bundle Artifacts
I struggled to put bundle artifacts in the list. This is because it really is not one artifact but a grouping of several together that comprise a particular deployment or need. Typically the destination for a bundle artifact is not a repository or artifactory but rather a drop zone such as cloud storage or a virtual machine (or cluster of virtual machines). The end result is the same though, a group of binaries and configuration are packed up for downstream deployment or usage.
It is worth noting that it is entirely possible to bundle up artifacts into container images that are then unbundled further down in a deployment operations. Most notably, the Immutable Configuration pattern for Kubernetes deployments would employ this technique.
Pipeline Artifacts
- Azure DevOps Pipeline Artifacts
- Github Workflow Artifacts
- TektonCD Pipeline Task Input/Output Artifacts
- Jenkins Archive (workspace) Artifacts
Often, a DevOps pipeline will rely on Pipeline artifacts when getting started or when a centralized artifact repository simply is not available. This is complicated by the fact that some platforms (most notably, Azure DevOps Classic Pipelines) have created whole ecosystems around treating them as first class citizens within the DevOps process.
DevOps Artifact Best Practices
You can see that the plan bundle can be pushed into a pipeline artifact as a holding area. This is where dedicated reviewers would approve/deny the deployment into an environment. A Terraform plan bundle is perfectly suitable for a pipeline artifact as it meets both criteria I personally use to justify pipeline artifact usage:
- Is the artifact part of a one-way or single use operation? (Yes. It is unlikely you would use a prior Terraform plan bundle to roll back operations.)
- Does the artifact only apply to a single target environment? (Yes. It can ONLY ever be used on a single target environment.)
Do yourself a favor, don’t design your pipelines around using pipeline artifacts as your sole artifact type. Relying heavily on such ephemeral artifacts can weaken your overall pipeline independence and reduce the pipeline code base re-usability. They also makes rolling back to older versions of a deployment difficult, especially if your executed pipeline history is only kept for a short period of time.
While understanding the specific use and limitations of pipeline artifacts is crucial, it’s also important to consider broader artifact management practices. These general strategies complement a detailed approach that ensures both precision and efficiency in managing DevOps processes.
Regular Artifact Cleanup
Keep your artifact space clean and efficient by regularly removing old or unused items. You will declutter your repository and ensure that your team focuses only on relevant and current artifacts.
Naming Conventions
A clear naming system helps everyone. Using consistent and understandable names for artifacts helps team members to quickly find what they need without confusion.
Audit and Usage Tracking
You should keep an eye on how artifacts are used by performing regular audits to provide insights into their impact and effectiveness.
Prioritize Security
Security should never be an afterthought. Ensuring your DevOps artifacts are protected against unauthorized access and breaches is crucial, especially when they contain sensitive information. Implement strong security measures to keep everything safe.
Controlling Access to Artifacts
It’s important to manage who can see and use your software artifacts. This translates to setting up rules to control access, so only the right people can use or change these important pieces.
