How can we make applications easy to operate? The 12-factor methodology is about 13 years old. How did it age in the cloud-native era? Do we need a 13th factor? In this blog, Tibo explains why you should keep an eye on the 12-factor and how it's still useful in this day and age.
In a presentation about CI/CD I gave recently, I briefly mentioned the 12-factor methodology. Somewhere along the lines of “You might find some good practices there”, and summarizing it as:
artifactconfiguration +---------------deploymentAfter the talk, a colleague of way back came to me and said: “You were way too mild in suggesting it. It’s mandatory, people should follow those practices."1
And yes, he was right. There are a lot of good practices to get from the 12-factor methodology. But do all parts still hold up? Or might following it to the letter be actually counter-productive in some cases?
In the past, I have onboarded quite a number of applications into Kubernetes, that were already built with '12-factor' in mind. That process usually was fairly smooth, so you start to take things for granted. Until you bump into applications that are tough to operate, that is.
Upon closer inspection, such applications are usually found to violate some of the 12-factor principles.
The 12-factor methodology has been initiated almost 13 years ago at Heroku, a company that was ‘cloud native’, focused on developer experience and ease of operation. So, it’s no surprise it still is relevant.
So, let’s glance over the 12 factors, and put them in the context of modern cloud-native applications.
One codebase tracked in revision control, many deploys
Looking at the image, these days we would add artifact between codebase and deploys. Artifact being a container, or perhaps zip file (serverless).
code -> artifact -> deploy- versioned - container - prod - zip - staging - localIt’s worth noting that for local development, depending on the setup, some form of live-reload usually comes in place of creating an actual artifact.
Explicitly declare and isolate dependencies
This is something that has become more natural in containerized applications.
One part of the description is a bit dated though: “Twelve-factor apps also do not rely on the implicit existence of any system tools. Examples include shelling out to ImageMagick or curl.”
In containerized applications, the boundary is the container, and its contents are well-defined. So an application shelling out to curl is not a problem, since curl now comes with the artifact, instead of it being assumed to exist.
Similarly, in serverless setups like AWS Lambda, the execution environment is so well-defined that any dependency it provides, can be safely used.
Store config in the environment
This point is perhaps overly specific on the exact solution. The main takeaways are:
Confusingly, and especially with the rise of GitOps, the configuration is in a codebase, but detached from the application code.
As long as the above concept is followed, using environment variables or config files, is mostly an implementation detail.
Using Kubernetes, depending on security requirements, there might be considerations to use files instead of environment variables, optionally combined with envelope encryption. On this topic, I can recommend:
Treat backing services as attached resources
This has become common practice. In Kubernetes, it’s usually easy to configure either a local single-pod (non-prod) Redis or Postgres, or a remote cloud-managed variant like RDS or Elasticache.
There can be reasons to use local file system or memory, for example performance, or simplicity. This is fine, as long as the data is completely ephemeral, and the implementation doesn’t negatively affect any of the other factors.
Strictly separate build and run stages
From Kubernetes to AWS Lambda: It will be hard these days to violate this principle. Enhancing the aforementioned summary:
Build -> artifactRelease -> configuration +--------------------------Run -> deploymentExecute the app as one or more stateless processes
In the full text, there is a line that better summarizes the point:
Twelve-factor processes are stateless and share-nothing
Some takeaways:
Somewhat overlapping with factor 4, this factor implies using external services where possible. For example: Use external Redis instead of embedded Infinispan.
Export services via port binding
This holds up for TCP-based applications. But it is no longer applicable for event-driven systems such as AWS Lambda or WASM on Kubernetes using SpinKube.
Scale out via the process model
Make your application horizontal scalable. This is somewhat related to factor 4, which result in share-nothing application processes.
Furthermore, the application should leave process management to the operating system or orchestrator.
Maximize robustness with fast startup and graceful shutdown
In a way this can be seen as complementing the previous factor: Just as it should be easy to horizontally scale out, it should be easy to remove or replace processes.
Specific to Kubernetes, this boils down to:
SIGTERM signal in the application, or setup a PreStop hook (more info).OK when the application is actually ready to receive traffic.maxSurge (rolling updates) and PodDisruptionBudget(scheduling).Keep development, staging, and production as similar as possible
This is a broad topic and as relevant as ever. At a high level it boils down to ‘Shift left’: Validate changes as reliably and quickly as possible.
Solutions are many, and could include Docker Compose, VS Code dev containers, Telepresence, Localstack or setting up temporary AWS accounts as a development environment for serverless applications.
Treat logs as event streams
Don’t store logs in files. Don’t ‘ship’ logs in the application.
The operating system or orchestrator should capture the output stream and route it to the logging storage of choice.
Where the 12-factor methodology shows its age a bit is that there is no mention of metrics and traces, together with logs, often referred to as “the three pillars of observability”.
Extrapolating the approach to logging, consider systems that ‘wrap’ an application instead of requiring a detailed implementation. OpenTelemetry zero-code instrumentation could be a good starting point. APM agents of observability SaaS platforms such as New Relic or Datadog can be applied similarly.
Run admin/management tasks as one-off processes
This fragment in the full description might summarize it better: “Admin code should ship with the application code”.
This is about tasks like changing database schema, or uploading asset bundles to a centralized storage location.
The goal is to rule out any synchronization issues. Keywords are:
As long as we try to grasp the idea behind the factors instead of following every detail, I would say most of the factors hold up quite well.
Some recommendations have become more or less common practice over the years. Some other recommendations have a bit of overlap. For example: Externalizing state (factor 4) makes concurrency (factor 8) and disposability (factor 9) easier to accomplish.
There is a point not addressed in the 12-factor methodology that in my experience has always made an application easier to operate: Backward and forward compatibility.
These days we expect application deployments to be frequent and without any downtime. That implies either rolling updates or blue/green deployments. Even blue/green deployments, in large distributed platforms, are hardly ever truly atomic. And deployment patterns like canary deployments, simply being able to roll back.
So, getting this right opens up the path the frequent friction-less deploys.
This is about databases, cached data and API contracts. We need to consider:
N and N+1 are running simultaneously?N+1 to N?Some pointers:
What will happen with data in the transition period? Store the data in old and new format? Do we need to store version information with the data and support multiple versions?
This can be complicated for applications provided for others to operate, unlike applications operated by the developing team itself, and released via CI/CD. External users often don’t follow all minor releases, making it more likely to not have backward compatibility.
Some of the above recommendations might take additional effort. However, in my experience that is worth it and will be paid back (with interest) by ease of operations, peace of mind and a reduced need for coordination of releases.