How I improved my application configurations

How I thought I found a silver bullet for application configs, but it turned out I overcomplicated things again for my tiny application.

By the way there is a Youtube video I made recently where I show all the details of my Pkl experiments and it is based on the same notes this article was written.

Years ago, when I started working as software developer I had no major choice of how to configure my application. I was using PHP and Javascript and mostly frameworks or platforms, like Symfony and Magento. Back then (~ 10 years ago), uploading files over FTP was not a completely obsolete strategy yet, however it became classy to configure CI/CD for handling deployments.

While Symfony had some degree of choice, between plain PHP config with associative array (map), YAML and the king of that days: XML. Magento 1 was less loose on this: layout was controlled via XML files, and there was the env.php for application configuration, like database connection and similar configurations.

When I started to make friends amongst DevOps team I learned this is a huge pain in... surprisingly raising the headache. The was the moment I learned about 12 factor app. If you have not heard or revisited it too long time ago I encourage you to go there and refresh the manifest. It is still very important for modern development, despite I have not noticed it has changed or has been updated recently.

Today I am sharing my story of learning & experimenting on the topic from chapte #3: Config

"Store config in the environment”

A very straightforward and simple definition. Everything, that might change during the deployment should become a part of environment. How? I bet you’ve been using environment variable at least once. If not - you should try, but I must warn you it is so easy to get used to it.

Getting back to my story it is hard to recall all the details, but I bet there were some scripts that were trying to merge configurations for the environment with configuration that was already passed (commited). And that did not work as a robust solution, causing issues from time to time and requiring a lot of attention from DevOps or team leads to get new team members familiar with the details of the deployment. Overall: it was getting us towards separating config, but it was such an overhead.

When Magento 2 arrived it introduced support for environment variables and it was such a great change. But it turned out there are now multiple configuration files, one containing the sensitive information and the other - overall application’s configuration. A while later - a new possibility of building the application without a full configuration was introduced. But again - it had some quirks over there, so we had inject a very minimalistic, dumb config.

As a CTO back than I was working on implementing Docker containers, that should support a lots of differences and reduce the time of onboarding new team members and bootstrapping new projects. And it is hard to measure how much effort and time I have spent to figure out that quirks and find out workarounds. The worst part of it is still related with Composer plugin, that silently was destroying the production deployments crafted by DevOps. But I’ll save it for the other article.

So after spending all these weeks on figuring out how to configure the Magento 2, what config it needs to build succesfuly and many fixes we have done it. I and a small team of colleagues we introduced a set of scripts and docker-compose files along with documentation and pre-defined environment variables, so docker based setup could be enrolled everywhere. Exception was production environment, as usually it was resource hungry and infrastructure was crafted based on requirements.

A while later I drifted away from Magento, but stayed with PHP for a while longer. Environment variables were adopted by Symfony and other frameworks, but it is hard to get rid of things and approaches we’re used to, isn’t it?

Starting my journey with Go

Despite a couple of attempts to start working with Go it was not an easy move to switch. Luckily I got a chance to work on a side project part-time, where I could spend a lot of time on research not wasting client’s resources and not being worry about being unproductive. I was fine with going an extra mile so I made a switch.

I thought that was the best thing I ever did. I still enjoy working with Go, despite it has own weak sides. But configs... since any Go app is compiled into a single binary the best way to configure it is to use environment variables. I was so happy about that. “Finally I won’t have to create stupid scripts to override values with regexps, sed or other workarounds” - these were my thoughts. And it worked well for a while, while the project was fresh.

But even small projects are growing, more options are introduced and it turned out that organizing environment variables requires some attention. Maybe you’d like to prefix it, maybe you’d like to follow a convention but in any case you still need to document it. And it becomes a crucial part of the app. how-to-deploy.md might become a nightmare, I realized that working on a different projects. You constantly need to think about it, you must share the knowledge and decisions with anyone: your team, DevOps team, clients... And the worst part of any documentation: to keep it up to the date.

I thought I found a bright light

Turned out it was just someone’s flashlight, highlighting the best aspects of it. Earlier this year I found out press release about Pkl language: a language dedicated for configurations. The best part: it is possible to define validations and even logic, based on properties or parameters, it is capable of reading environment variables!!! and... it had bindings for Go. And even more: it is always 1 command away from becoming a JSON or YAML. Open source, released by Apple.

“Aha, I found it!” I thought. Went over the documentation to figure out the set of features I’d like to start and I implemented it for my small project. If you have another approach to learning new technologies let me know about it.

It did not went very smoothly, while I got used to the way Go structs are generated based on pkl definitions, but overall I got what I wanted. Until I deployed it.

First of all: there are some bugs there, that just crashes the runtime which evaluates config. But that’s not the biggest problem. The disaster for me was when I passed the wrong config at some point. And my app crashed. Fatal. I was experimenting, so I knew what I’ve done, but there was no way to prevent it. Because my root config struct was relying on embedded properties, that were pointer to other structures. If you’re familiar with Go you already know what was the problem. All the pointers were nil’s, while config itself was absolutely legit from Pkl perspective.

And yes, envvars was working just fine. But in my wish to organize and create reusable module configs I went with not enough parameters, that were read from envvars. After I realized my mistake and revisited the definitions I realized that most of configuration properties should be passed as environment variable, so they could be controlled without rebuilding images.

At that point I realized my application must be huge. It should be much bigger with a multum of internal configurations, that could be more static and passed and kept as files. These files should not change much, so they won’t require image rebuilds too often. I understood that I forgot the lesson I had with Magento and went into another round battling own wrong decisions.

Why Pkl was looking so shiny to me?

I pointed out above, that managing environment variables requires additional attention, mostly for supporting team members including keeping documentation up to date. With Pkl I saw a possibility to reduce that effort with configurations that are self-documented. It is easy to define enum and make myself perfectly clear on the options, that are accepted for a specific property. It is easy to define validation, that is descriptive as well and provide a clear feedback (error). And it is all possible. But I decided to leave the Pkl on my devtool shelf for a better times.

It is still a part of the project, but I have reduced the structure to the bunch of files and will be migrating back to pure environment variables pretty soon. And in the next section I will provide my reasons for that.

Staying with env vars

The project I am referring to is my personal, relatively small. The team is just me. I am perfectly fine to re-visit example.env to figure out what are the envvars available and to remember what the values are. And even figuring it out going through the source code if my assumptions won’t work. There is no issue with documenting envvars for a single-dev-ops team

I am used to pack my software into the container. I’m loving go for the fact, that it can be shipped as a single binary, despite I rarely use it (i.e. building on my Mac apps that are running 24/7 on my Raspberry Pi server). And with Pkl I have to do so many additional moves:

• I have to take care of configuration files themselves, these must become a part of the build. This is easy, until I don’t want to override it. And if I’d like to do it for environment? Means I’m doing something wrong, and it should be envvar. As I said - this is 95% of my existing config, so I am just dragging more files into the docker image.
• Environment must have Pkl runtime. It is necessary to install and probably update in the future. Breaking changes? Good luck with migrating configuration module. I have not even mentioned modules are supposed to be published. And this is a good thing for a bigger team, but definitely not good enough for a solo developer.
• Even configuration is parsed and read into Go structs I can not rely on it. I have described above why exactly it happens, but if I am using configuration for some delayed tasks I may find out my application being crashed for hours due to misconfiguration, despite all validation have passed. The binging itself is not reliable. And the only suggestion to deal with the problem I found out was to have a test that regenerates Go’s bindings (config structs) and validates if VCS has not changes. I have a feeling this is rather a suggestion for layman.

Simplicity is my choice

Taking into account:

• almost entire configuration should be available for configuration with envvars;
• there is an overhead to bring Pkl runtime everywhere my app goes;
• and the fact configuration is not reliable, even it has all the validations, fuses and flexibility; I decided to not continue with the technology.

Please don’t get it wrong, if I would have more scenarios or more complex app to configure most probably I would be happy to use it further. If I would go with Kotlin or Swift, that might have better bindings and supports that might works as well. Pkl definitely requires some scale, bigger scale to stand out, not a tiny Go application that could be shipped by a single developer with a few dozens of variables.

There are couple of similar products on the market like, at least Cue was the one that popped up pretty fast. However I had no experience in implementing such solutions from day 1. I’ve been a developer of the platform, where all decisions were already made and I had to figure out which compromise would work the best. Now when I have real choice and more experience my choice is to keep it straightforward, staying with environment variables. Solution that might require additional cognitive effort in the beginning, but is straightforward and reliable.

Thank’s for your attention and keep GO-ing! :)