Unable to understand the benefits of r10k dynamic environments..

asked 2017-05-25 18:42:22 -0600

Vishwa
1 ●2 ●2 ●2

We use Puppet, hiera, r10k with a number of forge modules, roles and profiles patterns to perform an automated deployment of a Java/NodeJS Application.

I find r10k to be very useful in fetching all dependent modules from a Puppetfile. However I am confused with the benefits of the r10k dynamic environments. We have 8 environments, so we have created 8 branches in Git. The important feature is that all these 8 environments are identical except for heira properties.

Whenever we make a change in one of the puppet manifests(roles or profiles), we manually have to merge(sometimes cherry-pick) these changes from 1 branch to the other, when we are ready to do a new deployment in another environment.

Managing code in 8 branches is appearing to be a time-consuming and an error-prone process to me. I am thinking of moving away from r10k dynamic environments usage and have only 1 branch (master). Whenever we deploy anything to Production env, we just tag the appropriate commit with a Git Tag.

When the business application code that the development team are writing does not have a Git branch for every environment, why should R10K mandate that we create a branch for each environment containing the same puppet code?

I was wondering if any body has benefited from using R10k dynamic environments for automated deployment of an Application and can share their experience.

edit retag flag offensive close merge delete

Comments

I second that. In addition, I would be interested whether I can use `r10k` just for installing and uninstalling modules listed in my `Puppetfile` (as a drop-in replacement for `librarian-puppet`, if you want so), without the "side-effect" of maintaining environments for feature branches.

bittner ( 2017-05-27 12:01:55 -0600 )edit

I believe what we see is a concept implementation that has gotten out of control. Using branches for configuration management is an anti-pattern. Git isn't meant to be used as conf mgmt tool. There should be a single branch for all environments.

bittner ( 2017-05-27 12:27:53 -0600 )edit

add a comment

2 Answers

Sort by » oldest newest most voted

answered 2017-05-28 10:30:16 -0600

Thomas
11 ●2

updated 2017-05-28 10:31:55 -0600

The important feature is that all these 8 environments are identical except for heira properties.

So I would suggest you to think about your hiera hierarchy. Maybe this needs looking into an external node classifier to provide the data to introduce a layer for the differences. Then you could eleminate 7 envs.

edit flag offensive delete link

add a comment

answered 2017-05-28 05:37:36 -0600

bittner
1 ●1 ●1

updated 2017-05-28 05:42:50 -0600

While I can't give an answer to the question (because I fail to see a benefit from using branches for deployment) I would like to present my assumtions on why this approach has probably been chosen, and why this is actually wrong. Saying this, I hope that the approach r10k is taking is changed in a future release.

I suspect the idea to use Git branches for deployment comes from gitflow (as a role model). The gitflow branching model, though, does not state that branches are meant to be long-lived.

The ultimate goal of gitflow is merging changes into master. All the other many branches are only there to accommodate changes that happen in parallel (bugs that need to be fixed, features that need to be developed, a release that needs to be prepared), but one day or the other any of those changes will land in master.
r10k follows a very different idea. Any branch is meant to have a parallel existence. Branches represent (permanent) environments. Hence none of them is meant to be merged with any other branch, ever. This contradicts the very idea of version control, and CI, which is to "merge changes into the mainline".

Using Git (branches) for maintaining environments that are meant to live in parallel is wrong. There are other strategies that fit this scenario better. Especially the problem of inheritance and mixing in behavior is made impossible to solve with Git branches.

The correct approach is putting all environments in the file system - just what the directory layout of default Puppet (/etc/puppetlabs/code) looks like -, and use a single branch under version control - just like in regular software development. Hiding them in version control branches is only making things very, very complicated.

edit flag offensive delete link

Comments

I believe the intent is that you should have a single long lived branch 'production', and any other environments should be short lived. (testing/exploration). If the only difference is your hieradata, then it seems like you should have a hierarchy which can have all of your data in it

DarylW ( 2017-05-30 10:53:15 -0600 )edit

Even if you are doing something similar to gitflow, you should have the same info (at different stages) in each branch with the expectation to converge.

DarylW ( 2017-05-30 10:54:07 -0600 )edit

I do feel that r10k feels 'inside out', you should pull in all your dependencies and create a deployment which has your code, that you then deploy to the appropriate server(s) (puppet master). Similar workflow was described in a talk from pconf2016, Scaling Puppet with ECS

DarylW ( 2017-05-30 10:56:35 -0600 )edit

thanks, we have merged all the branches into a single master branch. All environment information has gone into respective folders/files in hiera. We have short-lived branches (feature branches for developing/testing purposes). Source code management in Git appears to be a lot better now..