Ask Your Question

Migrating from Puppet 3.x to Puppet 4 at scale.

asked 2017-08-01 17:04:56 -0600

ramindk gravatar image

I have a very large and very old code base. 500k lines of manifests code. The code base also goes back to 2007. What are the steps to migrate this codebase to Puppet 4?

edit retag flag offensive close merge delete

4 Answers

Sort by ยป oldest newest most voted

answered 2017-08-01 17:29:24 -0600

ramindk gravatar image

updated 2018-04-19 20:13:06 -0600

Hi self. You are in for a world of hurt.

work in progress

Consider this a live blog

Links I found useful

Planned upgrade sequence

The Actual plan we followed was

  • fix code and change all masters to 3.8.7 with future parser enabled
  • Fix code again and upgrade to puppetserver 2.8 masters which supports Puppet 3 agent.
  • Upgrade to puppetagent 1.10.10
  • Cake.

Upgrade Notes

Upgrade to Puppet 3.8.7

Do any puppet 3.x upgrades now. This will make any future work simpler. You can run puppet 2.7 agents against your Puppet 3 masters and even against 3.8 with the future parser enabled. However you do not want to manage differences in agents and masters.

We update to 3.8.7 because it has the best future parser support before going to puppetserver.

Upgrade to Facter 2.4.6

Standardize on one version of facter. At some point you'll need to turn off stringified facts and best to have a single facter version.


I recommend starting with puppet-lint 2.3.x if you haven't been linting your code. The best way to run puppet-lint is with the --only-check and --fix flags. This allows you to test one set of changes at time. As you individually test checks you can add them to your commit hooks.

puppet-lint --only-checks=trailing_whitespace --fix modules/
puppet-lint --only-checks=hard_tabs --fix modules/
puppet-lint --only-checks=trailing_whitespace,hard_tabs --fail-on-warnings modules/

The order you run these checks in can be important. In a large codebase that has never been linted I suggest the following progression. For example it's easier to check for vars when your quoting is correct. --fix will get you 95% of the way there, but you will see corner cases you'll need to manually edit.

--only-checks=trailing_whitespace,hard_tabs,duplicate_params,double_quoted_strings,\ unquoted_file_mode,only_variable_string,variables_not_enclosed,\


You can run this across an entire codebase with the future parser turned on. Defaults to the Puppet that is installed locally. puppet-syntax is great for catching syntax mistakes, but will not catch data mistakes such as regex-ing a int, numeric operations on strings, etc.

Data types will be the majority of your problems in my experience.

A word on running 3.8.x with the future parser

It's a terrible idea

If you can spin up a new Puppet 4 infra and point your code at it, do that instead. Due to the size and history of our codebase we decided to take the incremental path. This was the correct choice for us. Our codebase depended on a number of patterns that were not going to be Puppet 4 safe. Rather than do a large rewrite we instead are trying to incrementally update Puppet to a point where we have access ... (more)

edit flag offensive delete link more


Hi. Thanks for this. Very informative. Are the processes/steps/preparation described here applicable to a jump from Puppet 3.8.7 -> 5.5? i.e. skip pupprt 4 altogether?

jsilverman_blispay gravatar imagejsilverman_blispay ( 2018-06-01 10:30:07 -0600 )edit

answered 2017-08-09 04:52:34 -0600

Henrik Lindberg gravatar image

Puppet catalog-preview is a very useful tool when migrating 3 -> 4. Here is an excellent presentation that shows you what it can do for you:

edit flag offensive delete link more


The presentation says it is PE only, but that is has been released as open source (free).

Henrik Lindberg gravatar imageHenrik Lindberg ( 2017-08-09 04:53:51 -0600 )edit

Go here to get it (also where the documentation is)

Henrik Lindberg gravatar imageHenrik Lindberg ( 2017-08-09 04:58:15 -0600 )edit

Thanks for the heads up. I had heard of the tool, but did not realize it was open source now.

ramindk gravatar imageramindk ( 2017-08-09 14:16:57 -0600 )edit

answered 2017-08-16 11:45:10 -0600

Jeremiah Powell gravatar image

updated 2017-08-24 08:51:25 -0600

Much of moving code from Puppet DSL 2 or 3 to Puppet 'full programming language" 4 is almost script-able. If you do native type development and lots of custom facts then you are going to need a lot more help than the following list.

The following psuedo-checklist applies to Puppet Enterprise, Open Source or whatever mix you run. If you are going from 2 to 3 you might just go straight to 4 with these changes.

  • Drop Modulefile for metadata.json
    • Remove checksums as these are now calculated separately during puppet module build
    • Use the new proprietary keyword for non-F/OSS licenses (you do have a LICENSE file, right?)
  • Replace environment and type as these are keywords
  • Replace import withinclude
  • Remove or replace code that used search()
  • Qualify namespaces on classes: include foo becomes include ::foo (puppet-lint will complain loudly)
  • Add the @ symbol to variables in templates that previously did not or that used scope() tricks
  • Update your dependencies
    • Update .fixtures.yaml if you use rspec-puppet
    • Update dependencies in metadata.json
    • Include both minimal and maximal versions (e.g. "version_requirement": ">=a.b.c <x.y.z")
  • Replace any if ($string) tests that depended upon empty strings being False
    • These are now True
    • When passing parameters in rSpec, the nil keyword from Ruby becomes the string 'nil' in Puppet class parameters
  • Replace any code that depended on an undef or empty title for a resource as this is now a compiler error
  • Check class parameters
    • Start parameters with letters or underscores
    • Use lowercase letters, numbers and underscores only
    • Add missing documentation in older puppet-doc format for puppet-lint
    • Optionally add newer puppet-strings format documentation
  • Remove any purge options for the cron resource
  • Quote all octal permissions on file resources, this type's API changed in 4 but it won't hurt in 3 clean this up

Others are more subjective and depend on your intended design and site policies

  • Replace complicated anchor use with contain
  • Replace facts with $::facts[] references
  • Consider replacing or adding Puppet Templates in place of ERB
  • Check that all use of file has specific owner, group and mode parameters
  • Stop using the params pattern
    • Drop params.pp, defaults.pp or whatever you called it
    • Use hiera data-in-modules in Puppet 4 (hiera 3 and later)
    • Feel free to form complex hiera trees inside the module for defaults
    • Cut down the amount of complex conditional code since hiera lookup can do it
    • Laugh at the people who told you to switch to Ansible since your configuration is all YAML now anyway
  • Replace all validate_x calls with types (depends on what puppetlabs-stdlib version you want to use)
  • Rework manifests to not use class inheritance (composition and specialization is still okay)

Do you use rSpec? Rubocop? puppet-lint?

  • Check any Gemfile or Rakefile for tasks impacted by Puppet 5's release
    • The semantic-puppet changes will generate warnings for you when bundle install or bundle update is run
    • This affects you if you're using Puppet 2, 3, 4 or 5
  • Use is_expected ...
edit flag offensive delete link more


The comment that "the nil keyword from Ruby becomes the string 'nil' in Puppet" is not correct - where did you get that from?

Henrik Lindberg gravatar imageHenrik Lindberg ( 2017-08-18 03:12:23 -0600 )edit

This nil promotion to string is quite correct if you actually test it (Puppet 5.1.0, rspec-puppet 2.6.8): manifests/init.pp: ``` test ($testparam) { notify {$testparam: }}} ``` spec:/classes/test_spec.pp: ``` let(:params){{:testparam=nil}} ... it { contain_notify('nil') } ```

waveclaw gravatar imagewaveclaw ( 2017-08-19 21:02:56 -0600 )edit

Nil might be string nil in rspec, but it's something completely different when Puppet runs the code. Been banging my head on nil, undef, and empty string problems for most of the past week.

ramindk gravatar imageramindk ( 2017-08-27 21:17:58 -0600 )edit

That `nil` becomes the string `"nil"` is something that rspec-puppet seems to do in special circumstances. Puppet does not do this. From Puppet's perspective, a given undef parameter for a resource means to take the default value (or fail if there is none), and an undef resource title is illegal.

Henrik Lindberg gravatar imageHenrik Lindberg ( 2017-08-28 01:43:17 -0600 )edit

I adjusted the comment to clarify. The impact to unit testing is very real. Be aware that PDK creates a supported release of the rSpec-related toolchain. I still have to test that for this behavior.

Jeremiah Powell gravatar imageJeremiah Powell ( 2017-08-28 09:30:02 -0600 )edit

answered 2017-08-02 07:24:33 -0600

slk gravatar image

I found that using rspec-puppet helped a lot with making code changing for a migration like this. This will also help you trim the fat from your current code base. When that is done run the tests against puppet 4 and adjust accordingly.

edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools



Asked: 2017-08-01 17:04:56 -0600

Seen: 1,176 times

Last updated: Apr 19