# Weird 'cache' issue when calling roles via dynamic role assignment

Puppet 5.3

I've loosely followed this post to structure my system roles and profiles. This means I can say I want 'X11' on a box and it'll then call a set of profiles to set up the box depending on eg environment and other facts pulled out of node facts list.

Recently, I decided I wanted to move everything out of my site.pp and put all my configuration in Consul. Previously I just specified all my roles per-node in the site.pp. Now, my entire site.pp is simply:

node 'default' {
$this_roles = lookup("roles").split('\n') include$this_roles
}


lookup() is calling my hiera backend, which in turn calls Consul (via lynxman's hiera-consul plugin). I have some boxes that have a 'base' role and some that have specific profiles (which all inherit 'base').

in my hiera.yaml:

hierarchy:
- name: Consul
hiera3_backend: consul
options:
host: localhost
port: 8500
ignore_404: true
paths:
- "/v1/kv/puppet/nodes/%{trusted.certname}"
- "/v1/kv/puppet/common


Problem: When I run tcpdump on the requests being made to Consul, I can see that the %{trusted.certname} parameter is sometimes a previous FQDN - ie, it's being cached somewhere by Puppet, and the host therefore ends up being given the wrong set of roles. Am I therefore abusing Puppet and is this an anti-pattern? I see R10k etc are leaning towards versioning everything in scm and I'm sitting here trying to shoehorn dynamic role lookups via Consul into my setup, so I probably just need someone to tell me "hey, that's batshit, don't do it".

on Consul: eg path/to/_hostname_/roles

role::www
role::x11


path/to/common/roles

role::default


Edit: I think I've found my answer, which is to instead use an ENC, which can call Consul itself. It would still be good to know (for future reference) whether this definition straight in the site.pp would have ever worked though.

edit retag close merge delete

Edit 2: ENC implemented and resolving correctly, but looking at tcpdump, ${fqdn} lookups still use hostname for the box I run the agent on immediately after master restart, eg host 'bananas1.local' comes along and performs the correct lookup but then 'bananas2.local' ends up looking up 'bananas1'... ( 2018-11-01 16:59:10 -0600 )edit the 'trusted.$factname' values are set once, when the certificate is created for the instance, and not changeable without regenerating the certificate used for client/server communication. This is to prevent an attacker from gaining control of a node, and being able to tell it to be a different ...

( 2018-11-05 08:20:34 -0600 )edit

... node, to see passwords in files or something like that. You would need to either just use an ENC like you mentioned, or a 'dynamic fact', or just use the plain old facts['fqdn'] style fact to get the value, instead of trusted.fqdn

( 2018-11-05 08:21:31 -0600 )edit

Sort by » oldest newest most voted

Turns out that the plugin I was using did not utilise the 'config' parameter correctly (within which the 'paths' has was being looked up) ... it was effectively being cached. Have put together a pull request for the plugin.

more