Validating puppet code changes using octocatalog-diff

During Puppetconf 2016 Github announces it was releasing one of its internal test tools called octocatalog-diff as open source. What the tools basicly does is compile the catalog for a certain machine using your old and new puppetcode to show you the diff output. This allows you to see the impact of your code without actually deploying and running it on puppet clients.

As this could potentially save me a lot of time, I decided to delve into the tool to see what it could bring me. This post describes to the proces of setting the tool up and the surprises I came across. Environment

For my initial testing I used a vagrant provisioned box running Ubuntu 16.04. Next to that I installed the puppet agent 3.8.7 that is in the standard Ubuntu repositories. Preparation

The following gems were needed during my trails. Just run ‘sudo gem install [GEMNAME]’ to get them on your system. When using Puppet v4 take note to use the gem binary in /opt/puppetlabs/puppet/bin/gem

hiera-eyaml
r10k
bundler
rspec

The tool can query the puppetdb instance of your setup, or you can use the facts in yaml format for the node(s) you are testing. As the puppetdb did not work for my at this point, I copied the contents of /var/lib/puppet/yaml/facts from my puppetmaster to my local test environment.

The tool also needs some develop utilities upon install. Get them using your systems package manager :

cmake
pkg-config
ruby-dev

Installing the tool

After trying the gem, I’ve installed the source version following the instructions on https://github.com/github/octocatalog-diff/blob/master/doc/installation.md . The rake test gave me a a couple of errors that I reported.

Setting up

There were three files I added to my puppet repo :

• hiera.yaml configuration valid for my setup
• bootstrap script to run r10k on my Puppetfile and to symlink local modules to the common modules dir (see below)
• .octocatalog-diff.cfg.rb configured as per the instructions on github Please note the the current example file has a key settings[:hiera_yaml_file] that should read settings[:hiera_config] . (Github issue)

Running the tool

/vagrant/octocatalog-diff/bin/octocatalog-diff -f production -t [your current branch] -n [NODENAME] --fact-file /vagrant/puppetdb_facts/[NODENAME].yaml --to-fact-override vagrant_puppetrole=nagiosserver --from-fact-override vagrant_puppetrole=nagiosserver --bootstrap-script repo-bootstrap.sh

Installation paths may vary. Note: the fact override thing is my way to assign a role to host without a real ENC / foreman. This value is used by a small piece of code in my site.pp :

node default {
## This is a small hook to support local vagrant
## development. This special var get set as part
## of the vagrant provisioning process.
if $vagrant_puppetrole != undef {
class { "roles::${vagrant_puppetrole}": }
}
}

When removing a single package from my ‘baseline’ it resulted in the expected output:

screen-shot-2016-11-03-at-14-26-40

diff production/NODENAME fea-puppetv4/NODENAME
*******************************************
Package[tcpdump] =>
parameters =>
ensure =>
- present
+ absent
*******************************************

Contents of repo-bootstrap.sh

r10k puppetfile install Puppetfile
for i in site/*; do BLA=`echo $i |sed -e 's#site/##'`; ln -s ../site/$BLA modules/$BLA; done

Final notes

When trying to use the tool, I started out using Puppet 4.8. I run into some trouble with puppetlabs firewall module 1.8.1 (( [Puppet Error] Could not autoload puppet/provider/firewall/iptables: undefined method `value’ for nil:NilClass). As soon as I downgraded to puppet 3.8.7 the firewall module stopped producing errors. Not sure if this is related to the puppet version or the combination with octocatalog-diff.

I will use the tool the coming weeks. The next step could be integrating it in the build pipeline(s).