Puppet Module Testing

Why?

How?

Who am I

Jan Vansteenkiste

A story
of Puppet

(testing)

In the beginning

  • Simple modules:
    Install/Config/Service
  • Testing?
    • What is it?
    • Why?
  • Hacking on production

Start Growing

(adding complexity)

  • Configuration has parameters
  • Dealing with different distros
  • Hacking on production
    Vagrant

Vagrant

http://www.vagrantup.com/
  • Local testing
  • Exact environment
  • Vagrant up, down, up, down, up, provision, down
  • Host interactions (puppetmaster -client)

Vagrantfile

Vagrant::Config.run do |config|
  config.vm.define :puppetmaster do |vm_config|
    vm_config.vm.box = 'puppet3'
    vm_config.vm.host_name = "puppetmaster01.virtual.vstone.be"
    vm_config.vm.network :hostonly, '192.168.13.2'
    vm_config.vm.provision :shell do |shell|
      shell.path = "scripts/run_puppetmaster.sh"
    end
  end
end

Links

Poor mans testing

Hey, at least it's not on production

  • Still no real puppet module tests.
  • Losing time:
    • VM booting (SSD helps...)
    • Puppet stack

(Original xkcd.com - CC BY-NC 2.5) pic.twitter.com/CVX3gjxN

Testing for
silly mistakes

  • Syntax: Catch errors early
    Before a vagrant run
    puppet parser validate <filename>
  • Style: Catch common mistakes
    • Unexpected behavior
      • Quoted booleans
      • '$strings' in commands
    • Pretty code debugs easier: especially somebody else his code.
    puppet-lint <filename>

Automating

We tend to forget...

  • GIT Hooks: Validate syntax, trailing spaces, tabs, ...
  • CI Tools: Jenkins/Travis/...

Hello
Jenkins

Warnings Plugin Groovy

So far, so good...

But...

Still not testing the
functionality of our modules

Complexity+

Functions and facts

  • Copy/paste FTW!
  • Getting to know ruby
  • Tired of vagrant up, provision, down, up, provision, provision, provision, down ...
  • The need for spec tests rises
  • How hard can it be?

The toolkit

  • Ruby + Rubygems </captain-obvious>
  • Bundler: Dependency Management
  • Gemfile: Describes dependencies
  • Rakefile: Running rake tasks
  • Tests!

Gemfile

source :rubygems

gem 'rake'
gem 'puppet-lint'
gem 'rspec'
gem 'rspec-puppet'

## Will come in handy later on. But you could just use
# gem 'puppet'
puppetversion = ENV.key?('PUPPET_VERSION') ? "~> #{ENV['PUPPET_VERSION']}" : ['>= 2.7']
gem 'puppet', puppetversion
gem 'puppetlabs_spec_helper'
              

Bundler


# Install rubygems first
gem install bundler

# Install all required gems in .vendor/
bundle install --path .vendor
# Ignore this folder in .gitignore or globally.

# We can already use puppet-lint.
bundle exec puppet-lint
              

Rakefile

  • Copy paste
  • Use rpsec-puppet
$ bundle exec rspec-puppet-init
 + spec/
 + spec/classes/
 + spec/defines/
 + spec/functions/
 + spec/hosts/
 + spec/fixtures/
 + spec/fixtures/manifests/
 + spec/fixtures/modules/
 + spec/fixtures/modules/example/
 + spec/fixtures/manifests/site.pp
 + spec/fixtures/modules/example/manifests
 + spec/fixtures/modules/example/lib
 + spec/fixtures/modules/example/templates
 + spec/spec_helper.rb
 + Rakefile
              

Rakefile (rspec-puppet)

require 'rake'
require 'rspec/core/rake_task'

# By default this is :spec
RSpec::Core::RakeTask.new(:rspec) do |t|
  t.pattern = 'spec/*/*_spec.rb'
end
              

Running rake

$ bundle exec rake -T
rake spec  # Run RSpec code examples
              

Rakefile
(puppetlabs_spec_helper)

require 'rake'
require 'rspec/core/rake_task'
require 'puppetlabs_spec_helper/rake_tasks'
              

Running rake

$ bundle exec rake -T
rake build            # Build puppet module package
rake clean            # Clean a built module package
rake coverage         # Generate code coverage information
rake help             # Display the list of available rake tasks
rake lint             # Check puppet manifests with puppet-lint
rake rspec            # Run RSpec code examples
rake spec             # Run spec tests in a clean fixtures directory / Run RSpec code examples
rake spec_clean       # Clean up the fixtures directory
rake spec_prep        # Create the fixtures directory
rake spec_standalone  # Run spec tests on an existing fixtures directory
              

Something to test

Testing a function

./lib/puppet/parser/functions/mysql_password.rb
require 'digest/sha1'

module Puppet::Parser::Functions
  newfunction(:mysql_password, :type => :rvalue) do |args|
    raise(Puppet::ParseError, 'mysql_password(): Argument must be a string') unless args[0].is_a?(String)
    '*' + Digest::SHA1.hexdigest(Digest::SHA1.digest(args[0])).upcase
  end
end
              
./spec/functions/mysql_password_spec.rb
require 'spec_helper'

describe 'mysql_password' do
  let(:scope) { PuppetlabsSpec::PuppetInternals.scope }

  it 'should exist' do
    Puppet::Parser::Functions.function('mysql_password').should == 'function_mysql_password'
  end

  it 'should throw an error on invalid types' do
    lambda {
      scope.function_mysql_password([{:foo => :bar}])
    }.should(raise_error(Puppet::ParseError))
  end

  it 'should convert a password to a hash' do
    password = 'mypass'
    scope.function_mysql_password([password]).should == '*6C8989366EAF75BB670AD8EA7A7FC1176A95CEF4'
  end
end
                
Running the test:
$ bundle exec rake rspec
/usr/bin/ruby18 -S rspec spec/functions/mysql_password_spec.rb
...

Finished in 0.00693 seconds
3 examples, 0 failures
                  
Rakefile
RSpec::Core::RakeTask.new(:rspec) do |t|
  t.pattern = 'spec/*/*_spec.rb'
  t.rspec_opts = File.read("spec/spec.opts").chomp || ""
end
                
spec/spec.opts
--format documentation --colour --backtrace
Running the tests:
/usr/bin/ruby18 -S rspec spec/functions/mysql_password_spec.rb --format documentation --colour --backtrace

mysql_password
  should exist
  should throw an error on invalid types
  should convert a password to a hash

Finished in 0.00715 seconds
3 examples, 0 failures
                

Testing a fact

https://github.com/vStone/puppet-testing-example.git
./lib/facter/is_database.rb
Facter.add(:is_example) do
  setcode do
    if (Facter.value('fqdn') =~ /^(.*\.)?example\.com$/) == nil
      is_example = false
    else
      is_example = true
    end
    is_example
  end
end
./spec/facts/is_example_spec.rb
require 'spec_helper'
require 'facter/is_example'

describe 'Facter::Util::Fact' do
  before { Facter.clear }
  after  { Facter.clear }

  describe 'is_example for host' do

    it 'noexample.com' do
      Facter.fact(:fqdn).stubs(:value).returns('noexample.com')
      Facter.fact(:is_example).value.should == false
    end

    it 'some.example.com' do
      Facter.fact(:fqdn).stubs(:value).returns('some.example.com')
      Facter.fact(:is_example).value.should == true
    end
  end
end

So....

What else can we test?


Some common examples

  • Distro support: Setting custom facts.
  • Parameters: Testing the effect should they have.

Example: distro support

require 'spec_helper'

describe 'example' do
  describe 'on CentOS' do
    let (:facts) { { 
      :operatingsystem => 'CentOS',
       :osfamily => 'RedHat',
    } }

    it { should include_class('example::redhat') }
    it { should contain_file('foo').with_path('/etc/sysconfig/example') }
  end
end
              

Example: Parameters

require 'spec_helper'

describe 'example' do
  context 'with foo => false' do
    let (:params) { {
      :foo => false
    } }

    it { should_not contain_file('foo') }
  end
end
              

Conclusion?

  • It's not that hard
  • It's faster than vagrant or puppet apply runs
  • Limited scope.
    A test is a lot easier than randomly placing notifies in your manifests
  • Less resources used. No need to bring up VM's for testing

Freebies

  • New features without worries:
    Backwards compatible: check!
  • Help others help you
    Document what you expect.
  • Automated tests are useful now
    • Proper testing of modules and functionality
    • (Automated) Feedback on pull requests

Using Travis-CI

  • https://travis-ci.org
  • Test multiple ruby versions
  • Test multiple puppet versions
  • Simple:
    • puppetlabs_spec_helper
    • .fixtures.yml
    • .travis.yml
  • Bonus feature: integrates nicely with github

What do we need

.fixtures.yml

  • No need to mess with submodules
  • Clean fixtures every time
fixtures:
  repositories:
    stdlib: "git://github.com/puppetlabs/puppetlabs-stdlib"
  symlinks:
    example: "#{source_dir}"
              

.travis.yml

  • File must exist on the master branch
language: ruby
rvm:
  - 1.8.7
  - 1.9.3
script: 'rake spec'
env:
  - PUPPET_VERSION="2.7"
  - PUPPET_VERSION="3.0"
branches:
  only:
    - master
    - develop
              

Sign up using your github account

*clickety click*
Your Profile

In the end

Not-that-hard

Faster testing

Easier for debugging

Limited Scope


...


But needs some docs/examples

Questions?