Augeas with Puppet

Augeas is a great tool for changing only parts of a configuration file. It has a few problems though, it's slow and hard to learn and does lack some documentation. That said, unless you can use file_line or inifile, Augeas is the only game in town. The alternative is an exec with sed or awk or something terrible...

More information on augeas is available on the project home page and the puppet augeas page

To make Augeas perform quicker, it's better to define the context and lens rather than have Augeas load all files and lenses. This also works well when you want to apply a lens to a file that isn't in the location augeas expects it. In this example we'll be changing the configuration of logback for puppetserver to change from the default logging level from info to debug. Logback is configured with an xml file, so we'll use the Xml lens in augeas. To start though, we'll try our change with augtool.

Augtool lens speedup

By default augtool loads up your entire filesystem and parses every file you have...it takes a while. Here is a short demo of the time difference. First, we'll run augtool as normal and see how long it takes to figure out our nameserver in /etc/resolv.conf

$ time augtool get /files/etc/resolv.conf/nameserver
/files/etc/resolv.conf/nameserver = 10.1.1.1

real    0m1.881s
user    0m1.789s
sys     0m0.087s

Next, we'll create a command file to have augeas load a specific lens and tell it upon which file to apply that lens, here is our command file nameserver.aug:

set /augeas/load/xml/lens "Resolv.lns"
set /augeas/load/xml/incl "/etc/resolv.conf"
load
get /files/etc/resolv.conf/nameserver

Now we can run augtool against our command file and tell it not to load all the lenses:

$ time augtool --noload --noautoload -f nameserver.aug
/files/etc/resolv.conf/nameserver = 10.1.1.1

real    0m0.019s
user    0m0.015s
sys     0m0.004s

Convinced yet? I ran it three times and I got the following as an average speedup of 10,000% Ok, so it's worth it but it's hard to just start writing puppet code without knowing how to change the file in augeas first.

Back to the example, augtool first

I want to edit the logback.xml file and add a new appender (the syslog appender) but that file isn't known to augtool, but as we just saw, we can get augeas to load a specific lens using a command file, so we'll do that now, create a logback.aug file with the following contents:

set /augeas/load/xml/lens "Xml.lns"
set /augeas/load/xml/incl "/etc/puppetlabs/puppetserver/logback.xml"
load

Now we can run augtool using this file to see how augeas views the contents of the current logback.xml, the file currently looks like this:

<configuration scan="true">
    <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
        <encoder>
            <pattern>%d %-5p [%t] [%c{2}] %m%n</pattern>
        </encoder>
    </appender>

    <appender name="F1" class="ch.qos.logback.core.FileAppender">
        <!-- TODO: this path should not be hard-coded -->
        <file>/var/log/puppetlabs/puppetserver/puppetserver.log</file>
        <append>true</append>
        <encoder>
            <pattern>%d %-5p [%t] [%c{2}] %m%n</pattern>
        </encoder>
    </appender>

    <logger name="org.eclipse.jetty" level="INFO"/>

    <root level="info">
        <!--<appender-ref ref="STDOUT"/>-->
        <!-- ${logappender} logs to console when running the foreground command -->
        <appender-ref ref="${logappender}"/>
        <appender-ref ref="F1"/>
    </root>
</configuration>

Run augtool without autoloading everything and use our logback.aug file

$ augtool --noload --noautoload -f logback.aug -i
augtool> print /files/etc/puppetlabs/puppetserver/logback.xml/
/files/etc/puppetlabs/puppetserver/logback.xml
/files/etc/puppetlabs/puppetserver/logback.xml/configuration
/files/etc/puppetlabs/puppetserver/logback.xml/configuration/#attribute
/files/etc/puppetlabs/puppetserver/logback.xml/configuration/#attribute/scan = "true"
/files/etc/puppetlabs/puppetserver/logback.xml/configuration/#text[1] = "\n    "
/files/etc/puppetlabs/puppetserver/logback.xml/configuration/appender[1]
/files/etc/puppetlabs/puppetserver/logback.xml/configuration/appender[1]/#attribute
/files/etc/puppetlabs/puppetserver/logback.xml/configuration/appender[1]/#attribute/name = "STDOUT"
/files/etc/puppetlabs/puppetserver/logback.xml/configuration/appender[1]/#attribute/class = "ch.qos.logback.core.ConsoleAppender"
/files/etc/puppetlabs/puppetserver/logback.xml/configuration/appender[1]/#text[1] = "\n
...

So I can see that the appender is under the /configuration/appender tree in the augeas view, so I can write my set command using this location. Adding this to my logback.aug file, I get the following:

set /files/etc/puppetlabs/puppetserver/logback.xml/configuration/appender[#attribute[name="RSYSLOG"]]/#attribute/name "RSYSLOG"
set /files/etc/puppetlabs/puppetserver/logback.xml/configuration/appender[#attribute[name="RSYSLOG"]]/#attribute/class "ch.qos.logback.classic.net.SyslogAppender"
set /files/etc/puppetlabs/puppetserver/logback.xml/configuration/appender[#attribute[name="RSYSLOG"]]/syslogHost/#text "127.0.0.1"
set /files/etc/puppetlabs/puppetserver/logback.xml/configuration/appender[#attribute[name="RSYSLOG"]]/facility/#text "LOCAL0"
set /files/etc/puppetlabs/puppetserver/logback.xml/configuration/root/appender-ref[#attribute[ref="RSYSLOG"]]/#attribute/ref "RSYSLOG"
save

Now when I run augeas again with this new configuration file, I can see the new appender added to the file:

# augtool --noauto --noautoload -f logback.aug2 -i
augtool> save
Saved 1 file(s)
augtool> quit

And when we look at the contents of logback.xml, we see the new appender:

<configuration scan="true">
    <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
        <encoder>
            <pattern>%d %-5p [%t] [%c{2}] %m%n</pattern>
        </encoder>
    </appender>

    <appender name="F1" class="ch.qos.logback.core.FileAppender">
        <!-- TODO: this path should not be hard-coded -->
        <file>/var/log/puppetlabs/puppetserver/puppetserver.log</file>
        <append>true</append>
        <encoder>
            <pattern>%d %-5p [%t] [%c{2}] %m%n</pattern>
        </encoder>
    </appender>

    <logger name="org.eclipse.jetty" level="INFO"/>

    <root level="info">
        <!--<appender-ref ref="STDOUT"/>-->
        <!-- ${logappender} logs to console when running the foreground command -->
        <appender-ref ref="${logappender}"/>
        <appender-ref ref="F1"/>
    <appender-ref ref="RSYSLOG"></appender-ref>
</root>
<appender name="RSYSLOG" class="ch.qos.logback.classic.net.SyslogAppender"><syslogHost>127.0.0.1</syslogHost>
<facility>LOCAL0</facility>
</appender>
</configuration>

Put it in puppet code

Now all we need to do is translate this into puppet code, but the hard part is already done:

  augeas { 'logback syslog':
    incl    => '/etc/puppetlabs/puppetserver/logback.xml',
    context => '/files/etc/puppetlabs/puppetserver/logback.xml',
    lens    => 'Xml.lns',
    changes => [
      'set /files/etc/puppetlabs/puppetserver/logback.xml/configuration/appender[#attribute[name="RSYSLOG"]]/#attribute/name "RSYSLOG"',
      'set /files/etc/puppetlabs/puppetserver/logback.xml/configuration/appender[#attribute[name="RSYSLOG"]]/#attribute/class "ch.qos.logback.classic.net.SyslogAppender"',
      'set /files/etc/puppetlabs/puppetserver/logback.xml/configuration/appender[#attribute[name="RSYSLOG"]]/syslogHost/#text "127.0.0.1"',
      'set /files/etc/puppetlabs/puppetserver/logback.xml/configuration/appender[#attribute[name="RSYSLOG"]]/facility/#text "LOCAL0"',
      'set /files/etc/puppetlabs/puppetserver/logback.xml/configuration/root/appender-ref[#attribute[ref="RSYSLOG"]]/#attribute/ref "RSYSLOG"',
    ],
  }

Wordpress category: