Augeas with Puppet

Augeas is a great tool for changing only parts of a configuration file. It has a few problems though, it's slow and hard to learn and does lack some documentation. That said, unless you can use file_line or inifile, Augeas is the only game in town. The alternative is an exec with sed or awk or something terrible...

More information on augeas is available on the project home page and the puppet augeas page

To make Augeas perform quicker, it's better to define the context and lens rather than have Augeas load all files and lenses. This also works well when you want to apply a lens to a file that isn't in the location augeas expects it. In this example we'll be changing the configuration of logback for puppetserver to change from the default logging level from info to debug. Logback is configured with an xml file, so we'll use the Xml lens in augeas. To start though, we'll try our change with augtool.

Augtool lens speedup

By default augtool loads up your entire filesystem and parses every file you have...it takes a while. Here is a short demo of the time difference. First, we'll run augtool as normal and see how long it takes to figure out our nameserver in /etc/resolv.conf


$ time augtool get /files/etc/resolv.conf/nameserver
/files/etc/resolv.conf/nameserver = 10.1.1.1

real 0m1.881s
user 0m1.789s
sys 0m0.087s

Next, we'll create a command file to have augeas load a specific lens and tell it upon which file to apply that lens, here is our command file nameserver.aug:

set /augeas/load/xml/lens "Resolv.lns"
set /augeas/load/xml/incl "/etc/resolv.conf"
load
get /files/etc/resolv.conf/nameserver

Now we can run augtool against our command file and tell it not to load all the lenses:


$ time augtool --noload --noautoload -f nameserver.aug
/files/etc/resolv.conf/nameserver = 10.1.1.1

real 0m0.019s
user 0m0.015s
sys 0m0.004s

Convinced yet? I ran it three times and I got the following as an average speedup of 10,000% Ok, so it's worth it but it's hard to just start writing puppet code without knowing how to change the file in augeas first.

Back to the example, augtool first

I want to edit the logback.xml file and add a new appender (the syslog appender) but that file isn't known to augtool, but as we just saw, we can get augeas to load a specific lens using a command file, so we'll do that now, create a logback.aug file with the following contents:


set /augeas/load/xml/lens "Xml.lns"
set /augeas/load/xml/incl "/etc/puppetlabs/puppetserver/logback.xml"
load

Now we can run augtool using this file to see how augeas views the contents of the current logback.xml, the file currently looks like this:



%d %-5p
t] [%c{2}] %m%n</pattern>
        </encoder>
    </appender>
 
    <appender name="F1" class="ch.qos.logback.core.FileAppender">
        <!-- TODO: this path should not be hard-coded -->
        <file>/var/log/puppetlabs/puppetserver/puppetserver.log</file>
        <append>true</append>
        <encoder>
            <pattern>%d %-5p [%t] [%c{2}] %m%n</pattern>
        </encoder>
    </appender>
 
    <logger name="org.eclipse.jetty" level="INFO"/>
 
    <root level="info">
        <!--<appender-ref ref="STDOUT"/>-->
        <!-- ${logappender} logs to console when running the foreground command -->
        <appender-ref ref="${logappender}"/>
        <appender-ref ref="F1"/>
    </root>
</configuration>
</code>
 
Run augtool without autoloading everything and use our logback.aug file
<code>
$ augtool --noload --noautoload -f logback.aug -i
augtool> print /files/etc/puppetlabs/puppetserver/logback.xml/
/files/etc/puppetlabs/puppetserver/logback.xml
/files/etc/puppetlabs/puppetserver/logback.xml/configuration
/files/etc/puppetlabs/puppetserver/logback.xml/configuration/#attribute
/files/etc/puppetlabs/puppetserver/logback.xml/configuration/#attribute/scan = "true"
/files/etc/puppetlabs/puppetserver/logback.xml/configuration/#text[1] = "\n    "
/files/etc/puppetlabs/puppetserver/logback.xml/configuration/appender[1]
/files/etc/puppetlabs/puppetserver/logback.xml/configuration/appender[1]/#attribute
/files/etc/puppetlabs/puppetserver/logback.xml/configuration/appender[1]/#attribute/name = "STDOUT"
/files/etc/puppetlabs/puppetserver/logback.xml/configuration/appender[1]/#attribute/class = "ch.qos.logback.core.ConsoleAppender"
/files/etc/puppetlabs/puppetserver/logback.xml/configuration/appender[1]/#text[1] = "\n
...
</code>
 
So I can see that the appender is under the /configuration/appender tree in the augeas view, so I can write my set command using this location.  Adding this to my logback.aug file, I get the following:
<code>
set /files/etc/puppetlabs/puppetserver/logback.xml/configuration/appender[#attribute[name="RSYSLOG"]]/#attribute/name "RSYSLOG"
set /files/etc/puppetlabs/puppetserver/logback.xml/configuration/appender[#attribute[name="RSYSLOG"]]/#attribute/class "ch.qos.logback.classic.net.SyslogAppender"
set /files/etc/puppetlabs/puppetserver/logback.xml/configuration/appender[#attribute[name="RSYSLOG"]]/syslogHost/#text "127.0.0.1"
set /files/etc/puppetlabs/puppetserver/logback.xml/configuration/appender[#attribute[name="RSYSLOG"]]/facility/#text "LOCAL0"
set /files/etc/puppetlabs/puppetserver/logback.xml/configuration/root/appender-ref[#attribute[ref="RSYSLOG"]]/#attribute/ref "RSYSLOG"
save
</code>
 
Now when I run augeas again with this new configuration file, I can see the new appender added to the file:
 
<code>
# augtool --noauto --noautoload -f logback.aug2 -i
augtool> save
Saved 1 file(s)
augtool> quit
</code>
 
And when we look at the contents of logback.xml, we see the new appender:
<code>
<configuration scan="true">
    <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
        <encoder>
            <pattern>%d %-5p [%t] [%c{2}] %m%n</pattern>
        </encoder>
    </appender>
 
    <appender name="F1" class="ch.qos.logback.core.FileAppender">
        <!-- TODO: this path should not be hard-coded -->
        <file>/var/log/puppetlabs/puppetserver/puppetserver.log</file>
        <append>true</append>
        <encoder>
            <pattern>%d %-5p [%t] [%c{2}] %m%n</pattern>
        </encoder>
    </appender>
 
    <logger name="org.eclipse.jetty" level="INFO"/>
 
    <root level="info">
        <!--<appender-ref ref="STDOUT"/>-->
        <!-- ${logappender} logs to console when running the foreground command -->
        <appender-ref ref="${logappender}"/>
        <appender-ref ref="F1"/>
    <appender-ref ref="RSYSLOG"></appender-ref>
</root>
<appender name="RSYSLOG" class="ch.qos.logback.classic.net.SyslogAppender"><syslogHost>127.0.0.1</syslogHost>
<facility>LOCAL0</facility>
</appender>
</configuration>
</code>
 
<h2>Put it in puppet code</h2>
Now all we need to do is translate this into puppet code, but the hard part is already done:
 
<code>
  augeas { 'logback syslog':
    incl    => '/etc/puppetlabs/puppetserver/logback.xml',
    context => '/files/etc/puppetlabs/puppetserver/logback.xml',
    lens    => 'Xml.lns',
    changes => [
      'set /files/etc/puppetlabs/puppetserver/logback.xml/configuration/appender[#attribute[name="RSYSLOG"]]/#attribute/name "RSYSLOG"',
      'set /files/etc/puppetlabs/puppetserver/logback.xml/configuration/appender[#attribute[name="RSYSLOG"]]/#attribute/class "ch.qos.logback.classic.net.SyslogAppender"',
      'set /files/etc/puppetlabs/puppetserver/logback.xml/configuration/appender[#attribute[name="RSYSLOG"]]/syslogHost/#text "127.0.0.1"',
      'set /files/etc/puppetlabs/puppetserver/logback.xml/configuration/appender[#attribute[name="RSYSLOG"]]/facility/#text "LOCAL0"',
      'set /files/etc/puppetlabs/puppetserver/logback.xml/configuration/root/appender-ref[#attribute[ref="RSYSLOG"]]/#attribute/ref "RSYSLOG"',
    ],
  }
</code>

About the Author...

Slides from LISA 2019 Linux systems troubleshooting #LISA2019 https://t.co/D4dMKflK6R Tue Oct 29 05:59:30 +0000 2019

https://t.co/AGeihMALAv configuring grub2 with EFI Fri Sep 13 05:20:01 +0000 2019

I published a Thing on @thingiverse! https://t.co/IYpRyEb7Hz #thingalert Tue Jul 23 19:27:57 +0000 2019

Nokogiri install on MacOSX https://t.co/v3An0miW9L Fri Jul 12 15:06:49 +0000 2019

HTML email with plain mailer plugin on Jenkins https://t.co/Z6FSDMDjy8 Thu Jul 11 21:07:25 +0000 2019