Had an sssd process spinning and using 100% cpu. Did an strace on it and saw that it was complaining about too many open files.


pid accept(24, 0xaddress, [110]) = -1 EMFILE (Too many open files)


getting the number of open files for the process.


# lsof -p $(pidof sssd_pam) |wc -l
1065


Looking at the limits for sssd, I saw that the nofile was set to 1024, which appears to be the default everywhere I tried.


# cat /proc/$(pidof sssd_pam)/limits
Limit Soft Limit Hard Limit Units
Max cpu time unlimited unlimited seconds
Max file size unlimited unlimited bytes
Max data size unlimited unlimited bytes
Max stack size 10485760 unlimited bytes
Max core file size 0 unlimited bytes
Max resident set unlimited unlimited bytes
Max processes 5788 5788 processes
Max open files 4096 4096 files
Max locked memory 65536 65536 bytes
Max address space unlimited unlimited bytes
Max file locks unlimited unlimited locks
Max pending signals 5788 5788 signals
Max msgqueue size 819200 819200 bytes
Max nice priority 0 0
Max realtime priority 0 0
Max realtime timeout unlimited unlimited us


What I didn't know is that you can change it just by echoing the correct string to that limits file.


# echo -n "Max open files=8192:8192" >/proc/$(pidof sssd_pam)/limits
# cat /proc/$(pidof sssd_pam)/limits
Limit Soft Limit Hard Limit Units
Max cpu time unlimited unlimited seconds
Max file size unlimited unlimited bytes
Max data size unlimited unlimited bytes
Max stack size 10485760 unlimited bytes
Max core file size 0 unlimited bytes
Max resident set unlimited unlimited bytes
Max processes 5788 5788 processes
Max open files 8192 8192 files
Max locked memory 65536 65536 bytes
Max address space unlimited unlimited bytes
Max file locks unlimited unlimited locks
Max pending signals 5788 5788 signals
Max msgqueue size 819200 819200 bytes
Max nice priority 0 0
Max realtime priority 0 0
Max realtime timeout unlimited unlimited us


This works on a RHEL6 system, didn't work on my Fedora system. On Fedora there is a program called prlimit which can be used to modify the limits.


[root@burnaby 907]# prlimit --pid=907 --nofile=8192:8192
[root@burnaby 907]# cat limits
Limit Soft Limit Hard Limit Units
Max cpu time unlimited unlimited seconds
Max file size unlimited unlimited bytes
Max data size unlimited unlimited bytes
Max stack size 8388608 unlimited bytes
Max core file size 0 unlimited bytes
Max resident set unlimited unlimited bytes
Max processes 62617 62617 processes
Max open files 8192 8192 files
Max locked memory 65536 65536 bytes
Max address space unlimited unlimited bytes
Max file locks unlimited unlimited locks
Max pending signals 62617 62617 signals
Max msgqueue size 819200 819200 bytes
Max nice priority 0 0
Max realtime priority 0 0
Max realtime timeout unlimited unlimited us


Lightening talk I gave at SASAG on Sept 12, 2013.

Quick intro to using netcat and gnutls-cli to interact with HTTP, IMAP and SMTP and debug problems.

I Used reveal.js and shellinabox to do the magic. Shellinabox is run inside an iframe. I created a user to run the examples as during the demo (sasag), then started multiple shellinabox runs using a bash script. Shouldn't have needed to start more than one, but chrome wouldn't let me type in them all when they were running from the same instance.

Here is the shell script and css I used:


#!/bin/bash

SHELL="shellinaboxd -d -t --css=/usr/share/shellinabox/color.css --user-css /:+white-on-black.css --localhost-only --no-beep"
SHELL2="shellinaboxd -d -t --css=/usr/share/shellinabox/color.css --user-css /:+white-on-black-small.css --localhost-only --no-beep"
shellinaboxd -d -t --port 4200 --css=/usr/share/shellinabox/color.css \
--user-css /:+white-on-black.css \
--localhost-only --no-beep \
-s /telnet/:sasag:sasag:/home/sasag:/bin/bash &
$SHELL --port 4201 -s /listening/:sasag:sasag:/home/sasag:/bin/bash &
$SHELL --port 4202 -s /listening2/:sasag:sasag:/home/sasag:/bin/bash &
$SHELL2 --port 4203 -s /cat/:sasag:sasag:/home/sasag:/bin/bash &
$SHELL --port 4204 -s /links/:sasag:sasag:/home/sasag:/bin/bash &
$SHELL --port 4205 -s /socket/:sasag:sasag:/home/sasag:/bin/bash &
$SHELL --port 4206 -s /gnutlscli/:sasag:sasag:/home/sasag:/bin/bash &
$SHELL --port 4207 -s /socket2/:sasag:sasag:/home/sasag:/bin/bash &

white-on-black.css:


#vt100 #cursor.bright {
background-color: black;
color: white;
font-size: 16pt;
height: 16px;
}

#vt100 #console, #vt100 #alt_console, #vt100 #cursor, #vt100 #lineheight {
font-size: 16pt;
}

#vt100 #scrollable {
color: black;
background-color: white;
font-size: 16pt;
}

#vt100 #lineheight {
height: 18px;
}

#vt100 #scrollable.inverted {
color: #000000;
background-color: #ffffff;
}

#vt100 .ansi15 {
color: #000000;
}

#vt100 .bgAnsi0 {
background-color: #ffffff;
}

Talk I gave at puppetconf2013 on using exported resources to dynamically configure your system. Using augeas and concat the examples build up a working dns implementation that is automatic. The system was simplified a lot for the talk but the principles of using exported resources to configure the system is powerful.

Puppet posted the video.
The slides are here: http://goo.gl/nR9rti

There are some video examples embedded in the video as well. https://www.youtube.com/watch?v=sif_PJfLT3s and https://www.youtube.com/watch?v=RHGQQMFFReA

Handout version

I knew this script looked familiar, I just wrote the same script as I wrote almost a year ago. I guess I didn't do a good job getting it on the google's, cause I couldn't find it until I started posting about it...github:pass_to_shadow.py

Maybe this time I'll include some of the code in this post.


#!/usr/bin/python

# * Thu Jul 11 2013 Thomas Uphill
# - encrypt a password with sha512 by default
# - generate a crypt suitable for placement in /etc/shadow

import crypt
from random import randint
import string
import getpass
from optparse import OptionParser
import sys

# command line arguments
help_text = """%prog [options]
encrypt a password in a format suitable for /etc/shadow
sha512 is used by default
salt is calculated using randint if not supplied
"""
parser = OptionParser(usage=help_text)
parser.add_option("-s","--salt", dest="salt",action="store",type="string",
help="encryption salt", metavar="salt")
parser.add_option("-p","--password", dest="passwd",action="store",type="string",
help="password to encrypt", metavar="password")
parser.add_option("-a","--algorithm", dest="algo",action="store",type="string",
help="algorithm to use (sha512, sha256, md5",metavar="algorithm")

(options,args) = parser.parse_args()

# generate salt
try:
slen = len(options.salt)
salt = options.salt
if slen < 8:
sys.stderr.write("WARNING: Salt length should be at least 8 characters\n")
except:
salt_string = './' + string.ascii_letters + string.digits
salt = ''.join([salt_string[randint(0,len(salt_string)-1)] for x in range(8)])

try:
plen = len(options.passwd)
passwd = options.passwd
if plen < 8:
sys.stderr.write("WARNING: Password should be at least 8 characters\n")
except:
passwd = getpass.getpass()
plen = len(passwd)
if plen < 8:
sys.stderr.write("WARNING: Password should be at least 8 characters\n")

try:
len(options.algo)
algos = {'sha512': 6,
'sha256': 5,
'md5': 1,
}
try:
algo = algos[options.algo]
except:
sys.stderr.write("ERROR: Unknown encryption algorithm %s" % options.algo)
sys.exit(1)
except:
algo = '6'
print crypt.crypt(passwd, "$%s$%s" % (algo,salt))

This has come up 3 times in the last few days so I thought I'd share it. The situation is that there are files at /var/spool/mqueue that are part of the /var filesystem. But there is also another filesystem mounted at /var/spool/mqueue with it's own files. You want to access the files under the mount but you can't. So, there are two ways around this (in this instance).

Option 1

Mount /var somewhere else, get the mount device for /var
[me@host ~] $ mount |grep "/var " |sed -e 's/\(.*\) on \/var type.*/\1/'
/dev/mapper/vg_root/lv_var
[me@host ~] $ sudo mkdir /mnt/var
[me@host ~] $ sudo mount /dev/mapper/vg_root/lv_var /mnt/var


The files that were mounted on top of will be accessible again, yay (pretty darn simple no), just don't get confused cause now they'll be in /mnt/var/spool

Option 2

Use a bind mount to mount /var/spool somewhere else, don't need to figure out the device this way...

[me@host ~] $ sudo mkdir /mnt/spool
[me@host ~] $ sudo mount -t bind -o bind /var/spool /mnt/spool


I prefer this method, the bind mount is also a good clue to the next admin that something funky is going on here...

Overview of KVM
Intro to puppet

Thanks everyone who came to the talks, great conference this year. Thanks to puppetlabs for all the support. Getting better every year, looking forward to next year! (just hope it stays smaller than LISA)

The scenario is that users are controlled by sssd and you want to add a user locally for those times when your connection to your password backend goes down. Useradd won't let you do it though, cause the user already exists. I figured augeas would be the perfect way to do that but I kept hitting a problem, augeas would fail.


augtool> save
error: Failed to execute command
saving failed (run 'print /augeas//error' for details)
augtool> print /augeas//error
/augeas/files/etc/passwd/error = "put_failed"
/augeas/files/etc/passwd/error/path = "/files/etc/passwd/tuphill"
/augeas/files/etc/passwd/error/lens = "/usr/share/augeas/lenses/dist/passwd.aug:33.18-41.21:"
/augeas/files/etc/passwd/error/message = "Failed to match \n { /password/ = /([^\\001-\\004\\n:]+)?/ }{ /uid/ = /[0-9]+/ }{ /gid/ = /[0-9]+/ }{ /name/ = /([^\\001-\\004\\n:]+)?/ }{ /home/ = /([^\\001-\\004\\n:]+)?/ }{ /shell/ = /([^\\001-\\004\\t\\n ][^\\001-\\004\\n]*[^\\001-\\004\\t\\n ]|[^\\001-\\004\\t\\n ])?/ }\n with tree\n { \"shell\" = \"/bin/bash\n\" } { \"name\" = \"tuphill\" } { \"password\" = \"x\" } { \"uid\" = \"500\" }"


After a bit of staring I realized the answer was right in front of me, the order in which you add the fields to the record is important. The lens has the order set but when I add to the record using augtool, the order is just whatever I happened to do...so the following is the correct way to get everything in order.

augtool> ins tuphill after /files/etc/passwd/*[last()]
augtool> set /files/etc/passwd/tuphill/password x
augtool> set /files/etc/passwd/tuphill/uid 500
augtool> set /files/etc/passwd/tuphill/gid 500
augtool> set /files/etc/passwd/tuphill/name tuphill
augtool> set /files/etc/passwd/tuphill/home /home/tuphill
augtool> set /files/etc/passwd/tuphill/shell /bin/bash
augtool> save
Saved 1 file(s)


Now I can get rid of my stopping of sssd before adding the user. Next is to rewrite this as a defined type in puppet so I can just do something like

aug_user {'tuphill':
uid => 500,
gid => 500,
password => 'x',
home => '/home/tuphill',
shell => '/bin/bash'


I was trying to get my Samsung TV to play files from my Beefy miracle box and although it was able to connect, it would not show any files. It took a bit of hunting around but I found the following patch needed to be applied to get the files to display.

diff -ruN mediatomb.orig/tombupnp/upnp/src/genlib/net/uri/uri.c mediatomb/tombupnp/upnp/src/genlib/net/uri/uri.c
--- mediatomb.orig/tombupnp/upnp/src/genlib/net/uri/uri.c 2012-06-06 23:01:22.000000000 +0200
+++ mediatomb/tombupnp/upnp/src/genlib/net/uri/uri.c 2012-06-07 08:22:01.000000000 +0200
@@ -1042,7 +1042,8 @@
out->path_type = REL_PATH;
}

- if( ( ( begin_hostport + 1 ) < max ) && ( in[begin_hostport] == '/' )
+ //parse hostport only if scheme was found
+ if( ( begin_hostport > 0 ) && ( ( begin_hostport + 1 ) < max ) && ( in[begin_hostport] == '/' )
&& ( in[begin_hostport + 1] == '/' ) ) {
begin_hostport += 2;

@@ -1059,6 +1060,12 @@
out->hostport.text.size = 0;
out->hostport.text.buff = 0;
begin_path = begin_hostport;
+
+ //remove excessive leading slashes (fix for Samsung Smart TV 2012)
+ while( ( ( begin_path + 1 ) < max ) && ( in[begin_path] == '/' ) && ( in[begin_path + 1] == '/') ) {
+ begin_path++;
+ }
+
}

begin_fragment =

Next I needed to recompile mediatomb on fedora, to get that to work I had to modify the rpm and add another patch.

=== modified file 'src/metadata/ffmpeg_handler.cc'
--- old/src/metadata/ffmpeg_handler.cc 2010-08-25 17:07:03 +0000
+++ new/src/metadata/ffmpeg_handler.cc 2011-05-18 06:56:48 +0000
@@ -89,6 +89,35 @@

Ref sc = StringConverter::m2i();

+ /* Tabs are 4 characters here */
+ typedef struct {const char *avname; metadata_fields_t field;} mapping_t;
+ static const mapping_t mapping[] =
+ {
+ {"title", M_TITLE},
+ {"artist", M_ARTIST},
+ {"album", M_ALBUM},
+ {"date", M_DATE},
+ {"genre", M_GENRE},
+ {"comment", M_DESCRIPTION},
+ {"track", M_TRACKNUMBER},
+ {NULL, M_MAX},
+ };
+
+ if (!pFormatCtx->metadata)
+ return;
+ for (const mapping_t *m = mapping; m->avname != NULL; m++)
+ {
+ AVMetadataTag *tag = NULL;
+ tag = av_metadata_get(pFormatCtx->metadata, m->avname, NULL,
+ AV_METADATA_DONT_STRDUP_KEY | AV_METADATA_DONT_STRDUP_VAL);
+ if (tag && tag->value && tag->value[0])
+ {
+ log_debug("Added metadata %s: %s\n", m->avname, tag->value);
+ item->setMetadata(MT_KEYS[m->field].upnp,
+ sc->convert(tag->value));
+ }
+ }
+ /* Old algorithm (doesn't work with libav >= 0.7)
if (strlen(pFormatCtx->title) > 0)
{
log_debug("Added metadata title: %s\n", pFormatCtx->title);
@@ -131,6 +160,7 @@
item->setMetadata(MT_KEYS[M_TRACKNUMBER].upnp,
sc->convert(String::from(pFormatCtx->track)));
}
+ */
}

// ffmpeg library calls
@@ -178,7 +208,7 @@
for(i=0; inb_streams; i++)
{
AVStream *st = pFormatCtx->streams[i];
- if((st != NULL) && (videoset == false) && (st->codec->codec_type == CODEC_TYPE_VIDEO))
+ if((st != NULL) && (videoset == false) && (st->codec->codec_type == AVMEDIA_TYPE_VIDEO))
{
if (st->codec->codec_tag > 0)
{
@@ -209,7 +239,7 @@
*y = st->codec->height;
}
}
- if(st->codec->codec_type == CODEC_TYPE_AUDIO)
+ if(st->codec->codec_type == AVMEDIA_TYPE_AUDIO)
{
// Increase number of audiochannels
audioch++;

After applying that patch the package will build. You then need to change the config.xml and set some headers so that the Samsung will see the server.

After that, my samsung was able to see the fedora box and successfully stream files, even SBS 3d files work properly.

Here is my compiled rpm for f17 and the src rpm.

mediatomb-0.12.1-22.fc17.x86_64.rpm mediatomb-0.12.1-22.fc17.src.rpm

Here is the complete config.xml

<?xml version="1.0" encoding="UTF-8"?>

MediaTomb
uuid:e779c814-8ea1-4f2f-ab97-4bf09b2a24a2
/etc/mediatomb
/usr/share/mediatomb/web

mediatomb.db

localhost
mediatomb
mediatomb

redsonic.com
105
-->

101
-->

*

video

lastfmuser
lastfmpass

/usr/share/mediatomb/js/common.js
/usr/share/mediatomb/js/playlists.js

/usr/share/mediatomb/js/import.js

audio/L16
no
yes
no

video/mpeg
yes
yes
yes

A talk I gave at Lopsa NJ on linux-kvm, slides in a google doc here

John recorded the talk so maybe it'll be posted on lopsa's website soon...

The latest version of skype doesn't appear to work well on RHEL6, I found the following forum http://community.skype.com/t5/Linux/Skype-for-Redhat-Enterprise-6/td-p/… and followed the two things mentioned there to make it work.

• Download the ubuntu 10 version and extract the contents with dpkg
• Download the ubuntu 10 qtwebkit and extract using dpkg, put the libraries in /usr/share/skype
• rename the skype binary to skype-ubuntu and then run skype from a script which sets GTK2_RC_FILES=/etc/gtk-2.0/gtkrc
  

  <br />
  #!/bin/sh
  <p>export LD_LIBRARY_PATH=/usr/share/skype:$LD_LIBRARY_PATH<br />
  export GTK2_RC_FILES=/etc/gtk-2.0/gtkrc<br />
  exec skype-ubuntu $@<br />

You can get the contents of the debian package with dpkg

<br />
mkdir skype-debian<br />
dpkg -x skype-debian_4.1.0.20-1_i386.deb skype-debian<br />
dpkg -x libqt4-webkit_4.6.2-0ubuntu5.5_i386.deb skype-debian<br />
cd skype-debian<br />
mv usr/lib/libQtWeb* usr/share/skype<br />
rm -r usr/lib<br />
rm -r usr/share/doc/libqt4-webkit<br />
tar cjf skype-4.1.0.20.tar.bz2 *<br />

Then use the following spec file with that tar and you'll be able to make a src rpm for mocking...

%define __os_install_post %{nil}
%define _enable_debug_packages %{nil}
Summary: Skype is free Internet telephony that just works
Name: skype
Version: 4.1.0.20
Release: 1%{?dist}
License: Commercial
Group: Applications/Internet
URL: http://www.skype.com/products/skype/linux/
Source: skype-4.1.0.20.tar.bz2
BuildRoot: %{_tmppath}/%{name}-%{version}-root
Provides: config(skype) = %{version}-fc16, skype = %{version}-fc16, skype(x86-32) = %{version}-fc16
Requires: sqlite(x86-32)
Requires: alsa-lib(x86-32)
Requires: qt-x11(x86-32)
Requires: libXrandr(x86-32)

%description
Hack to get Skype 4.1 to work on Springdale Linux
Skype

Skype is software that enables the world's conversations.
Millions of individuals and businesses use Skype to make free video and voice
calls, send instant messages and share files with other Skype users.
Everyday, people also use Skype to make low-cost calls to landlines
and mobiles.

* Make free Skype-to-Skype calls to anyone else, anywhere in the world.
* Call to landlines and mobiles at great rates.
* Group chat with up to 200 people or conference call with up to 25 others.
* Free to download.

Originally done with rpm version 4.9.1.2,
built on du4-fc16-linuxui-02 at Mon Jun 11 19:38:07 2012
from skype-4.0.0.7-fc16.src.rpm with opt flags -O2 -g -march=i386 -mtune=i686

%install
rm -rf $RPM_BUILD_ROOT
mkdir -p $RPM_BUILD_ROOT
pushd $RPM_BUILD_ROOT
tar xjvf %{SOURCE0}
mv %{name}-%{version}/* .
mkdir -p $RPM_BUILD_ROOT/%{_docdir}/%{name}-%{version}
mv LICENSE README README.sdl third-party_attributions.txt $RPM_BUILD_ROOT/%{_docdir}/%{name}-%{version}
\rm -r %{name}-%{version}
popd

%clean
rm -rf $RPM_BUILD_ROOT

%files
%defattr(-,root,root)
%verify(md5 size link user group mtime mode rdev) %config %attr(0644L,root,root) "/etc/dbus-1/system.d/skype.conf"
%verify(md5 size link user group mtime mode rdev) %config %attr(0644L,root,root) "/etc/prelink.conf.d/skype.conf"
%verify(md5 size link user group mtime mode rdev) %attr(0755L,root,root) %{_bindir}/skype
%verify(md5 size link user group mtime mode rdev) %attr(0755L,root,root) %{_bindir}/skype-ubuntu
%verify(md5 size link user group mtime mode rdev) %attr(0644L,root,root) %{_datadir}/applications/skype.desktop
%verify(md5 size link user group mtime mode rdev) %doc %attr(0644L,root,root) %{_docdir}/%{name}-%{version}/LICENSE
%verify(md5 size link user group mtime mode rdev) %doc %attr(0644L,root,root) %{_docdir}/%{name}-%{version}/README
%verify(md5 size link user group mtime mode rdev) %doc %attr(0644L,root,root) %{_docdir}/%{name}-%{version}/README.sdl
%verify(md5 size link user group mtime mode rdev) %doc %attr(0644L,root,root) %{_docdir}/%{name}-%{version}/third-party_attributions.txt
%verify(md5 size link user group mtime mode rdev) %attr(0644L,root,root) %{_datadir}/icons/skype.png
%verify(md5 size link user group mtime mode rdev) %attr(0644L,root,root) %{_datadir}/icons/SkypeBlue_16x16.png
%verify(md5 size link user group mtime mode rdev) %attr(0644L,root,root) %{_datadir}/icons/SkypeBlue_32x32.png
%verify(md5 size link user group mtime mode rdev) %attr(0644L,root,root) %{_datadir}/icons/SkypeBlue_48x48.png
%verify(md5 size link user group mtime mode rdev) %attr(0644L,root,root) %{_datadir}/pixmaps/skype.png
%{_datadir}/skype

%changelog
* Mon Nov 25 2012 Thomas Uphill 4.1.0.20
- update to latest version

* Tue Feb 05 2008 Skype Technologies S.A. 2.1.0.10
- bugfix: #98304, Groups: No menus without surprise key.
- bugfix: #98312, Debug logs have different Filenames.
- bugfix: fix for tabs issue with qt 4.4.0 forward ported from 2.0.
- internal: disable prelink for skype on fc9 and fc10.
- internal: lib update

Sorry for making you jump through hoops but I doubt skype would like me posting the src rpm. Good luck.