Updates from February, 2013 Toggle Comment Threads | Keyboard Shortcuts

  • Jeff Forman 8:57 pm on February 8, 2013 Permalink  

    Wireless, now with more 802.11′s… 

    With nothing else to do around here tonight while the whole state is shut down thanks to a blizzard, I should catch up on some blog posts.

    On my list of home network upgrades for the past several months was the wireless. As my wife and I add to our collection of smart phones, laptops, tablets, and wireless streaming devices (I am looking at you EOL Logitech Revue with Google TV) the amount of latency and available bandwidth started to show signs of strain. I had been running the wireless for several years on an Asus WL-500G Premium v2 router/wap, which only ran 802.11b/g over 2.4Ghz. It was time for an upgrade.

    Welcome our new Asus RT-N66U 802.11b/g/n router/wap that handles dual channel 2.4Ghz and 5Ghz wifi.

    I did some very unscientific comparisons before and after I performed the hardware upgrade. I pushed and pulled a ~763MB Ubuntu ISO across the wireless, through a 10/100Mb switch, via rsync over SSH from a server on the LAN. The following table shows rsync’s average speed and transfer duration from the point of view of a 15″ Macbook Pro connected via the wifi.

    Old Wifi New Wifi 2.4Ghz New Wifi 5Ghz
    Upload  1.87MB/s (6:46)  7.69MB/s (1:39)  5.57MB/s (2:17)
    Download  2.61MB/s (4:51)  10.81MB/s (1:10)  10.39MB/s (1:13)

    Needless to say, I am keeping the new router.

     

  • Jeff Forman 5:29 pm on October 6, 2012 Permalink  

    Nagios and Git hooks, a redux 

    A while back I blogged about how I hooked up Nagios and Git to run the Nagios preflight checks before restarting with a new checkin’s worth of configs. But the more I looked at how it all fit together, the more I knew it could be improved. A sed hack, expecting a certain pattern in the nagios.cfg? Bad bad bad. Most of the improvement revolves around Nagios’s ability to reference relative paths for its config files. Given the path of the ‘main’ nagios.cfg file, you can then reference directories that contain your services, hosts, and other custom commands, in relation to that main file. With this functionality I significantly improved the Git->Nagios pipeline.

    First, the pre-receive hook

    #!/bin/bash
umask 022

while read OLD_SHA1 NEW_SHA1 REFNAME; do
  export GIT_WORK_TREE=/tmp/nagiostest-$NEW_SHA1
  mkdir -p $GIT_WORK_TREE
  /usr/bin/git checkout -f $NEW_SHA1
  sudo /usr/sbin/nagios3 -v $GIT_WORK_TREE/nagios.cfg
  if [ "$?" -ne "0" ]
  then
    echo "Nagios Preflight Failed"
    echo "See the above error, fix your config, and re-push to attempt to update Nagios."
    exit 1
  else
    echo "Nagios Preflight Passed"
    echo "Clearing temporary work directory."
    rm -rf $GIT_WORK_TREE
    exit 0
  fi
done

    Using the GIT_WORK_TREE environment variable, which specifies Git’s working directory, I check out the new set of potential configs to a temporary directory. This provides a temporary ‘waiting room’ for the proposed configuration to be tested, before before being put into production. Imagine never (intentionally) breaking Nagios again because of a broken host or service specification. The main thing remember is that all references in the nagios.cfg to other config files (hosts, commands, etc) must be relative paths. I.E., I have lines that look like “cfg_dir=configs” in the nagios.cfg. Note the lack of absolute paths. We now run the Nagios pre-flight check (nagios -v) on the nagios.cfg in the Git work tree. Depending upon the exit value of ‘nagios -v’, 0 for success and 1 for failure, we either proceed or die immediately. If success, clean up our temporary run directory.

    Now the post-receive hook:

    #!/bin/sh

echo "Updating repo /etc/nagios3"
sudo /usr/bin/update-gitrepo /etc/nagios3

    The post-receive hook merely runs a script, noted below, on the Nagios configuration directory.

    Update-gitrepo:

    #!/bin/sh
umask 022

REPO_DIR=$1
cd ${REPO_DIR}

/usr/bin/git pull origin master

    Given the Git checkout’s directory, we fetch the most recent push to the repository.

    For the final step we have to fix some permissions (given that my setup runs the repository through Gitolite as the git user). This hook is located in the actual checkout itself, /etc/nagios3, in the post-merge hook.

    #!/bin/sh

sudo chown -R nagios:admin /etc/nagios3
sudo /etc/init.d/nagios3 restart

    A full commit and restart looks like this:

    jforman@merlot:/mnt/raid1/personal/git/monitor/nagios/configs$ git push
Counting objects: 7, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (4/4), done.
Writing objects: 100% (4/4), 414 bytes, done.
Total 4 (delta 3), reused 0 (delta 0)
remote: Previous HEAD position was c80fa03... turn off test notifications with notifications_enabled 0
remote: HEAD is now at f088dbc... Example: Add boilerplate header that file is managed by Git.
remote: 
remote: Nagios Core 3.2.3
remote: Copyright (c) 2009-2010 Nagios Core Development Team and Community Contributors
remote: Copyright (c) 1999-2009 Ethan Galstad
remote: Last Modified: 10-03-2010
remote: License: GPL
remote: 
remote: Website: http://www.nagios.org
remote: Reading configuration data...
remote:    Read main config file okay...
remote: Processing object config file '/tmp/nagiostest-f088dbcebf194edbce78068b6004cbbfca703432/commands.cfg'...
remote: Processing object config directory '/etc/nagios-plugins/config'...
remote: Processing object config file '/etc/nagios-plugins/config/ftp.cfg'...
remote: Processing object config file '/etc/nagios-plugins/config/mail.cfg'...
remote: Processing object config file '/etc/nagios-plugins/config/snmp_int.cfg'...
remote: Processing object config file '/etc/nagios-plugins/config/nt.cfg'...
remote: Processing object config file '/etc/nagios-plugins/config/http.cfg'...
remote: Processing object config file '/etc/nagios-plugins/config/real.cfg'...
remote: Processing object config file '/etc/nagios-plugins/config/check_nrpe.cfg'...
remote: Processing object config file '/etc/nagios-plugins/config/snmp_storage.cfg'...
remote: Processing object config file '/etc/nagios-plugins/config/disk.cfg'...
remote: Processing object config file '/etc/nagios-plugins/config/mysql.cfg'...
remote: Processing object config file '/etc/nagios-plugins/config/snmp_load.cfg'...
remote: Processing object config file '/etc/nagios-plugins/config/fping.cfg'...
remote: Processing object config file '/etc/nagios-plugins/config/dhcp.cfg'...
remote: Processing object config file '/etc/nagios-plugins/config/ssh.cfg'...
remote: Processing object config file '/etc/nagios-plugins/config/rpc-nfs.cfg'...
remote: Processing object config file '/etc/nagios-plugins/config/mailq.cfg'...
remote: Processing object config file '/etc/nagios-plugins/config/breeze.cfg'...
remote: Processing object config file '/etc/nagios-plugins/config/dummy.cfg'...
remote: Processing object config file '/etc/nagios-plugins/config/netware.cfg'...
remote: Processing object config file '/etc/nagios-plugins/config/hppjd.cfg'...
remote: Processing object config file '/etc/nagios-plugins/config/load.cfg'...
remote: Processing object config file '/etc/nagios-plugins/config/mrtg.cfg'...
remote: Processing object config file '/etc/nagios-plugins/config/apt.cfg'...
remote: Processing object config file '/etc/nagios-plugins/config/snmp_cpfw.cfg'...
remote: Processing object config file '/etc/nagios-plugins/config/snmp.cfg'...
remote: Processing object config file '/etc/nagios-plugins/config/snmp_process.cfg'...
remote: Processing object config file '/etc/nagios-plugins/config/snmp_env.cfg'...
remote: Processing object config file '/etc/nagios-plugins/config/news.cfg'...
remote: Processing object config file '/etc/nagios-plugins/config/ntp.cfg'...
remote: Processing object config file '/etc/nagios-plugins/config/telnet.cfg'...
remote: Processing object config file '/etc/nagios-plugins/config/users.cfg'...
remote: Processing object config file '/etc/nagios-plugins/config/snmp_mem.cfg'...
remote: Processing object config file '/etc/nagios-plugins/config/procs.cfg'...
remote: Processing object config file '/etc/nagios-plugins/config/ifstatus.cfg'...
remote: Processing object config file '/etc/nagios-plugins/config/games.cfg'...
remote: Processing object config file '/etc/nagios-plugins/config/disk-smb.cfg'...
remote: Processing object config file '/etc/nagios-plugins/config/tcp_udp.cfg'...
remote: Processing object config file '/etc/nagios-plugins/config/snmp_win.cfg'...
remote: Processing object config file '/etc/nagios-plugins/config/ping.cfg'...
remote: Processing object config file '/etc/nagios-plugins/config/pgsql.cfg'...
remote: Processing object config file '/etc/nagios-plugins/config/ldap.cfg'...
remote: Processing object config file '/etc/nagios-plugins/config/flexlm.cfg'...
remote: Processing object config file '/etc/nagios-plugins/config/dns.cfg'...
remote: Processing object config file '/etc/nagios-plugins/config/radius.cfg'...
remote: Processing object config file '/etc/nagios-plugins/config/snmp_vrrp.cfg'...
remote: Processing object config directory '/tmp/nagiostest-f088dbcebf194edbce78068b6004cbbfca703432/configs'...
remote: Processing object config file '/tmp/nagiostest-f088dbcebf194edbce78068b6004cbbfca703432/configs/timeperiods.cfg'...
remote: Processing object config file '/tmp/nagiostest-f088dbcebf194edbce78068b6004cbbfca703432/configs/services.cfg'...
remote: Processing object config file '/tmp/nagiostest-f088dbcebf194edbce78068b6004cbbfca703432/configs/commands.cfg'...
remote: Processing object config file '/tmp/nagiostest-f088dbcebf194edbce78068b6004cbbfca703432/configs/hosts.cfg'...
remote: Processing object config file '/tmp/nagiostest-f088dbcebf194edbce78068b6004cbbfca703432/configs/abstracts.cfg'...
remote: Processing object config file '/tmp/nagiostest-f088dbcebf194edbce78068b6004cbbfca703432/configs/contacts.cfg'...
remote:    Read object config files okay...
remote: 
remote: Running pre-flight check on configuration data...
remote: 
remote: Checking services...
remote: 	Checked 95 services.
remote: Checking hosts...
remote: 	Checked 10 hosts.
remote: Checking host groups...
remote: 	Checked 7 host groups.
remote: Checking service groups...
remote: 	Checked 0 service groups.
remote: Checking contacts...
remote: 	Checked 3 contacts.
remote: Checking contact groups...
remote: 	Checked 2 contact groups.
remote: Checking service escalations...
remote: 	Checked 0 service escalations.
remote: Checking service dependencies...
remote: 	Checked 56 service dependencies.
remote: Checking host escalations...
remote: 	Checked 0 host escalations.
remote: Checking host dependencies...
remote: 	Checked 0 host dependencies.
remote: Checking commands...
remote: 	Checked 181 commands.
remote: Checking time periods...
remote: 	Checked 4 time periods.
remote: Checking for circular paths between hosts...
remote: Checking for circular host and service dependencies...
remote: Checking global event handlers...
remote: Checking obsessive compulsive processor commands...
remote: Checking misc settings...
remote: 
remote: Total Warnings: 0
remote: Total Errors:   0
remote: 
remote: Things look okay - No serious problems were detected during the pre-flight check
remote: Nagios Preflight Passed
remote: Clearing temporary work directory.
remote: Updating repo /etc/nagios3
remote: From monitor:nagios
remote:  * branch            master     -> FETCH_HEAD
remote: Updating c80fa03..f088dbc
remote: Fast-forward
remote:  configs/commands.cfg |    2 ++
remote:  1 file changed, 2 insertions(+)
remote:  * Restarting nagios3 monitoring daemon nagios3
remote: Waiting for nagios3 daemon to die..
remote:    ...done.
To git@monitor:nagios.git
   c80fa03..f088dbc  master -> master

    Note that I do keep the Nagios package bundled commands in the /etc/nagios-plugins directory and have purposely not put those in the Git tree. This allows for updated Nagios packages from Ubuntu to update those commands accordingly without interfering with the Git repo.

    Enjoy.

     

  • Jeff Forman 2:44 pm on March 11, 2012 Permalink
    Tags: networking, smokeping   

    The Internet is slow. Is the Internet down? 

    We have all heard the same questions at one point in our careers, “Is the Internet down?” or “Getting to X site is slow.” You scramble to a browser to see if Google, ESPN or the NY Times websites are up. Then you fire up traceroute. In some cases, the pages might load slowly, in other cases not at all. These two situations are often downstream fallout of two connectivity issues: latency and packet loss. Latency is the time it takes for a packet to get from source to destination. The speed of light says the latency for one packet to get across the USA from New York to San Francisco is normally between 70-90ms [1]. Packet loss occurs when packets do not make it from their source to destination, being lost along the way. Many factors can contribute to packet loss, including overloaded routers and switches, service interruptions, and human error.

    When diagnosing network issues between source and destination, it is helpful to have data to backup your suspicions of slow and inconsistent network performance. Insert Smokeping.

    (More …)

     

  • Jeff Forman 7:39 am on February 18, 2012 Permalink
    Tags: sregonewild   

    A home network overengineered: dhcpd, tsig keys, ddns 

    I started to write this post, explaining how I upgraded my home network setup with a dhcpd server, multiple dns servers communicating securely via tsig keys along with dynamic dns, but the post became unwieldy and would have been thousands of words. Instead, I’ll post some links and gotcha’s and hints on how to make it work a lot easier.

    Links scoured and re-read in the process:

    Hints:

    Manage the key files distributed to each of your DNS servers with some sort of config management system (I use Puppet). That way if you ever need to change a key or add a new one, it makes things a heck of a lot easier.

    Don’t stick the TSIG key files inside your named.conf. This posses a security risk because anyone who can read your named.conf, now has access to your TSIG keys and can potentially update your zones. Instead, put them in their own files inside your bind etc directory, mark their perms as 640 (bind:bind, or the like) and use an include statement to get them into your named.conf

    Following on that last point, use dns’s allow-update statement inside zone definitions on the master. You can either lock things down via IP (less secure) or via Key (more secure) so that only authorized processes or people can update your zones.

    Gotcha’s:

    If you have FreeBSD clients, don’t forget the ‘hostname’ parameter in /etc/rc.conf. Otherwise you’ll request a lease from the dhcp server, but never tell your hostname, and therefore won’t get a record added to the ddns zone.

    Notes:

    Yes, this is a completely over-engineered solution on how to run a home network. It came to be because I play around with lots of VM’s at home, and to pique my curiosity bug, wanted to try to get things working end to end. Being able to ssh into the various Linux/OpenBSD/FreeBSD VMs by name made it a lot easier.

     

  • Jeff Forman 3:11 pm on April 23, 2011 Permalink  

    Remind Me: Initial Data in a Django class-based Form 

    I love Django‘s class-based way of handling forms. You name the class, articulate each field (data point of your form), and attach it to a view. Voila. But what happens when you want some initial data in the form?

    Initial to the rescue!

    What your class might look like:

    class PersonForm(forms.Form):
    first_name = forms.CharField(max_length=100)
    last_name = forms.CharField(max_length=100)
    gender = forms.CharField(max_length=1)
    hair_color = forms.CharField(max_length=256)

    If you now wanted to initialize your form for males with blonde hair, include this snippet in your view:

    form = PersonForm(initial = { 'gender' : "M", 'hair_color' : "blonde" } )

    Then pass that form as part of your render return:

    return render_to_response('add_person.htm', { 'form' : form })

    This post is brought to you by #neverwantingtosearchtheinternetforthisagain, and StackOverflow for inspiration.

     

← Older posts

c
compose new post
j
next post/next comment
k
previous post/previous comment
r
reply
e
edit
o
show/hide comments
t
go to top
l
go to login
h
show/hide help
shift + esc
cancel