DevOps w/ Perl @ Linode.pdf (2.1MB)

I wanted a solution that would allow me to receive Growl notifications without having to forward any ports and no matter what the public IP for my laptop may be. I use App::PersistentSSH to maintain a persistent SSH control master connection to my Linode so the ideal solution uses a reverse tunnel over this connection to get the notifications too my laptop.

My Current Solution

• Added
  --ssh_opts -R 127.0.0.1:22992:127.0.0.1:22

  to my App::PersistentSSH command line. This connects localhost:22992 on my Linode to port 22 on my laptop.

• local-growl.pl — an Irssi script that uses growlnotify to send growl notifications
• growlnotify.pl — an ssh proxy for growlnotify, passes it’s arguments to growlnotify on a remote system via ssh

Simple push growl notifications from my Linode -> my MacBook Air whenever it is online. The scripts and an irssi.png to use in the notifications are available on GitHub.

Configuration for local-growl.pl is via Irssi’s in built settings, type

to see a list of possibilites:

Configuration for growlnotify.pl is via a set of config variables near the top:

Hopefully you will find this useful.

At Linode we have a wiki page where we list 5 or more tasks we wish to accomplish during the week. There is a heading for each employee and below the heading we place our tasks. Throughout the week we can add additional tasks or mark existing ones as done.

I’ve used Hiveminder for some time. When we started the weekly task lists at Linode I found that taking a few minutes to figure out which tasks I wish to complete in the coming week works quite well for me. I started marking these tasks with the ‘week’ tag in Hiveminder. I quickly ended up writing a perl script, week.pl, to help me manage hem.

My Weekly Workflow

First thing Monday morning I run:

This prints a report with two sections. The first section lists tasks that currently have the ‘week’ tag with a line through the task ID if it is completed. This gives me a nice summary of what I planned on accomplishing the previous week and how I did. The second section lists all of my tasks currently visible in Hiveminder. I hide tasks that I know I’m not going to work on in the next few weeks so this list is usually no more than 20 or 30 items.

I take this report into the Monday morning meeting with me. During the meeting, I’ll glance over this list and select items for the upcoming week. I also use this page to take notes on during the meeting, writing down any new tasks that come up in the meeting.

After the meeting, I add any new tasks generated in the meeting that I won’t be working on this week to Hiveminder.

todo.pl comes from App::Todo, a command line Hiveminder interface. The braindump command launches $EDITOR where I add new tasks, one per line. The braindump syntax allows for specifying tags, setting priorities, and other things as well.

Next, I prepare the task list for the upcoming week:

The edit command iterates over each task tagged with ‘week’ if the task is not marked completed. It prompts whether or not I wish to carry the task over to this week (leave the tag). Any tasks marked completed have the tag removed automatically.

The add command works the same as todo.pl’s braindump command except the ‘week’ tag is automatically applied to the newly created tasks. The full braindump syntax is available for specifying other properties of the tasks.

The update command grabs my tasks tagged with ‘week’ and formats them one per line started with ‘ * ‘, a wiki bullet list. It grabs the current wiki page, finds my heading, substitutes the formatted task list under the heading, and submits the change. It also stores the path given in the YAML config file.

This opens the stored URL for this week’s tasks wiki page in my default browser, allowing me to confirm week.pl did what it’s supposed to.

Later in the week once I’ve done something:

This retrieves the task(s) tagged ‘week’ that are not marked completed containing the given string. If there was only one match, the script marks it as done and then outputs the current state of tasks tagged ‘week’.

Same as update before, except when no URL is given, the URL is read from the configuration file. This way I only need to worry about the URL once per week, the first time I update for the week.

Conclusion

So there you have it. If you are also using Hiveminder, maybe aspects of my work flow will make sense for you and pieces of the Perl script may be useful. If you aren’t using Hiveminder, maybe you will be inspired to check it out. I use michael@thegrebs.com on Hiveminder in case you feel the need to assign me a task or wish to gift me another year of Hiveminder Pro ;)

week.pl

Usage looks like:

The online documentation is available via search.cpan.org.

Git Web: http://git.thegrebs.com/?p=WebService-MobileMe
Git Clone: http://git.thegrebs.com/git/WebService-MobileMe

So I added a quick function to .bashrc:

Now I just run:

I recently received a nice Honeywell weather station. It turns out they are actually manufactured by a company called Irox which also sells stations under it’s own name but they seem to be more popular in Europe. Being a geek, one of the first things I did was check out Linux or OS X support. Unfortunately only one piece of software supports this weather station under Linux, Weather Display. There Linux software is freeware but not open source and leaves quite a bit to be desired but at least it would talk to the weather station and collect data from it.

With a bit of Googling I found Saratoga Weather which seems to be based on a template from Carter Lakes. Unfortunately the PHP code is a bit of a mess with 43 separate php files all in a top level directory and including each other. With a bit of work I was able to bend it to my will but every little change was a pain to accomplish.

I started slowly re-implementing things in Perl. I found an online WD data-file parser which greatly helped in figuring out what the fields in Weather Displays native log files meant. Fast-forward a couple of weeks and I think things are pretty much done feature wise. I lack NWS warning/watch/advisory notification, the thermometer image with today’s information is still PHP generated, and I don’t yet have a wind graph. These last few details shouldn’t take too much work when I get around to it. I also have a few features the original lacks, graphs, a daily tweet with today’s high/low, and sane code :p.

The information from my weather station is available online at http://weather.thegrebs.com/ and the source is available via git clone from http://git.thegrebs.com/git/wxperl. You may also browse the repo via Git Web at http://git.thegrebs.com/?p=wxperl.

is the most awesome Debian package /ever/.

Done.

Dependencies

If you have

installed and Some::Module requires Lib::Foo for which a Debian package is available, the dependancy will automatically be created. If Lib::Foo does not have a Debian package already available, you will likely need to manually run

on Lib::Foo first.

I haven’t seen the

trick mentioned elsewhere. Honestly when someone mentioned it on IRC I didn’t believe him. Sure enough, looking at the source for

l revealed that it does indeed check apt-file for each module that Some::Module lists as required in the

.

Why?

Now dpkg/apt know that you have Some::Module installed. If an official package for Some::Module comes out, it will likely have the same name and if it is for a newer version of Some::Module aptitude upgrade will automagically upgrade you to it. The real win is ease of cleanly uninstalling though. How many times have you found what you thought was the perfect module for task XYZ and installed it. Only to find that it isn’t so hot 5 minutes after beginning to use it. Making sure ‘

‘ really uninstalled every thing can be a pain.

Implementing is easy (stolen from the docs):

MakeMaker:

    WriteMakefile(
        (MM->can('signature_target') ? (SIGN => 1) : ()),
        # ... original arguments ...
    );

Module::Build:

   Module::Build->new(
        (sign => 1),
        # ... original arguments ...
    )->create_build_script;

Don’t forget to add

to your

if needed.

Then when running

you will be prompted for the pass phrase for your gpg key. For extra goodness, add 0-signature.t to your tests. It includes a single test that verifies the package cryptographically during

if the

environment variable is set.

If you know waltman and haven’t heard of this module, yell at him for not telling you about it, he’s mentioned in the

file for his stellar documentation patches.

Adding a DS to a bunch of RRD files is a big pain, RRD doesn’t have any sort of native mechanism for doing this. The usual procedure is exporting the RRD data to XML, creating a new rrd file with the old datastores plus your new ones, then importing the data back in from the XML.

A while back I needed to do this for a load of RRDs so I wrote a quick and dirty bash script for it. Today I had reason to add a DS again but couldn’t find my script. A bit of googling was no help either, surely other people have this problem too? I discovered Nicola Worthington’s RRD::Simple contains add_source method so a few moments in TextMate later we have:

#!/usr/bin/perl

use strict;
use warnings;

use RRD::Simple();

my $rrd_path = "/path/to/dir/of/rrds/";
my @rrd_files = <$rrd_path/*.rrd>;

my $rrd = RRD::Simple->new();

for my $file (@rrd_files) {
    print "Processing $file...";
    $rrd->add_source($file, 'DS_name' => 'DERIVE');
    print " ok.\n";
}