Pete Shanahan's Shack

This is one of those odd things. For several years I have paid a data plan on my mobile phone to read my email. I think at the first time I paid it, they were looking for about €10 for 10MB of data (up and down). As time has passed I have still been paying this internet tax for the phone, only in the last two years with the purchase of a Nokia N95 has it become something of a complete fraud.
I've just lost this entry due to it being (a) a web 2.0 item and (b) not capable of dealing with the downing of my internet connection. Not making me a happy camper.
I bought the phone, while I still had my €13 for 10MB data connection plan. There was nothing better available. I assumed (more fool me) that there would be an option for a better data plan as I was buying a phone which required a half decent packet connection. Apparently, my mobile company were unaware of the fact that my phone was using the packet connection until I was in excess of €300 in the hole, and that was a paltry 2 weeks after getting the phone. Something to do with them being complete pillocks.
After a while, a plan became available that allowed for 1GB data per month. I barely use 100MB on the phone (I have a broadband dongle and am paying for broadband access in two locations simulataneously). I will be using my phone to tether my laptop. This is a given.
Apparently, the new phone plans for the iPhone and tethering will ask for a supplemental €15 which is epletivingly ridiculous. You cowboys have been charging us 10cent per 160 7bit message which is 64cent a KB or €655 a MB. All the code to deal with these is built into the system. The messages themselves pass through a mostly unused D-channel. They are free. Stop taking the piss. Data access should be a right - just like food and shelter.

This is a list of the top players of a game from GameTap. Kind of similar to the stats that are maintained by steam. The only issue is that those numbers are a little large - after all there's only 744 hours in a 31 day month. It's not actually a good piece of information to be displaying - 2032 hours is almost 3 *solid* months of gaming. If you flip over to another game you see 10274 hours - which is 428 days. These stats are kind of scary.
The total number of hours this month for may: 1053020, Number of sessions: 10376 which is 101.5 hours per session for the month of may. Assuming that a session is one launch -> termination of the game this is a really insanely long time.
I question the veracity of the statistics.
Anyone got any World of Warcraft stats?

Not a lot of code. We create a rewrite rule for apache to remap any 5 character requests at root to this script.
RewriteRule ^/([A-Za-z0-9][A-Za-z0-9][A-Za-z0-9][A-Za-z0-9][A-Za-z0-9])$ /cgi-bin/shorten.cgi?$1 [PT].
Requests for http://site/..... lookup the entry in the database, requests to shorten.cgi?URL return the shortened uri in a text/plain output when it works.
There isn't a lot of checking, and you probably need to create the db/ directory with mode 777 so you can update the database under cgi, but it... works on my box ;)
#!/usr/bin/perl -w
# shorten or unshorten a url passed in
use strict;
use DBI;
use Sys::Hostname;

my %keys;
my $value=0;
# 26 + 26 + 10 = 62
my $keys = join("", 'A'..'Z') . join("", 'a'..'z') . join("", '0'..'9');
my @keys = split(//, $keys);
for my $i (@keys) {
    $keys{$i} = $value++;
}

my $file = "shorten.db";
my $dir;
my $var;

if (defined($ENV{'SCRIPT_FILENAME'})) {
    $var = $ENV{'SCRIPT_FILENAME'};
} else {
    $var = $0;
}

($dir) = $var =~ m/(.*)\/[^\/]+/;

$file = $dir . "/db/" . $file;

if (! -d $dir . "/db") {
    mkdir($dir . "/db", 0777);
    chmod(0777, $dir . "/db");
}

my $dbh = DBI->connect("dbi:SQLite:dbname=$file", "", "") || die "Could not open $file";
chmod(0666, $file);

$dbh->do("create table if not exists mapping (id INTEGER PRIMARY KEY, url TEXT)");
$dbh->do("create index if not exists mappurl on mapping(url)");

exit(0) if (!defined($ENV{'QUERY_STRING'}));
my $qs = $ENV{'QUERY_STRING'};

if (length($qs) == 5) { # from short -> long
    my $key = 0;
    map { $key = $key * 62 + $keys{$_} } split(//, $qs);
    my $ary = $dbh->selectall_arrayref("select url from mapping where id = $key");
    if ($ary) {
        my @ary=@$ary;
        print "Location: " . $ary[0][0] . "\n\n";
    }
} else {
    my $sth = $dbh->prepare("select id from mapping where url = ?");
    my $ret = $sth->execute($qs);
    die "Failed to execute " . $sth->errstr unless($ret);
    my @row = $sth->fetchrow_array();
    my $value;
    if (!@row) {
        $sth = $dbh->prepare("insert or replace into mapping (url) values (?)");
        $sth->execute($qs) or die "Failed to insert" . $sth->errstr;
        $value = $dbh->last_insert_id("","","","");
    } else {
        $value = $row[0];
    }
    if (defined($value) && ($value > 0)) {
        my $op = "";
        while(length($op) != 5) {
            $op = $keys[$value % 62] . $op;
            $value /= 62;
        }
        my $base;
        if (!defined($ENV{'HTTP_HOST'})) {
            $base = hostname();
        } else {
            $base = $ENV{'HTTP_HOST'};
        }
        print "Content-Type: text/plain\n\nhttp://" . $base . "/" . $op . "\n";
    } else {
        print "Content-Type: text/plain\n\nFailed to shorten $qs.";
        exit(0);
    }
}

# vim: ts=4:sw=4:et

Apparently I forgot to plug out my null before quitting the application. That's a shame as I thought I had two or three nulls floating around

Unlike my default shell (zsh), bash has a wonderful feature where it doesn't keep variables that are set at the other end of a pipe, so for example:
i=
cat foo | while read bar; do
    i=$bar
done
echo $i
Yields an empty line. I've been stung once or twice on this as I prototype the code initially in an interactive shell, which doesn't exhibit the issue.
The simplest solution is to use a named pipe.

i=
mkfifo /tmp/foo$$
cat foo >/tmp/foo$$&
pid=$!
while read bar; do
    i=$bar
done </tmp/foo$$
This gives the last line of the file in the i variable.

So there I am looking at the sysinfo from a particular machine and I look at the content of the procs field. It looked to be a bit out. Went hunting through the kernel source and noticed that the procs field is filled with the value of the number of threads in the system. This is a little bit odd, as I'm used to separating my threads from my processes.
Turns out that there is an nr_processes() call, which returns the number of processes in the system, rather than the number of threads. A little bit of a change, rebuild and testing now gives me the correct number of processes from the proc field, and I also have a separate result for the number of threads.
There we go, much more sensible ;)
diff -Naur linux-2.6.25.18/include/linux/kernel.h linux-2.6.25.18.new/include/linux/kernel.h
--- linux-2.6.25.18/include/linux/kernel.h      2008-10-09 03:58:32.000000000 +0100
+++ linux-2.6.25.18.new/include/linux/kernel.h  2009-03-16 16:23:39.000000000 +0000
@@ -415,7 +415,8 @@
        unsigned long totalhigh;        /* Total high memory size */
        unsigned long freehigh;         /* Available high memory size */
        unsigned int mem_unit;          /* Memory unit size in bytes */
-       char _f[20-2*sizeof(long)-sizeof(int)]; /* Padding: libc5 uses this.. */
+       unsigned int threads;           /* Number of current threads */
+       char _f[20-2*sizeof(long)-2*sizeof(int)];       /* Padding: libc5 uses this.. */
 };
 
 /* Force a compilation error if condition is true */
diff -Naur linux-2.6.25.18/kernel/compat.c linux-2.6.25.18.new/kernel/compat.c
--- linux-2.6.25.18/kernel/compat.c     2008-10-09 03:58:32.000000000 +0100
+++ linux-2.6.25.18.new/kernel/compat.c 2009-03-16 16:43:31.000000000 +0000
@@ -1031,7 +1031,8 @@
        u32 totalhigh;
        u32 freehigh;
        u32 mem_unit;
-       char _f[20-2*sizeof(u32)-sizeof(int)];
+       u32 threads;
+       char _f[20-2*sizeof(u32)-2*sizeof(int)];
 };
 
 asmlinkage long
@@ -1076,7 +1077,8 @@
            __put_user (s.procs, &info->procs) ||
            __put_user (s.totalhigh, &info->totalhigh) ||
            __put_user (s.freehigh, &info->freehigh) ||
-           __put_user (s.mem_unit, &info->mem_unit))
+           __put_user (s.mem_unit, &info->mem_unit) ||
+           __put_user (s.threads, &info->threads))
                return -EFAULT;
 
        return 0;
diff -Naur linux-2.6.25.18/kernel/timer.c linux-2.6.25.18.new/kernel/timer.c
--- linux-2.6.25.18/kernel/timer.c      2008-10-09 03:58:32.000000000 +0100
+++ linux-2.6.25.18.new/kernel/timer.c  2009-03-16 16:20:02.000000000 +0000
@@ -37,6 +37,7 @@
 #include <linux/delay.h>
 #include <linux/tick.h>
 #include <linux/kallsyms.h>
+#include <linux/sched.h>
 
 #include <asm/uaccess.h>
 #include <asm/unistd.h>
@@ -1166,7 +1167,8 @@
                info->loads[1] = avenrun[1] << (SI_LOAD_SHIFT - FSHIFT);
                info->loads[2] = avenrun[2] << (SI_LOAD_SHIFT - FSHIFT);
 
-               info->procs = nr_threads;
+               info->procs = nr_processes();
+               info->threads = nr_threads;
        } while (read_seqretry(&xtime_lock, seq));
 
        si_meminfo(info);

The Full patch.

That's a huge amount of fragmentation. I think I may not be able to survive that percentage. It took < 5 seconds to defrag. Please people, percentages are supposed to mean something ;)

Using the natty little webtext script (updated to use an $HOME/.webtextrc file for the username/password) and a little bit of procmail magic, we can now send messages containing the subject line of a specific email message once it has been received.
Firstly, there's the .procmailrc file. The recipe I'm using looks like:

# send text messages
:0 W
* ^subject: IM:
| $HOME/bin/sendim

The script sendim is a simple bash script that checks the subject line contains "IM: " and then sends on the remainder of the subject line as a text message to my mobile phone.
As a security precaution, I've added a special header 'X-apikey' which is checked by the sendim. If the apikey doesn't match then the rule doesn't fire. You should replace the XXX item with your own value generated using echo <some text here> | sha1sum. By not putting the api check in the .procmailrc file you can quietly drop messages that don't have the correct key instead of keeping them in your inbox.
#!/bin/bash -p

export PATH=$HOME/bin:$PATH

subject=
apikey=
apikey_c="XXX"

while read foo; do
    [[ -z $foo ]] && break
    subject=${subject:-$(echo $foo | sed -n "s/[sS]ubject: IM: //p")}
    apikey=${apikey:-$(echo $foo | sed -n "s/X-apikey: //p")}
done

if [[ -n $subject && $apikey = $apikey_c ]]; then
    webtext -t 'YYY' "$subject" >/dev/null 2>&1
fi

Source of sendim. Don't forget to replace the XXX and YYY with your chosen items.

I've been testing the resizing of the drives located on a Dell MD3000, and i've seen errors when resizing after the 2TB mark. This is on the new firmware which supports > 2TB logical drives. I wrote a script to write to random locations of a block device. It can then read them back and verify that they're still the same as what was written. Rather than writing to the entire device I use random sampling, with a few fixed points on the block device. I pretty much get consistent failures. If I put in the failed locations into the next write run they come out again in the subsequent run. Kind of makes resizing a dangerous operation, even though it is stated that resizing is non-destructive.
I realize that the array is nothing more than a rebrand of another device, but it would be great if it was tested in a lab before something this bad got out to the customers.
#! /usr/bin/perl -w

use strict;
use Getopt::Long;
use Digest::MD5 qw(md5_hex);
use File::Basename;

my $fs;
my $readfile;
my $writefile;

my $numpatterns = 2048;
my $seed = undef;
my $size;
my $real_size;
my $help;

my %vars;
my @def_offsets = (0);

sub usage($) {
        print <<EOM;
Usage: $0 --fs=<filesystem> --read=<file>|--write=<file>
        [--num=<number of blocks>] [--offset=<offset to test>]
        [--seed=<random number seed>]
EOM
        exit ($_[0]);
}

my $result = GetOptions( 'fs=s' => \$fs,
        'num=i' => \$numpatterns,
        'seed=i' => \$seed,
        'read=s' => \$readfile,
        'offset=i' => \@def_offsets,
        'write=s' => \$writefile,
        'h|help' => \$help);

usage(0) if defined($help);
warn "Need file system to use" if (!defined($fs));
warn "Need either a read or write file" if (!(defined($readfile) || defined($writefile)));

usage (1) if (!defined($fs) || !(defined($readfile) || defined($writefile)));
my $base = basename($fs);

open (IN, "</proc/partitions") || die "Could not load partition tables";
while (<IN>) {
        chomp();
        my ($major, $minor, $blocks, $name) = m/(\w*)\s+(\w*)\s+(\w*)\s+(\w*)$/;
        next if (!defined($major));
        if ($name eq $base) {
                $real_size = $blocks;
                last;
        }
}
close(IN);

die "Could not get size" if (!defined($real_size));

# Write to the offset in blocks
sub write_to_offset($$) {
        my ($offset, $buffer) = @_;
        sysseek(INFS, $offset * 1024, 0);
        my $write = syswrite(INFS, $buffer, 1024);
        if (!defined($write) || $write != 1024) {
                warn "Failed to write: $offset $!\n";
        } else {
                $vars{$offset} = md5_hex($buffer);
        }
}

sub read_from_offset($) {
        my ($offset) = @_;
        my $buffer;
        sysseek(INFS, $offset * 1024, 0);
        my $read = sysread(INFS, $buffer, 1024);
        if (!defined($read) || $read != 1024) {
                warn "Could not read 1024 bytes at $offset $!";
                return (1);
        }
        if (md5_hex($buffer) ne $vars{$offset}) {
                warn "Data at offset $offset was not the same as expected";
                return (1);
        }
        return (0);
}

sub get_buffer {
        my $i = 0;
        my $buffer = "";
        while ($i++ < 256) {
                my $randval = int(rand(255 * 255 * 255 * 255));
                $buffer .= chr($randval >> 24) . chr(($randval >> 16) & 255) .
                        chr(($randval >> 8) & 255) . chr($randval & 255);
        }
        (length($buffer) == 1024) || die "Buffer was " . length($buffer);
        return $buffer;
}

if (defined($readfile)) {
        # reading from previous file
        open (INPUT, "<$readfile") || die "Could not open previous run log";
        while(<INPUT>) {
                chomp();
                my ($key, $value) = m/(.*)=(.*)/;
                if ($key eq "patterncount") {
                        $numpatterns = $value;
                        next;
                }
                if ($key eq "size") {
                        $size = $value;
                        next;
                }
                if ($key eq "seed") {
                        $seed = $value;
                        next;
                }
                $vars{$key} = $value;
        }
        close(INPUT);
} else {
        $seed = time ^ $$ ^ unpack "%L*", `ls -l /proc/ | gzip -f` if (!defined($seed));
        $size = $real_size if (!defined($size));
        open (OUTPUT, ">$writefile") || die "Could not open new run log";
        print OUTPUT "patterncount=$numpatterns\n" .
                "size=$size\n" .
                "seed=$seed\n";
}

print "Size: $real_size [$size] Seed: $seed\n";
srand($seed);

my $mode = "<";
$mode = "+<" if ($writefile);
open(INFS, "$mode$fs") || die "Could not open raw device";

if ($writefile) {
        map { write_to_offset($_, get_buffer()) } @def_offsets;
        write_to_offset($size - 1, get_buffer());
        while($numpatterns > 0) {
                my $offset = int(rand($size));
                print "Writing pattern: $numpatterns           \r";
                next if defined($vars{$offset});
                write_to_offset($offset, get_buffer());
                $numpatterns--;
        }
        map { print OUTPUT "$_=" . $vars{$_} . "\n" } keys(%vars);
        close(OUTPUT);
} else {
        my $failcount = 0;
        my $tocount = scalar(keys(%vars));
        map { $failcount += read_from_offset($_); printf("To Count: %0.7d\r", $tocount--); } sort(keys(%vars));
        print "Count difference: $failcount\n";
}

consistency.pl.txt

I bought an internet radio for the mother for Christmas - it means that she can listen to BBC Radio 4 without it sounding like complete rubbish over long wave. It worked fine for a few hours in the morning on Christmas day then it malfunctioned - the volume started to act as though the volume up button was jammed down. I can't reset it - the behaviour makes it completely unworkable. It seems to be some form of short circuit. After I powered it off overnight, it seemed to work again - for about 10 minutes, then I got the same behaviour. With sadness I shall be returning it to the store to get a replacement unit, which, I hope, will work much better.

The issue it that it's frustrating, I don't think that it's a problem endemic with the model as there seem to be a lot of people with the same model, none of whom seem to be complaining about it.