Recently in Programming Category

signal versus sigaction

| No Comments | No TrackBacks
the use of the
signal(int signum, void (*handler)(int))
is a smidgin dangerous on various operating systems. Under Solaris, for example once the signal has been delivered to the process the signal handler is reset, so a typical piece of code that wants to reuse the signal handler repeatedly will typically set the signal handler again when receiving the signal. This leads to a minor race condition where upon receipt of the signal and the re-setting of the handler the process receives another copy of the same signal. Some of these signals cause Bad things to happen - such as the stopping of the process (SIGTSTP for example). Under Linux it keeps the signal handler in place, so you have no fear of the event triggering an unwanted event.
The manual page for
signal
under Linux makes it clear that the call is deprecated in favour of the much more functional
sigaction(int sig, const struct sigaction *restrict act, struct sigaction *restrict oact)
call, which keeps signal handlers in place when you don't pass the SA_RESETHAND parameter as part of the sa_flags parameter of the sigaction structure. So you get to explicitly choose to accept a signal once, and then have the system deal with it in the default manner afterwards.
Signals, are of course a real pain in the ass when dealing with sub-processes. For example the use of ptrace to perform profiling works well until you fork. If another SIGPROF signal arrives before you can create your signal handler then the child process is terminated as that's the default behaviour in that situation.
Under Solaris (and Leopard) you can make use of dtrace to perform profiling on a set of processes without needing to deal with vagaries of signal handling, making this a non-issue. For those of you stuck in LD_PRELOAD land, probably the only thing that can be done is to set the signal disposition to be ignored before execing the new process. you have a small window where the profiling is missing, but the overall increased stability of the application is improved by preventing it from accidentally being terminated due to a profiling signal being received too soon. I know the accuracy nuts would hate that, but it's part of the price of dealing with standards.
#!/bin/bash -p

if [ $# -eq 0 ]; then
    echo "Usage: $(basename $0) " 1>&2
    exit 1
fi

while [ -d /proc/$1 ]; do sleep 0.5; done
If I implemented it using inotify, I presume I can get rid of the sleep, but that entails compiled code.
Ugly cuts to the bone...
Over at Coding Horror, young Jeff is complaining about the abuse of the words 'Beautiful Code' wherein the best example of an essay that is in the book is from Yukihiro Matsumoto, who doesn't even have the benefit of english as his native language, and he's not really mentioning his own language in the essay.
It's true, the beauty of the code isn't in the language specifics, it's in the expression that is being made.
Ugly code, however is ugly code, regardless of the language. It looks small, brutish and nasty.

Where's the SDK, huh?

| No Comments | No TrackBacks
I'm just wondering when Apple will be shipping the SDK for the ipod touch/iphone. Just being nosey really. I've veered away from jailbreaking it simply because I don't want to end up with an expensive brick next firmware update.
They say February, but of course remembering that the Leopard launch being the end of the month it could be anywhere up to the 29th.
Another thing I'd like to see is the UI guidelines. I'm a bit of a nerd when it comes to reading design guidelines, simply because there are a lot of good points in them. Mind yo, you should not be slavishly obeying them, as, after all, they are only guidelines, and not commandments.
On guidelines, I'm get miffed with applications that require the use of the mouse to accomplish things. Vista's keyboard usable everywhere is a charm to use, even while it's gobbling up all those cpu and disk resources with the indexer.
It turns out there's a bug, due primarily to testing on zsh and putting the script into bash.
function find_lowest_subprocess() {
    local -i parent=$1
    local pids
    typeset -a pids

    pids=($(pgrep -P $parent))
    while ((${#pids[@]} > 0)); do
        if ((${#pids[@]} > 1)); then
            parent=${pids[0]}
            local i=0
            while ((i < ${#pids[@]})); do
                local sub=$(pgrep -P ${pids[$i]})
                [[ -n $sub ]] && parent=$sub
                ((i=i + 1))
            done
        else
            parent=$pids
        fi
        pids=($(pgrep -P $parent))
    done
    echo $parent
}
Someone, somewhere forgot about the parenthes around the array assignments. Shame on me!
I was reading the 'Basics on how shims work', and all I could think about was the old linux/unix trick of using LD_PRELOAD to intercept library calls in dynamically linked applications (used to great effect to bypass timebombed applications).
It is one of the reasons why I think that static linking is just horrible - after all, it breaks things, and makes you have to use terribly complicated tricks like disassembling and binary patching to fix problems because you don't have the source code. You don't get the advantages of page sharing, which is kind of important when you consider how many libraries are loaded by the typical gnome application (last check on nautilus was 117 libraries, excluding VDSO and the binary itself, firefox has 151, excluding fonts and other pango related nonsense). Yes, indeed, winners don't use static linking.
As a tangent, it's one of the reasons for not using application compression tools on binaries - after all, the only thing you save is on the distribution size, not on the run-time. The binary itself acts like a static linked blob, and doesn't share the text segment (program code) amongst the other instances, like other binaries, so every instance can use a lot of private memory, leaving less physical memory on the machine for other things like playing high resolution video.
The GPL is a difficult beast.
all those MODULE_EXPORT_GPL symbols.
F**k them.
I'm just going to release a patch that remaps them all to MODULE_EXPORT.
The patch is GPL.
F**k you I'm a pragmatist, not an idealist.
This was written at 3am, I am drunk and I'm bitter. You know who I am.
I don't prefix my posts with 'yes, I drink...', you should know what I'm like, after all; you're here
How can you tell if your website is usable by anyone? Well one of the simplest ways is to use a text mode browser. It gives you a good idea as to how easy it is to navigate without having to spend the effort getting in a UI designer at infinity$/hour just to tell you that your links should be purple, and not help you in the least bit.
links is a pretty damned good text mode browser. It makes it obvious if you've made a mess of the UI for navigating, which is always a good thing. It makes it obvious when you've forgotten to put the bloody ALT tags into all your images - like a good designer should. Spacer gifs are evil and must be stopped :)
We need to extend the linkit script to support (a) Solaris and (b) spaces/escape characters in the file name.
Sorry, my shell environment just isn't cooperating with me - BASH_ARGV[0] == {}, yet I cannot dereference the variable correctly no matter what I try. No wonder people use perl. God this is so frelling irritating.
Some of the locations have changed, some have been removed (XP)

Baggy sweatshirt problem

| No Comments
<ramble>
It's a problem when programming. You don't have to make a one size fits all solution to a problem. It just isn't worth the effort. I'm being reminded of the Simpson's episode parodying Mary Poppins... 'If you cut every corner you'll have more time to play'. i.e. if you can get away with it then fake it.
Polishing crap still leaves you with crap. Spend a half an hour experimenting with something to decide if it will work.
Learn to shoot your baby in the crib or It's too easy to get locked into a commitment that is really too much work.... i.e. even when you're half way through the experiment and you realize it's going to drive you mad trying to get it to work then issue a big old 'rm -rf'.
</ramble>
This little piece of shell script attempts to find the 'lowest subprocess' of a passed in process. It works well with a straight tree of processes, and if there are the occasional pipes in the command tree then it will miss them most of the time.
Maybe tomorrow I'll talk about how to fix it.
function find_lowest_subprocess() {
        local -i parent=$1
        local pids
        typeset -a pids

        pids=$(pgrep -P $parent)
        while [[ -n "$pids" ]]; do
                if (( ${#pids[@]} > 1 )); then
                        local i=0
                        while (( i < ${#pids[@]} )); do
                                local sub=$(pgrep -P ${pids[$i]})
                                [[ -n $sub ]] && parent=$sub
                                ((i=i + 1))
                        done
                else
                        parent=$pids
                fi
                pids=$(pgrep -P $parent)
        done
        echo $parent
}
FeatureBourne ShellBusybox ShellBash
Subprocess Execution`` (the backtick)`` or $()As Busybox
Math Evaluationuse expr (not builtin)$(( ))As Busybox; adds ((var=math))
ConstantsNoneNonetypeset -r
IntegersNoneNonetypeset -i
Evaluation[[[
Extended Evaluation/bin/[[ (1)/bin/[[ (1)[[

(1) The [[ operator is not the same as the program /bin/[[ as a program you need to still use double quotes around the variables to be expanded; thus defeating the reason for having them in the first place.

For Example:

	# file="/tmp/let me go"
	# touch "$file"
	# [[ -e $file ]] && echo "There"

Yields [Busybox]:

	[[: me: unknown operand
- as $file is word split before handing it to the command [[

Yields [bash]:

	There
- due to $file not being split when passed to the test -e.
being at work again raises a few interesting issues. One of them involves the period over Christmas. It seems like there's nobody in the office (I am one of three/four). I have 0.6 days holiday left and I am troubled. Damn my work related guilt.

Subversion for the oblivious (svn)

| 1 Comment
I'm a big fan of source code control systems. They help you not shoot yourself in the foot when making changes and it is an essential part of any form of development. Heck, it even comes in handy for managing changes to configuration files.
Solaris ships with SCCS, the source code control system for pretty much all forms of unix. It's a file based source control system, and is used as the binder for other products such as teamware.
On Linux there's rcs for file based source code control and then there's CVS or Subversion for the remote repository work.
Subversion really beats the pants off CVS for features that make it more sensible. If you've ever tried to move a file in CVS you know what I mean.

debian makes life more realistic

| No Comments
It's very simple really. Ubuntu uses dash as the default shell for executing init scripts as it is faster (there's less overhead). dash is, of course the debian almquist shell, which is a bourne shell compatible shell. Not bourne again, but simply bourne, this means that features such as [[ are not present (see a previous blog entry for a complaint about that). We all like the fast boot, but it kinda breaks Makefiles that assume sh == bash (ieeee80211.sf.net, ipw3945.sf.net).
I'm too used to writing makefiles under Solaris to use features that are not present in the original bourne shell.

Makefile default shell

| 1 Comment
This one is a bit of a gotcha that I encountered in the ieee80211 and ipw3945 source. The make files used some convoluted shell syntax to check the state of the kernel. The problem is that it uses particular non bourne-shell isms, such as the [[ syntax, which simply doesn't work in bourne, because it's so bloody simple.
The short solution was to put SHELL=/bin/bash at the start of the Makefile, which causes the shell that is used to be bash instead of sh. This is of course one of the regular issues between the different versions of Linux that float around. Some use bash as sh, which technically speaking is a bit of an evilness. It can hit you time and again in writing rc scripts as well, where the choice of shell is not generally determined by the magical '#!' on the first line, but by the rcX script (X is generally S, 1, 2, 3, ...).
Again, the quantity of run levels is determined by the provenance of the Operating System - for example the machine I'm currently working on(Ubuntu) claims a run-level of 2, while the fedora core desktop I've just checked claims a run-level of 5. Solaris makes thing really fun by informing you that tour run-level is a legacy state that you should stop considering to be useful, after all it's about the services that you have enabled, not the run-level.
Probably one of the handiest features of C++ is the ability to create an object on the stack, and have it destroyed once the class has gone out of scope. This is because of the design of the language.
When you create an object using the syntax 'ObjectT foo' the object is instantly initialized, and you refer to each of the items in the object as foo.<whatever>. When the function returns the ObjectT's destructor is called to clean up the object. This happens for every class.
Borland have seen it fit to make their compiler barf when you use one of the Visual Class Library(VCL) classes to create an object. Personally, I find the fact that you have to then wrap the code in a __try__ __finally__ block to be a waste of my time. After all there is no rational reason for preventing me from using the variable on the stack, as stack memory is just as good as heap memory (I'm old skool me!).
All you're going to have on the stack is a pointer to a VTBL and the data concerned with the object; nothing more. If you have to cast it to a lesser object, then cast it to a lesser object. If you are using this object in another object (for example adding it to a collection), then use the proper syntax (in this case the ObjectT *foo = new ObjectT()). As a programmer you should know these things.
I would argue that the compiler should not protect us from such annoyances, but the reality is that to make better code we need more assertive nannies. All my C code is compiled with -Wall -Werror, which catches a lot of stupid mistakes, but doesn't catch a lot of normal problems. Sometimes I think I would be better in a garbage collected, reference managed, array overrun protected world... but where would be the fun in that? I like my assembly language, I'm more careful as I know every instruction counts. That and the fact that a review of assembly code takes significantly longer than the same review of C code makes me pray that the developers are paying more attention.
I'm getting annoyed with the number of programs that I have to exclude from DEP. Practically all of the palm simulators fall into this category along with all the securom games I've used.
Grrrr.
Trivial, I know, but some people need to know these things.
From the start we have the header, appinfo (optional), sortinfo (optional) followed by the record entry headers.
The header is:
FieldSizeValue
Db Name32Ascii Database Name
attributes2See DataMgr.h for meaning (dmHdrAttrResDB)
version2Version number an application would use this to decide if the content was compatible
creationDate4Creation date of the database - time from the palm epoch when it was created
modificationDate4Last modifiied date of the db. creationDate by default.
lastBackupDate4date last backed up ina hotsync
modificationNumber4when changed this gets bumped
appInfoID4Offset in this DB to the appinfo block (0 if it does not exist).
sortInfoId4Offset in the DB to the sortinfo block (0 if it does not exist)
type4the 4 character type of the pdb
creator4the 4 character creator of the pdb
uniqueIDSeed4Stumped - I don't know what this is for, leave it at 0
This is the end of the header structure, the next is the record entry information
Record list:
FieldSizeValue
nextrecordList4Address of the next record list (only used for really bit pdbs)
nRecords2Number of records in the recordlist
pad2Number of records in the recordlist
nRecords items4 * nRecordsAddresses of the records in the pdb

The address of the first record usually lies immediately following the appinfo and sortinfo data. As this data is consumer defined, the only way of determining it's size is to use relative calculations. For example, the size of the appinfo structure it's local address up to min(addr(sortinfo, Address of any data records)) - 1. The size of the sortinfo structure is it's local address up to the min(Address of any data records) - 1. They are application defined, and as such should not be messed with.

This is really a WTF

| No Comments
I discovered this one in a state management routine - save the address of the database corresponding to the module that's loaded. It works until the program is restarted or the operating system is rebooted. I'm going to have to change it to use the name of the module instead of it's address. The only problem I can forsee is the presence of multiple copies of the same named module in the application path; That's just another thing to work out.
Firstly, it's cool that they are there. Rather than having to do something surreal involving runas, explorer and a couple of other things to allow you to run an application at a privileged level you now encounter the ubiquitous shield icon, which tells you that to perform this operation you need to acquire the appropriate privileges. It's a lot like the linux sudo, except by default you just have to click the 'continue' prompt instead of a password.
Pretty cool, even if you're an administrative user, you don't start with all the privileges that your group memberships provide.
Here comes the rub - I've stopped reading the prompts, I just find the one that tells me how to get to the next step and click on it. I'm not positive, but I think this is probably par for the course for other users as well.
Shame that, nice idea, but hamstrung by having too many things need administrative privileges.

Palm Binary Compatibility

| No Comments
I find the Palm platform convenient for programming. You can build something using the latest and greatest Palm 5 based SDK and then run it on the oldest cruddiest palm you have. As long as you make sure to not use unsupported APIs your program just works. It's like static linking under Unix/Linux - all you need to make sure is that the underlying system calls are there (or in this case entries in the trap table.

Depends, really

| No Comments
Some people think that make is a terrible piece of software. Honestly, it is just awful for modern projects with large numbers of dependencies. Header file dependencies become a problem. There are options to auto-insert the dependencies on header files into the makefile. Subdirectories are a problem - isolating certain code in directories is tricky. You can use the VPATH feature to provide a certain level of automatic path traversal without over-complicating the makefiles, and for trickier features you have the :sh= [svr] or $(shell ...) [gnu] options to pass the work off to the shell or a script. Still the best thing about make is that it completely evaluates the dependency graph for targets; explicitly forbidding loops (or self reference).
Then we come to software releases, and their dependencies. You can't install X without Y. you can't remove X without remving Y. Removing X will break Y therefore you can only remove X and Y together. Good idea. Difficult on customers, though. They want an 'add/remove programs' option which installs and uninstalls all the needed components. I recently bought Sin Episodes from steam. It auto-installed Sin 1/Sin Multiplayer. I wanted to uninstall Sin 1/Sin Multiplayer. You can't Sin 1 depends on Sin Multiplayer. Sin Multiplayer depends on Sin 1. Perfect cyclic dependency preventing you from uninstalling. Uninstalling means going in and deleting the .gcf file. Ah well, that's life, I suppose.
One of the links for The Best of Software Writing II linkfest is to an article entitled 'Primum non Nocere', or First, do no wrong. It's an important principle that should be followed when writing software. On the non paranoia side of things we have the 10 places you must and must not use AJAX, which is a good consideration of appropriate cross cutting of client/server interaction. There's the 'Fractal nature of UI design problems', where I've reached step 5 in the problem being addressed before just saying 'good enough'. How I came to despise AJAX, part of a long rant that I've experienced myself. Does Visual Studio rot the mind? I'm a vim user. Why I hate Frameworks - an argument against complicated frameworks. Simple ones for me thanks.
Bit of a linkstravaganza really.

XP on the mac

| No Comments
It's unofficially doable; a few folks won the $14,000 odd prize for it. Practically nothing on the machine works, though. I am reminded of the situation of Solaris on X86 - noone wants it because the hardware support is so poor. The reason for the poor hardware support is that there isn't anywhere the number of driver developers as there would be for such beasts as Linux or Windows.
640x480 (or 800x600 I think) VGA graphics drive, no wifi, no networking, no bluetooth. Pretty much useless from the usable laptop front. I'd take Linux on it before Windows if that's the case. Of course theres a fully functional unix machine under the hood for Mac OS X, and while someone will probably want to shoot me for it, the fact that it's proprietary isn't too much of a big loss.
I was talking to my mate Mark on St. Paddys day about Hyper Threading processors, and he was mentioning that they're not the best at high-performance computation (without extensive and expensive hints in the code). I agreed, mentioning that the latest generation of multi-core processors offer roughly equivalent cost and scale almost multi-processorly. I then went on to explain that the multi-threaded processors are better for I/O workloads, you shove a lot of the scheduling cost back into the silicon where it belongs, rather than having the OS deal with it in software.
For a big server, performing lots of I/O, a multi-core and multi-threaded processor would be the best of both worlds, and based on the direction that Sun is taking with the Niagra system, one can see that this can be taken to a scary extreme - consider 8 core with 8 threads per core all on the one processor module. The power-savings alone would be enough to warrant buying these machines.
Im still waiting for quotes on a few more laptops. I can wait, I just don't know for how much longer. Meanwhile I'll probably buy a phone. Nokia 6230i looks like a cheap and easy option - buyable from €260.19. Or maybe an annoying smartphone like the iMate PDA 2K (it's the original of the O2 XDA IIs).

Slow composition

| No Comments
I have a set of hand written pages of the next 20-30 features to implement/changes to make to pocketcity in order to make it 1. better and 2. more easy to port. One of the things I'm planning on doing is moving the water pipes underground. This will change things quite a bit.
I'm considering getting a new laptop (again) and I've reached the point where the hardware that's available is reaching what I want. It needs dual-core processor; I'm not getting another uni-processor machine again. Acer have dual core centrino models, with a decent graphics card. The only problems are the DVD drive and the video card. It's a DVD-RAM drive. It won't region free! it's the same problem as the Ferraris. I don't want to re-code all the non region-2 discs I have. That would be a pain in the ass. The video card isn't supported in Linux either, which is another annoyance. I hope that ATI release a driver for the X1000 family soon.
Alternatives are Alienware and their Aurora m7700, which has an athlon processor. Has a more supported graphics card, but it's about twice as expensive as the Acer. Then there's the Widow PC laptop. Still the price tag problem.
Until I get a new laptop, I'll have to be happy with the one I have. It's an early generation centrino, so I've only got a/b wireless, and an integrated GPU. It works well for what I'm doing (programming, watching movies, occasional game).
Shame that Dell haven't caught up on dual core for the gaming laptop.

Network protocol design

| No Comments
Let's make this one easy for the protocol writers. Front load the important information in the packet. That way we can more easily detect it and send it on to the correct handler. Stop putting the decision making information in the middle of the packet. We don't have infinite processor time on these handlers.
This complaint was brought to you by people who care how network bandwidth is being consumed.
I thought I had it, but I didn't. I still can't find out the reason. Currently, after each post I re-read the table by closing and opening it. The table is a small, local, temporary table for recording information before posting it to the real database so I don't really care that the exception is triggered.
Now, I'm getting DbgBreakPoint exceptions. This was in a simple showmodal call, so I have no idea why it's there. Apparently it might have something to do with opening the table concerned.
I was experiencing this occasionally when implementing the delete functionality for a browse window. It turns out that I was not positioned on a record, but instead was either before first or after the last record. Simple problem, really but a bit of a pain to discover the reason as the error isn't really informative.
The solution was to do move to either the first or last record. Thanks to the magic of the 'meta bof/eof records'. I have to use the description loosely, as they definitely ain't records and the correctness police would be rapping my knuckles for such a statement.
Discovering the solution was not helped by the browse grid control I was using - it seemed to indicate that I was on a record when in reality it wasn't. Another case of model does not match the implementation.
This one: Secure Java apps on Linux using MD5 crypt
Firstly, the encrypted string is: $<mechanism>$..., where a mechanism of 1 is MD5. My desktop has 2a, which indicates I'm using the Blowfish algorithm - I see no reason to compromise.
Well, guess what - this document won't work for me because I use blowfish locally, and secondly, the only accounts in /etc/shadow (and passwd) are local accounts - if you're using nis, nisplus, or ldap (solaris more so) for your name services, then you're SOL with this mechanism.
This is another reason for not using the crypt mechanism is that this is trying to solve the problem from the wrong level. You should not be trying to compare the encryption strings, you should be using an alternative to the OS provided security mechanism. So in this case you should be using jaas, and a PAM plugin. The use of yet another 'well it works on my box' mechanism is so cripplingly annoying that it angries up my blood.

Who's living in what apartment?

| No Comments
It's the COM apartment models. They're related to the threads that make use of COM objects. What happens is that when you initialize COM for a specific thread you declare that it's either Apartment Threaded (AKA Single Threaded Apartment) or Multi Threaded.
When you use the Apartment threading model, it means that the COM object is isolated within the thread that created it. The most important piece of information about this model is that you should never use that object in another thread - it causes brokenness.
When you use the multi-threading model, what you're pretty much saying is that I'm probably going to use this COM object in several threads. The way it works is that a multi threaded model, then the context is shared within the process.
The model you support also puts extra complications on you, the creator of the object. COM objects with a declared MT support must use some synchronization to protect shared information within the object, otherwise you'll suffer from data corruption due to threads walking over the data. You don't have any of these considerations in a Single threaded model - you're guaranteed safe and sane interactions.
Additionally, when you're in COM land, remember never just WaitFor*, but instead MsgWaitFor* things. This also applies to using DDE. This is because the Apartment model uses windows messages under the hood.
[Listening to: TWiT 38: MacWorld Expo - Leo Laporte and the TWiTs - this WEEK in TECH (1:19:01)]
The laptop does suspend and resume to disk, but I have issues with the video card - I use the 855resolution program to allow the video card to set the native 1400x1050 resolution of the screen. When I resume once X kicks in and can't set the video mode correctly I am booted back to a login screen.
I put in an entry in the powersaved post resume from suspend2disk script, and it correctly repatches the video mode before the X server tries to reinitialize the video. The problem is that I need to use the powersave command to shut it down correctly.
I took the alternate route and installed the software suspend2 patch. It provides a cleaner interface to hibernation (the hibernate command); it compresses the file; it supports using a file on disk instead of the swap partition; you can encrypt the hibernation file. It will unmount the windows paritions when I suspend, remounting them when I resume (corruption issues avoided) and runs the 855resolution command before X kicks in. All in all a nicer way to deal with hibernation.
I still haven't got suspend to ram working. Standby works, though.
Now all I need is to get 855resolution working on Solaris and most of my complaints will just vanish into the distance.
Well it seems to be a problem for me - I'd love to get more work done on Pocketcity, but these bloody episodes of CSI on UK Living (I used to work there) are distracting me horribly until really late at night.

Valid windows file names

| No Comments
I keep forgetting this when I decide to accidentally create files with names that turn out to be undeletable under windows. What makes a valid Windows file name? is a blog entry on Brian Dewey's blog and it reminds me that using the \\?\ feature bypasses the MAX_PATH check for a file name's length when trying to manipulate it (or delete it). I've accidentally created files with long names a few times, and been unable to delete them. Thankfully I've created a little application that allows me to do this now.

Least Significant 1 Bit

| No Comments

This can be useful for extracting the lowest numbered element of a bit set. Given a 2's complement binary integer value x, (x&-x) is the least significant 1 bit. The reason this works is that it is equivalent to (x & ((~x) + 1)); any trailing zero bits in x become ones in ~x, adding 1 to that carries into the following bit, and AND with x yields only the flipped bit... the original position of the least significant 1 bit.

Alternatively, since (x&(x-1)) is actually x stripped of its least significant 1 bit, the least significant 1 bit is also (x^(x&(x-1))).

Integer Selection

| No Comments

A branchless, lookup-free, alternative to code like if (a<b) x=c; else x=d; is ((((a-b) >> (WORDBITS-1)) & (c^d)) ^ d). This code assumes that the shift is signed, which, of course, C does not promise.

Integer Minimum or Maximum

| No Comments

Given 2's complement integer values x and y, the minimum can be computed without any branches as x+(((y-x)>>(WORDBITS-1))&(y-x)). Logically, this works because the shift by (WORDBITS-1) replicates the sign bit to create a mask -- be aware, however, that the C language does not require that shifts are signed even if their operands are signed, so there is a potential portability problem. Additionally, one might think that a shift by any number greater than or equal to WORDBITS would have the same effect, but many instruction sets have shifts that behave strangely when such shift distances are specified.

Of course, maximum can be computed using the same trick: x-(((x-y)>>(WORDBITS-1))&(x-y)).

[Listening to: Queen Bitch (live) - David Bowie - RarestOneBowie (3:15)]

Normally, a dual-linked circular list would contain both previous and next pointer fields and the current position in the list would be identified by a single pointer. By using two current pointers, one to the node in question and the other to the one just before/after it, it becomes possible to store only a single pointer value in each node. The value stored in each node is the XOR of the next and previous pointers that normally would have been stored in each node. Decoding is obvious.

Unfortunately, using this trick in C is awkward because the XOR operation is not defined for pointers.

[Listening to: Space Oddity - David Bowie - Changesbowie (5:17)]
I never really read the documentation for the built in quick sort function for Palm OS. The only thing I ever looked at was how to swallow the extra parameter for the function. If I had read the documentation properly I would have noticed that it possesses several very handy optimizations that come (mostly) for free. The first is that it automatically uses an insertion sort if the number of records are 'low'. This is very handy, as it means you don't need to decided to use one algorithm over the other. It also swaps to an insertion sort if at a point in the sorting it discovers that the stack is about to be consumed completely, which means no mysterious application crashes due to the recursive nature of the quick sort.
The only real problem with it is that it, like all other versions of quicksort is that it's unstable. If two items have the same key then they may appear on the output in a different order than they were input. Not a big pain really.

There's NUMA perhaps we need NUPA

| No Comments
a minor addendum to the previous post, perhaps we need to work on Non Uniform Processor Access :)
What's in a file? If you're to believe Windows, a .doc file is a Word Document, a .xls is a spreadsheet, a .jpg is a jpeg file. The reality is in the business you will encounter a bunch of anonymous files. If you'fe fscked a file system or ran chkdsk /f you can end up with a bunch of unlabelled or labelled badly files. Identifying them is a bit tricky. Some people don't care, they just wipe the recovered files, but if you care, probably the first thing you want to do is figure out what file type they are.
For Unix and Linux we have the file command. This command determines what a file is based on the content of the file. How it does this is based on the contents of the /usr/share/file/magic file, which describes in simple means how we identify the file. It's not 100% accurate, and regularly makes a pigs ear out of identifying text files properly (curses and your free format), but it works most of the time for images, programs and most of your annoying microsoft files.
Convenience aside, most of the work on the file command is actually performed by the libmagic library. You, as an application developer can take advantage of this library to provide useful information about a file to the consumer.
Apple Macs (and PalmOS devices) have it easy. Each file carries along with it identifying marks of it's creator application and file type. This information is stored in the resource fork of the file, which for Macs can mean problems when transporting it from platform to platform. Self identification goes a long way though. As creator codes are registered with Apple, it means you generally avoid treading on other applications and co-opting their file types, as happens on windows all the time.

Bad Motorola, no cookie for you

| No Comments

My old, unreliable mobile phone died a couple of days ago, and I needed to buy a replacement. I'm not able to upgrade my phone for another 2 months so I decided to buy a pay-as-you-go phone and just slip my SIM into it (same network, no issues with locking). People who know me, will understand that I have a liking for flip-phones, so I went for the Motorola V3 (Razr). Aargh! christ, but the phonebook is the biggest piece of shit I've ever come across. I understand this misbegotten need to have the phone book maintain some compatibility with the SIM, but for christ's sake, they need to get their act together on this. Practically every contact in my phone book has two entries, in fact most have 3. On the motorola phone book every contact is a separate entry on the person list. This means I have to wade through 2+ entries per person to scroll from one person to the next. Factor in that I store people's full names on the phone, in surname order, wading through the 9 Shanahan's in the book takes a long time. It always integrates the SIM contacts onto the phone book. There is no visible option to disable this 'feature' (my SIM contacts always were a backup of my phone entries), so I end up with loads of duplicates (or purge my SIM). I can't send my entire contact details to someone, I have to send it piece by piece. It shows the email addresses interspersed with the phone numbers which is pointless most of the time.

The next complaint is really a bit of a click-fascist thing. You know what I mean by this - it just seems to take an extra click or two to perform some tasks. Just enough to annoy perceptibly.

Then there's the syncing software. Following the really annoying splash screen - slow, irritating and serving no purpose, we are presented with a rendering of my phone on the bottom right corner of the screen, a bunch of icons and no idea what does what without mousing over one of the icons and seeing what it does, based on the tooltip! Come on people, tooltips cannot replace text! Apparently I can dial numbers from the number pad (never would have guessed at that). The only way to pop up the menu is to hit a box that's about 20 pixels square, replicating the menu button on the phone itself. Too small to hit easily, and there is no keyboard navigation, unless you can guess at the magic hotkeys. Most of the hard work is farmed off to other applications, none of which share details of the current state of the phone (contacts, calendar), each sub-application launch causes the data to be re-read, which takes ~30 seconds each. None of these sub-applications are keyboard navigable (bugbear of mine). Quitting the application requires either clicking on the really small off switch, or doing the acme Alt+F4 close the window trick. Practically everything visual about this application could do with a rewrite.
On the plus side, it does synchronize, which is it's primary role, but I just wish it wasn't so annoying about it.

Badly implemented phonebook aside, practically everything else about the phone is good. It's small, neat and call quality is great. I've not tested the bluetooth functionality very much so I can't say either way on it. Over all, I'd consider it a good replacement phone, but unless something good happens with the phone book, I'm not planning on buying another Motorola phone in the forseeable future after this one.

Now if only I could make my own phone book. I wonder if it's even possible on these kinds of phone. Maybe I should check this out.

This one is beautiful. If you get an unhandled run-time exception in the public void main(String args[]) method then your application simply won't launch. You should make sure to intercept all exceptions in the main class and then throw up a simple error dialog.
Of course this doesn't work if you are missing a class in the distribution, the exception happens before you get to the main method.
Quick tip: enable and show the java console (JavaCPL on linux, java control panel under windows) it's under Java console in the Advanced tab.
[Listening to: Funny Break - Orbital - Orbital (4:56)]
There's an article on Computerworld that Microsoft is telling us that multicore chips are changing PC software design, but that not enough people are programming multi-threaded applications to take advantage of this feature.
Let me tell you, writing multi-threaded code is really easy. Writing correct multi-threaded code is the tricky part. Most development frameworks are not Multi-Thread safe. This means that you can't use it willy-nilly from multiple threads at the same time (it's primarily a resource assignment issue). So you have one thread that performs all the GUI work. Then you have to coordinate to have either the data or something close to rendered detail for the GUI thread. Then you have a barrage of threads performing various other bits of work. Of course, don't forget that making an application too multi-threaded has negative effects.
There is a subtle difference between multi-processor and multi-threaded processors which means that an mt-processor isn't the same functionally as separate cores/processors (shared resources, this being the whole -threading implication behind the name), so just throwing arbitrarily extra work at the mt-processor won't gain you much. The OS needs to know this information to schedule more intelligently, so adorning the threads with informatiion about the related data-affinity can gain you significant performance boosts (the OS schedules different threads more intelligently). The problem is that you need to export this concept to an application programmer. Guess what, it's generally too complicated for anything less than the most processor intensive tasks.
Generally, having the extra threads/cores/processors means that you get an overall system performance boost, it's just that the OS stops the isolation granularity at the process level. Operating systems have been designed around the 'complicated process, simple thread' principle. You don't want to change the balance of complexity moving back into the threads, we'll just end up with a sub-thread concept, and my head just hurts from that (atoms, quantum elements).
So what does the average joe programmer do? How do you find places that are suitable for parallelizing? How do you then 'fix' them up? Well, unless your program needs parallelism in the first place, it's actually difficult to retrofit it into a pre-existing design.
Well, there's a ton more stuff on this that I would like to get down, but it's 2am, and I need to get some sleep. More in the morrow.

Annoying Delphi 2005 OTA issue

| No Comments
Well, this one is just lovely. The first problem is that it does not support introspecting the Environment Option Names. This is annoying, but the next issue makes it even more fun. When we go looking for the options for, for example the package output directory variable (PackageDPLOutput) it looks at the .NET version before looking at the win32 version i.e. it's got overloaded option names.
Oh well, it will need to be worked around again, I suppose.
[Listening to: SSREP13 The Veldt - Andy Doan - Spaceship Radio (29:48)]

Page Style Chooser

| No Comments
I put in the 'set page style' options on the right hand side of the page. They seem to work just fine, mind you there is also the choice in the View->Page Style menu option in Firefox. I'm going to have to change some of the styles to make them more fluid.
I like fluid pages - I tend to resize and get annoyed when they only occupy a small portion of the screen.

How to steal focus on 2K/XP

| No Comments
You used to use SetForegroundWindow, but that only blinks your window now, this is because stealing focus is evil. It can be done and it's evil.
You use the AttachThreadInput API call. The process is attach, focus, detach.
AttachThreadInput(GetWindowThreadProcessId(
  GetForegroundWindow(), NULL),
  GetCurrentThreadId(), TRUE);

SetForegroundWindow(hWnd);

AttachThreadInput(GetWindowThreadProcessId(
  GetForegroundWindow(), NULL),
  GetCurrentThreadId(), FALSE);
Again, this is doable, the problem is that it's not good UI design as this will make your window focus even if your application wasn't in the foreground, which as we all know is a no-no.
[Listening to: Purple Haze - Groove Armada - The Best of Groove Armada (4:03)]

Default make rules

| No Comments
The default make rules for solaris are in a configuration file. This file is /usr/share/lib/make/make.rules
gnu make wanders around with the file built in. You use gmake -p to get it's default rules (and a whole load more besides).
You would think that people would remember this. C keeps things as an integer unless you tell it otherwise. This means that unless you tell the computer otherwise, it will keep blithely working on integers. so to get a proper answer cast one of the values to a float, and the entire operation gets done as a floating point operation.
Simple rules for simple languages.

First rule of profiling

| No Comments
The first rule of profiling is that by profiling you change the behaviour of a system. This can mean intermittent bugs, performance slowdowns, or general wierdness. Nothing earth shattering, but you just have to keep reminding yourself of it occasionally.
[Listening to: The Lion and the Cucumber (The Doctor and the Rockit Remix) - Dr Rockit - The Spirit of Vampyros Lesbos (5:48)]
Just a hint for those who might forget, because the new layout manager is not standard in Java, you need to make sure that your application ships with the new layout code in tow. To find the location of this layout manager on your system, open the library manager and look for the 'Swing Layout Extensions' library. This will give you a pointer to the .jar file needed to run under the new layout manager. In my case that's J:\Program Files\netbeans-5.0beta\ide6\modules\ext\swing-layout-0.7.jar, Your jar file name and directory mileage may vary.
java.lang.NoClassDefFoundError: org/jdesktop/layout/GroupLayout$Group. Then again if you don't know why the error is happening I'm slightly shocked.
[Listening to: Little Earthquakes - Tori Amos - Little Earthquakes (6:52)]
image of the layout manager
The new layout manager for the Netbeans 5 designer. This form designer kicks ass on so many levels. It's really quick and easy to get things into the correct places without having to worry about baglayouts within flowlayouts within gridbaglayouts. Truly a joy to make use of.
[Listening to: U Boat - Kasabian - Kasabian (10:50)]

Good old PromptDataSource

| No Comments
This one makes building ADO connection strings in client applications very easy. PromptDataSource is a member of the IDBPromptInitialize COM object. Creating it involves a small bit of C++ code:
  IDBInitialize *pIDBInitialize = NULL;
  IDataInitialize *pIDataInitialize = NULL;
  IDBPromptInitialize *pIDBPromptInitialize = NULL;
  LPOLESTR strConn;

  CoCreateInstance(CLSID_DataLinks, NULL, CLSCTX_INPROC_SERVER, IID_IDataInitialize,
    (void **)&pIDataInitialize);

  CoCreateInstance(CLSID_DataLinks, NULL, CLSCTX_INPROC_SERVER,
    IID_IDBPromptInitialize, (void **)&pIDBPromptInitialize);

  pIDBPromptInitialize->PromptDataSource(NULL, 0,
    DBPROMPTOPTIONS_PROPERTYSHEET, 0, NULL, NULL, IID_IDBInitialize,
    (IUnknown **)&pIDBInitialize);

  pIDBInitialize->Initialize();

  pIDataInitialize->GetInitializationString(pIDBInitialize, true, &strConn);
There are no checks anywhere here! You should add your own. C# makes things quite a bit easier.
  MSDASC.DataLinks oDL = new MSDASC.DataLinksClass();
  ADODB.Connection conn = new ADODB.ConnectionClass();
  conn.ConnectionString = "Provider=sqloledb";
  object oConn = (object)conn;
  oDL.PromptEdit(ref oConn);
  OleDbConnection xconn = new OleDbConnection(conn.ConnectionString);
  xconn.Open();
Of course, you need to add a reference to the com types: 'Microsoft ActiveX Data Objects 2.8 Library', and 'Microsoft OLE DB Service Component 1.0 Type Library', which provide access to the MSDASC and ADODB COM objects. You just extricate the Connection string at the end. I'm explicitly setting the provider in the second instance as I'm lazy, and I'm always accessing sqloledb. The 2.0 .NET framework may introduce an easier mechanism for doing this.

Debugging LoadPackage

| No Comments
Well this one is a complete pain in the ass. I've been trying to debug plugins in Delphi. It looks like the use of LoadPackage isn't allowing us to debug the plugin. This is really annoying; it makes work difficult.

Annoying installations

| No Comments
This one is a real pain in the ass.
For some reason the installation of the Visual Fox Pro 8 SP1 OLEDB provider left all the registry keys installed were not readable by ordinary users, so when I tried to execute my application as an ordinary user didn't work as the 'provider wasn't installed', whereas it's just permissions.
Registry Keys that needed permission changes:
HKCR\CLSID\{50BAEED9-ED25-11D2-B97B-000000000000}
HKCR\CLSID\{50BAEEDA-ED25-11D2-B97B-000000000000}
HKCR\CLSID\{50BAEEDB-ED25-11D2-B97B-000000000000}
HKCR\VFPOLEDB
HKCR\VFPOLEDB.1
HKCR\Vfpoledb.ConnectionPage
HKCR\Vfpoledb.ConnectionPage.1
HKCR\VFPOLEDB.ConnectionPage
HKCR\VFPOLEDB.ConnectionPage.1
and the SOB still won't work. Permissions on the files in OLDEB directory seem OK. Then I had to copy registry information into the user's environment:
Windows Registry Editor Version 5.00

[HKEY_CURRENT_USER\Software\Classes]

[HKEY_CURRENT_USER\Software\Classes\CLSID]

[HKEY_CURRENT_USER\Software\Classes\CLSID\{50BAEED9-ED25-11D2-B97B-000000000000}]
"OLEDB_SERVICES"=dword:ffffffff
@="VFPOLEDB"

[HKEY_CURRENT_USER\Software\Classes\CLSID\{50BAEED9-ED25-11D2-B97B-000000000000}\ExtendedErrors]
@="Extended Error Service"

[HKEY_CURRENT_USER\Software\Classes\CLSID\{50BAEED9-ED25-11D2-B97B-000000000000}\ExtendedErrors\{50BAEEDA-ED25-11D2-B97B-000000000000}]
@="VFPOLEDB Error Lookup"

[HKEY_CURRENT_USER\Software\Classes\CLSID\{50BAEED9-ED25-11D2-B97B-000000000000}\Implemented Categories]

[HKEY_CURRENT_USER\Software\Classes\CLSID\{50BAEED9-ED25-11D2-B97B-000000000000}\Implemented Categories\{D267E19A-0B97-11D2-BB1C-00C04FC9B532}]
@=""

[HKEY_CURRENT_USER\Software\Classes\CLSID\{50BAEED9-ED25-11D2-B97B-000000000000}\InprocServer32]
"ThreadingModel"="Both"
@="C:\\Program Files\\Common Files\\System\\ole db\\vfpoledb.dll"

[HKEY_CURRENT_USER\Software\Classes\CLSID\{50BAEED9-ED25-11D2-B97B-000000000000}\OLE DB Provider]
@="Microsoft OLE DB Provider for Visual FoxPro"

[HKEY_CURRENT_USER\Software\Classes\CLSID\{50BAEED9-ED25-11D2-B97B-000000000000}\ProgID]
@="VFPOLEDB.1"

[HKEY_CURRENT_USER\Software\Classes\CLSID\{50BAEED9-ED25-11D2-B97B-000000000000}\TypeLib]
@="{50BAEECA-ED25-11D2-B97B-000000000000}"

[HKEY_CURRENT_USER\Software\Classes\CLSID\{50BAEED9-ED25-11D2-B97B-000000000000}\VersionIndependentProgID]
@="VFPOLEDB"

[HKEY_CURRENT_USER\Software\Classes\CLSID\{50BAEEDA-ED25-11D2-B97B-000000000000}]
@="VFPOLEDB Error Lookup"

[HKEY_CURRENT_USER\Software\Classes\CLSID\{50BAEEDA-ED25-11D2-B97B-000000000000}\InprocServer32]
"ThreadingModel"="both"
@="C:\\Program Files\\Common Files\\System\\ole db\\vfpoledb.dll"

[HKEY_CURRENT_USER\Software\Classes\CLSID\{50BAEEDA-ED25-11D2-B97B-000000000000}\ProgID]
@="VFPOLEDB.ErrorLookup.1"

[HKEY_CURRENT_USER\Software\Classes\CLSID\{50BAEEDA-ED25-11D2-B97B-000000000000}\VersionIndependentProgID]
@="VFPOLEDB.ErrorLookup"

[HKEY_CURREN