The dangers of upgrading

One of the guys in the office was having trouble since the upgrade to python3.6 – a bunch of test code was breaking when run under the debugger.

The issue seemed to be related to the use of pexpect 4.6 in the new environment as opposed to pexpect 3.3 in the old environment.

This is made very difficult to debug as the ptyprocess code closes all the file descriptors of the child process before attempting to exec. As a result random exceptions were being swallowed, the entire thing was crashing out, and the code was locking up in an error read loop.

I hemmed and hawed about downgrading to the 3.3 version of pexpect, but decided to investigate further, rather than leave the problem as is.

Addressing the debugging problem involved replacing all the code that closed all the file descriptors with code that marked all the file descriptors as close on exec, so that when I saw the exception, I was able to deal with it. This was done in the ptyprocess module. The solution is linux only, but TBH at this time it’s all I’m concerned with.

Addressing the pexpect problem involved just removing the code that re-encoded the arguments when the encoding argument was passed, and just leave them as-is.

The confusion is because encoding was for the I/O, not for the arguments on the command line, and when the change was made it relied on this argument, rather than adding an extra argument to deal with it.

Fixes the problem in my case, but it was a complete pain to debug.

Hope in every box

Imagine if life stretched out in a single span from birth to death, and all you have is a long stretched out span between the start and then end, with no pause in between.

It would be absolute hell on earth. How could you bear to survive in a world like that? A never ending stretch until the precious final release of death.

However it’s not that case — life is, instead, broken down into little boxes.

Each box is a day. Each box is separate and distinct. Sometimes when you’re in a box it seems like that’s all there is and there’s no way out. Sometimes you look at another box and think that it’s an impossible goal because it looks so difficult to get to as you don’t have the skills or experience to get there.

The trick is though, every day you have choices as to where you want to go — there are exits to the box doing in different directions. Some of the directions are positive, and some of the directions are negative. Sometimes, you may have slipped so far into the negative that you cannot conceive of getting to the positive.

However, the thing about the individual boxes is that you only need to deal with the situation one box at a time.

Whether the box is getting practice in your writing,

or the box is managing to to another day without a drink,

or the box is learning more about yourself…

It’s only a small box that you need to get through; you’re not trying to deal with the entirety of your life, you’re just trying to deal with this one small thing called now.

An important thing to remember, though, is that you don’t always have to ‘make progress’. Sometimes the only thing you can do is just get through that single box. The best thing about all these individual boxes is that you get the opportunity to try again the next time. Because no matter how bad it seems to be at this moment in time, you will have another opportunity the next time as long as you get the fact that there will be another time. It’s just a single box away.

We need to aim for progress, not perfection; one day at a time.

Originating Video

(Floating point) math is hard

On websites like StackOverflow, if you post something about floating point math, it will get closed very quickly as a duplicate of the ‘is floating point broken’ question. Very often it is a duplicate of that question, however there are occasions when this is not the case.

The general question is typically how come 1.1 + 2.2 != 3.3?. There are a lot of resources about this. The long and the short of it is that binary representations of floating point numbers are not the actual numbers, but close approximations to them, so as a result equality sometimes isn’t, and you end up having to do ‘fudge’ math (a == b becomes fabs(a – b) < fudge). It’s great fun (and I mean that in the most sarcastic manner possible).

Sometimes, it’s about the display of floating point numbers, and it’s really frustrating when the question gets closed for the wrong reason. This can be especially frustrating for the OP, who may be new to the site and gets a poor impression from this.

The issue, in this case was about missing precision on the display of java floating point values.

In java the code:

float f = 2.0f / 3.0f;
System.out.println(f);

Will only display 7 decimal places. This is because it’s a 32bit float, and can only represent up to that level of precision.

The code:

double d = 2.0 / 3.0;
System.out.println(d);

Will display 15 decimal places. This is because it’s a 64bit float, which can represent up to that level of precision.

So, if you want 15 decimal places in the default number math, you need to use the double.

This is all moot if you’re dealing with money. I really do hope you’re not using float to represent money. That’s how you get an audit. In the words of a great philosopher, that’s a German car, the ‘T’ is silent 🙂