Freesteel Blog » 2009 » January

Tuesday, January 27th, 2009 at 8:48 pm - - Machining

Suppose I have a function f(t) which returns a pair of 3d points given the scalar t. In mathematics speak it is a mapping from R to R6 where R is the set of real numbers. But from our point of view the mapping is to a pair of points

(fp(t), fn(t))

because fp(t) is the point that is a distance t along a toolpath, and fn(t) can either be the contact normal of the tool, or the tool axis direction for a 5-axis toolpath, depending on what you are doing.

The task is very simple. I need to sample this function (or toolpath) at a sufficient rate to satisfy a tolerance given by a set of parameters.

In this example, the return value will be a sequence of t values, (t0, t1, ..., tn) which are bounded by the conditions:

1. ti+1 - ti < maxSampleRate
2. Angle(fn(ti+1), fn(ti)) < maxAngleChange
3. If ti+1 - ti < minSampleRate, then no other conditions apply.

for all i < n.

If the values fn(t) are unit vectors (because they represent tool axes), then Condition 2 can be tested by taking the dot product between the two vectors and testing if the result is greater than cos(maxAngleChange), so you shouldn’t need to perform a trigometric calculation at every step to actually determin the angle between the two vectors. Often when talking about computer algorithms the implementation is different to the way it is described.

Condition 3 provides the lower bound for subdivision. Usually it’s set to 2 microns, the motion tolerance of the machine tool.

If maxSampleRate > minSampleRate, then Condition 1 can’t conflict with Condition 3, but if there is a discontinuity in fn(t) (for example, the contact angle suddenly changes due to the toolpath crossing an inside corner), then it becomes important.

Oh, and the last input parameter is T, where t ranges from 0 to T.

The simplest answer that solves all the conditions (by satisfying Condition 3) is to set

n = int(T / minSampleRate)+2, and ti = i T / (n-1)

but if the functions f are expensive (slow) then it will also be the slowest.

My usual algorithm is to do it two parts. First, define the function:

bool ShouldSplit(ta, tb)
    if (tb - ta < minSampleRate)  
        return false; 
    if (tb - ta > maxSampleRate)  
        return true; 
    double d = Dot(fn(ta), tn(tb));
    if (d < cos(maxAngleChange))  
        return true; 
    return false; 

As usual, the implementation is quite different from the description, where we would complicate matters by carrying around structures based on the tuple

(t, fp(t), fn(t))

so that there would never be unnecessarily recalculated, but it makes the description much harder.

In real code there are many conditions tested in this ShouldSplit() function. Usually t does not correspond to the distance along the curve, and Condition 1 becomes

|fp(ti+1) - fp(ti)| < maxSampleRate

and so on.

The second part of my usual algorithm works by subdividing the pair (0, T) until ShouldSplit() returns true for every adjacent pair, like so:

result = array(0, T);
i = 0;
while (i < count(result) - 1)
    if (ShouldSplit(result[i], result[i + 1]))
        t = (result[i] + result[i + 1]) / 2;
        result.insertBefore(i + 1, t)
        i = i + 1;

This was my state of the art for a good few years when I started programming this sort of thing. The ShouldSplit() can get as complicated as it likes, and the outer function would maintain the results in a linked list and can be parallizable when it is written in a different form, like:

result = list(0.0, T); // indexed by iterators
stack = Set();
stack.add(result.iteratorFront);  // points to the 0.0

        if (stack.empty())
        i = stack.pop();
    if (ShouldSplit(result[i], result[]))
        t = (result[i] + result[]) / 2;
        result.insertBefore(, t)

... and it goes on to get even more complicated when it is done properly. Obviously, there has to be a delay before sending in the second and third threads, or the first one won't have time to have created some space for them to get to work. The key is we've separated problem into two self-contained halves that can be developed independently.

Unfortunately, there is an inefficiency here. What happens if, for the sake of simplicity, we ignore Condition 2 and 3 and set T=64.0 and maxSampleRate=0.999. Then this algorithm is going to return a list of 129 evenly sampled points spaced by the distance 0.5 because we would have subdivided the single gap of width 64.0 into two gaps of width 32.0 and so on, until there are gaps of 1.0, which is only just wider than 0.999, causing the next subdivision to take it down to 0.5; whereas the most efficient sampling rate would be to set n = int(64.0/0.999)+2 = 66, resulting in a sample spacing of 0.984.

This is a factor of 2 inefficiency. Admittedly it's a worst case example, but on average it's going to produce about 50% too many points, depending on what you take for an average pair of numbers T and maxSampleRate.

The obvious way to fix it is to begin the algorithm with a loop which samples all the points at the even spacing driven by maxSampleRate, and then start the subdivisions between the pairs of points until Conditions 2 and 3 are satisfied. But this is inelegant and obstructs all that interesting, hard and rewarding parallel processing that could be developed into the outer function, as described above.

There, however, an easy tweak. And it goes like this:
Replace the line which bisects the two sample points:

t = (result[i] + result[i + 1]) / 2;


d = result[i + 1] - result[i];
m = (int)(d / maxSampleRate);
if ((m > 1) and (m is even))
    lam = m / (2*m + 2.0);
    t = result[i] + lam * (result[i + 1] - result[i]);
    t = (result[i + 1] + result[i]) / 2;

In other words, instead of bisecting in the middle of the interval all the time, we divide it to one side of the halfway point in certain cases, and these end up aligning with the optimal solution. In the case above, the first point between 0 and 64.0 will be 64/130.0*64=31.50769 which is an exact multiple of 64/65.0=0.98461.

For even more performance and self-containability, allow the ShouldSplit() function return the point of bisection when it returns true. It could also squeeze some further optimization by carefully examining the directions of the contact normals at ti and ti+1, and predict an approximate point of change where there is likely to be the discontinuity in fn.

However, this is complicated because the result demands a sample on either side of the discontinuity, separated by a distance less than minSampleRate, and if you consistently hit the wrong side of the discontinuity you'll produce a sequence of convergent samples running towards the value maybe in an infinite loop. (I think I've had that bug once.) So it's often better to alternate subdivisions at this predicted discontinuty, and at the halfway point, or pick an average of the predicted value and the half-point, or all kinds of variations of this that don't seem to make a lot of improvement, partly because discontinuities are rare (though important) compared to the overwhelming majority of other sample cases.

When I started this work, the guys working on the old system ran a sampling function which calculated the point fn(t) between fn(ti) and fn(ti+1), tested whether it was colinear to within a certain tolerance, and discarded it if it was. This meant that twice as many points were calculated during the toolpath creation than showed up in the result. After exploring the idea of looking beyond just pairs of points in the sequence to test colinearity without throwing away valuable calculation results, I settled on the above way of doing it, when I observed that the geometric distance sample rate defines the machining tolerance. This gave my code an immediate two-fold advantage, though no one in the office seemed particularly interested in this discovery at the time or any time later.

It takes dozens and dozens of variations and tricks of this kind to make one CAM system perform better than another. You've got to be pretty dedicated to carry on doing it for year after year, even though no one gives a damn. Out in the academic world, each one of these advances could result in a published paper and another credit to your name. But in the commercial programming business in the small niche that CAM is, these things just get lost, forgotten, unnamed, unrecognized, not passed on and not shared. And in the end the people who are able to do them get discouraged, lose interest and stop caring, even when they keep coming to work every morning just to pay off the mortgage. What a waste of a life. Without any conferences of programmers at this level technicality -- totally at odds with the misguided "commercially confidential" culture within almost all of the CAM businesses these days -- the software becomes an exercise in stagnation.

Friday, January 23rd, 2009 at 7:49 pm - - Whipping

The U-turn on MPs’ expenses is all over the blogosphere. The political mugging to keep them secret was being organized at the highest levels between the leaders of the two main parties. The event shows that in this democratic system even they can be over-ruled by less senior members of Parliament, which goes to show that they all do have direct and fundamental responsibility and influence — even when they pretend they don’t.

In this political system the top dogs, such as the Prime Minister, have got so many different things to cover and respond to and weild power over, that their cognition is always going to be shallow and prejudiced on any single issue. Widespread bad governance is therefore inevitable.

That and having the same dominant financial policy-maker (Gordon Brown) running the country for over a decade who has to pretend and act as though nothing of the current crisis has any relation to his years of ignorant and misguided deregulation policies. Such a core lie has the unfortunate effect of ruling out many of the affordable solutions, leaving only the of stupid ones that are consistent with the axiom that he has always been right.

Isn’t it funny how, when finance was booming, London had to be at the heart of it with its lax controls, bloated business activity, and the skewing of all parts of the national economy towards serving its needs? And now when all the miscalculations don’t add up and the game stopped working, the City of London is just a tiny insignificant snowflake buffeted around in a world-wide hurricaine.

Let’s take a look at Wednesday’s Prime Minister’s questions and put the key statements into context:

Carswell (Harwich): What are the Prime Minister’s official engagements for Wednesday 21 January?

Brown: In Afghanistan the people there have managed to kill another three men out of the thousands of foreigners in that country who have been waving their guns around and shooting at people for no reason for the last seven years.

Also, can I congratulate the new US President. We’ll try to suck up to him like we did to the last president when we provided him with the essential political cover and know-how to get his country deep into two ruinous wars. I’m sure he will thank us for our services.

Carswell: Why is the Prime Minister whipping his party to keep MPs’ expenses secret?

Brown: “I should tell the hon. Gentleman the real facts: our proposals are for more transparency than the Conservative party’s proposals were and for more transparency than is the case in most Parliaments in the world. That is why we will publish a revised Green Book with clear rules, and there will be enhanced audit by the National Audit Office. We will put the proposals to the House on a free vote. We thought we had agreement on the implications of the Freedom of Information Act as part of this wider package. Recently, the support that we believed we had from the main Opposition party was withdrawn. I believe that all-party support is important on this particular matter, on which we will continue to consult.”

Waltho: Will you commemorate Holocaust memorial day on 27 January?

Brown: I will commemorate Holocaust memorial day by holding a Parliamentary debate.

Cameron: I too agree we should only count the number of armed British government employees who get killed in Afghanistan, and never waste our breath mentioning the number of Afghanis they’ve massacred over the last seven years.

But what about the recession?

Brown: Every job lost and every redundancy is a matter of regret and sadness for us all… The one thing that President Obama did not say in his speech yesterday was, “Fellow Americans, let’s do nothing.”

Cameron: Obama will not be putting up national insurance contributions on people earning £19,000 or £20,000 because the country is so bust.

Brown: I’m not taking lessons from you. I’m taking them from the banks.

Cameron: The recession is getting worse.

Brown: The Mayor of London agrees with me.

Cameron: When it comes to these great infrastructure projects, who just put back the carrier programme? Who cancelled the widening of the motorways? It was this Government, because they have run out of money.

Brown: Every country understands that when the private sector and the markets fail, and particularly when banks fail, the Government have a duty to pay them compensation.

Cameron: The Prime Minister is the only person in the country who thinks he is doing a good job.

Brown: I am sorry that the Conservatives do not want to understand economics.

Cameron: If you want to ask us questions, have an election!

Brown: The right hon. Gentleman has not one idea about how to begin to sort the problem out.

MacKinlay: Were the Serious Organised Crime Agency, the Financial Services Authority or our security and intelligence services fast asleep, or were they part of a cover-up in relation to Lloyds TSB’s illegal handling of money from Iran to get round sanctions?

Brown: Our sanctions policy against Iran has been one of the toughest in the world.

Clegg: I also single out for special mention the predictable deaths of the trained British killers in Afghanistan, because that is what is expected of all serious party leaders.

The British economy is now standing at the edge of a cliff.

Brown: I shall just repeat what Richard Lambert, the head of the CBI, said.

Clegg: Full temporary nationalisation of our weakest banks without any further delay.

Brown: That is the problem that we are trying to deal with. The problem is the resumption of lending; that is still the problem that has to be dealt with, whatever the status of the banks.

Abbott: Will the Prime Minister agree with me that people all around the world will have been moved by the inauguration of President Obama?

Brown: The White House was built by slaves, and it is now occupied by the first black American President.

Garnier: In the very week when unemployment is heading to 2 million, and when our constituents and our constituency businesses are deeply concerned about their finances, the Prime Minister demands that his own Members of Parliament keep their expenses secret.

Brown: “The hon. and learned Gentleman misunderstands what we are doing and the transparency that we are providing in the expenses. I said earlier that the main Opposition party and the Government had discussions about the statutory instrument. The Opposition party gave the impression that it was supporting that statutory instrument but it has now withdrawn its support for it. It is right, then, to seek all-party agreement on that, and that is exactly what we are doing. Far from spending a great deal of time this week on the issues that he is talking about, I am spending my time dealing with the problems of the British economy, and that is what I will continue to do.”

Skinner: “Those Tory fat cats at the banks, represented by the Conservative party, cannot blame the trade unions this time for the almighty mess that this country is in. In another generation, they would have been described as the enemy within.”

Brown: It is a global banking crisis that must be dealt with by global co-operation.

Spink: Will the Prime Minister meet me and Age Concern so that we can look at ways of further helping pensioners on issues such as meeting their fuel costs and on savings interest?

Brown: The Sun said only a few days ago that there was only one MP who was more independent of the Conservative party Front-Bench positions than him, and that was the right hon. and learned Member for Rushcliffe, now the shadow Business Secretary.

Ladyman: What does the Prime Minister going to do, besides the sanctions that he has already imposed, to neutralise that influence of Iran whose toxic influence is the cause of the massacre in Gaza?

Brown: The international community has to show great unity in isolating Iran, not only for the position that it takes on nuclear weapons but for its attitude to Israel.

Beith: Get EU funding for my constituency.

Brown: We have successfully lobbied for the European Investment Bank to double the support for cars that have green technology and manufacturing processes.

Campbell: Nationalize the banks.

Brown: President Obama and many countries around the world will be putting forward programmes for public investment in the future: public works programmes, investment in environmental technologies, investment in roads, transport, schools and hospitals.

Lait: The value of Railtrack was talked down by the previous Deputy Prime Minister so that the company could in effect be renationalised. Is that what the Prime Minister and the Chancellor are doing to our banks?

Brown: “No.”

Gilroy: I also believe that three British men with guns in Afghanistan have a far greater right to life than any of the people who actually live there on our battlefield.

Many in the banking sector who have covered things up and given bad advice have have got off scot-free.

Brown: “That is exactly the matter that is being discussed now internationally to agree common standards of disclosure, common standards of transparency, common standards of risk management by the banks and common standards of responsibility to be taken by board members. We will do better by getting an international agreement on these standards and I believe that that will feature in our discussions at the G20 summit on 2 April.”

Do all PMQs begin by mentioning dead British soldiers whenever possible in a way that makes it tactless to ask: “What the f*ck are they doing over there in the first place?”

From 17 December 2008:

Harriet Harman: I have been asked to reply. As the House will be aware, my right hon. Friend the Prime Minister is in Iraq today. He will make a statement to this House on his return.

I am sure that the whole House will wish to join me in sending our profound condolences to the family and friends of Lieutenant Aaron Lewis, of 29 Commando Regiment Royal Artillery, who was killed in Afghanistan on Monday. To those who never shy away from danger and who never shirk from their duty, to the families who will be apart from our troops this Christmas and to those who have died in the service of their country, we owe an enormous debt of gratitude.

From 10 December 2008:

Gordon Brown: Before listing my engagements, I am sure that the whole House will wish to join me in sending our profound condolences to the family and friends of Lance Corporal David Wilson of the 9th Regiment Army Air Corps, who died in Iraq on Thursday. We owe him and all those who have lost their lives serving our country a deep debt of gratitude.

From 26 November 2008:

Gordon Brown: Before I list my engagements, I am sure that the whole House will join me in sending our profound condolences to the family and friends of Marine Alexander Lucas, who was tragically killed in Afghanistan on Monday. We owe him and all those who have lost their lives in conflict our grateful thanks for this service and for their sacrifice.

From 19 November 2008:

Gordon Brown: I would advise Conservative Members not to talk down the pound, and I advise them to take the advice of Lady Thatcher, who said that

“trying to help the speculators”

and talking sterling down was

“the most unBritish way.”—[ Hansard, 15 June 1989; Vol. 154, c. 1119.]

Quite what the hell Gordon Brown is doing memorizing Thatcher’s senile rantings from 20 years ago during an earlier fiscal disaster, let’s take a look:

Mr. Kinnock: Why cannot the Prime Minister simply answer yes to the question, “Is the Chancellor going to retain his job after the forthcoming reshuffle?”

The Prime Minister: I answer on precisely what the Chancellor said on Wednesday of last week in the economic debate. The Chancellor set out the Government’s position clearly and in some detail. he said : “Our overriding”

–I repeat “overriding”–

“objective is to bring inflation back down.”

We shall not be diverted from that course.

Mr. Kinnock: Is that “gladly, joyfully, generously, fully, fully, fully” a refusal to guarantee the future of the Chancellor of the Exchequer?

The Prime Minister: I repeat what I said last week. If the right hon. Gentleman would like a little longer lecture I will read out the entire speech.

Sir Peter Tapsell: While the Leader of the Opposition regards all this as a joke, is it not about time that we all began to take the sterling situation rather seriously and that the Leader of the Opposition ceased to try twice a week to talk sterling down?

The Prime Minister: I entirely agree with my hon. Friend. The Leader of the Opposition is normally trying to help the speculators and talks sterling down in the most unBritish way.


Meanwhile, the Afghanistan war condolences carry on pretty routinely for most of the year, according to my random sampling. If I had the time I’d write a cross-referencing parser and compile a list of all such “Glorious Dead” who received condolences from self-serving politicians who have been allowed to dissociate themselves from direct responsibility. It is, of course, illegal to read out these same names beside the official national war memorial.

Speaking of this Afghan war, the central justification was: “they attacked us (more specifically, the United States) on 9/11″, and so we had to go over there, cluster-bomb and invade the hell out of them, arrest loads of guilty men “off the battlefield” for not wearing the correct clothes when shooting the invaders, and torture them in Guantanamo Bay until they told us about their future planned terrorist attacks. That’s the story, right?

But there is a counter-story, which is that the 9/11 operation was funded and staffed by Saudi Arabians, and that that county couldn’t be prosecuted because their government are friends with the Bush dynasty and they have all our oil. And Donald Rumsfeld wanted to go straight to Iraq, but couldn’t give any evidence that didn’t pass the laugh test, so he order the army into Afghanistan instead. Owing to the fact that there was no one to find there, we swept up a lot of random guys “from the battlefield”, framed them as guilty agents, and held them in Guantanamo Bay within reach of incriminating news footage but out of reach of any legal investigation that could even find out who they were. They were severely tortured (one third of the camp is on hunger strike and being force-fed) in the hope of getting some false confessions out of them (that’s what torture is for), because there’s no chance there’s going to be any other evidence against them to justify the fact that they need to be guilty.

Guantanamo Bay with its Afghanistan-sourced prisoners is the official repository of evidence for why young British men were sent half-way around the world to shoot at people in Afghanistan.

Last night I went to a public meeting given by two former prisoners and a former very young camp guard of Guantanamo Bay re-united following this traumatic boondoggle staged wholly and entirely for the purpose of starting a war in Afghanistan, which was the warm-up act to the invasion of Iraq.

The mainstream press, particular the US press, are wholly culpable for this crime. If they had had the capability of smelling a rat this big and stinky, having had any understanding of law and government and lies and the purpose of “due process”, this whole sorry show could have been closed down on its opening night.

Yesterday (22 January) the new US President signed the executive orders to close the Guantanamo Bay prison camp.

The day before, the same US President signed a Presidential Executive Order to revoke Bush’s Executive Order of 1 November 2001 which was designed to conceal Ronald Reagan’s presidential records beyond the twelve year limit (which expired in 2001), whose contents would have been inconvenient because many members of the Bush administration were from the Reagan administration, and these records would have exposed their history as absolute liars and scoundrels and the last kind of people who should ever be allowed to hold office.

Barack Obama’s supposedly with-it White House internet team have, with their new site, managed to break all the links to the old President Bush White House website, which is pretty ignorant of them because now that Wikipedia article about Bush’s executive order can’t take me to his signing statement where I could have watched him smirk and make his pathetic excuses.

Also on the same day, the new US President issued a Presidential memorandum on the Freedom of Information Act instructing:

All agencies should adopt a presumption in favor of disclosure, in order to renew their commitment to the principles embodied in FOIA, and to usher in a new era of open Government. The presumption of disclosure should be applied to all decisions involving FOIA.

We could do with a bit more of that attitude over here, with our sorry-ass, tired and wasteful political elite who seem only concerned about keeping the lid on all their serious lapses of judgment and comfortable greed to allow for anything that’s right.

The lid is officially off this mess in America. Wave after wave of scandal is going to come out. It’s a shame there’s so few investigative journalists left around to coordinate the facts. Our only hope will be structured computer databases and collaborative research tools like Wikipedia as the only means of assigning the piles of malfeasant acts onto the political actors themselves. The legal system seems unable to do it fast and efficiently enough to stop them.

Tuesday, January 20th, 2009 at 4:03 pm - - Whipping 2 Comments »

Following on from my earlier rant, the momentum is increasing in time for a vote this Thursday.

You’d think that MPs were experienced enough political operatives to know a fragmenting land-mine issue when they see it. No one cares about their so-called “compromise” position. I noticed that the earlier changes to the FOI Act to exempt expenses for MPs’ personal security equipment (eg CCTV cameras, stockpiled atomic weapons for deterring intruders) bought for their spare houses (claimed on expense) whose addresses are could not be disclosed, was probably the hard work of the nuclear bomb lover Julian Lewis MP.

Although politician’s home addresses are declared on the ballot during an election and so must be in the public domain, ready to be gathered together in a fashion that winds them up, they’ve decided that this information is going to have to be redacted from any FOI disclosure of housing expenses, because, as the paranoid Lewis explained:

[T]he fact that we have to [disclose] some of our addresses, some of the time, is no reason whatever for our having to do it with all our addresses, all the time, in an easily accessible form. Of course, if someone who is targeting a particular MP and means to track him or her down in order to do him or her harm puts in enough effort, it will be possible to do so. However, that does not cater for a situation in which someone with a grudge, someone with an obsession, a follower of a political cause or a self-taught follower of al-Qaeda, at home or abroad — who will not even have heard of most of the Members of this House — goes on the internet, conveniently finds 646 addresses and sends 646 packages containing something explosive, horrible or, at the very least, abusive to 646 unprotected mail boxes. The proposal is absolutely insane.

You have been told.

Our politicians make a virtue out of deciding matters on gut instinct, rather than considering them scientifically, rationally and transparently, so it’s no surprise when their attitudes in personal life extend to their style and prejudices in public administration. What other input are they going to use?

Trivial though the question of MPs’ expenses are in the grand scheme of things, the political fight over their disclosure is crucial for two reasons.

The first is that this is one of the very few instances of there ever being an on-the-record decision in favour or against the conditions that allow for political and financial corruption at any level. Normally these matters are safely kept out of the public decision-making process, be they top executive’s pay-packages that are answerable only to non-executive boards staffed by other top executives, or doomed-to-fail government contracts overseen by private advisors from the same industry, or non-returnable bonuses to bankers whose banks get bailed out only a few months later. This is one of the only times the public gets a voice. And the message has got to be made absolutely clear that we really don’t approve of this shit.

The second reason is that if MPs’ are going round deciding things on their personal prejudices and experiences, and not really paying attention to the evidence, then this provides a personal lesson about the power of public disclosure of financial details for clearing up questionable practices. Maybe they’ll see the value of extending the principle to other areas of the economy where things aren’t working too well, such as the railways, and in fact all government contracts. Perhaps on the day that Barack Obama becomes the President, they’ll have time to take a look at a law in the United States which he passed when he was in the Senate called the Federal Funding Accountability and Transparency Act.

The MPs’ expenses issue centres on the second house in London whose mortgages and furnishings are paid for on expense, but which then wind up belonging to the MP rather than to the State. This is an irregular practice, because in business things bought on expense belong to the Company unless they are consumed eaten; if you want to keep them you have to buy them back.

It’s this buying back of used Company property (the stuff that was bought with the Company’s money that still exists) that is often hazardous. For example, Conrad Black got busted when he bought a flat in Manhattan on expenses in 1994 as proprietor of his newspaper company, and then wanted to keep it in when it time to sell it in 2000. He decided he only needed to pay the 1994 price rather than the greatly inflated 2000 market price. (Obviously, if the price had gone down, he’d have chosen to buy it at the market price.)

But MPs don’t even need to go through that. The furniture and houses they buy on expenses are theirs to keep — under the rules they made.

We need these slack standards, which are even worse than in business, tightened up for the people at the top, so they are able to enforce better results on everyone lower down. Corruption is not just about the money that was opportunistically taken; the real damage is the abuse of policy done to create the opportunities for corruption. For example, we could have had an efficiently computerized NHS years ago if the business community wasn’t so corrupt that it recommended, supported and covered up for iSoft who, for four lost years, blocked anyone competant from doing the work as they took the money and produced nothing at all. If all Government IT software was provided live and open source, this could not have happened because anyone like me could have logged onto their servers on any day of the week and seen exactly how much work they had or hadn’t done, not just the amount it was costing.

No, the place for corruption is the House of Lords, home of Lord Conrad Black (mentioned earlier), Lord Smallpox, beneficial funder of the New Labour Party and Her Majesty’s official arms dealer, Lord Mandelson of the undeclared interest-free loan because Parliamentary expenses are not enough for the lifestyle, Sir Digby Jones who helped business to make money out of iSoft’s failure and
said that being a junior minister was “one of the most dehumanising and depersonalising experiences” anyone could ever have, and his replacement Mervyn Davies, an established banker from the banking industry appointed as a Lord and a Minister on the same day last week to look after the government rescue of the banking industry and who will no doubt be sympathetic to the immense hardships our wealthy bankers are having to suffer in this crisis of their own making as he drafts government policy on the Prime Minister’s behalf.

We don’t get any say regarding this business-as-usual at the rotting economic heart of the British political economy, built as it is from a vast interlocking conflict of interest. This MPs’ expenses business is but one tiny chink in the ediface of unaccountable maladministration. Our one opportunity to say No to something, anything.

Which way is it going to go on Thursday? Will we be able to do anything about it when they do what they have threatened?

Tuesday, January 20th, 2009 at 3:26 pm - - Machining

I must remain interested in work. I must remain interested in work. Must not get distracted by the complete ban on the sale of all tobacco products (I thought this was just a western world thing, like animal rights and car safety). As well as the Report of the International Court of Justice, which is not to be confused with the Report of the International Criminal Court. So interesting.

My job this week is to make this five-axis toopath result look good.

Saturday, January 17th, 2009 at 12:27 am - - Whipping 1 Comment »

The most important Parliamentary news yesterday was Geoff Hoon, currently the Transport Secretary, spewing awesomely huge lies to justify decisions that will take away vast numbers of human lives, allbeit over a longer term than the million or so lives that have been lost over the Iraq war, for which he lied morning, noon and night for about a year.

There’s always a compromise:

[W]e intend that new slots at Heathrow will have to be green slots. Only the cleanest planes will be allowed to use the new slots that will be made available.

Yes, and baby seals will be clubbed to death for only three days a month, so you’ve got 90% of what you wanted, haven’t you?

The entire policy continues to rest on the Air Transport White Paper which was as wrong five years ago when it was written as it is now, because:

The long-term real GDP growth assumption for the UK used in calculating forecasts was 2.25 per cent per annum… Aviation fuel prices were assumed to stabilise at $25 per barrel in real terms in year 2000 prices.

But in politics the connection between the policy and its justification is broken once the “right” answer has been established, and anyway back then we needed more air travel because the economy was booming. Now we need more air travel because we’re in a recession. In either case the air travel lobby buys a better dinner for the Minister.

But that’s not the real news out of Parliament yesterday. The news they failed to bury, but which all the newspapers picked up on (going to show that they don’t really care about the wrath of the press when it comes to something really important, like their own expenses) was announced:

[N]ext Thursday there will be a statutory instrument on freedom of information, followed by motions. The draft statutory instrument is in the Table Office and the motions are on the Order Paper.

Apparently the poor hard-pressed officials are struggling with the requirements of having to scan and redact a million pages of receipts in order to comply with a final high court ruling from May 2008. Far easier to say it’s too difficult and change the law than deal with the issue of what the heck these expenses claims are doing on scrappy little bits of paper in the first place.

One imagines that the refunds are made to their bank accounts electronically and in a timely manner (since they are the boss), so how hard would it be to enter the claims into a simple web-form so everything is above board and not lost in a blizzard of scrappy paperwork? They do know we don’t trust them, don’t they?

Let’s recap. In 2000 the UK finally got its Freedom of Information Act, decades after equivalent laws had been passed in all other civilized countries. It didn’t go into force until 2005, at which point Heather Brooke began fighting for public accountability for MP’s expense claims, which citizens have by right from officials in America.

The MPs, especially the Speaker, have been fighting the case constantly, howling at how unfair it is, while demonstrating at every step of the way that they are incapable of preventing corruption when these matters are kept confidential, and are only pretending at being honourable gentlemen.

In 2007 there was one underhanded attempt to change the law when David Maclean MP introduced a Private Members Bill to exempt the House of Commons and MPs from everything to do with the Freedom of Information Act and repeatedly lied that the Bill was protect private correspondence relating to a constituent being released against their will.

How do we know this was a lie? Firstly, all the legal experts (without a conflict of interest relating to the involuntary disclosure of their personal expenses) said the Freedom of Information Act wouldn’t allow this. And secondly, when this precise concern was addressed so that constituents’ communications would be explicitly exempt, but MPs’ expenses were not, they voted against it! And then the Government got caught lying about how they had no position on the Bill when a suspicious number of ministers turned up to vote on a Friday afternoon in order over-ride a filibuster.

Gordon Brown, who was about to ascend to the office of the Prime Minister, pretended to be above the fray, until he briefly saw the light while on a holiday in Wales ten days later.

But now he’s older and wiser he’s going to just get it done through an Executive Order, known in this country as a Statutory Instrument. The Freedom of Information (Parliament) Order 2009 says (approximately):

The Secretary of State makes the following Order in exercise of the powers conferred by Section 7(3)(a) and (b) of the Freedom of Information Act.

Substitute paragraph 2(d) for:

    “(d) information relating to any expenditure in respect of which–
    (i) a claim has been made to either House of Parliament by a member of either House, or
    (ii) a payment has been made by either House of Parliament to or on behalf of a member of either House.”

Hmm. There is no part(d) in paragraph 2 of Schedule 1 of the Freedom of Information Act. All it says is:

2. The House of Commons.

However, with some nimble web-searching I discovered an earlier Statutory Instrument called The Freedom of Information (Parliament and National Assembly for Wales) Order 2008:

In paragraph 2, after “The House of Commons” insert–

    “, in respect of information other than–
    (a) information relating to any residential address of a member of either House of Parliament,
    (b) information relating to travel arrangements of a member of either House of Parliament, where the arrangements relate to travel that has not yet been undertaken or is regular in nature,
    (c) information relating to the identity of any person who delivers or has delivered goods, or provides or has provided services, to a member of either House of Parliament at any residence of the member,
    (d) information relating to expenditure by a member of either House of Parliament on security arrangements.

    Paragraph (b) does not except information relating to the total amount of expenditure incurred on regular travel during any month.”

You track enough of these down and you get the feeling that the compiling of the law is about as obscure, unnaccountable and deliberately disorganized as MPs’ expense claims.

That’s why we need to support the free our bills campaign, because the problem is even worse than anyone can imagine. It should be a “just give us a goddamn copy of the law!” campaign.

It’s unacceptable. As you can see, the statute law database for the Freedom of Information Act doesn’t have that change updated, so how is anyone going to know?

I’ve an idea: Except in exceptional circumstances, no law can apply until it is committed to the Statute Law Database. (The Freedom of Information Act was delayed by 5 years from going into force, so it shouldn’t be a problem.)

And similarly, until an MP’s expense claim has been declared on-line and in the public domain, it doesn’t get reimbursed.

I’ll bet they would quickly discover how easy it is to organized these things efficiently if that was the case.

Thursday, January 15th, 2009 at 7:24 pm - - Machining 3 Comments »

I am trying to knuckle down for some machining work for a change, faced with yet more scallop algorithm problems obstructing getting on with anything more productive. It appears that my independent income (if I eventually get any) is likely to be the only source of funding for projects like publicwhip and since all the supposed public interest funding bodies appear not to give a damn.

Here’s a set of offset contours that are linking up incorrectly. You can see it in that sliver of a rectangle turned to an isometric view instantly. It’s impossible to subdivide the cell in the XY plane to reduce the complexity (the number of entries into the cell) below eight because they are all on a vertical wall.

This comes with offsetting a set of contours that bounded the horizontal areas outwards by several milimetres so that they were all on the wall, and then attempting to offset them back inwards (as done in the mother of all scallop bugs).

You have to offset them both ways and then discard the pieces that are on the side you don’t want, because the condition of which side the contour is is often very difficult to determine locally — something I finally concluded for sure at the end of 2007. If I examined the code for anyone else’s constant scallop algorithm, I would look for this characteristic. You can tell the sidedness of the offset in almost all cases, but the final cases where it fails are entirely unsolvable; the information is not available locally.

So that’s the explanation for why the contours need to be so complex — they are doubled up at this stage, before the ones on the wrong side are discarded.

Here are two ways to connect up eight incoming paths in a single cell with four contours.

The example on the left could be coded as [(0,7), (1,6), (2,3), (4,5)] and the one on the right could is [(0,7), (1,6), (2,5), (3,4)].

In the example above I am getting the pieces connecting like the right hand cell (which is why there is that annoying disconnection in the upper branch of the main picture), when I really want the left hand version.

How many ways can you connect these eight edges?

Well, four edges can be connected in two different ways.

[(0,1), (2, 3)]
[(0,2), (1, 3)]

Six edges have five ways:

[(0,1), (2, 3), (4,5)], [(0,1), (2,5),(3,4)]
[(0,5),(1,2),(3,4)], [(0,5),(1,4),(2,3)]

The grouping should give an idea of how I’m counting them. When you get to eight ways you can either start with (0,1) or (0,7), leaving a six-way, or you can start with (0,3) or (0,5) which divides the area into a four-way and a two-way, which gives a total of 14 ways.

I’m sure someone has worked out the general formula that counts the number of ways for each 2*n, so I’ll resist spending the rest of the day proving it.

But here’s the issue. When I created the algorithm I expected only to deal with four-way cells, because I thought I could always subdivide the cell enough to get rid of anything more complicated.

Then along came the six-way cells, which is only one step up. I changed the record of the cell connection to work with a three bit integer that could encode the five options.

For higher order cells (eight, ten and twelve — offsetting three contours on the same wall) I encoded only some of the options, the ones I called “striped”. The connecting diagram on the right is striped because you start with (0,7) enclosing nothing, and crawl outwards along both sides until you get to (3,4). Or you could start with (0,1), or (1,2), or (2,3) — four options.

The one on the left, which is correct for this case, is not striped, so I can’t encode it. What am I going to do about this?

If I decide it’s important, I’m probably going to have to create an exceptional connecting case. In the eight-way cell where the data structure can only encode for the 4 striped connections (out of the possible 14), I’ll add in a fifth one which says to the function: “Go look up this cell’s coordinates in another data structure where the connection is recorded in its full list of pairs, rather than encoded into a three bit number.”

I don’t expect there to be more than two such exceptional cells in the whole subdivided surface, so the speed implications are negligable. However, carrying around some special decoding thing in place of a simple integer code requires a heck of a lot of messing around with function calls that were previously simple and self-contained (operating only on the cell itself). I don’t have a way of referencing the cells of one of the structures while it is being subdivided — the indexing is applied to them later when it becomes read-only. So my look-up table is going to have to use the actual floating point XY coordinates of the corner of the cell as the reference index. This is normally not done, because floating point values are considered approximate. But if the number doesn’t change (it’s stored in absolute form in a list), it’s like a bit-patterned reference, which isn’t going to shift if cells further to the left of it get subdivided and its integer value cell-number increases. This is critical, because the cell subdividing of the surface is done with multiprocessors running in parallel. The only limitation is that cells subject to this simultaneous subdividing operation can’t share the same row or column, because it’s not safe for two concurrent threads to change the size of the same array at the same time.

Update: I couldn’t stop myself looking at the n-connection problem. I’ve reduced it to a recursion relation in the following way:

Define c(n) as the number of ways to connect all the corners of a fixed n-sided polygon with curves inside the shape that don’t cross each other.

Set c(0) = 1, c(1) = 0, c(2) = 1


    c(n) = SUM{ c(i – 1) c(n – 1 – i) | i = 1, 2, …, n – 1 }

Results are: c(4) = 2, c(6) = 5, c(8) = 14, c(10) = 42, and c(2i + 1) = 0

Some extra notes. If you’re allowed to rotate the polygon, then the numbers are smaller because the two cases in c(4) are the same, but rotated by a quarter, and the c(6) case would only give 3 unique shapes. It’s a different question, and I don’t know the recursion formula.

It’s easy to see that odd values are zero, since you can’t connect the lines up without leaving one spare. The zero value is one — one valid solution. These are always like the initial conditions of any recursion, and there’s no justification for deciding how they should be, other than to make the equations easier. You can try and make a recursion equation for c(2n) but it gets really complicated because you have to take account of whether n itself is odd or even.

If you look at the problem hard enough, it starts to look like it’s counting binary trees, where the first connection cuts the area in two, and the next connection cuts those two areas in two (when possible), and so on. If I could prove this problem was equivalent in some way to counting partitions, then it would be answered. It probably isn’t, but it gives a flavour of its potential for very hardness that this could be.

Someone on the maths.newsgroup can deal with this, and come back with which Paul Erdos theorem answers it. I’ll erase it from my mind by thinking instead about the tensioned thread algorithm.

Also, there’s the one-line Python program that gives you all the numbers you need:

c = [1, 0]
while len(c) < 5000: c.append(sum([ c[i - 1] * c[len(c) - i - 1] for i in range(1, len(c)) ])) print len(c) - 1, c[-1] # prints values as they are created

Cripes! New scary feature in Python: thousands of digits in a single long integer, as if by magic.

Wednesday, January 14th, 2009 at 7:19 pm - - Whipping

The Department of “Not fit for Purpose” (aka The Home Office), whose remit is to give foreigners, criminals, and other outcasts a hard time (so it doesn’t affect most of us until they do something superbly foolish like take their universal ID Card system beyond the drawing boards of their unbelievably expensive and corrupt consultants), wants to make improvements to the services they provide.

Apparently, their service isn’t to institute socially damaging and unworkable schemes that responsible politicians would say No to, but are gimmicky enough to spread all over party political rags in order for irresponsible politicians to get elected in their place.

No, their “aim is to provide clear, accurate and prompt responses to your queries”. The message goes on:

In order to make improvements to the service we provide, we need to know what you thought about how we answered your correspondence and what you think we could do better. Please note that we are interested in your views on how we answered your correspondence, not on the actual Home Office policy.

We have designed an online survey that will give us the feedback we need. It should take around 10 minutes to complete and is your chance to influence how the Home Office communicates with you in the future.

You will receive an e-mail from [email address] <mailto:[email address]> within the next 24 hours. This will provide you with a unique ID number and password so that you can complete the survey on-line. You will need to do this by 9 February 2009 so the responses can be collated by FDS International Ltd, an independent research company, who will forward the results to the Home Office for action.

You’ll note that this was posted in duplicate on the site for filing Freedom of Information requests.

The website was put together last year and works by hosting an email discussion in public through a unique email address randomly generated for each request.

I am still sort of unconvinced that this will work in the long run, having seen its flaws exposed by Liverpool City Council where the FOI Officer sometimes picks any old email of yours on the desk and replies to it with an attached Word document a month later, so it can fail to get filed properly.

Anyhow, there are ways round this, like giving all the senders unique middle names (eg Julian Hoola8ed Todd) and extracting those.

In any case, the Home Office must have contracted this FDS International (“guiding business success in a changing world”) who knows a thing or two about animating waterfalls on their front page. I don’t know what that silly little red paper boat is supposed to represent. Is that the Home Office somehow staying upright on the water by luck? I don’t see any “guiding” going on. These little PR details are what the top management really gets excited about, while their underlying technology is more embarrassing than a primary school computer project.

The next message comes as promised, through an alert generated by the considerably more technologically competent mySociety website.

You should have received an email from the Home Office within the last 24
hours inviting you to take part in a short online survey.

We are FDS, an independent market research agency which has been commissioned by the Home Office to carry out the survey. If you have any questions about it, please do not hesitate to contact myself, Brian Westra at FDS on 020 7272 7766.

To begin the survey, please click on the link below.[password]

Your ID and password will be entered automatically. In certain circumstances, you may be asked for your ID and password. They are:- Your ID:- 2177 [password]

I would like to thank you in advance for taking part in this survey. Please
note that the deadline for completion is Monday 9th February 2009.

Why does it need an ID and a Password anyway? And why is mySociety replacing all the passwords with the word “[password]” in the text to stop people having any fun? Perhaps it should not substitute the word “[password]” for the password when it’s me who’s logged in and looking at my own requests!

Luckily, I got into the system before they slipped in that feature between yesterday and today, and took the survey:

Q: Firstly, can we confirm that you did email the Home Office within the last 5 – 6 months?

A: Yes

Q: Did you receive a reply from the Home Office to your email?

A: Yes

Q: Was your email simply a request for a publication, a freedom of information request or a job application form?

A: Yes

Q: In that case, we are sorry to have troubled you. Please disregard this questionnaire.

End of interview. Thank you for your participation.

This raises a number of issues.

Firstly, the Freedom of Information Act is quite clear:

Any person making a request for information to a public authority is entitled–
(a) to be informed in writing by the public authority whether it holds information of the description specified in the request, and
(b) if that is the case, to have that information communicated to him.

And so is the Code of Practice:

All communications in writing to a public authority, including those transmitted by electronic means, may contain or amount to requests for information within the meaning of the Act, and so must be dealt with in accordance with the provisions of the Act. While in many cases such requests will be dealt with in the course of normal business, it is essential that public authorities dealing with correspondence, or which otherwise may be required to provide information, have in place procedures for taking decisions at appropriate levels, and ensure that sufficient staff are familiar with the requirements of the Act and the Codes of Practice issued under its provisions.

In other words, the FOI Act enforces standards of response to any request for information, even if you don’t invoke the magic words “Freedom of Information Act 2000”. It’s not possible to single them out.

Secondly, to trawl through all the email lists in the Home Office, and to not know enough to strip out the ones that end in “” for something you want to exclude FOI requests from is astoundingly dumb and an indication of the lack of expertise of these FDS International consultants.

Not that these consultants usually get contracts on the basis of expertise, at least on all the past evidence. Someone with some actual competance would have pointed out that the Home Office ought to have all the information that they need inside the department, if only they were properly organized.

I mean, the FOI Act envisages that a department the size of the Home Office would establish a rigourous computerized system for dealing with and recording the processing stages of incoming emails and letters, rather than just forwarding them around the office in crufted-up and disorganized attachments in Microsoft Outlook. Such a system would look not dissimilar to the system, but might cost twice as much, like, say, £100k.

Unfortunately, in UK government consultant-land £100k wouldn’t even pay for producing a speculative tender proposal, while the actual project has got to cost an order of magnitude more to justify the cost of the proposal, so the correct answer is already ruled out from the beginning.

Such an internal system would save a lot of pain and provide immediate feedback as to how long the letters are taking to answer, and whether anyone is happy with them. In fact, if you needed to, why not file out a feedback email to 1 in 100 correspondents at random at the time, not from some bogus market research agency six months later who imagines that it is reasonable and sane to separate people’s views on Home Office policy from how they answered the correspondence. It’s like asking me how well I appreciated the quality of paper, typography, and embossed Queen’s crest at the head of my warrant for arrest.

But who asks the question: What’s marketing to do with the government business in the first place? Our interaction with their core business is not voluntary, so there’s no need to systematically lie to usadvertise at us to fool us into buying from their business, rather than, say, the competition from who we can alternatively get our passports.

But we can go a little deeper.

Do you want a job with FDS International? Well, this marketing company has the barest blank on-line application form in the business. Obviously, the pre-computer literate Managing Director doesn’t see this part of their web-presence, like she sees the front page waterfall, so it doesn’t get a make-over. How about telling us what types of jobs are available at your company so people can apply to them not-blind?

Some people would think you really could do with some web-programming skills in the office, given the quality of the product. But they’d be wrong, because you’re only offering industry standard SPSS Dimensions mrInterview crap web software and probably can’t do and won’t do it yourself. After all, success comes from sales and networking; no one is interesting in technical curiosity and excellence. You buy that in like you buy cardboard-encased Brakes Bros frozen dinners into ye olde worlde country pub. No one cares about home cooked food any more if they want to make money.

FDS International appears in the Hansard records a few times. For example, this written answer and this written answer differ by one column, and we learn that the Department for Communities and Local Government contracts FDS International from 12 May 2008 to 30 February 2008 (apparently for 5 months!) for £47,303 under the “Fire Protection” programme. (Don’t ask.)

FDS International also did a Comparisat benchmark survey, according to the Home Secretary, which found that the UK Passport Service was number one for customer satisfaction.

That’s how you market yourself. By complimenting them!

(Page down one screen to see a late comment from Chris Lightfoot.)

Not surprisingly, if you believe you have a quality brand, like the passport service, any marketing consultant will tell you that you can repackage your low-grade toxic horseshit under that brand and sell it at a profit. I give you Identity and Passport Service: Identity Service Proposition – A joint venture with the Criminal Records Bureau, prepared by FDS International:

The trial was held over nine dates in June, following a pilot exercise in May. Participants were volunteers who were mainly members of Registered Bodies and most had previous experiences of going through the current CRB process.

Having agreed to take part in the trial, participants attended one of nine venues and:
* proceeded through a trial of the short-term process based on the UK passport, then
* went through a trial of the long-term process based on the ID Card.

Like the questionnairre about responses from the Home Office, the marketeers feel it is legitimate to separate the enjoyability of the process from the underlying policy and purpose. From that document:

At the end of the interview about the short-term process, and before going through the second, ID Card-linked process, participants were asked their opinion in general of the ID Card scheme. The timing of this question (at the end of the feedback interview about the first process, but before going through the second process) meant that their later views about the CRB ID Card-linked process could be examined in the context of their overall opinions about ID Cards.

The main value of this question was as a check to see if negative reactions to the process linked to the ID Card correlated with negative views of the ID Cards in general.

In fact, the majority (65%) are supportive of the ID Card scheme, with only 10% not supportive.

Older participants were more likely than younger ones to be very supportive with 74% of the over 50s scoring 4 or 5, compared with 50% of the under 40s.

Overall, reaction to the two new potential services was overwhelmingly positive. The short-term Passport-linked service is considered a major improvement on the current process, for its speed, efficiency and simplicity.

The longer-term ID-linked process is rated even more highly and is thought to be much more robust than the existing process.

These research results represent a very strong endorsement of both these new potential services.

I went back to their crappy on-line survey, ticked No for the question “Was your email an FOI request?”, and filled it in. It’s tediously not interesting and not worth repeating here.

I made an FOI request as well:

I have reason to believe that the Home Office has contracted the independent research company FDS International Ltd who is now conducting a Customer Satisfaction Survey online through email.

I have searched for information about this company and its relationship on the Home Office website and can’t find any information.

Under the Freedom of Information Act 2000 please can I have a copy of:

* The OJEU tender procurement documents, as well as details of the successful application submitted by FDS International, pursuant to this contract (if its value was above the £100k threshold making it liable to this procedure)

* The contract signed between the Home Office and FDS International for conducting this survey, including all details of its value.

* A record of what personal information (for example, email addresses) has been transferred from the Home Office to FDS International for its use in satisfying the contract, and under what privacy terms.

Further in my web-searching I found this posting on seenoevil from March 2006:

Dear Sir/Madam,

The Home Office is trying to improve the service it provides… We have designed a survey so that we obtain the feedback we need without taking up too much of your time… You will receive an e-mail from within the next 24 hours. This will provide you with a unique ID number and password so that you can complete the survey on-line. You will need to do this by 20 March so the responses can be collated by FDS International Ltd, an independent research company, who will forward the results to the Home Office for action.

So this is just money for old rope, and at least shows the value of recording on the internet any old cruft that comes out of the government.

The fundamental category error being made by the political elite which is causing such dreadful misgovernance is to think that it can be made to all work like a marketing gimmick. It’s true, elections can be made to work that way. If you sell more of your crappy product on the right day (election day) than your competitors, then you win the power.

The legitimacy of elections then means that those of us who disagree are supposed to put up and shut up, because we lost, and are therefore wrong.

But policies like ID Cards don’t work like that. It’s not enough to kid an overwhelming majority of people to buy into this horseshit. You need legitimacy. All it takes is for one obstinate granny to make a stand against it, having been aware of the crimes that governments have a long history of committing (systematic violence, theft, and the protection of unearned wealth) when good people let them get away with it, and for the police to haul her out into the mobile Identity Station van parked on the street to get her eyes scanned, and all that marketing effort to pursuade the public how sweet and friendly this system is will come unstuck like a lose scab. The crisis management consultants will then be on the line because their number is auto-dialed into the emergency switchboard at the Ministry.

I’ve not seen any recognition of this old political story-plot in any ID Card promoting document, even though it’s utterly predictable, and will happen. What do they care? Once enough rubbishy marketing consultants are in place, they efficiently drive away anyone with another story to spin until the whole Office is 100% crap. Money and time doesn’t only get wasted, it gets only wasted. Nothing of any good to compare it to can be permitted to exist.

A wise politician will make sure that the ID Card system remains always under construction, and never gets built. Consultants happy, because they’ve got the money for nothing. Labour Party elected politicians happy, because they can use it to bust politicians who are more responsible than themselves and are against it. And the marketing men and women can carry on dreaming that the UK Passport Service is a business which sells us passports, while ignoring the fact that they invented the requirement for this document in the very first place.

The ID Cards are like passports, except they’ll be required to get a job or visit the library. For the next few years they’re only required as a license to have been born in another country. For some reason, that otherwise untraceable historical incident beyond your control means you don’t belong here and can piss off if you don’t like it.

Here’s a public service youtube by the Home Office. Enjoy.

Thursday, January 8th, 2009 at 2:51 pm - - Whipping 2 Comments »

Back in 2005, about a year after PublicWhip and TheyWorkForYou were getting going on the back of significant unpaid volunteer work by people like me who were expecting to, you know, get some kudos for it in the form of, say, at least one newspaper article about the how’s and why’s of the project, and which would be able to get across the sense that maybe more people should try getting involved in this sort of activism, the Guardian Newspaper produced its Ask Aristotle site entirely independently.

Great, we thought, another source of structured data to link to from the TheyWorkForYou pages under the more useful links for this person to things like the constituency voting results back to 1997.

  • Never mind the fact that The Guardian didn’t put any links back to the TheyWorkForYou site or support it in any way.
  • Never mind the fact that nobody could ever find the people behind it who did the work, much of which replicated what we had done, and we have always wanted to collaborate.
  • Never mind the fact that all our work was open source and free, consciously, so that other organizations like The Guardian could re-use it and integrate with it. We never thought of investigating why they weren’t, because, well, they were free not to.

I always wondered why it was called “Ask Aristotle” anyway. It’s a stupid name. Whenever I puzzled about it out loud to my friends, I always got some mumbo jumbo about some ancient Greek philosopher back, and look, there’s a sort of a grey-haired dude in the icon.

You can look as hard as you like, but you won’t find anyone on the Guardian website explaining this philosphical choice of a name.

They don’t explain it anywhere.

Everyone outside seems to have been satisfied with not asking the question, or sticking with the answer that they made up themselves. We’re used to seeing stupid names for internet things.

Doing some research on what’s possible for the next General Election, I’ve found better theory:

25 Years of Excellence in Politics and Technology

Aristotle was founded in 1983 by John Aristotle Phillips and Dean Aristotle Phillips. The company’s principle activity was the development of specialized software and databases for elected officials. With early success and rapid growth throughout the 1980’s, Aristotle is largely responsibly for developing the Political Technology field as we know it today…

With our base philosophy of providing power tools for political professionals, Aristotle has continued its growth by expanding its portfolio of services, adding several new employees and opening offices in Toronto and London… Our International Services division has bought the company global acclaim for its work overseas, including Campaigns & Elections International Campaign of the Year for its work on the Victor Yushenko campaign for Ukrainian Prime Minister in 2004.

Aristotle International, Inc. is a private company which has made one set of detailed filings for the year 2000 at the Securities and Exchange Commission.

What’s neat about these filings is they are supposed to inform investors about the risks associated in gambling on the stock investing in the company:

As we expand the scope of our Web services, we may face greater competition from a number of Internet companies and other media companies across a wide range of different Web services. Many of our competitors may have advantages over us in expertise, brand recognition, size, financial and personnel resources and other factors. Several companies offer competitive products or services through Web advertising networks, including 24/7 Media, Inc., DoubleClick, Inc., Flycast Communications Corporation and L90, Inc. In the Internet authentication services industry, our principal competitors are Equifax, Inc. and ChoicePoint Inc. Our Internet advertising business may also encounter competition from providers of advertising inventory, database management products and related services, including AdForce, Inc., DoubleClick, Inc. and Engage Technologies, Inc. In addition, we may face potential competition from a number of large Web publishers, search engines and ISPs, including America Online, Inc., Excite, Inc., Infoseek Corporation, Juno Online Services, Inc. and Yahoo! Inc. Competition may also materialize from political or public policy websites, including,,,, and Moreover, we compete with television, radio and print media companies for a share of companies’ overall advertising budgets.

Hm. No Google. This was back in 2000, after all.

But what about their business income?

For the year ended December 31, 1999, we derived approximately 21% of our revenue from a contract with the National Rifle Association. We have completed this contract with the NRA and, although we have entered into a new contract with the NRA, we do not expect to derive substantial revenue from this client in the current fiscal year or under the terms of the new contract.

For the six-month period ended June 30, 2000, we derived approximately 12% of our revenue from a contract with MatchLogic. We recognized $350,000 from this contract in the three-month period ended June 30, 2000 and will recognize the remaining $100,000 of services over the next 24 months.

We rely on a third-party database program to store our voter and departments of motor vehicle records, and our ability to deliver our products would be impaired if we were unable to use that database program or replace this program in a timely manner.

Our voter records and departments of motor vehicle records are stored on a standard, commercially available database program that we license from Microsoft Corporation. If we were to lose access to this database for any reason, including our breach of the license agreement with Microsoft, we would be required to transition our database to an alternate commercially available database program, which could cause delays in our ability to offer our products and services associated with the voter records and departments of motor vehicle records. This delay could lead to dissatisfied clients, potential liability to clients, damage to our reputation and loss of any competitive advantage we enjoy.

Anyway, we’re talking about the US of A here. They are a big sophisticated advanced FUBAR democracy. What does it mean to us?

License of United Kingdom Voter Records

On February 21, 2000 the Company entered into an agreement with a third party to license voter records in the United Kingdom. For the use of these records, the Company will pay 50% of its after tax revenues generated from the use of the licensed data. The contract also provides for minimum per record royalties based upon the total number of records licensed to third parties. The Company made a non-refundable deposit of approximately $15,000 which will be used to offset the initial royalties. The agreement expires in 2003 with the right to use the voter records expiring one year later.

We have licensed the voter records of the United Kingdom, and although we have made only one sale in the amount of approximately $10,000 through June 30, 2000, we may collect voter data and expand our sales efforts in the United Kingdom and other countries, if appropriate opportunities arise.

Use of our products and services in foreign jurisdictions may be heavily regulated. For example, in order to collect campaign contributions online in the United Kingdom, a website operator may have to comply with the legislative requirements contained in the Political Parties, Elections and Referendums Bill, which has not yet been finally approved by the British Parliament. We believe that this legislation could place restrictions on access to and use of information about registered voters in the United Kingdom. However, we cannot foresee the scope or nature of these restrictions or their ultimate impact on our operations, if any, because the legislation has not yet become law or been made publicly available.

Further, the terms of our contract for our license of the United Kingdom voter records provide that the licensor can terminate our license if the licensor reasonably determines that the contract is no longer commercially viable, as a result of a change in circumstances. If our collection and use of voter records were ultimately too heavily restricted to make our products commercially viable, or if our license to use the United Kingdom voter records were terminated, we would lose our investment in acquiring the license to this data. Similarly, other countries into which we intend to expand may enact their own privacy regulations that may limit the collection and use of voter information, which, if applied to the sale of our products or services, could negatively impact our results of operations. Given the early stage of our discussions, we have not yet engaged legal counsel to research the existence or extent of any foreign restrictions that may be applicable to our operations.

These days activity seems to be thin here, though maybe things will be ramping up for the next UK election after two years of all money being in the United States. I don’t have time to check whether Aristotle International Europe Ltd has anything to do with them.

Other trivia

The founder was the kid in Princeton University who famously got in trouble with the FBI for designing an Atomic weapon from publicly held available information in 1977. His supervisor was Freeman Dyson, father of Esther Dyson, who stood on the board of Aristotle International, among her other interests.

This alarmed me, because Esther Dyson is on the board of The Sunlight Foundation, so to see her involved in what appears on the surface of it to be a completely evil anti-democratic sell-out-to-anything corporation doesn’t bode well.

But a Vanity Fair article about the company explains it:

“It seemed really intriguing,” she says of the company’s mission, “using market mechanisms to foster political action.” But she gave up her seat after concluding that Phillips was more interested in making a profit. “The focus changed,” she says.

From the same December 2007 article:

“You can target Labour Party voters in the business section of London with the same systems you use to target people in the Virginia primary,” Phillips says, the excitement rising in his voice. “There are going to be global referenda in the not-too-distant future, I think, where people all over the world are going to vote at their A.T.M. machine, from their mobile phone. We’re getting to an age when there’s going to be that kind of direct democracy, and direct democracy requires ready access to the ability to communicate with voters.”

What are the conclusions?

One of the most effective ways to win in business, politics, or even sport, is to destroy the competition. Poisoning or starving your competitors makes them easier to beat and therefore win the pot of money reserved for the “best”. Do nothing to help them.

Clearly, by this logic, Aristotle International isn’t going to want to have anything to do with us (citizen activists developing our own web technologies), because that cuts into their own business/profit interests. If they can, they’re going to want to prevent such grass-roots activities from taking hold — at least until their business has become established and dominant (but then it becomes even easier to suppress and discourage).

As I have discovered, political engagement during elections doesn’t seem to happen in the UK in the same way as it does in the US where political lobbies routinely attack a candidate on a particular Parliamentary vote. So their services might not have been so successful as expected. Who could have predicted that back in 2003 when we all thought that the Iraq War would have some tiny effect on Tony Blair’s re-election.

In any case, the allegation I can make, based on the evidence collected and presented so far this morning, is that Ask Aristotle may have been a joint venture between The Guardian Media Group and Aristotle International, Inc to establish a UK business in the sector of corporate manipulation of electoral behavoir for profit, and in the process of this they discouraged, failed to engage with, and failed to publicise the exciting nascent web activities of democratic activists in their country (by not even putting in back-links) in a way that the most forward-looking, high-quality, dynamic on-line web-enabled media outlet would have been expected to have done had they not been planning to make money out of it by instead selling all of their brand and expertise to an outside US corporation whose intentions would be to make politics here as absolutely shitty, expensive, dire and misinformed as it is over there.

I don’t like this thought at all. I hope it’s wrong. Everything can be explained by the answer to the question “Why was it called ‘Ask Aristotle’?”

An on-the-record article explaining how it is connected to the philosopher and not to the corporation brand name derived from some greedy Princeton graduate’s middle name would do the trick.

Update 1: shows that Ask Aristotle began on 23 March 2008 and was “sponsored by Syntegra” probably for the election. I’ve no idea what Syntegra was in 2001, but it’s now BT Consulting. Anyway, it makes this conspiracy theory look a little weaker, but doesn’t completely get to the bottom of these questions.

Update 2: The April 2001 of Syntegra says it was part of BT “dedicated to creating winners in the digital economy”, but with no mention of The Guardian being one of its customers.

From the Sponsorship opportunities on the Guardian:

There are various search boxes across the Guardian Unlimited network which are available for sponsorship. As part of their campaign to promote their brand as ‘The brains behind the scenes’, BT’s Syntegra sponsored the search boxes on the front of the Guardian Unlimited network, and the Ask Aristotle search boxes across the Politics site. The three-month package included integrated statements of association, and Syntegra was the exclusive banner advertiser on the results pages generated by using the search tool. The package also included buttons on the front of the network and across the Politics site.

Usually banner advertisements are hyperlinked to the sponsor, but not according to the HTML source. It’s not yet clear whether employees of Syntegra, the Guardian, or Aristotle International are responsible for the sub-site. Syntegra could have the connections to do it, given their association with the Statute Law Database as far back as 1993. Ultimately it would take someone involved in its creation to make themselves known.

Saturday, January 3rd, 2009 at 2:07 pm - - Whipping 1 Comment »

I kept an eye on Francis last night till four in the morning as he moved all the files from old seagrass server to new improved server with bigger disk space, lower power consumption, and no added fish. Electricity prices are now so high in relation to the cost of a new server (not including the hassle of moving everything from the old one to the new one when you haven’t done it before) that the pay-back is less than one year.

Seagrass has also been moved to Cambridge for cheapness. There’s some inconvenience due to changes in ip-numbers which I don’t understand. I hope to take delivery of the old hunk of historical iron in due course.

While waiting for millions of files to copy, I tested the functioning of Public Whip by motion text editing the utterly ill-considered vote on reducing the VAT rate from 17.5% to 15% that was all over the news recently.
According to the Chancellor, who made the order on 24 November, this will put £12.5billion into the economy for the purposes of encouraging more shopping.

One of the very few MPs to know anything about economics, Vince Cable MP, held a Parliamentary debate and the vote (linked to above) to annul this order because it’ll be a wasted, ineffective fiscal stimulus that will have too little an effect on prices to affect consumer behaviour, was predicated on rosy forecasts of economic recovery, and the money will be consumed by higher margins in major retailers, not to mention the administrative costs of conversion for numerous small businesses.

This is not America, where the sales tax is added on at the till to the cover price of a book so it’s like paying a tip in a restaurant, we don’t see it. We don’t actually feel the tax rate. We, the public, are even more ignorant than most of the MPs about economics — if that’s possible. That means there’s lots of space for lying about it during an election campaign and imposing major misunderstandings about who’s paying, how much, and what the effects are.

To Francis, I sounded off about the irrationality and ignorance embodied in the party election leaflets for the Crewe and Nantwich byelection and how it was all hitting on the 10p tax rate — which is one of the least relevant issues you could choose out of all those that an MP has jurisdiction over.

It’s that, or the home family values of the candidates.

Sometimes personal characteristics matter. Like being right in their predictions and judgments about what would happen in economics over the years.

Unfortunately, “Vote for me, I foresaw a house price crash and the wholesale corruption in the banking industry, unlike my opponents,” doesn’t seem to get any votes. Consequently, we elect people who don’t know anything about economics, they get into Government not knowing anything about economics, and they hire consultants to help them run the economy. Unfortunately the consultants are experts at money, which is not the same thing, and are usually working as Economic Hit Men, whose measure of success is making more money, not saving the economy from ruin or misallocating precious resources towards inappropriate projects.

So, the consultants know about money, but not about economics, just as our successful politicians know about elections, but not about democracy.

You can make a lot more money as an economic consultant by lying to government ministers and getting them to buy into bogus economic theories (eg that the economy is all about shopping and share prices, and is fundamentally self-correcting and is up to the challenge of long-term planning), than you can by working out the truth of what matters and attempting to implement it fairly.

You can get a lot more votes by lying to the public about the facts of democratic representation (eg that elections are all about the price of petrol and the cost of out-of-town hospital car-parking charges), than you can by connecting people to the policy choices available.

I’m concerned that the very worst policies are like that on purpose.

New Labour’s launching of numerous Terrorism policies way over beyond the looniest wing of the opposition Conservative Party, like the 90-day detention of suspects without telling them of their crime, or the later 42 day detention proposal, caused chaos. Had they got these through, all the election campaigns would have been:

The Conservative Party: Soft on terrorists, hoodies and knife crime

We may be getting astonishingly bored with this terrorism shite from the politicians, but they can’t predict that. They knew that there was a good chance of an incident that would have an effect among the British electorate, so these political insults were being planted in the ground so they could be harvested in the event that we passed through that landscape again.

That is the story of how elections are won. Accidents and bloody-minded opportunism.

This VAT rate change looks to me like this story all over again. The Crewe and Nantwich byelection suggested that confusing tax changes flogged like a dead horse can win votes. Nobody really knows why, but it’s like marketing, or battery chicken farming — it’s an experimental science. You can’t predict how far it can be pushed. Those who win in the business world have absolutely no scruples and will simply pack as many chickens as they can into the box until the production of eggs per square metre of cage starts to go down. What do they care what it feels like on the inside?

What any cynical and desperate politician who intends to play the game of dumbing down beyond all comprehension a tax issue for the next election campaign needs is a tax cut which their party can be instructed to vote for, but is so bad and ignorant that not even the opposition Conservative Party can support.

I’m seeing it now:

The Conservative Party: Against cutting taxes for hard-working families to help them afford their weekly shopping bills.

Millions of these leaflets will be designed and ready for shipping for the next election, which is on the Prime Minister’s mind all the time.

It probably doesn’t occur to the political elite to have any interest in the actual economic circumstances or long-term outlook of things. All they need to know is that this wheeze converts into votes, no matter how irrational.

It’s just things gut feelings that are often wrong, and can be used.

For example, no one says: “Who put the damn hospital outside the town so everyone has to drive to it?”

A human attention span is literally seconds, and the effort — mostly not taken — to make logical connections, means we experience a little friction of anger at being ripped off by those car parking charges, and it gets added into a feeling in the gut — without that anger transferring to those who created the policy in the first place which moved the hospital outside of town, and replaced it with shops that were easier to get to and go shopping at.

Because shopping is what our purpose in life is now.

There is a systematic inbuilt political miscalculation; as there are regarding our systematic miscalculations of probability that give rise to the existence of the gambling industry; as there are regarding the addictions of certain substances that give rise to the existence of the tobacco industry — which this political structure relies upon.

If tobacco, or bad politics, didn’t actually kill, they would merely be an irritation.

What we need is a science that discovers the intellectually toxic substances which allow the wrong people with the wrong ideas to continually win elections and be in power.

This has to come first. Unless we know (and want to know) what these toxic substances are, there’s almost no point in attempting to design treatments for the affliction, because they will almost certainly be attacking the wrong symptoms, and it will be predictable that projects like those coming out of mysociety will be counter-productive, like alternative medicine which, if you believe it works, can dangerously delay the application of effective treatment.