Freesteel Blog » 2008 » November

Friday, November 28th, 2008 at 12:38 pm - - Machining 1 Comment »

So much to get distracted from, what with codewikis to invent, people teaching me to use Pylons instead of Django, yet more templating languages I didn’t know about with so many layers of components that nothing makes clear exactly what each one of them does, as well as getting make-over done to — that when you face bugs like this:

it’s easy not to want to do any more. But I’ve been going through things like this for about two weeks now.

What’s going on is that the scallop machining algorithm for 3-axis toolpaths somewhat fudges the issue when it comes to toolpaths on the vertical walls. It’s a special case. Until recently I couldn’t have more than one point of a toolpath on the same vertical wall at a time. This needed to be upgraded, because the Danes came up with a nasty little trick.

What you do is start with some sensible contours made around the shallow areas of the part. If you scalloped inwards from these, everything would be fine, because the scallop algorithm works well on these predominantly flat areas.

However, they’re sending the contour areas outwards a certain distance to smooth them. And what this does is place the contours all on top of one another on the vertical sides of the part where the scallop algorithm has a hard time getting it right, especially where it’s twisted simultaneously up and down a saddle shape.

So I spend a week debugging it, upgrading its capabilities (I can have up to 12 contours passing through the same XY area — half of them are invisible), get it all working lovely, send back the improvements.

And then all they do is double the offset outwards and it breaks all over again.

Although it is difficult to concentrate, so far I have been very lucky, because the fundamentals of the algorithm are sound. It’s just turning out that the some of the details, special cases, and submodules are misfiring.

But this, however, is the worst example I’ve seen. I haven’t started work on it, so I don’t yet know if my luck has run out.

Onwards to Euromold shortly. I’ve printed out a 5-axis algorithm on a sheet of paper to wave at people, to see what they think of it. The conventional response to the invention of an apparently new algorithm is to detail it incomprehensibly into an academic paper, include 5 pages of introduction reviewing all the other established 5-axis algorithms out there that don’t work (although the academics who think they know everything think they do and are going to argue with everything you say), include shed-loads of irrelevant citations, print it out in triplicate with the images on separate sheets of paper, sign it over to some evil Elsevier publication, wait three years, and get it thrown out by some distinguished South Korean professor who’s refereeing the paper anonymously (for no good reason, because anonymity was only ever justified to protect junior researchers from the wrath of senior professors who think they should get away with writing rubbish papers). With this drag of astonishingly mediocre past work which anything new has to answer to (never the other way round), you don’t wonder why there is so little progress. So I’m not bothering. I can afford to publish things independently of this broken academic accredited-publication system because my job doesn’t depend on it.

Wednesday, November 26th, 2008 at 12:44 pm - - Whipping 1 Comment »

On the BBC you get a different quality of propaganda from the commercial networks. Whereas Channel 4 gave us a programme that lied on camera from start to finish and received lawsuits from one of the interviewees for misrepresentation, the BBC lies by ommission, amnesia and fraudulent misrepresentation of the opposition.

The Horizon episode was called Jimmy’s GM Food Fight where our tame scientist/flying pig farmer was taken on a journey around the world through a carefully scripted series of staged encounters, which included:

  • A nice lady professor doing the genetic modification itself with the simple act of pouring a test-tube of natural bacteria onto a barley seed to create more nutritious and drought resistant varieties
  • A visit to the traditional Amish farmers in Pennsylvania who, even though they reject new-fangled technology such as electricity, are quite happy to grow GM maize crops
  • An ecologist in Arizona who spoke in suspiciously perfect PR nuances, claiming to be studying just how much more wildlife was now growing on the landscape because the pest-resistant GM cotton crops didn’t need to be sprayed with chemicals every other week
  • Small-hold farmer women in Uganda desperate for new GM banana varieties to save their staple diet from the infections that reduces the yield and could threaten the country with famine

The opposition to GM crops came in the form of an interview with the ridiculous Lord Melchett who didn’t know anything and has proved himself too dippy to care about the damage cause by his acceptance of employment with global PR firm Burson-Marsteller, and a knowledgable member of the Union of Concerned Scientists who might have had more to say if it weren’t for the lack of time in this hour-long info-mercial.

Instead, the only fact the independent scientist could get across was how GM products had been put into every single foodstuff in the USA in the last ten years without any testing, pause to study or check out its effects on the population. This, he pointed out, was a sign of regulatory weakness with the corporations as usual conducting their experiments on the unwitting public. It’s just a matter of luck and good fortune that something really bad hasn’t happened yet.

But our farmer Jimmy turned this round and concluded that obviously this showed the products were perfectly safe, since nobody had noticed any bad effects in ten years of widespread use. But we already know that when there are on-going issues, such as with Monsanto’s recombinant bovine growth hormone, the US corporate media and amazing state legislation like the food libel laws is quite capable of covering it up for years. Would you like puss and diseased blood in your milk? It’s pasteurized, so it won’t kill you. You’ll notice that there are a lot of changes going on regarding the policy of BGH use in America this year (2008), which is a sign that something real is happening, even if the media is not willing to report it.

The laws against this product in Europe centre around the matter of animal welfare. In other parts of the world where animals have no rights, it’s entirely decided on the basis of cost-benefit analysis. By turbocharging your cow with a massive dose of mammary gland hormone, some of them are going to get very sick. How this affects the bottom line depends on the proportion that die.

Perhaps it’s because European citizens more often come into contact with farm animals than Americans do, because of the nature of the landscape. Imagine if we relaxed the laws against drug taking for Olympic athletes. These shows would rapidly get pretty ugly too, with steroid-fueled freaks occasionally dying during the medal’s ceremony. How would that work for your entertainment?

Meanwhile, the whole policy is pressured at us from the government and the corporate lobby for three reasons: money, profit, and nothing else.

If there are any positive benefits resulting from the corporate application of technology in the market, it’s a pure coincidence. It’s been established that if it’s profitable while at the same time being seriously harmful to life, they push it just as hard anyway. The so-called right-wing corporate ideology rests on the axiom that profit always implies public good — in spite of the fact that those who do make the big profits clearly understand that in order to get there they must: hide the damages, create monopolies, and abuse power to kill the competition.

The corporate GM lobby does all three, especially with their use of the patent system and systematic contamination of the alternatives. Almost the entire political issue would disappear overnight at the stroke of a pen if the EU simply ruled that all patents of genetic materials were invalid. Unfortunately, that’s not up for debate, because any of the public benefits that might be had from this technology can wait while they fight it out to capture the obscene profits they believe are available.

These corporate lobbyist’s brothers in the software patent industry weren’t quite so successful at creating these monopolies, because, unfortunately for them, people were already making software quite successfully before the patent regime went insane in the 1990s, and so they could easily expose the lie that adding patenting would represent anything of an improvement. The biotechnology revolution, however, happened after patents became silly, so there aren’t enough people around without a conflict of interest who can inform us that it just ain’t necessary for the technology to flourish. They’re necessary for huge unearned profits, but they’re probably irrelevant for the technological advancement itself.

The attempted sale of GM crops into Europe back in 1998-99 was an absolute debacle and an insult to the public intelligence. Now, ten years later, they’re gearing it up again, hoping we’ve forgotten the sorry episode and that enough young people will have come into the political market since then who don’t remember the story. Coz we have to remember the story, since the BBC is not going to remind us of it.

It’s like running an entire Republican Party presidential campaign without mentioning George Bush. The ban on GM foods in Europe has Monsanto’s name written all over it, and any programme about the politics of GM crops that doesn’t mention Monstanto once is clear evidence of propaganda.

And indeed it was, because apart from Lord Melchett and a couple of minutes from the UCS, the rest of the opposition were painted as those bad Europeans with their ignorant prejudices against perfectly safe and beneficial progress, who are so technologically backward they make the Amish look like spacemen, and who wouldn’t care if millions of Africans died from malnutrition just to preserve their anti-scientific principles.

The main weapon of PR disinformation on this subject is the establishment of the false category “GM crops”. It works like this.

  • Hello. Are you against “Things Made Of Metal”?
  • Well, I’m not sure. Sometimes “Things Made Of Metal” can be dangerous.
  • Look at this egg-beater. It’s made of metal, and it’s lovely and very useful. See: we can make pancake batter with it. You like pancakes, don’t you?
  • Yes, I really like pancakes.
  • So you’re not against “Things Made Of Metal”.
  • Well, I guess not.
  • You do know that all “Things Made Of Metal” use the same basic technology to make them, however big or small.
  • Really? I didn’t know that.
  • We’ve been making “Things Made Of Metal” for the last 6000 years since the end of the Stone Age.
  • That’s great. Thank you, “Things Made Of Metal”, for all you have given us.
  • Right, now please sign this petition against the ban on the sale of new “Things Made Of Metal”.
  • Uh, okay… Hang on, this is a petition about swords. I really don’t like swords. My brother was killed by one.
  • But you said you were okay with “Things Made Of Metal”.
  • Not with swords. They’re very bad, and they always seem to be in the wrong hands.
  • I’m afraid there aren’t any distinctions in the law between different sorts of “Things Made Of Metal”, they’re either banned or not banned, and since we can’t ban all “Things Made Of Metal” — you do like your pancakes, don’t you — we have to un-ban them
  • Wait a minute, wait a minute.
  • How ’bout this. We’ll compromise. You help me lift the ban on “Things Made Of Metal”, and then we’ll let the government license which ones are legal. Does that sound okay?
  • Well, maybe. But don’t we have to trust the Government?
  • You do, but the Government always works in your interest. The BBC says so.
  • But the Minister in charge doesn’t always seem to be fair.
  • He tries his best.
  • I’ve never met the Minster. Do you think I’ll be able to talk to him if he makes a wrong decision?
  • Don’t be ridiculous. There are millions of people like you. It’s not realistic. The Minister only has time to speak to important people.
  • Like who?
  • Owners of large corporations. Such as the Sheffield Sword Factory Institute, and other arms makers.
  • That doesn’t seem fair.
  • Well, the Government needs to buy weapons. Otherwise how do you expect it to do its job?
  • What’s it’s job?
  • To protect you from harm. You don’t need to worry about it. It’s all in good hands.
  • Oh. That doesn’t make me feel very happy.
  • Here, have a pancake. That will cheer you up.

Meanwhile, the secret licensing of new crop trials has begun. Nothing to worry about. I’m sure it’s not con-trick, like those Hawk “Training” Jets sold to Indonesia for giving novice pilots a taste of what it is like to fly (bombing raids against defenceless civilians), or even Japan’s “scientific” whaling programme whose scientific experiments seem to appear in school cafeterias throughout the land.

I mean, what do they take us for?

Back in 1998, the PR wing of Monsanto told us that:

GM foods were necessary to preserve farming yields

until the logical contradiction between this and terminator gene technology became too grotesque.

Then they told us that:

GM crops required less chemical spraying

until the properties of their Roundup Ready soybeans that made it resistant to their own-brand of herbicide meant it could be doused more.

Then they told us that:

Farmers in the third world were crying out for their products

Which they were, when they weren’t committing suicide after having been ruined by debt to the GM seed companies. (Though the Guardian now reports otherwise, somehow generalizing to the whole field from a study only on a particular strain of GM cotton.)

Then they told us that:

It’s too late. There is no alternative. The world’s soy commodity is already 5% GM contaminated, and there’s nothing you can do about it.

Unfortunately for them, the supermarkets found some sources that were not pre-contaminated and were able to offer their customers a choice.

Then they said:

You’ve got to let our scientists do this, or they’ll go to America and take the benefits of this new biotechnology with them

Yeah, and all their stem cell researches with them too.

Undoubtedly, the European ban on GM crops is too blunt, unfair, ineffective, and insufficiently targetted against the bad stuff. We import large amounts of GM soy to feed to animals, which we then eat.

However, I would continue to support it, not on the basis of prejudice against GM technology itself (though that’s how the companies misrepresent the opposition), but on the matter of serious concerns about the ability of the European politicians to regulate it properly.

They, their corporate minders, and the PR industry, through the vehicle of this BBC programming, have shown no sign or willingness to distinguish between particular GM products that are potentially beneficial, and those that are seriously detrimental to the public interest (which may sometimes quite profitable).

Without this discrimination, any regulatory regime is going to be nothing but a joke. We’ve got many jokes like this. We don’t need another one.

No campaign to get the public to accept the lifting of this ban should get by without a truthful statement about the legitimate public concerns supporting the ban in the first place, and an accountability for the loss of confidence which was the direct result of the systematic lying by the government and the corporations during the first round of this debate.

Tuesday, November 25th, 2008 at 1:02 pm - - Whipping 1 Comment »

I’ve bought a few day-return tickets to Manchester from Liverpool in my time, and sometimes received £15 performance-related pay for my troubles. (You have to ask me after a few beers to explain this because it relates to a wholesale refutation of the axioms of evolutionary psychology.)

Anyways, I popped over to see Substance in a fast-decaying Urban Splash building, where the cheap laminated floors which probably looked good on Day Zero were now as dimpled a cow field from the tracks of high-heeled shoes. I was there to pitch my Codewiki project which is going to be the next big paradigm shift on the internet, after such phenomena as YouTube, Twitter and Wikipedia. Not many people are going to use it directly, but the effect of this infrastructure project is going to be profound.

The demo and meetings went kind of okay, as these things do, not great, not bad. A good idea that actually looks good to other people would have been done years ago. But I have a knack for finding ideas that look bad, even after I have done all the work to show that it works and it should be really good.

I can summarize my application as the data equivalent of OCR, which is predominantly applied to documents that were printed off a computer in the first place so why didn’t they keep the original electronic file backed up then, eh? Solving problems that shouldn’t have been a problem if things had been done sensibly is always a totally thankless task because no one notices it. It’s like that really bad hang-over you don’t have right now, which makes life such a joy to live, as you’re able to appreciate only at the times when you do have a bad hang-over.

I left the office at about 2pm and wandered through Manchester feeling quite sad. I’d missed lunch, but that wasn’t the reason. It took a while to work out what the problem was, and I think it was this:

Barrington J. Bayley
14 October 2008

Fittingly, the guy who inserted Bayley’s death notice onto the Wikipedia page has been indefinitely blocked, because the bearer of bad news has to watch out what they do.

Barrington Bayley is my favourite Science Fiction writer, without a doubt. His general lack of recognition pretty much convinces me that I’m never going to make it, because I could barely aspire to aim for that level of quality, and he’s nowhere — even way back in the old days of SF when having original ideas was actually once important in the genre.

In all my years of attempting to write fiction, my closest claim to something genuinely interesting and Bayley-esque is the short story Mine the Primes written in 2002. Bayley could really do it, capturing the horror of this outrageous phenomenon we call life and the way we handle it.

They say, “Truth is stranger than fiction” — damn right, these days it is, with most of today’s writers not able to swallow the fundamental reality of the situation, continually flying around the planet instead of using the internet, and breeding like they think we’ve actually got a future.

You’ve got your mainstream fiction literature which systematically does not tread into the wide-open and very real territory of the future, and then you’ve got your Science Fiction literature which has the license to go there, but always seems to bung in some space travel to other planets without any consequences like it’s just turning over and using the other side of a dirty napkin. The real story is that we’re not going anywhere soon. We are going to have to live through the consequences that have been clearly mapped out for us by Science and are as real as the graphic details of our own individual death from a diagnosed terminal disease with no prognosis — whether or not we are prepared to imagine it.

Barrington Bayley could really communicate a sense that life was like being on a run-away train, even when the adventures spanned the galaxy and involved major battles between time-travel ships deep in the strata where they warred against the Hegemonic empire encroaching from the future. There is always an Out There. In the book The Fall of Chronopolis Captain Aton of the Third Imperial Time Fleet finds out a crucial fact underpinning the whole war which renders it entirely meaningless (much more profound than pathetic lies about Weapons of Mass Destruction), and takes his news to the emperor following a dangerous journey through the strat, minus ship, (such people are normally executed for religious reasons as soon as they return to the surface), chats up a princess and gate-crashes the palace.

Aton and the princess were mingling with the courtiers surrounding the throne. Boldly Anton stepped forward to confront the emperor and the prince.

“Your Chronotic Majesty” he said in a loud voice.

Both men turned to look at him. Philipium II appeared cold and supercilious, the emperor merely startled.

For one instant Aton looked into his ruler’s tired, feverish eyes and knew that his mission stood no chance of success. Behind those eyes was … nothing. The emperor was dead inside. There was nothing but bigotry, prejudice, set patterns of thought. Even if Aton were to persuade him of the truth of his story, which seemed unlikely, nothing at this stage could possibly cause him to alter his decision.

Aton glanced from him to the younger Philipium, and again from him to Arch-Cardinal Reamoir, who was hovering as always by the emperor’s side. As before he found that his new perceptions laid bare their inner natures. In Philipium II there was only a blind arrogance that was a sort of later version of his father’s unctuous religious humility. And in Reamoir there was ambition of truly shocking proportions: ambition that was prepared to sacrifice whole worlds, to cheat, lie, and kill in the pursuit of personal and religious aims.

He stood, tongue-tied and white faced, as the awful realization struck him.

“What is it, young man?” Philipium said sharply. “Who are you?”

“Captain Aton of the Third Time Fleet, Your Majesty.”

“Then you should be helping defend the frontier. On leave, are you? Why?”

“… The action for Gerread, Your Majesty,” Aton said after a momentary effort.

“Ah, yes. Take courage, young man. Eventually we shall regain Gerread, together with all the other possessions that have been lost since.”

An official slid through the circle and murmured something in the emperor’s ear, who then turned and began a conversation with someone else. No one took any notice of Aton. His rude intrusion had been forgotten.

The plot turns more often than the page turns, but this incident stuck with me to the extent that I was compelled to skim through the pages of the book to find it and then type it in. Probably because I know that if I barged into the Prime Minister’s audience one day with some vital insight, this is exactly how it would go.

I have a recollection of meeting Barrington Bayley in the summer of 1990 when I was part of the Cambridge University Science Fiction Society’s attempt at running the 11th University Science Fiction convention called Uniconze in the college formerly known as New Hall (before it was tastelessly renamed like a bride marrying someone rich).

I must admit that I didn’t understand a lot about what was going on, and I was tasked with being in charge of the cooking competition in which I had been instructed to grab Barrington Bayley, one of the far-too-numerous Guests of Honour at the convention (we’d made an infamous con-running mistake) to act as a judge. Don’t ask me to explain it further, because this and nothing else is inexplicably lodged in my memory. I distinctly remember having to lock everyone out of a classroom while I and this nice old man Mr Bayley surveyed a number of dishes, many of which had cakes on them, tasting them to determin which one was the winner at a Science Fiction convention! What the hell? It’s not like any of them were shaped like Captain Kirk’s ears or something. I have no idea why this was on the schedule. Costume-making I can understand, but cakes make no sense at all.

I had read nothing of Barrington Bayley’s fiction at the time, except for a rather pointless story in a recent issue of Interzone which involved a scene where some kids in the background were performing formation hang-glider flying up on a hill. I remember taking him to task for this, because I had just learned to hang-glide that year, and it seemed as ridiculous an idea as formation fishing. You fish where the fish are, and you glide where the wind currents go upwards, or you’ll be on the ground in short order.

While on the subject of me, and my triumphs of creativity, I also talked to him about my ZX Spectrum game Fat Worm Blows a Sparky written all in machine code with fancy solid 3D graphics, state of the art pushing the boundaries, whatever. (Francis Irving apparently bought a copy with his pocket money when he was very young. Oh dear.)

Barrington Bayley said he spent a couple of years dabbling in Z80 machine code himself for a time, being fascinated with what you could do with these amazing micro-processor commands, such as the LDIR command which was able to shift huge blocks of memory with one instruction through an internal loop cycle feature. He had to stop programming suddenly one day because he realized he was wasting too much time.

Aha, I said, that’s not how we do it in real programming. The LDIR shifts one byte at a time, which takes a certain number of processor cycles. What we do is relocate the stack pointer to the start of the memory block you want to shift the data to, and then push the words onto the stack 16 bits/two bytes at a time. Then, obviously, the whole loop is unrolled all the way through, because you’re shifting memory from the back-buffer onto the screen as fast as you can, and you don’t want to have too much flicker. Also, every 10th push-back in the program has an if-statement where you can optionally toggle the speaker bit in order to produce bursts of sound during this memory transfer process. What do you think of that?

Later in the year I moved to Bristol and discovered a second-hand book-shop on Gloucester Road that for some reason regularly stocked copies of the different old Barrington Bayley books over the next three years. It was then I discovered that he was a genius. And when I did find one of his old books on some rainy afternoon when I wasn’t bunking off from my Maths degree to go fly hang-gliders (badly), it really made my day.

I often dreamed of meeting him again, and conversing with him on the level of philosophies, plots, and Actual Ideas, which is the whole point of it for me. I’d bet he’d like my Mine the Primes story. Normally, when you get granted a conversation with a living author, they go on and on about their life, their travels, and their getting or not getting paid enough in their goddamn royalty cheques, and you kind of humour them because this is your one and only chance to meet with the man, and they could be talking to anyone else in the room, and you’re hoping they’ll get onto something deep and philosophically meaningful at some point, like the sense you get from their novels, and it’s just never there in the person, and you don’t know why, and no matter how much you push the conversation around and around to different things it just never gets there. You’d get more enlightenment going home and sitting in bed reading their books for the evening than going out to dinner with them.

This would not happen with Barrington Bayley, because he appears to believe in this calling (unlike most writers seem to), given the interviews he’s made. He’s done writing for money at times, but it just didn’t feel worthwhile. I’d probably have blown it again had I met him. Maybe creative programming is my life’s work. It would be nice if more people took an interest on the philosophical level.

Here’s Page One of The Soul of the Robot, which begins with the Frankenstein myth before moving on swiftly in a matter of a hundred words:

Out of pre-existence Jasperodus awoke to find himself in darkness

Seldom can a sentient being have known such presence of mind in the first few seconds of its life. Patiently, Jasperodus remained standing in the pitch-blackness and reviewed his situation, drawing upon the information that had been placed in his partially stocked memory before his birth.

He became aware that he stood unaided inside a closed metal cabinet. The first intelligent action of his existence was to grope forward with his right hand until he found the knob on the inside of the cabinet’s door. He turned and pushed. Then he stepped out to inspect the scene that met his eyes.

A man and a woman, well worn in years and dressed in smudged work smocks, stared at him shyly. They stood close to one another, like a couple who had grown old in each other’s company. The room smelled faintly of pine, in which the same wood were fashioned workbenches and other furniture: chairs, cupboards, a table and an assembly rack. Cluttered on these, as well as on the floor, the benches and on hooks, was a disorderly array of components together with the curious instruments betokening the trade of an electronics craftsman.

Although the room was untidy and somewhat shabby it had a warm, homely atmosphere. Its disorder was that of someone who had his own sense of method, and Jasperodus already knew how efficacious that method was.

His glance went back to the elderly couple. They, in turn, looked at him with expressions that tried desperately to mask their anxiety. They were gentle and blameless people, and in Jasperodus’ eyes rather pathetic since their eager expectations were doomed to disappointment.

“We are your parents,” the wife said in a hesitant, hopeful voice. “We made you. You are our son.”

She had no need to explain further, for Jasterodus knew the story: childless, and saddened by their childlessness, the couple had chosen this way of giving their lives issue. They looked to Jasperodus now to give them as much joy and comfort as would an organically born flesh-and-blood child.

But like many and ungrateful son, Jasperodus had already made his decision. He imagined better things for himself than to spend his life with them. Jasperodus, the hulking, bronze-black all-purpose robot they had made, laughed harshly and moved purposively across the room to the door. Opening it, he walked out of their lives.

Friday, November 21st, 2008 at 11:29 am - - Machining 1 Comment »

The algorithm for doing the constant scallop stepover in HSMWorks is far more robust and speedier than the one I wrote in Depocam/Machining Strategist. Unfortunately this means it gets pushed much harder in the application.

For a start, bugs are a function of processor cycles, not time. Back in the old days when I was writing these algorithms on a 50 MHz Pentium, the users would have to run it for an hour, or all weekend, to produce the metres of toolpath they needed to run their machine tools. We called it High Speed Machining, because they were running their machine tools several times faster than they used to, which would require several times the length of toolpath. They would get this by halving or quartering the gap between each cutting pass of the tool on the surface, so the resulting surface had smaller ridges (scallops? cusps? the terminology has never been standardized! what’s this tell you about the degree of communication between practitioners in the industry?).

In the old days people would (I’ve never seen it) hire someone on low pay to polish these ridges down for hours to get a surface smooth enough to use for a plastic mold.

So anyway, what worked as a pretty reliable algorithm back in 1993 would begin to crumble quite badly in 1998 when the PCs being shipped with the application were 10 or 20 times faster, and the users adjusted to it by tightening the stepover, running it on bigger parts, and just generally executing 10 or 20 times the processor cycles per work day. While we, in the programming team, understood the problem of the software having too many users sending in too many bugs for us to deal with, this speed inflation was equivalent to the user base expanding at an exponential rate, even when the customer base and the size of the programming team remained constant.

There’s always been this mysterious phenomenon we called bit rot when the code you thought was fine suddenly starts breaking and having lots of problems, and you didn’t know why because you hadn’t touched it since last year when you thought you’d finally made it perfect.

If my conjecture is right, this will cease to be an issue as the speed of processors more or less stabilizes or exceeds a threshold where the code is being sufficiently tested in the way that it wasn’t when computers were much much slower.

It also means that there is going to be a difference between software that was designed early on in the processor speed inflationary process (in the 1990s), versus stuff that was put together in the later days.

This is the opposite of my original thinking: that older software would be better than what we write now when the computers are much more powerful, because back then the code had to be efficient and well-made to get the job done, and nowadays with so many more processor cycles to waste we can afford to become slap-dash and inefficient.

This intuition was consistently confirmed by Microsoft who made their products and operating systems more and more bloated and inefficient with each passing year so that a one-page plain text document you saved on a 386 machine in 1992 now required 200Mb on-board RAM minimum just read it. It was a joke. The hardware manufacturers were only just able to keep up with the rate of the software deterioration. What kind of sloppiness and memory leaks does it take to waste 199.99Mb of RAM to load a 10Kb document? How could it be getting this bad? We thought there must have been a conspiracy between Intel and Microsoft during those years driving the continual need for upgrades just to stay the same.

So the story is that later-designed software is going to be better than early software because it will be more in tune with the modern hardware and the way it is used. The old stuff is going to feel like you have changed your bike chain without changing the gears — it’ll skip all over the place.

It’s a pleasant diversion. I have decided I’m not going to fix the bug illustrated in the picture for now — along with the dozens of other ones in the same algorithm — because I am too annoyed after a week of hard work.

What’s happened here is he’s detected the shallow areas of the model using a 10mm diameter ball, and then offsetted all these areas outwards along the surface by 1.5mm so that the contours have wrapped down over the edges. The point is to smooth them in case your shallow area contours are fragmented at the threshold point and you’d them to merge. So here is a case I hadn’t thought about where three areas overlap in the same vertical axis and two of the contours are joining up incorrectly with a double vertical line. (There are four instances of it in the picture.)

It’s notable that I hadn’t thought of this case when I designed the local connecting algorithm (sort of like the patented marching cubes algorithm, only much harder, so no one’s interested), because now I’m thinking “How many other cases did I forget to think about?” Are there going to be thousands more, or is it just this one?

There are thousands more, because you could have a piece of a spiral staircase without the central pillar.

Wednesday, November 19th, 2008 at 11:27 pm - - Machining, Whipping

To be fair, there was also a human interest angle in this “working life of a software engineer story” to make it media-noteworthy enough: the software engineer in question was deaf.

Now, there are many activities in the world which it is more challenging to do if you are deaf than if you are not, and programming computers is probably not one of them. Nor is playing badminton for that matter, even though there is a whole squad of competition players, vastly outnumbering the negligable membership of the deaf lug (linux user group) mailing list, for example.

What if it is the case that careers counselling, as it is experienced in our schools and universities, systematically steers students away from free open source software businesses, and into the traditional channel of vocational training, followed by proper old-fashioned employment in a real company that does things the capitalist way where they talk only about how much money they’re making from the start to the end of their annual report? The company directors not even slightly interested in anything else.

I’d be willing to posit that people with disabilities probably get more than their fair share of careers counselling, and this would result in an under-representation of perfectly agreeable deaf programmers in the open source sector, compared to their peer group who will have been allowed to muddle along a bit more at the start of their lives.

Generally career counsellors are going to want you to seek employment with respectable and astonishingly expensive education software suppliers with notoriously bad service, bogus security regimes, tightly closed software, and who combine this with the willingness to convert these major customer disadvantages into financial vitality that makes them qualified to apply through the government’s gratuitously complex procurement systems that at no time take any account of coding quality or long term vision (except the vision to make more money).

So it was that the Guardian’s photography correspondent, Leo Bendictus, interviewed Xander Hurley, who also features on the net as a Deaflymic Badminton champion. The article begins:

So … I pause, because this is a question I have dreaded asking. Erm … what does a software engineer actually do? Xander Hurley stares quizzically across the table. He is a long-limbed young man, with floppy blond hair and blue shirt. He has already explained how he meets clients, discusses with them what they need, and then goes out and makes it for them.

“Can I borrow your pad?” he asks. I slide it over, with a pen. “For example, if I was to create a calculator to add two numbers together.” He pauses, looking suddenly concerned. “I hope this doesn’t sound patronising?” Not at all, I assure him, and probably impossible. “OK. Bear in mind I’m using C#, which is a language, part of Microsoft .NET.” Consider it borne.

Hurley writes “value1 = 10” on my pad, followed by “value2 = 20”. “This is pseudocode, by the way,” he explains, “not actual code.” I nod fraudulently. “Now I want the code to take these two numbers and add them together.” He writes “int result = value1 + value2”. “Now, ‘int’ stands for integer. It’s a whole number. And that,” he points to the word “result”, “is a placeholder, which is like a bracket for holding numbers. It’s just a name. It could be anything.” He crosses out “result” and replaces it with “Xander”.

“So you add value1 to value2, and that gets assigned into there.” My pen hops about the page in his large hand. “So ‘Xander’, after doing this, will contain 30.” He looks up like a maths teacher who has just made everything clear, and sees, as maths teachers must often do, an expression of poorly simulated comprehension looking back at him. What I am still struggling to decide is whether software engineering is in fact much simpler than I had hitherto imagined. Or far, far more complicated.

Certainly what Hurley does with C# most of the time would be beyond me. He is a member of the data exchange team at RM, a large education software company, in the dowdy bowels of whose Oxfordshire headquarters we are sitting now. Here Hurley spends most of his days building computer programs to help schools manage their information. At the moment he is working on a way of enabling pupils and parents to access their own data themselves.

The reporter went on to explain how Hurley took six years to get through his software engineering course at university because of the difficulty lip-reading the lecturers from 20 metres away, and how he pulled out of exams at the last minute twice to avoid getting a bad marks in his final degree.

Now, I know a lot of programmers, and this is not how any of us learned our trade. Exams in software engineering the butt of jokes. When I used to work for a company and we had to hire somebody, we’d give them a one page coding quiz and that pretty much sorted everything out. Oddly, this was not standard practice in other companies, as though their chosen criteria for the job of being a programmer did not include the ability to programme. I flunked every single job I’ve ever applied for, and one thing I can do well is programme.

Programmers generally teach one another one on one. We practice for hours, sometimes all night long. We debug other people’s code. We search for answers on usenet. And, at the last resort, we RTFM. All this social activity over the decades has given rise to the open source community from which the whole world has benefited.

The ironic thing is that Hurley is permitted to talk about actual in the boundary of a newspaper story, but he works for RM in Oxford, one of the notoriously evil PFI contractors which doesn’t care one jot about code. Check out their web-page and any news. It’s all about money money money money money.

For contrast, you could inspect the website for moodle which is the leading competitor of RM for software to run schools. One click and they’re talking about the code, the features, what they’re trying to do with it, and how you can learn about it and use it. It’s all about friendship, not business. In a business, they occasionally take a break from the money and talk about people. Watch this video where some lobotomized zombies unfortunate enough to have been TUPED across to RM’s PFI business explain how they’re having a wonderful time doing exactly the same job as before for exactly the same pay and conditions while only costing the taxpayers only four times as much for a lot of senseless paperwork and stagnant technology.

The PFI system bundles up the publicly utilized IT infrastructure into a system of bogus shell companies and tax-efficient asset vehicles, such as Local Education Partnerships (LEP), using the same crooked logic of obscuring things illustrated with the sub-prime loan banking crisis. While the job of installing an operating system or wiring a network router is something that fundamentally anyone can do and is only ever going to get easier as the hardware and (open source) software technology improves, the result of packaging this work within the exclusive remit of one particular trade employee of one particular company for the next thirty years is only going to make things cost much more than they should. That’s what big profits are all about — the ability to invoice for much more than something is worth. It used to be that we’d laugh about how many union workers it took to change a lightbulb, because the union was only ever interested in preserving jobs. Now the question is going to be: how much money is it going to cost to change a lightbulb? The act of changing the lightbulb is going to be done perfectly efficiently, but you will pay. That’s because it’ll be declared as an Emergency and this will cause the need for Unprogrammed Maintenance.

It’s all there in the ICT Services Contract for Partnership for Schools:

8.8.8 If, as a result of an Emergency, the need arises for Unprogrammed Maintenance, the LEP may carry out such Unprogrammed Maintenance provided that the LEP informs a member of the senior management of the School(s) affected by the Unprogrammed Maintenance as soon as possible and the LEP notifies the Authority’s Representative as soon as possible (and in any event within five (5) Business Days of the occurrence of the Emergency) of the extent of the necessary Unprogrammed Maintenance and the reasons for such Unprogrammed Maintenance. The LEP shall take all reasonable steps to minimise the duration of, and any interference caused by, such Unprogrammed Maintenance.

8.8.9 The carrying out of Unprogrammed Maintenance shall not be construed as relieving the LEP from providing the ICT Services or as entitling the LEP to any relief from Deductions.

8.8.10 Notwithstanding that there has been no objection to a schedule of Programmed Maintenance submitted in accordance with clause 8.8.3 (Maintenance of the ICT Assets), the Authority’s Representative may, at any time, require the LEP to accelerate or defer any Programmed Maintenance by giving written notice to the LEP, (unless otherwise agreed) not less than forty (40) Business Days prior to the scheduled date for carrying out such Programmed Maintenance (where applicable, as accelerated), which notice shall set out the time and/or periods at or during which the Authority requires the Programmed Maintenance to be performed. The LEP shall, within ten (10) Business Days, notify the Authority of the amount of any additional reasonable costs which it will incur as a direct consequence of such acceleration or deferment (the Estimated Increased Maintenance Costs). The Authority shall, within a further period of ten (10) Business Days following receipt by the Authority of notification of the amount of the Estimated Increased Maintenance Costs, at its option, either confirm or withdraw its request to accelerate or defer the Schedule of Programmed Maintenance. If the Authority does not respond within this ten (10) Business Day period, the request shall be deemed to have been confirmed. The Authority shall reimburse the LEP the direct and reasonable costs actually incurred by the LEP as a consequence of such acceleration or deferment up to, but not exceeding, the amount of the Estimated Increased Maintenance Costs.

The Guardian article ends with a story of how Hurley was once called home by his mother to fix her computer and how he travelled two and a half hours only to discover that all he needed to do was turn on the plug. We smile at this comical situation, because it has often happened to us. It’s a weakness when humans and computers come into contact. The problem of the plug being turned off is common even to a desk lamp, but because there were so many other things that could go wrong with the computer it is easy to over-look it.

As with any other human weakness in our interaction with the modern world, our economic system is designed to reward entrepreneurs who prey on and exploit it to the maximum extent the law allows. We have a bad intuition with probabilities: mega-casinos. Our appetite is satisfied by cheap processed food that will kill us: McDonalds. We get addicted to chemicals and ape what the stars do in the movies: Tobacco. We are greedy to make lots of money for nothing: Ponzi schemes. (There are regulations outlawing overt versions of the Ponzi scheme because it is so damaging to the economy, but they didn’t catch up in time to prevent the current financial crisis.)

These vast PFI service contracts now gobbling up the public sector IT are merely institutionalizing the lack of knowledge at the current state of technology, locking it down as effectively and systematically as Coca-cola targets children, getting them hooked on the brand and the sugar while they’re young and impressionable. The fundamental purpose is to disempower people.

Think about it: any employee unfortunate to work for one of these companies is basically forbidden from talking to anyone about the “company secrets”. What are these company secrets? Well, they’re often on the face of them secrets kept from the customer about how they could do things themselves. It is not in the company’s commercial interest for these secrets to get out, or it will no longer be able to charge people for doing things for them.

This is fundamentally the opposite attitude to open source programming which is why it is doomed in the long run.

It would help, however, if newspaper reporters were able to stoop to the level of talking about what programming is all about with open source people for a change. But they won’t because it’s deemed too boring and geeky.

Unless it is framed in terms of making money.

Or it intersects with something worthy, like deaf culture. Hacker culture just doesn’t deserve respect.

Friday, November 14th, 2008 at 11:40 am - - Whipping 2 Comments »

I stayed up last night to scrape and parse the Merseyside Police Force helicopter logs using my new codewiki. (I’ll buy anyone a beer if they find an incident sillier than foot problem at 1:15am 2008-04-30.) The columns in the table are Date, Time, Area, Incident, and Outcome, and there was a typo about every other month in the format of the date or time, often with the delimeter ‘/’ or ‘.’ shifted over one place, but none whatsoever in the over-all table lay-out. This suggests that it’s the output from some sort of a spreadsheet whose entries are not verified. If it was a database you’d have used a DateTime field, and there would never be these sorts of errors in the info-dump.

This is good news, because it probably means they’re not doing graphical analysis and mash-ups of the data, so if I did it, it would be news.

For mash-ups you need GPS positions, and all I’ve got are district names. I can count how many occurances of each name string there are across the logs. Here’s the table of the top 50 Liverpool districts who have sometime been in the small hours of the morning by an incident known as “High visibility policing” whose outcome is “Public reassurance and the prevention and detection of crime” (I’m sure all the other call-outs were equally useful):

506 Huyton 208 Southport 126 Netherton 74 Aigburth 49 Birkdale
489 Kirkby 188 City Centre 119 Everton 69 Hoylake 48 Woolton
376 Bootle 179 Speke 117 Birkenhead 64 Whiston 47 Ainsdale
340 Norris Green 174 Aintree 110 Litherland 62 City centre 46 Rainford
338 St Helens 173 Toxteth 105 Fazakerley 59 Childwall 46 Garston
293 Anfield 161 Maghull 100 Halewood 58 Dingle 45 Haydock
259 Walton 147 Wavertree 98 Wallasey 58 Allerton 44 Netherley
247 Crosby 140 Formby 93 Kirkdale 57 Stockbridge Village 38 Vauxhall
229 Croxteth 137 West Derby 92 Old Swan 57 Seaforth 35 Dovecot
223 City 130 Kensington 88 Tuebrook 53 Prescot 34 Hightown

Converting these names into Lat-Lon locations is a tricky problem, but making them into links into Wikipedia pages as I have done above looks like it will lead to GPS coordinates of 90% of them, thanks to someone’s diligent work out there (I don’t know who). Unfortunately, Wikipedia resists webscraping, but you can download the whole thing and parse it yourself.

Now, I know what you liberal mash-up people are thinking: just do a map of the city and show how those nasty police are tending to buzz the poorer communities more than the wealthier ones.

Well that’s where the crime is, they’re going to say. Someone’s doing drugs? Send the helicopter up in the air to sort them out. What’s it going to do if they don’t happen to be sitting in an open park with no trees surrounding them? Shout at them from above the rooftops?

The compromises that have to be made to account for the physical reality of helicopter flight may mean that what is left-over from this great idea in the pursuit of community policing is of no use whatsoever. But who’s going to prove it when all that would result would be the embarrassment of senior officers and the curtailment of joy-rides throughout the Force? There are some good and hearty conflicts of interests when it comes to assessing the merits of certain pieces of expensive and potentially pointless kit.

Liverpool is a mixed-up in city with the “good” and “bad” areas very intermingled, and they change every couple of years as regeneration continues. I don’t think there’s much at a crude enough resolution for the police helicopter to be able to focus on certain communities who are going to be deprived of their sleep. We Brits put up with it, like busy roads or noisy airplanes.

Meanwhile, the rest of us can do it virtually. For example, Freesteel central (Martin’s house) can be seen at high resolution on the Microsoft bird’s eye view here (I think it’s one of the houses that still has its roof). No expensive and ridiculous glass and steel corporate work-office building floating in a sea of parked cars is necessary for this high-performance machining kernel to be developed. (I wonder which major CAM company will be the first to go to the wall in this recession when they stop making enough money to cover costs. We’re safe: we’ve never made any money in the first place anyway to afford having any expensive costs.)

My freezing cold stone house in St James Cemetery is here, or it can be viewed from above here. The google maps version tries to go for the same resolution, but they haven’t got it, so it’s blurry. I have heard that their street-level view camera car has been sighted earlier in the year.

Which gets us back to the police and their toy factory. Five years ago they installed dense network of CCTV cameras on very tall poles that block various pavements and really offended me at the time. But like most Brits, I’ve had to get used to it. The cameras have little rotating motors and windscreen wipers, and I have never seen them move, so I wonder if they’ve rusted into place by now.

I would suppose that the police should install a nice virtual reality city in their offices using all these google/microsoft/live-maps technologies so they can fly around our houses and dwellings in the dead of the night to their heart’s content without waking us up (or disturbing my latest programming session) to reassure us of their ability to prevent and detect crime from a distant but very noisy gravity-defying vantage point.

Actually, what this will develop into will be a hybrid virtual reality and virtual presence system for flying unmanned drones up and down the streets of the city. The technology that’s being developed and deployed large scale in Iraq, where the people are utterly expendable, will come like so many things to the war in our streets.

You haven’t got a war in your streets?

What do you mean you haven’t got a war in your streets?

Well, we’ll find you one. We’ve got goods to sell here. All it takes is a little bit of fear to get this nationally vital industry going. The mainstream media loves scare stories. We can work with them. Did you know that fear is also an essential ingredient for winning elections? Works almost every time, if you get the mix right. I think we can do business in this economy.

Have you seen how much fun tasers are?

And we would have gotten away with it too if it weren’t for those pesky kids who also have access to lots of cheap camera equipment and the ability to publish the photos in places for everyone to see, giving their own side of the story.

Who knows? Maybe it will be enough to save this generation from the kinds of crap that used to go down in the good old days.

Tuesday, November 11th, 2008 at 1:37 pm - - Whipping 6 Comments »

Background rant (definitely skip this)

The great British newsmedia, on which so much of the nation’s self-evaluation depends, is notable for its unimagination and lack of systematic coverage. It is outrageous that there has never been any notable coverage of leading projects such Open Street Map, TheyWorkForYou, or FarmSubsidy — let alone in enough depth for people to hear about the underlying politics which they represent.

The lowest point in this never-ending news black-out came when Jeremy Paxman found time to interview a very bad political blogger on his own ridiculous terms. But to this day Newsnight has never spared a minute for a representative of mySociety to come on and explain why we exist, in spite of 5 years of sustained technical work to create the most user friendly and serious political website in the land without any cooperation from the inside.

One of the (many) reasons it is in the public interest for these projects to get some publicity is so that professors who need to know about them find out about them and value the data.

The Parliamentary transcripts (Hansard), which have been processed into usable form for the TheyWorkForYou project back to 2001, are interesting because they contain a near-verbatim record of the spoken words for a formal group of people covering long periods of their lives and the life of the nation.

Several years ago I wondered if a certain local university professor would be interested in using this data for running it through their fancy-schmancy grammatical analyzers and doing some research. Perhaps he’d be able to help fund us to parse and prepare the rest of the on-line data that goes back to 1988 and covers all kinds of legislation that was put in place then and affects us profoundly to this day. (Train privatization anyone?) It wouldn’t take much — just a sign that somebody out there actually cared.

So I visited Prof. Paul Watry (of the National Centre for Text Mining) across the road, spoke to him for an hour, met the programmers in the basement, gave them the links to download the Hansard data, and nothing came of it. I tried to download and run their software, but I wasn’t clever enough and didn’t get any help. Being an unpaid volunteer who has to earn money elsewhere, my time in this field is considerably more precious than those who are employed full time job with all the resources — even though our culture rates things the other way round: I am supposed to do all the running, and it’s entirely my fault if there’s no follow-up.

Had this project been shown on TV it would have probably been more interesting to the Professor. Money and celebrity are all that attracts attention these days, not technical results, even to people who should know better.


OCR-ing for Alzheimer’s (skip to here)

I was woken at 8:50am this morning by an interview on the Today Program with Dr Peter Garrard who was reporting some text mining from Hansard in an attempt to diagnose the early onset of Alzheimer’s disease in Harold Wilson and explain his sudden resignation in 1976.

It’s very early research, little more than a conjecture with no control studies — just the sort of half-baked science that the BBC likes to make prime-time.

The companion piece on the BBC website is here:

The study, published online in the Journal of Neurolinguistics, converted Hansard transcripts to digital format using optical character recognition software.

This allowed Dr Garrard to use markers to compare Wilson’s speech patterns, and the number of times he used certain words, with those of his parliamentary colleagues.

The analysis was based on techniques developed by literary scholars for quantifying the stylistic similarities and differences between authors, genres, and literary eras.

The findings showed that the content of Wilson’s speeches was identifiably different from those of other members of the House throughout his career as prime minister.

However, the difference was smaller during the months leading up to his resignation – a sign that he was losing his distinctive voice.

Dr Garrard said that could be a sign of the earliest stages of Alzheimer’s.

The paper is not available on the public internet. It’s been deposited here behind the eternally evil Elsevier’s pay-wall, so only people who work for institutions that pay the steep license fee for their dreadful web-interface have the right to access it.

This has been a long-running issue. In this country we privatize the dissemination science just like we privatize the law. Public money pays for the universities, the research grants, and the salaries of the professors who then write up their work in scientific papers and sign over their entire copyright for free to a no-value-added publisher like Elsevier following the necessary peer-reviewing by other professors (who don’t get paid), so it can be kept archived and inaccessible, and the universities have to buy it back at great expense.

Professors, just like those powerful lawyers in the legal system, don’t notice the problem, because the secretaries and lowly librarians deal with the issue of buying the materials back, and they don’t notice that no one else has access to this paid-for information (and anyway they don’t care because only Professors need to see these documents, and they all can).

Senior professors tend to have been brought up in an era when publishers actually did something expensive and skillful, like typesetting. They don’t notice that they and their new PCs now do this entire job themselves, and all the publisher does is run a web-server with an on-demand printing system while cashing in obscene profits and recycling some of the money back into political influence where it fights against reform of the system, for example by filling the Science Minister’s brain with the most bogus argument against Open Access publishing I have ever seen:

Lord Sainsbury of Turville: What the government does not think is right to do is to promote one model, open-access publishing, in the marketplace. It is not clear that on a like-for-like basis open-access publishing will have a lower cost base, and as it will transfer some of the payments from industry users to the authors, it is likely to lead to higher costs for universities and research institutes. Also, because Britain produces 5.3 per cent of articles in the world’s science journals while accounting for only 3.5 per cent of subscriptions, we would also lose out as a country. (2004-11-10)

Dr Garrard needs to pop over to his colleague Steven Harnad who has been banging away on this since about 1991, and has established the precedent that academics should upload their pre-press (no so-called publisher’s “added value” yet included) articles onto their public websites without any legal consequences.

Garrard’s research on evidence for Alzheimer’s disease in written text was first reported all over the place back in 2004 when he published a paper analysing the language of Iris Murdoch’s later novels. The old dear had just famously died, movies were being made about her illness, and this kind of thing piqued the literary interest of the old British media.

So it’s no surprise they picked up his similar research done four years later on Harold Wilson for the same deal.

Senile authors, senile parliamentarians: lots of text to use. Maybe when we invent life-blogging with microphones pinned to our throats containing reliable speech recognition systems that transcribe our entire spoken corpus from birth to death on a 100Gig memory card the size of a pin-head, it’ll be applicable to the rest of us.

Meanwhile, there are a whole lot more interesting questions to be finding out from this amazing corpus of text data, than looking for boring fragments of senility that don’t give us any clues about where we are going as a people or a nation. Everyone knew that Ronald Reagan was senile. Everyone knew that Tony Blair was a raving liar. The way that the political system is able to ignore and/or use these facts to its advantage its advantage group-dynamic-wise needs some serious looking into by thoughtful scholars. But who’s going to go there when our imagination is capped by narrow uninteresting questions that assume honourability and people trying to do their best. It’s as irrelevant as picking on George Bush’s speech impediments.

From Dr Garrard’s Journal of Neurolinguistics paper:

Transcripts of Prime Ministers Questions that were held while Harold Wilson was Prime Minister (ie firstly between October 1964 and June 1970 and secondly between March 1974 and April 1976) were obtained and converted to ASCII format using optical character recognition software. Markers were added to identify the date at each change of year and month, while the identity and party affiliation of every speaker was recorded at the beginning of any speech or contribution to debate.

The troubling thing, before I got into this whole rant, was that Parliament has already paid to have all the old transcripts scanned and OCRed, and people I know have been putting it online here. And here’s their development blog. Goddammit, if we can’t get these things together, what hope is there?

The Telegraph has the story from a yesterday too:

Last week the Medical Research Council awarded a research grant to Dr Garrard and Dr Celeste de Jager, a neuropsychologist at the University of Oxford.

The research team will collect and examine a large database of spoken and written language samples collected over the past twenty years as part of the Oxford Project to Investigate Memory and Aging.

Here’s the fancy website of the Medical Research Council. All kinds of stuff there to distract you from the fact that their one and only mission is the delivery of scientific grants to researchers.

I leave it as a difficult exercise for the reader to navigate through the website to the list of grants given.

What would make me really happy would be if the fruits of the vast work done into text mining software by academics were installed onto the TheyWorkForYou servers and ran live so that the results showed up in the Numerology section where it belongs.

Connect scholars to activists not via Elsevier’s dead tree vaults. Progress might then occur.

Saturday, November 8th, 2008 at 4:06 pm - - Whipping

Angus Macleod in The Times has the story:

Labour’s private Glenrothes poll had shown SNP heading for defeat

The Prime Minister was privy to the results of a mass canvass completed towards the start of last week which showed that voters who had previously said they were “don’t knows” were heading for the Labour camp by a margin of three to one as Thursday’s polling day approached.

The results of that canvass were kept to a group of five party top brass… They made a pact to tell no one else and, other than to say that the result would be “close”, were content to sit back and watch as the premature claims of victory from Alex Salmond and the SNP grew louder and louder, media pundits busied themselves estimating the Nationalist majority…

[So what went wrong?] It was, as the SNP alleged, negative campaigning. But as past masters at such tactics the party should not have been surprised that it became the dominant issue.

It tried hard to answer the Labour accusation that the council was forcing the vulnerable elderly to pay unreasonable amounts for items such as home alarms but never managed to neutralise the Labour propaganda.

…While the campaign concentrated almost exclusively on local concerns, it was played out to the doom-laden background of the global financial crisis and Mr Brown’s role in handling the fall-out.

Fifers are notoriously clannish and took pride that one of their own was being praised around the world for his role.

Meanwhile, back in media Pundit and planted-story land, Andrew Porter writes in The Telegraph:

When the polls closed on Thursday night the Prime Minister had just been told that Labour was likely to lose the seat it had held at the last election with a comfortable 10,000 majority. Mr Brown went to bed thinking Labour had lost.

An ally of the Prime Minister confided to another: “He will take it badly. He always does.”

Instead, in the early hours, the Scottish political narrative was spectacularly spun on its axis. Labour had secured a barely believable majority of 6,737.

But has it changed the political story across the wider canvass of UK politics?

There were some obvious reasons why Labour won. They fought as the opposition to the incumbent SNP. The unpopular policy of care home fees under the ruling SNP Scottish executive was remorselessly played by Labour.

And today the Guardian is literally wetting itself:

  • Brown hails Glenrothes triumph as vote of confidence in economic strategyThe SNP spent the four-week campaign ahead in the polls,[What Polls???]
  • Euphoria at No 10 greets stunning Labour win in seat it had written offDouglas Alexander, the Labour election coordinator – who in Bahrain on Thursday night had received a gloomy forecast of the result from the Glenrothes count as the polls closed – argued yesterday: “This has shown Gordon was right to argue that he had to focus on taking people through this downturn, and it has shown, above all, voters recognise that it is progressives that have the answers.”
  • Where did Labour get it right?Andy Sparrow This was the first byelection since the global banking system came close to meltdown. But in the event local issues seemed to resonate more than national or international concerns. Labour selected a prominent Fife headteacher, Lindsay Roy, as its candidate and focused relentlessly on the alleged failings of the local council, which since last year has been run by the Scottish National party. Home care charges, which for some users have risen dramatically, became the key issue. Although only a few hundred people are apparently being asked to pay under Fife’s new means-testing regime, the charges aroused considerable opposition to the doorstep.
  • A good night for Labour – phew!But coming so soon after some really bad byelections this result may also come to be seen as an indicator that the outcome of the next general election isn’t the forgone conclusion many thought.
  • A bounce in the backyardThe result was not just a surprise but a stunner – one for the record books. Glenrothes is the first Labour byelection victory in a Labour-held seat since 1997 in which the party increased its share of the vote compared with the preceding general election.
  • Sarah Brown: the real reason Labour won in Glenrothes?I consulted a senior Downing Street staffer, a cabinet minister who’d recently visited the seat and a couple of party staffers yesterday, all of whom predicted a narrow SNP win — and no, I dont think for once it was expectation management.
  • and so on…

It’s not as though the accurate story didn’t briefly get out, before this fake narrative became dominant. Back in October 23 reporter Andy Sparrow wrote for the Guardian: Gordon Brown ‘has been told Labour will win Glenrothes byelection’.

One should never underestimate the mendaciousness canniness of leading politicians in this world. They get there because they know how to work the electoral system with all its flaws. They will do any amount of misleading the public in order to get their vote. This skill does not necessarily translate into an ability for competant governance in the public interest, however. If it did, the entirety of Western democracy wouldn’t spend so much time in the sh*thole, to be honest.

Having trialed a set of tactics in this by-election:

  • getting the family into the campaigning
  • claiming to be competently solving a financial crisis you absolutely didn’t foresee and which is widely viewed as a consequence of deregulation, expressed in its most extreme form with the Abolition of Parliament Bill
  • targeting micro-issues, such as council charges to a small number of old people, that are of no relevance to the office that is being elected
  • misleading the media (and therefore the public) about facts — knowing that they are too stupid, disinterest and under-resourced to find the evidence to correct it

… we can expect to see a heavy amount of this crap rolling out over the next two years. Sanity doesn’t stand a chance.

The overwhelming majority of the public’s education about politics and governance comes from leading politicians, channelled through the media. No wonder we don’t know anything and don’t have a clue. Fighting this is like trying to sell athiesm in the time of the dark ages.

Friday, November 7th, 2008 at 12:34 pm - - Whipping

And so the Labour Party candidate has “unexpectedly” won Glenrothes — with an increased share of the vote (now at 55%) — and with the only difference from the 2005 election being that all the LibDem and Conservative voters switched to the SNP. Well done, folks. You have elected a new MP, Linsay Roy, who’s action plan is:

  • 1. Crack down on anti-social behavoir
  • 2. Fight for more opportunities for young people
  • 3. Sort out the roads and busses
  • 4. Help Fife families through tough times

We can count on the new MP voting for lots of horrible things sight-unseen for the next decade, owing to the fact that he was essentially chosen by the party machine for being a popular head-teacher, who probably has an average understanding about what goes on in Westminster (ie barely any), and will do exactly what he’s told. There has been no report of anyone asking him about where he stands on the controversial issues which have been so challenging to those few of us who have bothered to pay any attention to the ingredients as they are listed on the Bill for materials in Westminster.

Having just now reviewed the Guardian coverage of the campaign, as well as perused the BBC’s less easily navigated reports, one thing is obvious from this “upset” (after many pundits had predicted an SNP victory):

No public polling had been done

This is a complete disgrace. Clearly, the professionally run Labour Party with all its seasoned strategists from overseas and PM (Public Manipulation) companies must have run some polls and knew they were going to win. Polls are quite expensive to run, but are absolutely crucial if don’t want to fly blind.

Meanwhile, our dozy press corps (and this includes the BBC) have been relying on the betting odds down at Ladbrokes, knowing these are bogus on account of the fact that they have even reported the SNP sending out emails to members to place £10 bets on them winning. Not one poll has been taken where the results have been published, therefore giving this critical advantage to the Labour Party machine who can deliberately lie play down expectations to the gullible press and the misinformed public.

Actually, I have found two articles which are almost poll-taking. There’s this one from the BBC on 14 August 2008 which gives Two Labour interviewees to One SNP; and this one published in the Guardian on 5 November (the day before the election) where the reporter stood in the Kingdom Shopping Centre and reported incredulously:

“I spoke properly to about 20 people who could be voting tomorrow. The sample was so random that I don’t think the figures (SNP 3, Labour 6, Conservatives 2 and the rest don’t knows or wouldn’t says) mean anything.”

Two in-depth mood pieces by reporters covering the campaign (without any polling information) give a flavour of the issues which were used. As I observed, the SNP were hitting almost entirely on fossil fuel prices like there was no tomorrow (which there won’t be if this carries on). And the Labour Party campain was hammering on and on at a some insignificant and misreported imposition of charges in some home care services by the SNP-led local council (the leader of which was the SNP candidate). The Guardian outlined this too. The BBC did a tiny bit of fact-correcting in its article, with:

A retired ex-miner added: “They will be taking away free prescriptions next – I can see it coming.’ (The SNP, for the record, plan to extend the scope of free prescriptions, not restrict them.)

But in the long run this is hopeless. We, in Britain, don’t have the slightest understanding for how money adds up. That’s why supermarket managers can add one penny onto every single product in the shop, take 10p off the can of beans, and we think they whole place has suddenly become better value. We work by instinct, which is utterly flawed when it comes to mathematics, and the press who are similarly mathematically illiterate don’t bother to do the work to put us right. Something is sucking. It’s like an allergic reaction and you don’t know what food it is, and it’s going to take months of experimentation to work it out. The political establishment don’t want us to find out why we keep getting what we don’t want.

What are the results from the leaflet drop? Well, I can identify two instances where the URL must have been copied off the page and used on my website on the Sunday we were there: At 9:58 in Markinch, and at 12:10 near the Leslie Roundabout. That’s looks like it.

It’s not like people are unhappy with doing things on-line, because the BBC’s Have Your Say website about the by-election already has over 600 comments in just a few hours. That’s the power of the official media. It’s the default place where everyone gets their thoughts and ideas. This is why when they mis-report things — which they do — it’s so damaging. The mis-reporting here is that Brown has had an amazing bounce-back victory on his handling of the financial crisis, etc, when it was in fact all there ever was was a bunch of reliable Labour voters far away from Westminster with no interest in politics whatsoever voting for their local boy, Gordon Brown.

Until people who care start to get out of the policy issues and work out what is happening in the electoral system, it’s going to be hopeless. This is the place where there is the only source of a countervailing power against the ownership of the country by neo-conservative war-makers, arms dealers, oil companies, endlessly greedy bankers who don’t care about the consequences, and other corporate horror-shows. We need to find out why the House of Commons is continually being stuffed with turkeys who keep voting for Christmas week after week, and find a way to stop it.

Today is two days after the United States has elected a president who’s actually intelligent for the first time in ages, and who appears to be somewhat aware and concerned of the awesome train wreck that we are heading towards as a species. Sure, everyone can see a crisis when it is happening. Where’s the heck is the chance that things will get done before it gets bad when it’s possible to avoid the disaster, but at a time when special interests find it both in their interest and their power to suppress it?

Tuesday, November 4th, 2008 at 12:53 pm - - Whipping 1 Comment »

This article by Iain Macwhirter “the award-winning political commentator for the Sunday Herald and Herald who has also been a presenter of BBC political television programmes from Westminster and Holyrood for nearly 20 years” is based on 100% unmitigated and total bollocks:

Labour has put up a good fight in Glenrothes, but I don’t think they’ve done enough yet. They’ve hit the nationalists hard on the economy and council cuts. But the problem with Labour’s campaign, as with Glasgow East, is the sheer lack of bodies on the ground. No regal visits from Gordon Brown and the first lady, Sarah, can make up for activists knocking on doors. At the weekend, the nationalists piled 1,200 volunteers onto the streets and put leaflets through every one of the 40,000 homes in the constituency. You just can’t answer that kind of effort.

I was there all Sunday!

No goddamn wonder this country’s democracy is broken, with this news system just making stuff up to fit the chosen narrative. Any pro-democracy on-line project that could slowly be occurring, but is however inconsistent with the programme that these brain-dead zombies grew up with 20 years ago, will not be covered in favour of fabricated usual old horse-shit.

No wonder we aren’t getting anywhere. At one time we thought that attention would come if only we passed good enough work that no one else could do and which arguably addressed a systemic problem in the structure democratic engagement.

But now we realize that the only hope is to wait for this older generation of useless hacks to simply die off like so many incumbent MPs entrenched in safe seats in the absence of a system of primary elections.