Freesteel Blog » 2008 » September

Monday, September 15th, 2008 at 11:53 am - - University, Whipping 1 Comment »

In spite of many incoming links to this blog from this trivial post, Gary (the unnamed owner of that very-new-but-already-much-more-popular-than-this blog) didn’t make it to Pycon 2008, so I went there knowing only one person.

I’d gone to Cambridge (to hack on Tunnel) using someone else’s return train ticket to London and didn’t want to get screwed by a single train ticket back to Liverpool (prices hiked by another £20 last week), so caught the bus which involved a 40 minute stay in a layby in Milton Keynes where I could see the great avenues of horse chestnut trees dying due to global warming (the mild winters not killing off the parasites). Apparently, this isn’t news, though it was news to me, and I wonder how it connects to the business of planting trees to offset a gratuitous jet-set flight across the Atlantic for a handful of hours at a conference when it’s about time someone started experimenting with telepresentations.

I checked in at Birmingham Central Backpackers and spent half an hour walking around in the rain unable to find their second bunkhouse because it was in a converted pub which still looked exactly like a pub, so I walked past it.

The talk schedule was so-so. But some of the talks turned out to be very good and important, though I couldn’t tell from the titles. The keynote speakers were Mark Shuttleworth, who seemed kind of ordinary, and Ted Leung, whom I would have recognized as being very important if he had phoned his talk in from Sun headquarters on a big screen, instead of being flown over bodily, as is the standard, unquestioned, routine practice at geek conferences that are supposedly tuned in to the future of the internet and planet. Readers of this blog will know that this is a big issue for me, and I don’t want to hear arguments about how presenting in person is better than doing it as a giant video conference, because I Have Never Yet Seen It Attempted.

I quickly wrote up and scheduled a lightning talk about the Metroscope project and cunningly inserted it into the half hour slot before Mark Shuttleworth’s speech, making sure to mention my wholly unsupported but very farsighted work on undemocracy.com, (where you can find some of the international documents establishing the Registration Convention, among other things). The video channel failed to work when I tried to display my slides from someone’s machine which had recently been installed with Ubunto, so I was lucky enough to move my slot on by ten minutes and get them to show up on a Windows machine donated by someone in the audience.

undemocracy.com still has no supporters whatsoever, but a lot of people could see the point of the Metroscope. I have developed the notion that scrapers and parsers for these tiny sources of civic data should be run using a codewiki that runs in a sandbox using pypy. I have not found such a sandbox on-line. trypython.org is not it, because it’s python running in your browser with the right plugins. Who wants to work on it? I must get a new laptop urgently as I my old one is really beginning to hamper this work.

The conference dinner which came with free wine and beer was excellent. The conference hangover the next day was not so good. I got back to the hostel at 3:30am, got up at 8am, sat on the road for 20 minutes until they opened the door and I could have breakfast of lots of tea with peanut butter and banana sandwiches. Still felt terrible. I needed to find a talk that was boring enough to fall asleep in, but failed because instead I went to one on software complexity where the speaker made us all stand up and talk in pairs to our neighbours about the problems we’ve had with software complexity.

The second keynote speech (the Leung one) was on the same theme as Mark Shuttleworth’s, though in greater detail. These two speakers saw their duty as trying to drive the direction of this herd of cats known as the python programming community. Big changes are happening in computing and the language has to get out of its box to make itself relevant so it doesn’t get eclipsed by, for example, Javascript. This is the greatest threat. Javascript captured the browser and now there are ten times more Javascript programmers as Python programmers. Javascript has also been stealing a lot of ideas from Python as well. The catastrophe will be when people start using Javascript on the server and displacing Python from there. (I lost the argument with HSMWorks that postprocessors should be in python — they chose instead to use Javascript.) The place to look is at the weaknesses of the language, and don’t accept them as part of the design.

The issues are multicore processors, performance limitations, no clear recommended web-framework, and being able to run on the browser platform — these are things that Java (do not confuse with Javascript) dominates. Regarding the web-framework, Django is seen with a lot of hope and optimism. I have to give up going to talks about pylons, as I’m just not getting it. I have resolved to try Jython, which is Python running on a Java virtual machine, once I get a new computer with some space in which to install things.

To me, the most important talk was about pro bono work converting climate science software to python from its old, incredibly impenetrable FORTRAN implementations. You could guess it would be them from reading their company’s heartful goals. Far too little pro bono programming happens outside of the field. A huge amount of volunteer work is done by programmers to improve and promote the public free open source software tools used by programmers (eg Python), but the phenomenon rarely breaks out of this arena and into dealing with civic and government software and data. I don’t know why. It’s considerably easier as the quality can be so ultra-bad outside the open source communities, especially when it relates to billion dollar government contracts. But, on the other hand, nobody out there seems to give a damn about software quality, so you for sure won’t get any thanks. It’s like writing poetry in a language nobody understands. The belief that good quality software is a gift that keeps on giving is nowhere around. Consequently, there’s a disincentive to produce it in the professional world.

Next year there is no UK Pycon. Instead they are holding EuroPython. This is a very good thing and everybody should go.

Wednesday, September 3rd, 2008 at 3:31 pm - - Whipping 6 Comments »

In the current issue of Private Eye No. 1217 (not available on-line to take advantage of time limitations for libel proceedings when information is printed on impermanent paper which isn’t vellum) says:

Another multi-million pound government IT cock-up is quietly playing itself out — this time in the courts service.

Back in April, the lord chief justice, Lord Phillips, released his first review of how the courts are run. Buried inside was a criticism of HM Courts Service (HMCS) and its inability to provide an electronic document-submission system for the new Commercial Court by 2010, when it becomes the Business Court.

(The current Lord Chief Justice, Lord Phillips, is not to be confused with the former Lord Chief Justice, Lord Woolf, who moved on to more challenging appointments such as whitewashing our greatest weapons company by uttering the words “BAE Systems” and “ethics” in the same sentence.)

Now, it could be seen as a disappointment if your new fancy shmancy hi-tech modern world-class Business Court had to invent and run an entire paper shuffling system for the first two years of its life as it passes rulings on the finer aspects of software patents, but that’s what’s going to happen. Anyway, why would the very businesses who make so much money out of the government want an effective court system which could be used against them, eh?

Given my knowledge of the FOI archive, I wondered how hard and expensive could it be to build a document management system? What’s it going to look like when it is eventually rolled out (following the transfer of so much public money to the usual corporations)? Will it contain any great software modules that can be reused by the FOI archive? Are there going to be a compatible open standards with which we will be able to hook in with our outside interfaces?

I located this article in the Law Gazette which gives details of an FOI request about the Commercial Court (CCIT) project and the Electronic Filing and Document Management (EFDM) project.

CCIT has costed £1.47million, with nothing to show for it. The EFDM, however provides the following breakdown:

from to for consultants internal
2005-08 2005-12 Feasibility Study £136,300 £52,308
2006-01 2006-06 Programme Initiation £470,948 £73,601
2006-07 2007-06 Programme Definition £860,453 £377,483
2007-07 2008-03 EFDM £815,813 £454,022

The feasability study, commissioned in April 2005 and finished in December 2005, was produced by PA Consultants who “draw on the knowledge and experience of 3,000 people, whose skills extend from the initial generation of ideas, insights, solutions and new technology, all the way through to detailed implementation.”

Unfortunately, they didn’t draw on the knowledge of their consultants Nuala Moran [update: who is an FT reporter, not a PA consultant] or Peter Houppermans [update: see his comments below] and his Open Source knowledge group, because you will find not the slightest consideration of Free Open Source Software for the job. This is the UK Government, after all. You don’t even need to rule it out.

The feasibility study is worth combing over in detail because lays out the full horror of their plans within the DISC framework on page 25, which includes a full year of programme initiation, definition and full business case wherein the DISC supplier is put in place before talks begin, and loads of money gets made without any risk of a job getting done — which is always the best tactic for a contractors looking for long-term employment.

Now, you can see the point of treading carefully in the cases that you don’t want to add just another computer system into the mass of computer systems already in place, but this feasibility “study” doesn’t actually propose any ideas. All it says is it’s feasible to take more than a year to come up with some ideas.

From page 22:

* There is a commercial market and the commercial approach would fit with HMCS (HM Court Service) procurement rules
* Viable commercial models exist and have been shown to be robust and workable.
…but there are issues to address going forward:
* No particular procurement route is recommended at this stage, although the study had not seen a successful multiple vendor approach
* The emerging delivery options would need to be tested on the market before making a final decision
* The actions HMCS would need to consider the likely impact of its requirements upon commercial viability
* necessary to mitigate the risk of a monopoly supplier.

So, anyway, for this brilliant piece of work (Expert development of the EFDM business case), PA Consultants received £373,700 according to a Parliamentary Written Answer.

To clarify where we are at, the consultants have been paid over £2million for two years of seemingly unproductive work. According to the Law Gazette FOI,

Logica have just issued a prospectus and pre qualification questionnaire (enclosed) to the market which sets out our current assumptions. These assumptions include a high level timetable showing the implementation approach as well as the procurement timetable.

Since the Law Gazette chose not to use the FOI document management system of WhatDoTheyKnow.com, we’re going to have to ask for them again. However,

Documentation over and above the EFDM Business Prospectus and Pre Qualification Questionnaire that refers to potential further work within the CCIT project and the wider EFDM programme contains commercially sensitive information regarding business partners and sub contractors.

Disclosure of the information in a manner which fails to protect the interests and relationships arising in a commercial context could have the effect of discouraging the companies from dealing with the Department because of fears that disclosure of the information could damage them commercially particularly as the tendering process is still ongoing.

This could in turn deter companies from supplying commercially sensitive information, and ultimately undermine the ability of the Department to carry out a successful procurement exercise, adversely affecting the IT projects and any future projects. Whilst the tender exercise is still being carried out, the public interest is best satisfied by maintaining confidentiality in order to ensure the integrity of the process. The information is therefore exempt under Section 43 of the Act.

Oh, yeah, another important thing from that feasibility study:

The large legal firms interviewed would have no difficulty linking into or using EFDM.

Presumably because they’ll all be buying into the same closed software, and will happily be able to bill their clients for the full inflated costs whilst at the same time not providing them or the public with free open access to the official documentation, thus maintaining complete control for the guild.

So that’s where it’s at. An FOI request has been made, including a request for:

Any reports, documents or explanations of what was received from consultants and contractors who billed up to £2,147,214 between the publication of the January 2006 Feasibility Study and the initiation of this new tendering process. This would include any samples of partly complete ‘programme definitions’ that future contractors could use for technical referral.

The wording of my request is crucial, because if they’re going to cite commercial confidentiality, it’ll have to be within the bounds of the law. Public embarrassment resulting from releasing the answer and therefore detering such well-qualified consultants from applying for yet more work is not grounds for an exemption.

Commercially sensitive means there has to be commercial damage from commercial competitors obtaining this commercial information. Clearly, if one batch of consultants gets sacked and a different outfit of consultants are called in to pick up the pieces, they are in no position to argue that it’s commercially sensitive information which they have a right to withhold.

For now, the first step is to force these consultants who have been hired into the hollow shell that is our government policymaking framework to answer the case for open source development, rather than simply getting away with pretending that it does not even exist.

It’s easy. They have to start with this handy catalogue of open source CMSs and explain why none of them could be made good enough by the sinking of £2million into it.

After all if Stephen Fry can celebrate the 25th birthday of GNU, then there is hope that this country might eventually get it.

Monday, September 1st, 2008 at 4:54 pm - - Weekends

The following were intended to be included in last months blog post, but we had a bad version of wordpress installed that couldn’t upload photos.

I put two lights under the suspended floor of ice at the bottom of Tunnockschaft and used a 4 second exposure.

Monday, September 1st, 2008 at 4:47 pm - - UN, Whipping

Webcasts are cheap if you happen already to be filming the procedure and recording the live translations being provided (both of which are rather expensive). They are usually posted onto the un.org website long before the meeting transcripts come through.

Unfortunately they poorly archived and not linked with the written transcripts (which you can’t link to anyway) or to the documents they depend on, and there’s no good way to dip in.

The latest archive pages for the General Assembly are here and the Security Council are here. The latter includes Media Stakeouts where, ideally, the ambassador gets to rendezvous with an open microphone following his theatrical storming out from an important meeting.

Now I wanted to make links directly to these pages from the parsed transcripts on undemocracy.com, but these enormous tables don’t have any anchor tags, so I can’t link directly to the correct place. The feedback form seems to reject messages of longer than 100 characters, so I hope I got something through. Not that I expect anyone in the UN offices to take any notice, like anyone there has taken any notice all ready. I wonder how many hits they get on these webcasts in total anyway.

So in the meantime I’ve skinned these index pages by loading them into the server and writing them out again with id’s on each line corresponding to the dates. Subject to a few annoyances, such as the index pages going: sc2003.html, sc2004.htm, sc2005.html, sc2006.html, sc2007.html, sc.html, and the month names Janurary, Febuary, and Septmeber, I’ve got it working, so you can have the pleasure of dipping into a General Assembly meeting, such as this one from last December.

Click on the Webcast Video link in the top right and it takes you to the corresponding point in the skinned index. When you get bored, skip through to minute 18 of the webcast to see some voting as well as Mrs. Mladineo banging her strange hooked stick.

Security Council meetings are also very scripted and dull (except for the last couple ones on Georgia). However this one on Western Sahara has a rare display of dissent. Costa Rica pipes up. And so does the South African ambassador in minute 12 (you have to skip past all the media stakeouts) complaining about behavior the Group of Friends (the Permanent Members) who script the show in advance too much.

Anyone who is missing Tony Blair and his interminable rants about terrorism, “this ghastly game with our conscience… we must never forget that the events of 11 September 2001″ can get their fix from here (45 minutes into the webcast). That was a man obsessed. His name is now mentioned monthly in the Security Council by those expressing appreciation for his work as The Quartet representative, although he has never showed up to read a personal report. Otherwise we could see him in video as he is now without needing to be praying customers at the Yale Schools of Divinity and Management.