The Government is beginning to support its unique citizen-grown movement of… making new web-pages of data that they have managed to present so badly that any kid who knows javascript could do better at.

As reported on the BBC (with practically invisible links to the web-page the story is about hidden in the right hand side panel — rather than in the text of the article), Tom Watson, the first government minister to understand what a blog is for, announces a whole £20,000, yes, that’s a

“to develop the best ideas to the next level”.

Now, I’ve got the sort of track record of winning prizes and influencing people that means my effort is far better spent elsewhere on projects that are already done yet continue to be ignored. Someone who knows he’s cool can use the www.showusabetterwayentryform.co.uk. (can you believe it! Yet another URL domain here — what’s wrong with using the cabinetoffice.gov.uk domain for everything, including for that new powerlessinformation.wordpress.com blog?).

So while I resist the temptation of filling in their entry-form, I can certainly pass comments about the recommended information sources.

The first site is the neighbourhood stats from the Office of National Statistics. Put in your postcode, and see lots of results for various gerrymandered shapes of the districts you are in. This is what I get for where I live:

Explains how I afford it on my small income, doesn’t it?

Apart from that, the website is quite excellent, all things considered. You’re not going to hack something up to better in less than a year. Maybe the team behind it should run a blog documenting the features they are shovelling in, and allow for comments and discussions with them about it. I mean these extra-ordinary communication barriers between the communities (civil servants vs. the public) ought to be broken down. A good place to start is with UI matters relating to these technical web-pages. And that’s my best recommendation for today: the ONS needs to blog about their website features.

Next up: the NHS healthcare data where they keep a lot of information that has had a long history of being absolutely essential for epidemiological studies. Because of the proven high value of this data, the government has already given it all away wholesale along with millions of pounds to Dr Foster Holdings so that some businessmen can get rich by keeping the matter out of public control while obstructing the public interest. Rather like our railway system, isn’t it?

What’s next? The London Gazette (wot’s that?), edubase, the OS, and something about carbon footprint information, which is the most important thing in the world today and in the future — but who cares? Have I told you about my carbon footprint research projects reported here and here?

So Where Is The Money?

Actually, even though the entire country and every single private sector policy decision (such as the no-brainer practice of corporations providing bogus technical advice to the government on how to commission billion pound software projects so that they are guaranteed to fail and create huge profits) is driven by money. This vital fictitious substance doesn’t feature anywhere at all. It’s none of your business. It’s commercially confidential. Because we’re making money out of it, you’re not allowed to know!

In the United States — one of the most insanely corrupt legislatures in the world — the data is not confidential. They put it all on a website.

When the same idea was introduced here by Baroness Noakes, only 20 MPs showed up for the vote. This means it didn’t make quorum. Tom Watson wasn’t there, because he obviously doesn’t think that the government financial expenditures are an important source of data for the public to have access to.

What he did show up to were all seven votes directed towards exempting himself and his fellow MPs from the Freedom of Information Act.

Friends, don’t be distracted. The money is what matters. The pursuit of it by guys who mix socially with the elite, and wannabe up there jet-setting in the stratosphere of the super-rich elite, is at the hollow core of the British state. Greed is as institutionalized as laundering black propaganda from the United States government. And don’t you forget it.

Hang on… I think I’ve got a proposal to Show Them A Better Way. I’ll just put it in and see if it gets on their web-page.

We got away at 6:30pm, so it was no surprise we arrived in Marloes at mid-night completely knackered and cross. There didn’t seem to be anybody at West Hook Farm, the site of UBUC’s Skomer summer encampment for the last 30-odd years, as well as implied by the map on this page of their modern-looking but informationally-challenged website. The farm had changed hands and no longer did camping. But I didn’t know that, so we rolled a further kilometre down the road into East Hook Farm and moved into the late night arrivals field. I ran back along the bumpy Pembrokeshire Coast Path in the dark back to West Hook Farm and into the field just to check my memory wasn’t completely lost. The last time I’d been there was in 2003 — but many formative times before then because it was an annual migration.

I spent the whole of the next day (Friday) looking for where the f*** UBUC had gone. There were at least 20 of them, two boats, and everyone I spoke to recognized who I was looking for. The carpark attendant for Martin’s Haven said they always got here for an early start when they do. Perhaps they were over at Dale the bad weather diving spot in Milford Haven. No one was there either. I’d spotted the UBUC trailer on the beach of Martin’s Haven when I cycled to it in the morning, so they had to be somewhere.

During the day I implemented my own version of the Directionless by phoning up and bothering a friend at his computer on the internet and getting him to check out the dreadful, misinformative, and utterly useless UBUC calender website containing times but No Precise Locations, and he confirmed it was that bad. There were email contacts but no phone numbers. Great. No email reply was forthcoming during the day.

Now, the reason why I urgently needed to see UBUC was that they had their dive kayaks down on site, and it might have been the first time in 6 years of kayak diving that I would get the chance to kayak dive with another pair, apart from me and Becka. Why do I always get hooked on such astonishingly unpopular sports!

Between Marloes and Broad Haven there’s about a dozen campsites on the OS Landranger map. I checked them all.

It’s worth pointing out that the weather was also pretty windy, with a howling southerly gale and belts of rain the day before. Just so we got something done during the day, Becka and I paddled out of St Bride’s Haven round Nabbs Head and into the full force of a localized rain storm. This woke us up a bit and made us appreciate how small our problems really could be made if something went wrong. Especially when we turned around and it got much more difficult to ride on the downwind leg.

Back in Dale we had a cup of coffee in the cafe and the woman said, Yeah, that group of students were in here yesterday, wringing wet from the rain. I closed up at five and they all moved into the pub and sat there drinking pints of water all night trying to keep warm without spending any money. She thought they could be up at a farm called Hillside, which was not signed on the OS map as a campsite. Windmill Farm next door was not signed as a campsite either, yet I found a lovely laid out field with level plots and all. The farmer had seen UBUC on the beach when he took fuel down to the boat that carries tourists to Skomer Island, but didn’t know where they could be.

After dinner back at the campsite of West Hook Farm I cycled across the abandoned air field in the fog to Dale, double checking the fields around Hillside farm. The owner of West Hook Farm was convinced they’d be there, because that’s where he recommended they go after turning them down for his campsite. I had half a pint in the pub in case anybody showed up. To complete my perfect timing, Mr. West Hook Farm decided to phone Mr. Hillside Farm on our behalf and found out that the UBUC had f***ed off from Pembrokeshire that morning. He told Becka this news minutes after I had set off.

Bollocks!

The weather was apparently really good on Monday, Tuesday and Wednesday. It must have been the sudden shock of a day of rain that drove them away. Had it been drizzly for all that time, they’d have probably suffered it, because evidence suggests it was a very snap decision, possibly taken after I got there, but couldn’t find them.

Why do we bother trying coming out diving when the weather is so bad? Becka asked. Well, I would have preferred to have got out on Monday, Tuesday, or Wednesday. Anyway, we got on the kayaks on Saturday (without diving gear) and paddled around Wooltack Point and down the length Jack Sound. No I didn’t understand any of the tides, I just pretended I did to give Becka the confidence I knew what I was doing. I was sure it was going to be flowing north — against us — but I didn’t have any idea how strong it would be. Nor could I predict how far we were going to get. Or that Becka’s greatest whinge was that her kayak’s knee straps were the wrong length. There’s always got to be something wrong; at least it wasn’t: Oh my god we’re going to die out here!

We ran into Robert Bailey as we carried our kayaks back up the beach.

Then we headed over to Abercastle for another bit of paddling. Becka had been reading the thoroughly excellent Welsh Sea Kayaking guidebook and decided we should aim for Abereiddy, a mere 8kms along the rugged coast.

Once again I faked it with the currents, but we were going into the howling wind, so I was sure we could just be blown back. It was wind against tide, as they say, so it was a bit of a roller-coaster ride and we got about a kilometre along and decided to turn back at the first corner. The waves are always worst at the corners. Going round the corner further meant we’d be up-wind of some rather rocky cliffs, and anyway I was beginning to appraise the rather brisk current going with us. More concerning to me was the two fully kitted out sea kayakers whom we passed in Abercastle Harbour after they had turned back owing to the weather on their trip to Strumble Head. I was absolutely sure they were going to call out the life-boat having just seen two clowns on sit-on-top kayaks disappear to sea and not come back.

In my experience, sit-on-tops are a lot more seaworthy and robust than anything else. They are the mountain bikes of the kayaking world — not suitable for racing long distances across flat seas, but better for the bumps. We surfed back briskly against the current to the harbour. Becka said she preferred Jack Sound. The sun was out. We camped in Trefin for the night.

We went back to Abercastle in the morning to complete one dive on the Leysian to prove a point. This is a huge silty wreck in 10m and we carried the anchor the whole way. My suit leaks so bad around the wrists I have to do something about it soon. (Un)luckily we won’t do any diving for the next two months because time has run out and we’re now into the caving expedition season.

Oh well.

We cased out a few likely kayaking journeys on the north Pembrokeshire coast from the sea kayak guidebook before declaring the weekend done and heading for home. Becka observed that not one of the photos in the book was of weather as nasty as what we’d been out in. We’ll have to come back at a better time. With whom? I wish.

This blog posting has been marked up with links to geographical articles in Wikipedia. This means that it contains a form of unique-ids common across the world, and could insert this post into a cross-sectional blog sliced up from everybody else’s and recreated timesorted per location as a common heritage. It’s all going to work well unless those stupid students carry on balkanizing the web with their useless Facebook log-in shite pages, and it’s eventually not possible to access anything useful on the internet unless it is produced by (a) a corporation, or (b) a “friend”.

I had to write the Jack Sound article, which someone out there can expand, put some pictures onto and describe other stuff to make it nice. I realized it was legitimate when I discovered an article for an insignificant rocklet south of Ramsey Island. Keener people could try and revive Wikiscuba with some content.

As to the cost, which was at issue in 1984, it is clear that there would be substantial savings to public funds from such a change. The current cost of vellums is £27 per page (ie for two copies). Expenditure on vellums amounted to £59,000 for the 1997 Acts, and is projected to be £67,000 for the 1998 Acts. The Finance (No. 2) Act 1998 alone cost £11,502.[1]

However, the MPs voted 121 to 53 to keep the practice. The campaign to keep the tradition was led by Brian White MP, in whose constituency the sole remaining vellum manufacturer in Europe is based.

Now, to me it’s not a money issue. I’d quite happily see some of the millions of pounds paid to Dairy Crest diverted to support the vellum industry, so that we could all afford to have our business cards and birth certificates made on white leather. Printing those Acts of Parliament on the sides of cows seems like a waste, given their quantity and quality. No one wants to read that crap in the future when everything else has burnt down.

“Oh, but vellum sheets can last thousands of years, unlike acid-free paper, which can last 500 years max. And data in computer format rapidly becomes incompatible (wake up: not anymore, since the internet began). And anyway, there might be a war and our fragile computer technology will become obsolete when we go back to the precomputer age. Either that, or the PICT project will never complete.”

Well, in that case, the Computer Misuse Act won’t be much interest then, will it? And anyway, given the destructive effects of war, what good are two physical copies in London going to be — especially when there is no way trivial way to duplicate them through an electronic pipe. Is someone going to type the worthless text back in or something?

Dr Palmer MP: said:

It has been pointed out that vellum is less susceptible than paper to fire damage. However, the problem with that argument is that vellum and paper are both flammable, so security cannot depend only on the document’s material. If there is a serious fire, a document will burn whether it is of vellum or paper. I remind the House that all the records of the House of Commons were destroyed in the fire of 1834, whereas most of the records of the House of Lords survived. That was not because the House of Lords documents were recorded on a different material, but because officials, policemen and soldiers braved the flames and were able to save many of the documents by throwing them out of the window. We depend partly on that type of physical security; the Administration Committee and other Committees are anxious to maximise it. However, to rely only on the fact that the paper itself will not burn is not adequate.[2]

And so the useless, out-of-date, and frankly quite embarrassing tradition was saved for us. So proud of it are they that there’s scarcely a mention of it on the Parliament webpage, no photos, no patriotic declarations about how we record our laws “properly” in human-readable-only (barely that) unstructured form on the same substrate as the execution warrant for kings.

There’s one mention of vellum in the FAQ under: “I want… the text of an Act of Parliament”.

Apart from [post-1988 and some other historical exceptions], you will need to consult the paper versions - there are no free online versions available. Most Acts have been printed and will be available at a good public library or university research library near you. We can also make them available to you in our search room at Westminster. We also have our own set of digitised Local Acts 1799-1990 which can be consulted in our searchroom on CD.

If you are looking for a Private Act, for instance an Act that concerned a divorce, naturalisation or an estate, then it is unlikely to have been printed and you may need to consult the Original Act in the form of a parchment roll here at Westminster or order a copy from us - we will be the only place you can find it. Please note that photocopying of original act vellums is not permitted as per our reprographic guidance; all copying of vellum acts will be printed from microfilm.

A later FAQ caught my eye: “I want…the text of a debate after 1988 (Commons) or 1995 (Lords)”:

Hansard (the Parliamentary Debates) from 1988 (Commons) and 1995 (Lords) to present day is available online. If you know the exact date it is probably easiest to follow links to that specific date and browse the text from there. Otherwise use the Parliament website search engine, restrict your search to Commons and/or Lords Hansard and put in as many other factors as possible (e.g. date range, name of speaker).

How to find a division: Votes by MPs (divisions) are recorded in Hansard. Be aware that Hansard debates are long and can be split over many pages, which can make searching difficult. In particular the long lists of names of MPs in a division may cover pages and pages on their own, so (for example) you might find an MPs name recorded in a division on a different page from that of the word ‘Division’ or the name of the bill they were voting on. If you can use the search to locate a day where there was evidently debate on the piece of legislation you are interested in, it may be worth then following the debate page by page until you find the division.

Tracing debates on bills: If you can find out the exact dates a particular bill was debated on, you can follow links from the Hansard home page to all debates on that specific date and browse the text from there. You can find out the dates specific bills back to 1995 were debated by looking first at the Sessional Information Digest . Follow the link to the session you want and look under ‘Complete list of Public Bills and their stages in both Houses’. Once you have the dates, you can then go back to the Hansard page to find the actual debates on those dates.

Well, fancy that. Is it that the TheyWorkForYou and PublicWhip webpages are Not Invented Here? Or do they believe that these new-fangled computer indexes should be discouraged. After all, if the law and government business is such that it is both

• (a) in the public domain so that you can’t legally be ignorant of it, and
• (b) in such a disorganized state that only a qualified barrister with five assistant secretaries can find what he needs in it

then that’s the best of both worlds satisfied.

*Rip* Mooooo!

Update: FOI request made.

And so another entirely inexplicable hack-day (called mashed) in Alexandra Palace was served. (Last year’s event is written up here.) I had plenty of sleep on the night, lying across three triangular bean bags. It was the night before that was the problem, with the insane plan posted up on their event website of a bus pick-up from Liverpool at 3:15 in the morning, intending to reach London by 9am.

This seemed so incredibly stupid I had to go for it.

Unfortunately, the BBC had contracted out the booking of attendees (for its free 24 hour hacking convention) to a (sub)standard commercial package which didn’t collect people’s emails, mobile phone numbers, or give out reminder notices the day before to confirm to everyone that it really was going forwards on the specified times. According to the strict laws of economics, if the punters don’t show up after they have paid for their ticket, it’s a good thing, so why build helpful features for attendees into a commercial event-booking package?

I identified the bus on Lime Street, Liverpool at around 3:30am (I believe I was the only sober person in the surprisingly busy streets at the time), got on, and tried to make myself comfortable with the limited leg-room while resolutely ignoring the fact that the bus continued to remain parked for another 40 minutes with its engine running. I think I dozed off shortly after it finally moved on.

When I woke up it was daylight and the bus was parked in a back alley somewhere with its engine off — probably for the driver to take his statutory break off work or something, I thought. I tried not to be too concerned that the bus appeared empty, apart from me.

The driver was concerned. He was outside on the pavement complaining on his mobile phone, trying to find out why he only had one party on the bus for a weekend trip all the way down to London.

We were in Manchester behind the Oxford Road BBC building and it was 5:45am.

There was something on his job sheet about changed appointments (which explained why he’d spent so long uselessly parked in Liverpool), and those who were in Manchester at 4am were there when there was no bus. There had been no communication to or from these people — whoever there were — and no attempt to detect or rectify mistakes as they were occurring. The BBC guy, who was on the driver’s phone when he handed it to me, said there should have been about 20 people. Also, nobody could explain how I’d managed to catch the bus. Logically, everyone ought to miss it when there is a cock-up of this scale.

“I’ll catch the train,” I said. The BBC guy said, “You’re free to do so, but you have to understand that you’ll be on your own. We will pay for your ticket when you get here.” — Not.

Though contractually it could have been carried forwards, I considered the option of going all the way down to London on an empty bus more extreme than that taxi offer we once had in Esbjerg.

And I don’t see why the whole transport thing wasn’t being done on the train anyway, instead of a bus, other than as a result of the legendary dire crapness of the UK train fare system. In other places there’s such a thing as a group ticket to overcome the rip-off experienced when more than 5 people travel together on the same route and it becomes 90% cheaper to charter a crappy road vehicle, in spite of the fact that there’s plenty of room on the trains at 5 in the morning. In all other businesses there are hefty discounts for bulk-buying and wholesale. So it goes.

And so, in London, I met Dan, formerly of the CADCAM industry, whom I had insisted come over on the 4am bus from Bristol (which did exist), so I’d have someone to talk to. My plans of distributing a general-purpose, simple-to-use, fit-for-the-job parsing and mash-up database (the metroscope) rapidly deflated due to patent lack of interest, so I spent a day and a night teaching Dan about urllib, regexps, and how to scrape and parse the June 2008 Merseyside Police Helicopter flight page (chosen as an easy exercise), followed by uploading the records into my fit and general feature-complete but too-ugly-for-cool-people-to-be-seen-dead-near metroscope database on which I had suspended development exactly two weeks ago.

RP rolled in late in the afternoon, drank some beer for his hang-over cure, and began to write a screen-scraper for some other source of government data. He decided it was better to upload the structured data to his own version of the metroscope, written in PHP, designed incompatibly, and which he had begun work on exactly two weeks ago but didn’t complete due to time constraints that he had been aware of for at least the past week. As such, it was not possible to mash-up any of his scraped data on a map of the country in the time available, because no one had prepared a downstream webpage to render such images beforehand — as had been done for data in the metroscope. But that’s just the brilliance of building on top other people’s hard efforts and componentized systems, instead of just throwing it all away sight unseen, isn’t it!?

Late in the night I began building my unrolling-titles metroscope-front-end. Dan began work on scraping the missing person’s database, but by the morning there clearly wasn’t anywhere for this to go. I suggested Dan go chat with people at the other tables. Having been so long in the CADCAM industry where the companies are (a) highly secretive about what their programmers are doing, and (b) appear to express bugger-all curiosity for what their competitor’s programmers are doing (thus undermining the reason for (a) other than for the purpose of breeding a culture of sad isolation among their employees), the idea that people on other tables actually wanted you to come over and sit down with them to talk about what they were doing, came as a complete surprise.

A guy with a microphone, who claimed to be a radio reporter, approached me and asked what I was working on. At the time I didn’t really want to talk. I would have wanted to show him undemocracy.com, because it’s still important and still no one is interested in it, but I knew he wouldn’t be interested in it. So I made up something about mashing-up the locations of all these poxy police helicopter rides in the middle of the night, and letting people know where they had occurred.

I called Dan back and said we had to make a presentation. We got scheduled in for our 90 seconds of fame in slot 22. My computer broke down the moment I unplugged it. And anyway its 24 hour internet connection had predictably expired and refreshed the lovely dynamic Open Streetmap web-page with a Virgin logo. Luckily, we had prepared some static slides on Dan’s computer. But it was an Ubunto machine and unable to access its external monitor socket. The back-up back-up plan on-stage was for the camerawoman to direct her lens over our shoulders at the computer screen so that the text showed up in blur-o-vision on the big stage projection.

I let out a good healthy rant with a lot of genuine emotion about being kept awake by that f***ing police helicopter above my house at 4 in the morning. To sort this out I was going to run a screen scraper to read the log of pathetic excuses good reasons for being in the sky at that time of night. Our system was going to email everyone in the neighbourhoods affected automatically with a quote of their excuse and include a link to the police force web-page for making complaints.

It didn’t win a prize, or even an honourable mention. All of those freebies and respect went to incomprehensible word-soup TV subtitle unnecessary-OCR translating Lonely Fling-it find-your-favourite-music sound-tracks, again.

Meanwhile, the government has promised to produce a crime map, channel 4 has goes on about its £50 million to spend on things like on-line innovative public tools, and who knows what talent the BBC will report that they have painstakingly discovered and nurtured from across the country?

What all these nutty media corporations don’t see is that what we really need more than anything else is some engaged publicity — preferably in the form of a TV program where the stories are explained to the audience about what is happening in such a way that inspired more people that they could do it. You don’t have to waste your talent programming only what your boss tells you to. There are better things which you know you can do.

The fact that not one radio, TV or newspaper outlet has deemed it worth considering so much as a 10 minute broadcast on — for example — the development of the Open Street Map project is outrageous and completely inexcusable.

Same goes for other unique developments, like mySociety and TheyWorkForYou, whose segment, if ever made, will consist entirely without mentioning Public Whip.

Did I say I was horribly grumpy right now?

Must be the lack of sleep.

Meanwhile, Becka got flooded into the cave she was exploring on Saturday, and didn’t get out till seven hours late, missing the club dinner. I’m glad I wasn’t there to have worried about it.

It is an observable fact that when an area of software gets done to the point of being declared tolerably okay, it becomes totally static till the end of time. That’s why you have to finish things properly, or leave them in such a bad state that you cannot avoid fixing it later, if you don’t want to get stuck in a hole. It’s also why start-ups in the software industry can sometimes whip the established products with shockingly little effort, when in a sensible world they shouldn’t even be able to catch up.

However, what’s changed is that recently a small group have taken a block-copy of the TheyWorkForYou.com code and called it OpenAustralia.org for the purpose of republishing the Hansard transcripts of the Parliament of Australia. I’ve been winding them up over here recently.

While I’m very much in favour of reusing good code, what this means is that they’ve copied all the static flaws of the TheyWorkForYou.com project, when it would be preferable if they were a small start-up that whipped this established system by being a lot smarter. As the Australians are ruby programmers, you’d have expected them to borrow from the code-base of TheyWorkForYou NewZealand instead.

My unsupported attempts at producing a system with many of the features I want (and without any stupid “Add your comment” links) to see can be witnessed at undemocracy.com.

I wrote the parser for undemocracy.com (from PDFs) over several months, but the webpage suffered a serious setback because I tried to get someone else to build it on top of a content management system he was a very keen fan of called drupal. It didn’t work. I don’t know if any of these numerous big fancy CMSs are up to the job. It’s odd. You’d think there’d be something out there standard by now for noting down minutes of meetings and subcommittees, or court transcripts, or even historical plays, which could be adapted for this purpose. But it seems not.

In the end, after a really long hard weekend building it from the ground up in raw python, I got something ready in time to show to people at hackday last year. (Note: I’m staying up to catch a bus to the equivalent event this year, which leaves at 3:15am)

One of the important ideas is to parse everything into a standard HTML form, rather than this made up XML nonsense which I thought was a good idea at the time. After all, why have a line like:

<major-heading id=”uk.org.publicwhip/debate/2003-06-26.1220.0″>

when the semantically equivalent

<div class=”major-heading” id=”uk.org.publicwhip/debate/2003-06-26.1220.0″>

is readable in a standard HTML browser. Consequently, while the parsed files of ParlParse which feed into TheyWorkForYou.com found here are not a lot of use on their own, the undemocracy.com files here are quite serviceable with the addition of a trivial bit of CSS. In fact, you can design it so that this makes the job nearly done. All you need are some batch generated indexing pages and scripts to slice out the individual debates, and this is essentially what you get. No need to load all the paragraphs and speeches into a SQL database, only to print them all back out in the same order without any gaps — that’s just a long way round to get back to where you started.

What’s the problem with “Add your comment” then?

Well, as you can see, there are a billion times more empty comments than ones people have written on, and it’s always going to be that way.

The spread of locations where comments can be made is desperately uneven — there’s one per speech, whether it’s a substantial multi-page oration covering dozens of points, or a one word interjection. You might call this a minor implementation quibble that could be fixed by changing the unit to the paragraph, but the fact that this has not been done in the past three years is a hint that comments are not really being used.

What are comments on a debate speeches anyway? When you read a good debate, you have one person putting points to another person who responds to them in a process known as an intervention. Isn’t the intervention a comment on the first person’s speech? Is his response to that intervention a comment on the intervener’s comment, or a second comment in a pair of two comments on the point he was making at the time of the intervention? And how does a third person outside fit their comment in later? I know of no forum software where anyone, but the editors, can insert comments between two comments made earlier, such as with this example. Maybe a debate is a single comment thread on its own in the first place.

The most effective place comments can be used is to explain the back-story, such as with this example over the interjection: “So weak!” which witnesses had reported (across the front page of newspapers) as being “So what!”

These are extremely rare. Generally the quantity of data flowing out of the debates is such that, given the choice, people would prefer to skim down the comment column, hoping that others will have picked out the interesting speeches. They won’t have done. Someone has got to read it. And if it’s got no comments yet, no one will think it’s interesting.

Another use of comments is to point out contradictions or related speeches. Some person gives a speech in 2004 completely contradicting his speech from 2001, so you point to his 2001 speech from a comment on his 2004 speech. Then you add a similar comment to the 2001 speech pointing back to the 2004 speech, just in case someone finds that one instead of this one. It gets prohibitively annoying once you get up to three related speeches in a cluster.

As well as being very sparse in the data, comments are generally not worked over, because who is going to read them anyway? People who write good articles about speeches or disclosures in Parliament do it on their own blogs and professional newspaper articles. These are not going to appear on your TheyWorkForYou comment stream. It’s easy to find them for issues today on the Liberty website, here, here, and with a non-deeplink to TheyWorkForYou, and no link to PublicWhip (which should be to this one) here.

The ever-popular TheRegister often has snippets about events in Parliament, particularly when they report the latest publically funded IT debacle. (I posted my favourite exchange two years ago). Here is a recent TheRegister posting, with a link to a video that doesn’t work — especially at a time when TheyWorkForYou.com have been dealing with this issue. Just goes to show that most of the people who should know about it are going to be completely oblivious to the hard technical problems that you’re solving, because they don’t even know that what they’re doing now isn’t working.

Without trying very hard, it’s easy to find other respected places like Greenpeace, Oxfam, and the BBC who generate Parliamentary commentary, but will have nothing to do with your site.

So, comments hosted on a TheyWorkForYou system are going to go nowhere.

What’s the alternative?

Host only track-backs.

These are easy to harvest using the referrer in your incoming HTTP request, and turn out a handy live feed that is more dynamic than the “most recent comments” table, because it takes no effort to bump things up to the top of the list. Also, since the data is not intrinsic to your system, it makes it easier to develop the software because it doesn’t need to remain compatible with a huge blob of user generated data in a database.

You don’t need to respond to every blog; that would just allow the spam in. In fact, blogs and other publishers are kind of like individual users in this case, so when you ban one, all their messages can disappear. Wikipedia (of which more later) is a quality source as well. Say what you like about it, but it is astonishingly spam free in this day and age.

In fact, if you designated an open blog, or a forum, which you lightly integrated, it’s possible to recreate the whole execrable “Add your comment” feature in its entirety by making every link go to a new thread whose first sentence contains the link back to the speech. The comment is made there, and the back-link appears in its place as if you added your comment. The designated open forum software can then operate its own user-login environment so you don’t have to program it.

So that proves this is an enlarged system. What’s better is that one article or blog post can point to many speeches at once. So you can say, “Hey, my MP said this in a speech in 2004, but he said the opposite thing in that speech in 2001. I think it’s because of this entry in his register of interests.”

Now that’s three places that will all back-link to the same article. No more having to point one thing to another and to another and to itself again. Also, if someone else commented about that same register of interests, citing all the other MPs who had the same commercial interest, you can connect through from your post, to the register, back to his article, and then back forwards to another MP whom he says has the same interests.

This is a sort of zig-zag effect that could happen when a site automatically exchanges links with the outside world.

Okay, now this isn’t happening yet anywhere. But the technical implementation is not very hard. I can’t do it for TheyWorkForYou, because that system is already established, and you can’t make a rival one, and no one working on it sees this as a very pressing issue. Therefore, it won’t happen.

The undemocracy.com site, on the other hand, has limited resources and audience, but it makes the first step. Take a look at this link and click on the grey [link to this] block. This opens out into a block containing a copyable reference like so:

<ref>{{ UN document |docid=A-62-PV.15 |body=General Assembly |type=Verbotim Report |session=62 |meeting=15 |page=32 |anchor=pg032-bk02 |date=[[2 October]] [[2007]] |speakername=Mr. Gutiérrez Reinel | speakernation=Peru |accessdate=2008-06-21 }}</ref>

which is going to fit quite nicely here into a wikipedia article.

So that’s the first step — generate these high-quality links which will give people the support to add them into — for example — wikipedia articles. I’ve made up a few links, like in here, for the TheyWorkForYou system, including the necessary {{UK Parliament|}} template, but they don’t auto-generate them.

So much for that.

Like I say, I’ve given up trying to get this to happen. But if it did, the next step would be to create these back-links, and then make the back-links to the blogs. Now you will have a reason for people like Oxfam and Greenpeace and the BBC to link to you — because they’ll get something back: you will give people who read the Parliamentary transcripts a way back to their sites as good as a google-ad (which these organizations all pay for). And they don’t even have to associate themselves with you overtly by posting their original comments on your pages.

So, that’s the plan, as I’ve been expressing it for some time. It’s waiting for someone who’s cool, with more influence than me, to take it up as the basis for a robust general CMS that’s suitable for Parliamentary informatics. Then we can all load our different parsed data into one good flashy well-designed system so I don’t have to keep hearing about how crap I am at designing web-pages anymore, and it’ll be compatible across all continents. Wouldn’t that be great, eh?

That final one has some banjo music in it rather than the usual machine tool howl.

We desperately need to get in to a machine shop and make our own footage at some point. Must lean on the university a bit harder.

Below is a video of my failed attempt at rolling.

Unless you can see what you are doing, it’s hard to program it. I’m trying to make some algorithms which work, before I have time to make them fast. As you can see, the model has very few triangles to make the holder collision calculations quick. But it is also an extremely hard case.

Current strategy involves creating a waterline pass for a sphere (a ball-nosed cutter without the shaft) and then find a smooth trajectory of holder orientations that do not collide with it.

Step one is to find any trajectory that works. Step two will be to make it go smoothly.

Mr Garrett: about rebates for energy-saving insulation or about accelerating energy efficiency. Instead, we have had scaremongering from the Leader of the Opposition and symbolism from the member for Flinders, who in June last year presented two Wollemi pine trees to the King of Sweden — a gift that was described by the former government as ‘a symbolic gesture of action being taken to tackle climate change’. Wollemi pines became fig leaves when you examine the former government’s attitude and delivery on climate change.

Australia now has a government that is taking a new direction, taking responsibility for tackling climate change, taking responsibility for sound economic management and taking responsibility for ensuring that Australians have a sustainable future.

Interjection

Mr Hunt Mr Speaker, I ask that the minister table the statement from which he was reading word for word, including any evidence about the destruction of the solar sector.
Interjection

Mr Albanese Mr Speaker, I rise on a point of order. That was an abuse of the procedures.
Interjection

The Speaker The Leader of the House makes a valid point. That is why I ignored all after the request for the tabling of the documents. It is not a precedent, but I ignored it. Minister, were you reading from a document?

Interjection

Mr Garrett I was.

Interjection

The Speaker Was the document confidential?

Interjection

Mr Garrett Yes, Mr Speaker.

Interjection

The Speaker The document was confidential.

This section began at 2:30pm and the next section begins at 2:35pm, so there isn’t much room for things to have happened, but I am sure there are longer off-the-record intervals.

Anyways, that’s a set of work that can be done later, provided that they start archiving the video streams now.

The parsing doesn’t look like it was too desperate, as the pagination happens at the debate boundaries, rather than every 40 or so paragraphs with the Westminster Hansard. So the large-scale chunking is already done for them.

They also had it easy by the fact that the speakers are hyper-linked, so they don’t have the issue of name matching for Honourable Members. It would help, however, if they constituency-matched phrases such as

I remind the member for Sydney that she has been warned.

In fact, a simple tweak could add the constituencies into the text beside their photographs so that we could decode these to some extent ourselves from speeches in other parts of the debate.

Also, they are given hyperlinks to the Bill being discussed. Wouldn’t it be nice if Westminster did this for us too.

Detecting the ends of speeches needs more work, such as here where the final line of a speech shows up as:

Opposition members interjecting –

These run-ons are easy to deal with when you isolate all the different formulations of words and regexp them out into their own sections.

Where’s the heck is the Australian Public Whip?

It seems the pattern is repeating itself. While all the cool people are having a good time getting all the kudos diddling around with the speeches, the votes that matter are being completely ignored. After all, in Westminster there are 40 Honourable Members (known as Whips) whose job it is to police the votes. No one cares about the speeches, so long as the members vote the right way. Which is why the political class won’t have a problem encouraging everyone to be distracted by this entirely fruitless endeavour that everyone thinks is cool.

It took some time to find where they put the votes. I could tell some kind of decision was made in this debate, because it followed the pattern of a Westminster Second Reading debate, with an amendment proposed:

Christopher Pyne: I move:

That all words after “That” be omitted with a view to substituting the following words:

“whilst not declining to give the bill a second reading, the House:
notes

(a) that the increase to the Passenger Movement Charge is an unfair slug on Australian working families;

(b) that the Government has shown itself to be both tricky and cavalier in its attitude to Australia’s border security by cutting Australia’s Customs Budget by $51.5 million in real terms next year, while at the same time announcing a measure that will raise $459.3 million over four years, allegedly to offset ‘the cost of a range of aviation security initiatives’;…”

Then the debate ends with:

Question put:

That the words proposed to be omitted (Mr Pyne’s amendment) stand part of the question.

Somewhere there was a vote, which doesn’t occur in the original transcript. In fact you’ve got to look hard for it to get to the Votes and Proceedings which summarizes the debate, without the distraction of the speeches.

The order of the day having been read for the resumption of the debate on the question — “That the bill be now read a second time”

Debate resumed by Mr Pyne who moved, as an amendment — That all words after “That” be omitted with a view to substituting the following words: “whilst not declining to give the bill a second reading, the House:…”

Debate continued.

Question — That the words proposed to be omitted stand part of the question — put. The House divided (the Deputy Speaker, Dr Washer, in the Chair) –

AYES

Mr Adams Mrs D’Ath Mr Hayes Mr Price
Mr Albanese Mr Debus Mrs Irwin Mr Raguse
…

NOES

Mr Abbott Mr Haase Mrs Markus Mr Secker
Fran Bailey Mr Hartsuyker Mrs May Mr Simpkins
…

And so it was resolved in the affirmative.

Question — That the bill be now read a second time — put and passed — bill read a second time.

Leave granted for third reading to be moved immediately.

On the motion of Mr Debus (Minister for Home Affairs), the bill was read a third time.

In other words, this crucial information is in a separate parallel thread which is too boring and technical for cool people to be bothered with, but it must be parsed and aligned against the House Hansard debate speeches to make it any use.

If this doesn’t happen, then the party whips and high-performance lobbyists will continue to get their job done by getting the votes, while the public receives only fine political speeches that make us feel good about the process. This is the essence of modern politics — the art of successfully conning the public while selling out the to powerful interests. This website in its current form will help them do just that, because you could have an absolute disgrace of a law being debated in the Parliament, and everyone saying how bad it is, but a bunch of silent MPs come in and vote it through, and none of the people who are motivated enough to read and be moved by those great speeches finds out who the scoundrels are. We lose contact with the chain of events, get confused, and move off with the mere feeling that something has gone wrong, but we don’t know what.

So I wonder who is going to do this job, or if it gets onto their Open Australia development plan. I can tell anyone who does take it on that it’s an absolute thank-less task, because it’s very hard work, and nobody appreciates the results. At least they here in England they don’t.

There are some very important sell-out votes such as this one that I am still trying desperately to publicize, without anyone’s help. That particular one has 443 hits on it on the PublicWhip webpage — none of which come in referred from any environmental or sustainable energy group. Every reader of it has come in through the front page and my appalling web-interface that everyone complains about.

By voting against their government on this issue, 37 MPs sacrificed any chance of becoming a government minister or obtaining future party employment. Observing the so-far lack of any interest in this by the wider community, I’m wondering why they even bothered. They should have made their speeches, and then voted the way they were told to vote.

This web-technology enables you to access the information you couldn’t easily get to before. But it doesn’t mean you’ll go after the information you need. A wrongly interfaced system like this — in its current form — can be a step backward because it’s already filling the niche, and won all the kudos available in that department.

So I’d like to see them acknowledge the crucial importance of a Public Whip project in their system, and could easily propose some designs and possibly some collaborations on code — if they’re interested. Though I’m not sure they are.

Oh well. Nevermind.

At the end of April, Becka and I headed down to Babbacombe beach near Tor Bay on the invitation to teach a couple of divers how to dive from kayaks. As usual, they had bought the inflatable kind, probably because it was once advertised (and reviewed) in the national diving magazine. There has been never been coverage (since eight years ago) of the proper type of dive kayak, made from hard plastic and somewhat more sea-worthy, so few people know to get it instead. Ah, that corporate money-driven journalism attitude gets everywhere and prevents things getting reported on its merits. One day the world will catch up.

We had hoped it was the season for spring cuttlefish mating. Unfortunately, we missed seeing any, but we did find a patch of bivalves up to something or other.

In between discovering just how extraordinarily unfit divers can be at paddling kayaks, we ticked off a couple of dives in the bay, such as Morris Rogue. The guide writer for that site begins his description:

If I was a fish, and quite often I wish I was, I would want to live here.

No he wouldn’t. It’s wall-to-wall fat dalia anemonies. They sting.

It was an excellent dive.

We tried to find it using the transits, before giving up and heading for the pot buoy that marked it in the current.

That was a successful weekend.

The weekend before was a Becka’s choice. We went to Snailbeach mine in Shropshire, which was cold and full of loose rock. As you can see, the way to the next level is down a rope between the ore-cart rails. There was probably a solid floor there when they put in the tracks originally.

The week after Babbacombe and Tor Bay we visited Anglesey where a friend of ours was over from Ireland to attend the Sea Kayak Symposium. No one paid any attention to us, or asked us for tickets, probably because we had the wrong kind of kayaks.

The weather was less than ideal and there was an unimpressive upset in a big wave as we tried to get off the beach. We got by with our stumpy dive kayaks, but the proper sea kayak back-flipped and was sent towards the rocks in danger of a good scraping.

On Sunday the whole island was covered with a bitterly cold dense sea fog, which wasn’t very fun. We’d had enough of winter conditions for one year, so Becka and I departed and drove all the way round the north coast of Anglesey in and out of blazing sunshine searching for a break in the coastal cloud.

We found the cut-off point at Puffin Island, parked near Beaumaris, and paddled along the boring sandy beach, and around the corner on the wrong side of the light-house where the misty cliffs were spooky. Across the strait, just out of sight in the fog, sea kayakers playing in the over-falls like ghosts.

We pulled up for lunch by a small stream that dribbled on a huge slab of rock and made it slippery with moss. The tide had gone out significantly by the time we turned back, so we had to go round the light-house on the correct side, and then discovered this big ship-wreck near the beach where we had launched.

Many things are hidden under the sea. Next weekend we’ll go for a course to learn about this proper sea kayaking. Might take our dive kayaks for a play on Friday if the weather is good.