Freesteel Blog » Template:Email_document


Tuesday, November 20th, 2007 at 1:35 am Written by:

Allow me to propose the a Wikipedia citation template for referring to an email document, as follows:

{{Email document| 
    From= Adam Smith |
    From_Organization= Bristol City Council|
    To= Beth Smith |
    Date= 2007-10-10 09:35
    Subject= Re: FOI request for contract

    document_dump_location= MySociety|
    document_id= 883746

The template would do a #switch on the value of “document_dump_location” and expand it to a URL, such as

and the text would be laid out suitably for a footnote in a wikipedia article. In the event of the mySociety email archive melting down, you have two other repositories for the email, the sender and the receiver, such that there is still a chance it could be dug out, or at least obtained by a subpoena.

As some of you know, mySociety is in the process of building a Freedom of Information Archive, including a User Interface for people who don’t know how to make and follow up such requests.

The primary way in which this Free Information and its requests thereof are transmitted is via emails. These are going to be archived somewhere central, and the structure of what is a request, what is a reply, and what is a denial is going to be built within some kind of complex database.

Unfortunately, the design of the database is non-obvious, and no matter how well you try to set it up, there will always be missing cases. Possibly very important ones.

In the real world, difficult software is a process, not a finished product. It’s no good hoping for it to be great when it is complete, because it won’t ever be complete, therefore a design which allows the system to perform at various stages of incompleteness is going to be necessary if you are to avoid disappointment.

Now, it would be true to say that my theories of software design go against the prevailing practice. To me, databases exist only to make software run quicker, and generally provide a hindrance to the design and experimentation process. If premature optimization is the root of all evil, then databases are the devil’s post-it notes.

As it happens, the way the archive is initially scheduled to work will be as a series of dynamic web-pages, forms, drop-down address lists, exemption tick-boxes, log-in systems, and comment streams. At every stage of development before it is entirely complete (if that is indeed possible), there will be large gaps in the system. When you are faced with one of these gaps, your options are either to give up, or wait for the system to be redesigned to handle it, or, more often than not, drop out and complete the process by hand, thus starving the system of participation. Things that are created in your own records and email accounts will fail to get archived in public where we will ultimately all benefit.

My own design calls for the establishment of an email database with accessibility through the above Wikipedia template, combined with a copy of MediaWiki that makes everything possible by hand on Day 1.

Each request is made on its own page with an attractive info-box, with use made of the Category system. By hand, or using Wikipedia:Bots, categories of Pending Requests, Satisfied Requests, Denied Requests, Overdue Requests, and Appealed Requests could be maintained. Categories would also be for the Authority, Sender (if an institution), and according to Exemption.

The fluid structure of the InfoBox and the email templates therein are easy to hack about and scan using text-based scripts. A block of text with this structured data is equivalent to a database entry, but with one of the columns representing “everything else we haven’t thought of yet”. Except this way it’s hand-editable, quickly viewable, immediately functional, and suitable for experimentation. Think of it as a dynamic prototype we can all work with and show off different configurations using. And, most importantly, it works for everything from Day 1, so you never need to leave it.

Then, with this as the platform, you can start discussing and coding up a working user interface system for more novice users to access.

There is a precedence for this style of development. A while ago all User Interfaces were designed in programming languages like C++. Only later did we start to get graphical tools for designing these User Interfaces; and what they did was automatically generate the C++. This meant that they were built on top of a known-to-be-working layer, and when it turned out not to be able to do everything necessary, the programmer had the option of dropping down into the lower level to finish the job off in C++, if they chose to. It’s not compulsory to go into the hard stuff, but it remains there as a potential work-around, which is better than having no means of working around it at all when you hit a limitation.

Luckily, I have nothing to do with this project, so all this can be safely ignored. I am, however, a potential user, and have submitted the entirety of my Bristol City Council case as emails to them as data in the hope that the system will at least be able to handle all of that.

Hopefully an email archive component gets up and running at some point, with all these emails loaded into it. Then I can make my own Wiki page documenting the case and put all the above theory into practice.

Meanwhile, the mySociety crew will put together their proper database, web-form filling design however they like, and get lots of users into the system doing wonderful things. Speeches will be made. Awards will be given. And another amazing success story will be chalked up with the ease of something whose correct solution was completely obvious from the start.

Leave a comment

XHTML: You can use these tags: <a href="" title=""> <abbr title=""> <acronym title=""> <blockquote cite=""> <code> <em> <strong>