Freesteel Blog

UNdemocracy.com rejected by Knight News Challenge for funding

Wednesday, March 23rd, 2011 at 12:35 pm Written by: Julian

May as well copy here the words I sweated over from my submission before they take it off-line or something. I’ll bet this years winners are even more amazing than last years’.

Project Title:
UNdemocracy.com the accessible interface to all United Nations materials

Requested amount from Knight News Challenge:
$200,000

Expected amount of time required to complete project:
2 years

Total cost of project including all sources of funding:
$200,000

Describe your project:
Governments come together in the United Nations to make decisions that affect the futures of hundreds of millions of people across the world. Diplomats debate and vote on some of the key problems facing humanity: How to eradicate poverty and disease, stop wars and keep the peace. It is central to international relations and therefore news reporting.

The UN documents its meetings thoroughly. All the decisions, resolutions, transcripts of plenary meetings, voting records and mandated reports (tens of thousands of papers), dating from 1946 are held in its library. This is the raw data of international relations, but it is extremely difficult for journalists and citizens to get at it: although available on the internet, there is no index, it lacks structure and effective search capabilities.

UNdemocracy.com is the answer to this.

UNdemocracy.com will make it possible for everyone to see, search, track over time and be alerted about how diplomats have responded to international affairs. UNDemocracy works by converting the PDF files to structured HTML webpages that tagging bits of data about UN meetings. This includes the diplomats who attended, what the issue was, what they said, how they voted and when the meeting was. Video of the sessions will also be linked to the section. Any resolutions or documented mentioned in the text will be directly hyperlinked.

By putting the information into HTML webpages UNdemocracy will make the original source materials available for journalistic analysis (eg comparing all votes cast by the Marshall Islands along with the United States) and linkable to other web applications such as a newspaper websites.

Video data will be connected to meeting schedules and therefore become searchable and linkable. This is the most reliable technique for locating video resources which often contain “off-the-cuff” and timely information that has not been transcribed.

These techniques are already well developed and used to great effect by the hugely successful TheyWorkForYou.com in the UK, which makes Hansard (the Parliamentary transcript) accessible with 500,000 users per month. A working proof of concept site for UNdemocracy is up and running. It currently covers the UN Security Council and General Assembly, and is in English. The project will develop this site further in English, and then expand into the official languages of the UN. You can see a sample transcript of what has been achieved here (and in the attached images): http://www.undemocracy.com/generalassembly_63/meeting_64#pg005-bk03

How will your project improve the delivery of news and information to geographic communities?:
Creating easier information flows about the UN’s working and relevant historical record to news gatherers, particularly in fledging independent and citizen media, will help transparency in all countries. This leads to greater accountability especially where the UN, in all its guises, is intervening on an official mandate, sometimes at the scale of nation-building.

UNdemocracy will be a tool for journalists that enables civil society to participate in international relations. Once converted to a accessible format, the information is more easily processed and disseminated, referred to, and most importantly used by news organisations. This has already been demonstrated in the UK by the TheyWorkForYou.com project.

The unique feature of TheyWorkForYou.com was the availability of the voting record, allowing the public to see if their representatives’ deeds met their expectations. On the UN scale this would allow citizens to discover whether their government’s public policies are reflected by their actions. Probing, informed questions could then be raised in news briefings or in the national parliments, without the need for an army of supporting researchers. The votes in particular enable a new accountability to the citizens of any nation whose casting votes can be analysed in the context of public opinion.

News articles that link to otherwise difficult to access key documents would be interlinked through backlinks allowing for isolated researchers of the same issue to discover one another and establish productive collaborations — for example between a native journalist in the country receiving a foreign intervention and a native journalist in the country sending the troops (who is in the unique position to interview soldiers returning from the field and bring to them issues that known only to those who were in the field).

What unmet need does your proposal answer?:
The UN is the primary place where international opinion about global and regional crises are given voice, yet this voice is often heard only by those in the UN building. International disputes often have a long history where promises and commitments were made, and solutions offered which were either accepted or rejected.

In the unfolding of any political crisis, it is the large powerful governments that have access to the resources to expose or bury this information, depending on the current circumstance. This can lead to an over-simplification of the problem, especially in its reporting, leading to unsatisfactory outcomes that smoulder for years before re-igniting. These can range from disputes between a couple of states, to challenges to the survival of the species. Global warming or nuclear war is such a challenge.

The journalists in individual countries and their people holding their elites to account may be the only way to ensure an adequate implementation of the agreed resolutions. By making the tens of thousands of official documents easily available, UNdemocracy will allow journalists to bring constructive public participation in international relations in a form only dreamed about in the past. This will make it more difficult for public commentators to oversimplify and rewrite history to their own ends, as they often do in these circumstances.

How is your idea new?:
Nobody has combined these techniques to radically and permanently upgrade and improve the way in which public international institutions present information on the Internet. At a tiny fraction of the cost the UN would incur if it procured a custom product for itself, UNdemocracy will lay the UN’s workings more accessible than most public or governmental institutions anywhere in the world. It is also a good demonstration of how incremental and iterative development can lead to ground shift in the way government information can be made accessible. It will provide a trusted platform giving news organizations a tool to discover the information, as well as for reporters to discover potential research collaborators through backlinks from articles to common key documents.

The idea and framework is extensible to all other international treaty organizations that release official documentation (eg IAEA, Human Rights Council, International Whaling Commission).

What will you have changed by the end of the project?:
Although UNdemocracy will be an ongoing project, by the end of this phase the UN’s complete decisions and documented workings will be wired permanently into world wide media through search engines, Wikipedia and news articles. UNdemocracy will have made it efficient for journalists around the world to research and be alerted to topics they are following. Whenever keywords occur in speeches or documents of the UN Security Council and General Assembly debates become available journalists around the world will know. This will be a big step forward in transparency of the UN, stimulating debate within related institutions about how they can become more accessible online. UNdemocracy will be a leading light to show other institutions what is available and possible. For reporting international affairs journalists will rely on it a key tool for background authoratitive research of authentic source evidence to current situations, empowering them to gain material only previously possible with an army of researchers.

Why are you the right person or team to complete this project?:
I have developed a technical solution to deliver this project using techniques have been honed through my work on TheyWorkForYou.com. This project works on parliamentary informatics involving large scale data scraping and parsing. I am a director of Web innovation company that is currently developing and hosting and collaborative general-purpose public webscraping technology services (Scraperwiki.com). The management of this project will be undertaken within the company’s professional capacity. The prototype version which I built over four months in 2007/8 successfully parsed 14 years of Security Council and General Assembly transcripts into a structured format that indexed every speech made and vote cast according to ambassador name and country identifier. This made it possible to appreciate the technical challenges in detail in order to make this proposal.

What terms best describe your project?:
Democracy, Diplomacy, news credibility, international politics and law, parliamentary informatics, easy to use

Have you applied to the Knight News Challenge previously?:
No

How did you learn about the contest?:
Friend

What tasks/benchmarks need to be accomplished to develop your project and by when will you complete them?
The technical challenge of parsing was demonstrated in the proof of concept online at UNdemocracy.com The code for this needs to be revived and updated and put into a proper framework with documentation for re-use by a full time programming team. A redesign of the front end should be undertaken by a good designer to allow the best user experience possible. This will be completed within six months of the start of the project. UNdemocracy will also need time to receive feedback from journalists to develop it until it becomes an indispensible tool for researching and monitoring the UN. Initially, we will need to reach out to foreign reporters, thinktanks working at the international and regional levels, and associations of journalists. This initial outreach will be high priority and commence towards the end of the Beta version stage. This will be in progress after four months from the start. The website will be stable after twelve months. Furthermore we would like to extend the system to handle transcripts and votes within other chambers such as the First Committee of Disarmament and International Security, the IAEA, and to as many other international bodies as resources allow (such as the UN Human Rights Council). This will be guided by user demand.

How will you measure progress?:
UNdemocracy’s progress will be measured by number of documents that have been made available for linking, and the ability to find and create direct authoritative citations to the correct original document by any current or historical newspaper, blog, academic report of an event, or policy decision relating to the UN. This form of monitoring will be done by examining trackbacks from UNdemocracy. We will pay particular attention to geographical spread and the timely relationship of trackbacks to UNdemocracy to the media coverage of major global news events. A further qualitative indicator of progress will be based on the extent to which the UN itself engages with the ideas of open data in this area of its work: we will lobby the UN to improve the structure of the transcripts it makes public so they are naturally more accessible.

Do you see any risk in the development of your project?:
Legal and technical risks to UNdemocracy are minimal. No copyright issues have been encountered during the two year operation of the prototype website. Through Administrative Instruction 189/Add.9/Rev.2, the UN has actually given permission for its documents to be reproduced: http://en.wikisource.org/wiki/Administrative_Instruction_ST/AI/189/Add.9/Rev.2 The risk of failure due to technical problems is also minimal since the key challenge of scraping the UN’s site and getting the data into a usable form is already solved in a working as a proof of concept. The UN shows no sign of changing the formats and procedures for publishing its documents. However, if it did it would generally change towards a more open format which will be easier to process. (These are after all public documents.)

How will people learn about what you are doing?:
A sizeable proportion of the funding request will be allocated to outreach. So far people have learnt about the project by clicking on links to UNdemocracy that appear in Wikipedia which is currently populated by 900 links resulting in around two to three visits per hour from that route. The main challenge will be to recruit users from communities in the media, within UN bodies themselves, and international thinktanks or research groups that follow events involving the UN. Winning this grant will greatly increase media attention and use. The current UN document repository is also highly inaccessible and frustrating, so a well-designed and easy to use service like UNdemocracy will be quickly taken up by those who experience it. This has already be demonstrated in the UK by TheyWorkForYou.com, which has had little traditional publicity but has spread by reputation alone in a field where no other service competes.

Is this a one-time experiment or do you think it will continue after the grant?:
UNdemocracy will be a permanent, ongoing service. The project can continue indefinitely at a very low maintenance cost for as long as the UN continues to publish their documents in the same form. But if they produce documents in a new form, they would hopefully choose a more open standard where the efforts to import the data would be minimal, leaving the other function of UNdemocracy unaffected. Our ambition, however, is that UNdemocracy develop into a indispenble, ever-improving service that expands to cover every public international institution.