What public data is already available?

To give the more technically minded some raw material and a headstart on getting any prototypes up and running, we have brought together here a huge quantity of information for reuse.

Several exciting new datasets are being make widely available for the first time, including  the Royal Mail's Postcode PAF file, Neighbourhood Statistics from the ONS, Health care information, from NHS Choices, a list of all schools in England and Wales from the DCSF and the Official Notices from the London Gazette.

Below you'll also find listed a wide range of other sources of public data, as well as some other useful APIs.

Please leave a comment if you know of any more sources of useful public data, and we'll add them to the list below. Have fun, and good luck...

---

NEW! Neighbourhood Statistics API from the Office of National Statistics

What’s there?
Neighbourhood statistics about almost everything.

More specifically, Neighbourhood Statistics (NeSS) data including 2001Census, Access to Services, Community wellbeing/Social Environment, Crime and Safety, Economic Deprivation, Education, Skills and Training, Health and care, Housing, Indicators, Indices of Deprivation, People and Society, Physical Environment, Work Deprivation.

Where do I find it?
http://neighbourhood.statistics.gov.uk/dissemination/Info.do?page=nde.htm

You may want to look at http://hublog.hubmed.org/archives/001689.html for advice on what's in there and how to use it.

What do I need?
Contact the ONS for an API key at better.info@ons.gov.uk
Obtain a free Click-Use PSI Licence to re-use Crown copyright information from http://www.opsi.gov.uk/click-use/system/online/pLogin.asp

---

NEW! Transport information from Transport Direct

What’s there?
Location of bus stops, stations etc and descriptions of the locations
The National Car Park Register
Journey planning information
A carbon calculator
The National Public Transport Data Repository feed

Where do I find it?
Access to the data is via a secure FTP service or API. To support the competition, Transport Direct have agreed to help users access the data on an individual basis. Some restrictions apply as not all the data available is owned by the government.

What do I need?

Email Nick Illsley at Transport Direct to access the data, e.g. for the FTP address and password or to access the API.

Given some of the restrictions, it will help Nick if you can say what you're planning to do.

Obtain a free Click-Use PSI Licence to re-use Crown copyright information from http://www.opsi.gov.uk/click-use/system/online/pLogin.asp

---

NEW! Health care services and information from the NHS

What’s there?
Information about health care service providers, and “live well” health information from the NHS.

Where do I find it?
http://www.nhs.uk/nhscwebservices/Pages/Webservices.aspx

What do I need?
Fill in the Web Services Registration Form and agree to the terms of use.

{Warning: it may take a few days for the NHS syndication team to get back to you. We're working on reducing this lag.}

Obtain a free Click-Use PSI Licence to re-use Crown copyright information from http://www.opsi.gov.uk/click-use/system/online/pLogin.asp

---

NEW! Notices from the London Gazette

What’s there?
All notices published in the London Gazette, the Government’s Official Journal and newspaper of record, in XML from February 2007 to May 2008. Notices types covered include State, Parliament, Ecclesiastical, Public Finance, Transport, Planning, Health, Environment, Water, Agriculture & Fisheries, Energy, Post & Telecom, Competition, Corporate Insolvency, Companies & Financial Regulations, Partnerships, Societies Regulation.

Where do I find it?
http://www.londongazette.co.uk/mashup/gazettesrdf.htm

What do I need?
Obtain a free Click-Use PSI Licence to re-use Crown copyright information from: http://www.opsi.gov.uk/click-use/system/online/pLogin.asp

---

NEW! Address Data from the Royal Mail

What’s there?
Sample raw data from the Postcode Address File. This will give you a comprehensive overview of the type, structure and format of the address data held by the Royal Mail, as well as the different data sets Royal Mail maintain.

The full PAF file is available, but you will need to check out the sample first (the full Postzon file is not available as part of this competition)

Where do I find it?
www.royalmail.com/addressproducts

What do I need?

You should email the Address Management Unit at address.management@royalmail.com 

There are few things they need to deal with your request quickly:

• 'Show Us A Better Way' in your subject heading
• Your address (so they can post the CD straight away)

Once you have had a look at what's on the sample CD, you can get in touch to request the full data set if required. This is a commercial product so you will only get one copy per person and there will be some terms and conditions to sign. We have spoken with Royal Mail and they will accept a scanned signature. Royal Mail will also ask you to link to the idea or fill in a short form so that they can get some feedback on this initiative.

---

NEW! A list of all the schools in England and Wales from the Department of Children, Schools and Families

What’s there?
A list of schools from Edubase. The data is in Excel format and contains a list of the schools, the address, the Local Authority, type of school and the age range served.

Update: Now available in SQL form thanks to the kind people at Free Our Data.

Where do I find it?
http://www.opsi.gov.uk/advice/poi/edubase300608.zip (3MB)

http://www.opsi.gov.uk/advice/poi/SUABW_schools_list_02072008.sql.zip

What do I need?
Obtain a free Click-Use PSI Licence to re-use Crown copyright information from: http://www.opsi.gov.uk/click-use/system/online/pLogin.asp

---

Mapping from Ordnance Survey

What’s there?

More detailed maps than are available via the popular Google Maps API– though these OS maps are strictly for non-commercial use only and access is limited to a few thousand queries per day. (If OS's licence or usage restrictions prove too onerous, you can also try OpenStreetMap, which does what it says on the tin and gets better by the day.)

Where do I find it?

http://openspace.ordnancesurvey.co.uk/openspace/

What do I need?

Register for an API key, using the form provided by Ordnance Survey.

---

Carbon Footprint Information from Defra (via AMEE – a 3rd party aggregator)

What’s there?
Carbon Footprint information from Defra, various UK Government agencies and others around the world.

Where do I find it?
http://www.amee.cc/
http://www.dgen.net/amee/signup.php (to get an AMEE API key)

What do I need?
An AMEE developer API key. Within reason, AMEE doesn’t aim to charge for access to the platform for projects that have no budget and are non-commercial or won’t have significant scale.

---

Public Spending information from HM Treasury

What’s there?
Tables and Statistics showing:

Public expenditure overview

Analyses of budgets

Differences from previous plans

Trends in public sector expenditure

Functional and economic category analyses of public sector expenditure

Central government own expenditure

Local authority expenditure

Public corporations

Analysis of public expenditure by country and region

Country expenditure by sub-function

Where do I find it?
http://www.hm-treasury.gov.uk/economic_data_and_tools/finance_spending_statistics/pes_publications/pespub_pesa08.cfm

What do I need?
Obtain a free Click-Use PSI Licence to re-use Crown copyright information from: http://www.opsi.gov.uk/click-use/system/online/pLogin.asp

---

Other Public Information Sources

BBC 3 Day Weather Forecast
http://backstage.bbc.co.uk/data/WeatherFeeds?v=p12
(List of all weather station forecasts as OPML feed http://www.dracos.co.uk/play/bbc-weather/countries.opml )

UK Road Travel Data from the BBC
http://www.bbc.co.uk/travelnews/xml/tpegml_en/rtm.xml

UK Public Transport Data from the BBC (Currently not working - 2/7/2008)
http://www.bbc.co.uk/travelnews/xml/tpegml_en/pti.xml

National Public Transport Access Node database (locations for all bus stops, train stations etc.)
http://www.naptan.org.uk/

National Public Transport Gazetteer (names of towns and places etc.)
http://www.nptg.org.uk/

TheyWorkForYou.com API (Parliamentary debates, MP details etc.)
http://www.theyworkforyou.com/api/

Parliament Parser project (List of all MPs, constituencies, names of Lords etc.)http://ukparse.kforge.net/parlparse/

DowningStreetSays.com Feed ("Every day the Prime Minister's Spokesperson meets a small coterie of political journalists known as 'the lobby' for a topical chat, or 'briefing'." This RSS feed is a transcript of these lobby briefings.)
http://downingstreetsays.com/feed

TellThemWhatYouThink.com (Government Consultations API, courtesy of Harry Metcalfe)
http://tellthemwhatyouthink.org/api/

FixMyStreet.com (geoRSS Feeds of problems reported by the public to their local council)
http://www.fixmystreet.com/alert

The Government Says (feeds of announcements and press releases from dozens of Government departments and other public bodies).  By Sam Smith and friends.
http://www.thegovernmentsays.com/

PlanningAlerts.com API (planning applications submitted to most local authorities, courtesy of Richard Pope)
http://www.planningalerts.com/apihowto.php

BBC Backstage Feeds (BBC News feeds, BBC Search API, TV & Radio schedules etc.)
http://backstage.bbc.co.uk/data/Data

BBC Programmes data
http://www.bbc.co.uk/programmes/developers#findoutmore

Other useful APIs

Google APIs (Google Maps is particularly useful)
http://code.google.com/more/#products-geo-maps

Yahoo! APIs
http://developer.yahoo.com/everything.html
(FireEagle is particularly useful http://fireeagle.yahoo.net/)

Wikipedia Data
http://download.wikimedia.org/

Wikipedia API
http://api.futef.com/apidocs.html

Flickr API (Photo sharing API)
http://www.flickr.com/services/

MOO.com API (Printing API)
http://www.moo.com/api/

Programmable Web Directory (all APIs known to man or beast)
http://www.programmableweb.com/apilist

Comments

[Bill Chadwick]

Thanks Richard - I have posted a request.

There is a useful, downloadable (Creative Commons) data set here

http://www.ordnancesurvey.co.uk/oswebsite/ontology/

which shows country, county, authority, parish etc. relationships but lacks boundary polygons.

[Julian Burgess]

Has anyone had any luck with the postcode PAF file. I got the sample CD they sent and it is complex. I thought it would be a straight forward list of postcodes with latitude/longitude, but it doesn't seem to be that way. Let me know if I'm being dumb.

[Peter Stoner]

Normal Teasdale was looking for more information about NaPTAN bus stop data. You are welcome to look at www.travelinedata.org.uk/naptan.htm which I maintain as a resource for the people in local authorities who collect and keep the NaPTAN data up to date.

The timetable data in the public transport repository comes from Traveline and I would be interested in a discussion of what people could achieve if the timetable data for the whole of the UK were made available and updated on a weekly basis, instead of the once a year refresh at present.

[Jamie Thompson]

Still no word on access to the NHS Choices Web Service. I guess me app's not gonna be realised now. Thanks for that.

[Richard]

@Jamie: I was speaking to the NHS Choices people earlier today. They are currently in the process of reviewing their license - they hadn't anticipated the demand and needed to create a standard license rather than bespoke terms for each person.

[Norman Teasdale]

quote: "Normal Teasdale was looking for more information about NaPTAN bus stop data."

Not me - that post was from Mike Franklin I think.

My request (8th July) was for access to the Environment Agency's river level data for all rivers in England and Wales. In Scotland, SEPA have provided this information for a number of years.
Ideally I'd like to see the information updated on an hourly or perhaps two-hour basis. Daily updates would be better than what we have at present though.
Has there been any progress on this yet please?

Oh, and it's Norman; I'm not usually called Normal ;-)

[Dan Yates]

Has anyone been able to release the POI data from within the Ordnance Survey API? It's fine having it there but ability to manipulate makes it interesting.

Also having trouble with UKHO (tides) data - this is available in PDF or ASCII format only.

Met Office data is also not easily consumable - has anyone successfully cracked weather?

[Dan Yates]

No word on the OS data?

EA bathing water data is available from European Union site
http://ec.europa.eu/environment/water/water-bathing/index_en.html

[Rowland Shaw]

Royal Mail are no longer honouring this offer, which is a shame, as whilst the closing date has passed, that doesn't stop people having good ideas on how to use public data more effectively

[Richard Hughes]

The British Geological Survey's baseline geological map data, UK earthquake and other data are now available through a purpose-built web services page - go to http://www.bgs.ac.uk/data/services/.

[Mat Smith]

I have been in contact with Ofsted, the education inspection organisation for all ages of schooling, and they have some very useful downloadable summary data in Excel format from inspections made for a given academic year.

This could be of great use for a site featuring school performance metrics. The URN (unique reference number) for each school is also included.

The links are below.

For 2005/06:
http://www.ofsted.gov.uk/Ofsted-home/Publications-and-research/Annual-Report/2005-06/Inspection-judgements-2005-06

For 2006/07:
http://www.ofsted.gov.uk/Ofsted-home/Publications-and-research/Annual-Report/2006-07/Inspection-judgements-2006-07

For 2007/08 (up to the spring term):
http://www.ofsted.gov.uk/Ofsted-home/Publications-and-research/Annual-Report/2007-08/Inspection-judgements-2007-08

[Steve Hirsch]

Interestingly, those in the U.S. are not TOO far behind, but should use the UK as a model in getting started with this. I'd love to see much more data transparency (and in open formats so that developers can readily get at the data).

Steve Hirsch

[dave so]

Its really good news that the government will be/ have opened up more information for internet users. The data is ours anyway right, so we should be able to access it freely and develope applications that use the data to enrich our daily lives.

great stuff. i think i've got an idea for some of this.

[Jonathan Gray]

Readers may be interested in CKAN, an registry of open data (including UK and EU official data) that anyone can edit:

See, for example:

http://ckan.net/tag/read/uk
http://ckan.net/tag/read/eutransparency

[Alan Clouds]

I am looking for a system where I can enter aprox 50 post codes and get them sorted into street by street order rather than numerical order. any ideas ???

[Daniel Jenkins]

I'm trying to access the data from naptan - http://www.dft.gov.uk/naptan/index.htm but can't seem to be able to find the information, just the schemas to go with them. Can you point me in the right direction? Thanks

[Simon]

nationalrail has a SOAP API available at http://realtime.nationalrail.co.uk/LDBWS/