Thursday 22 December 2011

A snap

Today is the winter solstice here - the north pole was tilted directly away from the sun at 5:30 UTC today. This means the light level is low and the sun's angle is low, so photography has extra challenges. As we drove along a lane near Old Ellerby a sparrow hawk flashed along the hedge and whizzed across the road. I expected it to disappear, but it landed on a fence and stayed long enough to stop and snap a quick photo. The low light didn't help, but it is recognisable.

Saturday 17 December 2011

'Plane crash

We made a short trip out to eliminate some licence issues. It was a bright, cold winter's day with a lot of traffic on the roads (is there something going on?). We gathered the data to remap an area and headed home by a longer, quieter route that sticks to country roads. The light was still quite good, but in a field Jean saw an aeroplane that looked odd. I stopped the car and saw immediately the reason it looked odd was that it had crashed. It looked like a Piper. I called the police and while I did so another car pulled up. As it happened the driver was a paramedic. He went to the plane and contacted his control centre and was told they knew about it and no-one was hurt.


When I got home the police called to say the plane had crash-landed with engine failure and no one was hurt. Looking at the map the nearest airfield from the crash-site was less than three kilometres away at Mount Airey farm, maybe the pilot was trying to get there and didn't reach it. Having landed a plane there myself I know it is a small, tricky airfield with a sloping runway and a lot of trees around and it's not often manned, so not a great place to land with a problem.

The plane is probably a right-off. The starboard wing and engine look damaged. Looking up the registration it is a Piper PA 23 Apache and at least 35 years old, though planes that old are not uncommon in light aircraft.

Not quite an average mapping session.

Wednesday 14 December 2011

Ducking the wrecking ball

The long saga that is the OSM licence change process is picking up pace again. The plan now is to move to the new licence on April 1st next year. That will mean that anything in the database needs to have been added by someone who actively accepted the new licence and contributor terms. So anything else will be archived and removed from the live database.

Declaring the date has been like a light to moths for the few but vociferous deniers. Spreading FUD and generating bonkers conclusions from scraps of truth continues to be their main weapon against the will of the majority. Tricks like calling the change "the deletion" would be worthy of tabloid journalists. One tactic seems to be neither accepting nor declining the new licence to continue to spread uncertainty. I respect people's choices, I deplore the underhand tactics. The LWG continues to hold the high ground of not responding to the provocation.

I have been reviewing the area I'm most interested in, so that by next April there will be no problem areas here. The first weapon is to simply get as many mappers to accept the new licence and CT. So far everyone I have contacted have happily agreed. I have been as neutral as possible in contacting people, not wanting to agree is very much their choice and I respect that. The problem remains that a few people are not responding to messages sent to them. The main reason I suspect is that the email address they registered with is no longer valid. They may have used a work email address and no longer work there for example. I have tried to find people with Google+ accounts, Facebook and Twitter. I've searched for people by name in an area with Google but so far a few stubbornly remain as unresponsive.

There are a few objects created by people who have declined the new licence. These are easy to deal with: simply remap them. I risk losing details added later by acceptors, but having reviewed the bulk of the stuff, the loss is minimal. Indeed, gathering what I normally gather sometimes adds more than is there now, such as speed limits, weight restrictions, bridge heights, lit and so on. I'm leaving some stuff added by people I haven't been able to contact yet, in case they accept. I expect that there will be some places to replace, but I don't mind that. Some of the footpaths will take longer to replace and some will be easier with the better weather in the spring and summer.

I will be remapping stuff created by the FUD mongers too. If they accept at the last minute (which I doubt) there will be nothing of their work left to see, which will be a shame. I was out today gathering some traces and photos (in the gathering gloom) to start that process. I will be happy and a bit smug when I see an email on the lists that is clearly trying to be a wrecking ball and I know that my area is free from such problems.

Monday 28 November 2011

We paid a quick visit to Woodmansey today to check out a holiday home park that has been in the press recently and some other stuff. Apparently people have been living in the chalets all-year-round when the planning permission doesn't allow this. The place used to be a caravan site but now the caravans have gone to be replaced with smallish chalets on brick foundations. There were still plots for sale and the sales office is still advertising them as retirement homes which is not what the planning permission was for. 

The road ways have names and all showed up on the OS Locator anomalies list. We gathered much more information than just the road names: there were more named roads than OS locator indicates, one road name was wrong in OSL, there were a couple of lakes and a public footpath running through the site. It is always worth visiting the scene of these anomalies rather than just use the names, sometimes overwriting the existing surveyed stuff as has happened in the north of the county recently.

A few service roads and buildings on an industrial estate and a couple of extras added, including a newly-built gated community that I sneaked into while the gates were still open for the builders rounded off the visit. On the way home we found a new 40 mph speed limit on a very quiet, narrow and twisty road that is often used by cyclists and horse riders. There was also a bunch of notices up about compulsory purchase of land to the south of Beverley where a much talked-about southern bypass is in the planning stages. It seems a bit bold to compulsorily purchase the land when the planning has not been agreed and there is still, therefore, some debate about the exact route.

Friday 18 November 2011

Highest and Tallest

We took a ride to the top of Garrowby Hill on the A166. It was a bit wet in the unforecast rain, but it stopped long enough at the top of the hill to not see the view for the murk. This is the highest point in East Yorkshire at about 245m. We did find the trig point hidden away in the grounds of a small water works. I added the runaway lane on Garrowby Hill, but I don't know what others tag these as. I'll probably have to change the tag when I've worked it out.

On the way home we passed Market Weighton and noticed that the bypass has a new name: Giant Bradley Way. William 'Giant' Bradley was the tallest British man ever recorded at 7' 9" (2.108m). Market Weighton, where he was born and lived, is trying to use the Yorkshire Giant to promote itself as a centre for tourism. Which is nice.

Saturday 12 November 2011

Why don't Google credit Ordnance Survey?

I've been out checking names of OSM roads. ITO World's OS Locator tiles help to find the places where OS Locator and OSM don't agree. Often on rural roads the names are missing from OSM because there is no name board, but sometimes there are name boards and the difference is because OS Locator does not agree with the name board.

Normally I don't look at Google maps - I'm not interested. Their marking scheme clearly is different from OSM because farm tracks show up as roads for example. When I did look at a Google map page that someone linked to I noticed that an OS Locator name that is wrong on the ground appeared in Google maps. Recently I've checked a few more and whenever OS Locator disagrees with my findings on the ground, so does Google maps. In a couple of occasions OS Locator disagrees with OS StreetView and Google matches OS Locator. I have also used Google's StreetView to look at the name plates where they exist and sure enough the name plate disagrees with the road name on Google's page. Google have the photographs to show the real, on-the-ground names in the StreetView system, but they haven't used them.

Some people might say that the OS names are correct and the name boards might have errors, but I strongly disagree. When the council replaced some damaged name boards locally and one was wrongly spelled, there was a fuss from the people who's road name had apparently changed. They insisted the council put it right, and the council did replace it, though it took nearly three months. My point is that if a name is wrong on the ground people will have it corrected, but if the name is wrong in an OS database no one complains so it doesn't get corrected.

When it comes to Google's naming of roads, the only conclusion I can come to is that somewhere in the supply chain of data that Google uses, OS Locator or some similar OS data is used. Google, or one of their suppliers may have an agreement with Ordnance Survey, but if they are using the open data I wonder why they don't attribute OS?

By the way, why did OS choose "StreetView" as the name of one of their hurriedly-put-together OpenData when Google already used the name for a quite different product?

Monday 31 October 2011

Wifi coverage maps

I have been working on a small project for a Community Interest Company who deliver Fi-Wi and who want to display the coverage area of their wifi. They want to show the coverage as bands for their speed of connection: 10Mb, 20Mb, 30Mb etc. They cover a number of areas across the country, especially rural areas. I needed to display a polygon over an OSM map. Their website already has a map on it, using Cloudmade tiles, but with no coverage area on it yet. I took a look at GeoJSON as a way to handle description of polygons and looked at the way OpenLayers handles it. I created a GeoJSON feature collection by hand and tried to display it with OpenLayers. After a frustrating couple of hours I hadn't persuaded it to display my polygons, so I turned to Leaflet. In a few minutes, starting from scratch, I had a map with my polygon on it.

I needed to manage many polygons so I created an AJaX feed to send a GeoJSON feature collection for a bounds box. That way when a map is displayed any of the polygons that should appear on it get sent for that map. The coverage polygons are stored in a database with their maximum extents for north, south, east and west which can them be used to decide which polygons are served for the current bounds box of the displayed map. As usual AJaX is quick and effective. Debugging is fairly easy too since a call to the AJaX server returns a GeoJSON file which is easy to read.

The remaining job was to create a polygon editor so the guys who manage the website can manage the shape of the polygons easily as well as adding new ones as their business expands. This was more intricate, though I'm not sure that I have yet done as much as I would like.

I've enjoyed working with this problem, I hope it helps the company. It has also convinced me that Leaflet is a much more pleasant way to display maps with overlays than wrangling OpenLayers, which may be powerful and very flexible but it always causes me problems.

Saturday 29 October 2011

Chasing names

There is a list of roads sorted by county that compares the names on OSM maps to the names on the OS OpenData Locator. Chasing the names to get 100% is a bit phoney, but using OS Locator to find what is missing and especially if something has been added since the last OSM survey is very good. However much I think the name chase is phoney or even invites people to add stuff to OSM that isn't correct or surveyed, to make use of the idea that OS Locator help find new stuff, first you have to get a clean state to work from. The simplest clean state is to survey all of the roads that show up as anomalies and resolve the anomalies. In the process you see more details to add and things that have changed, such as closed pubs. I only add what I have seen for myself, though I will use aerial imagery and OpenData to assist with things like just what is the extent of that playing field that I now have the name of and can see is in use. Using OS Locator names for roads where the name board is not present is sometimes an exception, in that I have not seen the name, but I use it anyway. I do of course credit OS with a source:name tag.

Today we focussed on the southern part of the Holderness plain in East Yorkshire. It is a fairly flat area, with fertile aluvial soils making farming productive. The southern edge borders the Humber estuary which is miles wide at this point. Today the tide was out and the huge expanses of mud flats were exposed, which is what the wading birds like to feed on. The wild life is worth looking out for. It was a lovely place to take a break.

My aim is to clear Holderness steadily from the south, working past Withernsea towards Hornsea. I was contacted in the week by Stephen who mapped the whole of the town of Withernsea and then went quiet. He has now accepted the new licence which is very welcome; instead of starting again with Withernsea we can build on his good work and concentrate of places that need surveys, like Hornsea.

Saturday 22 October 2011

Kilham

A bright, fairly sharp day tempted us out. I set off to check a few names in the Wolds village of Kilham, and in the end just about redrew it. The existing work was by a few people who had added bits that didn't join up, were very, very, approximate and had loads and loads missing. Most road names were missing even though there were clear names on every road. There was a name issue near the school. One end of the road has a board showing "Millside", the other "Mill Side". There is a change of speed limit on the road, so each end gets the respective name.  It took a while, but it was a pleasant place to be. One thing us OSMers like is to find a substantial road that is missing. Seeing that GPS trace meander across an empty space is satisfying and today it happened twice for country roads, once where a rural road was marked as a bridleway and lots of times for small residential roads.

I usually draw areas such as sports grounds and school fields from Bing imagery when I can, but a very new recreation ground was just a field on Bing. I had to guess how much of the field was taken. There was also a short road with a few brand new, empty bungalows on it. The road has no name board and doesn't appear on OS Locator, so I left a fixme for the name.

On the way home I we passed some road works, with a footpath / cycle lane next to the road. There was a sign giving orders to cyclists. Clearly the person who erected the sign doesn't know the rules.

Tuesday 4 October 2011

The end of the strangely warm start to October saw us out onto the Holderness Plain looking for names of roads and any other stuff worth improving. Once again we discovered a road name that Ordnance Survey have just made up without checking what is really there. There is a road called Fox Covert Road just north of Otteringham. There is a sign which is not new, but is perfectly readable.

OS call the road Whinhill Road. The road is a minor lane which only goes to a couple of farms, the second of which is called Whinhill farm, so I guess that's where OS get the name from. Nearby OS show a road called North End road and there is a farm called North End Farm too, this time no sign boards to confirm or deny this. The road from Otteringham is called Station Road because it headed to the abandoned station on the Withernsea line, which fell under Dr Beeching's axe. The place where the name changes to North End Road is not clear, but near the farm seems a good idea. I suspect the road may have been called North End Road long before part of it was called Station Road, but I can't be sure.

Monday 5 September 2011

Of drain pipes and a polar bear

The junction of Princes Avenue with Spring Bank and Spring Bank West in Hull has changed a lot over the years. There used to be railway line scything across the junction but Dr Beeching saw that closed down. It ran to Hornsea and part of it is now the NCN 65 cycle track, though not until further east in Hull. On the south side of the junction was one of the first botanical gardens in the country, that has since become Hymers College in about 1890. The railway crossing was known as the Botanic Crossing long after the gardens had become academic. The road junction used to be a police- then traffic warden- controlled junction before traffic lights took over. The shape of the junction was better suited to human control than traffic lights, but it has taken thirty-odd years to redesign the junction to be better suited to the lights.

In nearby Beverley Road there used to be a pub called the Zoological Hotel marking the zoological gardens on the north side of Spring Bank. The pub shut down in the mid eighties. In the mid nineties the name was revived; the Old Zoolgical pub was built to replace the Botanic Station building. The station car park was where the coal yard used to be for the railway. The new pub, called the Old Zoological, is nearer the zoo than the old pub was.

The shape of the buildings shows how the road from Princes Avenue curved gracefully into Spring Bank West, to avoid the railway crossing. The old name boards survive on the wall, either side of a drain pipe which seems to mark the change. The place now doesn't seem to be quite the right place for the street name to change because of the new layout of the road, but the addresses of the buildings haven't changed so the drain pipe marks the boundary.

Close by is The Botanic pub, which also marks the existence of the long gone botanical gardens. Opposite the aforementioned zoological gardens was the Polar Bear pub, named after an inhabitant of the zoo. Sadly, the Polar Bear pub stands empty today, The Botanic looks like it might be going the same way but the Old Zoological seems to be thriving.

Sunday 21 August 2011

New roads and a castle

As the coverage of OSM improves, it is harder and harder to find roads other than small residential ones in East Yorkshire that are not in the database at all, but we found one today. It's always satisfying to see the red trace extending across the GPS in open space, showing it's a new trace.

We went to Skipsea, just south of Bridlington to see what was missing, quite a lot as it turns out. The previous mapper (tracer?) had used unclassified for residential roads, there were no road names, amenities like the pub, church, shop and school were all missing as well as most of the small residential roads, but of course it looked complete at a glance.


On the outskirts of Skipsea is a Norman church, All Saints. It was built fairly soon after the Norman conquest along with Skipsea Castle. The castle was a classic motte and bailey affair and all that remains today is the mound, now operated by English Heritage. The castle was built by 1086 but demolished on the orders of Henry VIII. Next time we're there I'm going to walk to the mound and take a closer look.

Tuesday 26 July 2011

A bit more cycle route

We discovered a section of cycle route signed with the NCN signs with the number 164. It seemed to be a newly opened route with the number 164. It seems to be part of a route designated the Yorkshire Wolds cycle route, but I didn't understand the numbering system.

We had followed a section from Beverley to South Newbald (see last post), so yesterday we then picked up from there, following the route onward. It headed through South Newbald and through North Newbald before turning up a rural lane called Stoneknowle Hill. The local council started to use a specific style of road sign about a year or so ago, so new signs are easy to spot and this sign is one of them. The road did not have a name tag in OSM, so I was pleased to add it.

The cycle route then leaves the road for a rather stony track, that in wet weather might also be rather muddy. This seems to be the only way of keeping the route off the A1034, which is not the safest road to cycle on. Another short stint on a quiet road and the route then crosses the A1079 - the main road from York to Hull which can be busy and carries fast-moving traffic.

Where did the 164 go?
The route coasts down a quiet road towards the disused railway line that runs from Beverley to Market Weighton which now carries the Hudson Way.  At the T junction under the old railway bridge there was an NCN sign but not for the NCN164, only for the NCN66. I guessed which way to go and followed the signs for the NCN66 towards Market Weighton, but then realised that the signs had the 66 label on them but also Yorkshire Wolds. This is where the confusion lay and I only sorted things out in my mind when I had got home. The cycle route is called the Yorkshire Wolds Cycle Route, but it uses various numbered routes, including the 164, 66 and 1.

The route goes through Market Weighton, climbs towards Londesborough and into Pocklington following the NCN 66, before striking out again on NCN 164 again. At this point it is joined by the Way of the Roses, which is a coast-to-coast cycle route. This is the best part of the Wolds route by far, meandering through the rolling hills and valleys around Millington and up to the village of Huggate. Near Huggate we found a sign for another cycle route, NCN 167, which was signed as going to Malton, but exactly what the route is I don't know, so there's another journey to investigate. The NCN 164 now drops down a long gentle slope to Tibthorpe, which is where we left it to head for home.

Marbled white
Clustered bellflower
The route was very well signed, once I'd understood that it wouldn't be signed with the same number throughout. The scenery in the Wolds is wonderful, with views through the hills and out over the Vale of York and the Holderness plain. We saw some birds, including yellowhammers, linnets, swallows and goldfinches. We watched a marbled white butterfly flutter through the flowers, which included a clustered bell flower.

I would recommend a visit and I really look forward to investigating the rest of the route and the other route too.

Saturday 23 July 2011

NCN 164

Out and about today in Beverley we stumbled on a new National Cycle Route, which it seems has recently opened. The NCN 164 circles the Yorkshire Wolds and has "Yorkshire Wolds" on the the bigger Sustrans signs. We followed a bit of it on a somewhat diverted journey home. I look forward to following some more of it, which seems to be a circular route mostly on rural roads.

If you are interested in a cycling in a quiet, rural space with gentle rolling hills, villages with B&B and pubs and wildlife still to be seen, try the Wolds. When this cycle route is added to OSM it will be worth following, but there are other delights in the Wolds too.

Wednesday 13 July 2011

Docks and footpaths

Shepherded between steel fences
Docks and public footpaths don't often mix. The modern dock area has to be secure, not only to prevent theft, but also to prevent unauthorised access to ships for people wanting to make unauthorised entry or exit of the country. The idea of having a public footpath running straight through a dock is a bit awkward, but there is an answer in Albert dock in Hull that is a bit unusual.
The walkway above the warehouse

The first part of the footpath is what you might expect, herded between steel fences to ensure you don't stray into forbidden areas. The next part goes over the lock gates, again carefully fenced to prevent any access to the main part of the dock.

The view along the warehouse roof
Then a surprise as the footpath mounts a steep set of stairs leading to the roof of a warehouse and the path follows a steel walkway along the roof of the warehouse to a short bridge and set of stairs that lead to the footpath along the outside of the rest of the dock. The view out over the dock and the Humber is pretty good but the novelty alone is worth the short walk from the Hull Marina to take a look at the footpath over the roof of a warehouse. You can see the map here.

A rather quiet Albert dock
Albert dock is fairly quiet now, mostly doing small refits on ships, including some large sailing boats. The view over the dock was particularly quiet today, but often there is a ship or two being worked on in there. The reason we revisited was to complete the footpath through the docks because the last time we tried the lock gates were open to allow a large ship out that only just fitted and since the footpath uses the lock gates we couldn't get across.

Saturday 2 July 2011

Crossing or not

I noticed a small change to OSM close to home, so, as usual, I looked to see what the change was. A guy known as Valentijn had added a gate to public footpath and a comment about it being locked. I contacted him to ask about this and following his helpful message I went to see what was going on.

There is a public footpath on a paved track from Dunswell Road to the service road that runs around the Creyke Beck substation just north of Cottingham, as shown here. What Valentijn had found was that the gate either side of the railway crossing was chained and padlocked, and that was what I found too. There were the usual notices that Network Rail put up on unmanned crossings so it looked as though the crossing was intended to be used, but that someone had locked the gates. It seemed to me that the public footpath had been blocked so I contacted the local council.

I got a prompt and helpful reply from an Assistant Engineer (definitive map) who explained that the footpath only went as far as the crossing gates at each side of the railway and that the crossing was a private crossing for a local farm. He sent me a definitive map, but I haven't copied it here since it bears the Crown Copyright notice.

He went on to explain that the farm has had a road built connecting it to Cottingham (part of the work on the substation expansion I suspect) and so the crossing is no longer the main way of getting to the farm. After some incidents elsewhere in the country Network Rail's Health and Safety Team looked at all such crossings and put up gates where the public right of way did not cross the railway. It is entirely clear that the public right of way that no longer crosses the railway is completely pointless, but Network Rail have clearly avoided a serious health and safety risk.

I and countless others have used the crossing for many years with no incident but we have now been saved from our carelessness by the timely intervention of the Muppets the Health and Safety Team at Network Rail, to whom I am indebted for doubtless saving my life.

I have, however, made a useful and helpful contact at the local council and thanks to Valentijn the map is now a more accurate reflection of the real situation.

Wednesday 8 June 2011

OS Locator and being thirty years too late

In May the latest Ordnance Survey OpenData was released for OS Locator. This has been turned into something useful by Robert Scott in his Musical Chairs application and ITO World's OSM Analysis. ITO also provide a tile layer which I have used on my site, oscompare. These tiles get used in the editor imagery layers too.

The ideas of these tools is to show the places where OSM road names differ from the road names provided by the data in the OS Locator dataset. They both take a different approach, but both showed that two new roads had appeared in the OS Locator data that were not there before. We set out today to investigate them, and what we found couldn't have been more different.

The first road that OS Locator showed up was Stanmore Close. This area is part of the redevelopment of the Ings Road area of Hull. I have been anticipating needing to survey this area because some of the old council homes have been standing empty waiting for a bulldozer. When we got there we found that not only has the bulldozer done its work, but the builders are in busily replacing the houses. Some of the roads from Mitcham road, which were cul-de-sacs now join Mitcham Road with Camberwell Way. They have names ending in Close, but they are not now a close, however the names stand. I have extended all of the new roads that I could get access to, it looks like a bit more to come yet.

This is by far the best use of OS Locator in my book. It showed me that there was an area of Hull that had changed, I went out to check what had changed and improved OSM in the process.

We then moved to look at the second new road in the city, Garrison Side. This is an area I know fairly well, and I was very sceptical about there being any new road there. As soon as we arrived it was clear that something odd was going on.  The site is close to the place the river Hull flows into the Humber. There was a defensive garrison built there in the seventeenth century, very long since ruined and disappeared. The road suggested by OS Locator would have been along the garrison boundary with the river Hull. A new river crossing over the Hull was opened in 1981 and the bridge cut through the place this road would have been. The photo shows one of the bridge piers slap bang on the place Garrison Side would have been. At the other side of the bridge there is a small remnant of a cobbled street in exactly the right place.

The last time it could have been there was before the bridge was built, so why has this obviously non-existent road suddenly appeared in the latest dataset from Ordnance Survey? If anyone needs reasons to check all the OS data on the ground before adding it to OSM, look no further. In fact if you're looking for Garrison Side you're thirty years too late.

Friday 20 May 2011

Leaflet

Edit: The code below is for a very early version of Leaflet and is not the best way to use Leaflet now. Much has been improved and extended. The range of plugins has grown enormously too.  Take a look at the Leaflet site for more information.

Vladimir Agafonkin announced that CloudMade have released Leaflet, a new open source JavaScript library for displaying map tiles as a slippy map. I've used OpenLayers up to now, so I thought I'd take a look at it.

I have a love-hate relationship with JavaScript. It is great that there is a powerful language that runs in browsers, but the environment it runs in is horrible: DOM, various browsers supporting various features, a fiddly run-time that makes debugging horrible etc. Firebug helps.

I decided that a reasonable test was to reproduce a slippy map page I already have to see how it works with Leaflet. That would help me compare with OpenLayers, but does run the risk of me using Leaflet in an OpenLayers way, which I hope I've avoided. It also only covers a small part of what Leaflet offers, but I seems like a start.

My page needs to display a slippy map with markers on it. A side bar will display details and photo for any marker that is clicked and a small message bar will explain the current situation. To get the layout I used a very simple HTML page and a simple style sheet. I tend to not embed styles in HTML or JavaScript, but to use either IDs or Classes to apply styles and declare these in separate style sheets, sometime more than one. I rarely embed JavaScript in HTML either, other than calling functions in events, so JavaScript goes into separate files too. All of these files are available to download, see the links at the end. The Leaflet JavaScript and style sheet are also included. The map is intended to show the locations and information about a series of sculptures around the City of Hull, as part of the Philip Larkin celebrations. They are no longer in place, but I had the data so I used it.

The first part seemed very easy: display a slippy map. The map is displayed in an HTML div. Ours has an id of map. The code to make the map appear is:

var map;
var hull = new L.LatLng(53.775, -0.356);
function initmap() {
map = new L.Map('map');
var osmUrl='http://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png';
var osmAttrib='Map data © openstreetmap contributors';
var osm = new L.TileLayer(osmUrl,{minZoom:8,maxZoom:18,attribution:osmAttrib});

map.setView(hull,11);
map.addLayer(osm);
}

That is a fairly simple piece of code to display a slippy map. The variables map and hull were defined outside of the function so they are available to other functions as global variables. The single most confusing part is probably the line

map = new L.Map('map')

The first part is a map variable used throughout the code. The second (L.Map) is the leaflet-defined object for a map. This defines how the map object behaves. The last part ('map') is because the div that displays the map has an id of map, which is in the HTML code. The URL includes {s} for the server prefix needed, {z}, {x} & {y} for the zoom, x and y tile coordinate. This allows various tile providers' tile structure to be used.  Now the map needs markers.

Adding markers with Leaflet is very straightforward when you have the longitude and latitude for each marker. However, displaying a map with hundreds or even thousands of markers slows things down a lot, so I like to only add the markers that are visible in the currently displayed part of the map. You can get this by calling the getBounds() method of the map object. This returns the north and south latitude and the east and west longitude that describe the edges of the displayed map. Of course this can change as the slippy map is dragged to a new position or the map is zoomed in or out. So the way to handle this change is to respond to map events. The one which seems to fit the bill for us is moveend. This is called whenever the map is moved, including zooming. We supply a function that is called whenever the moveend event is fired. This is achieved by adding the line

map.on('moveend', onMapMove);

Now, whenever the the map is moved our function onMapMove will be called. In our example onMapMove simply calls the function askForToads() which gets the location of the markers we want. I use this extra layer of functions to make debugging easier.

We still don't know where the markers are to be placed, I do that by using AJAX to request the information about all the markers that fall within the bounds of the current map display. askForToads calls the getBounds() method of our map object, extracts the north, south, east and west coordinates and wraps it up into a call to a PHP routine that finds the list of markers and returns this in a JSON format. This returned list is processed by the stateChanged() function which is called as part of the AJAX process. The JSON provides a neat way to return this data, because simply calling eval on the returned string turns it into an array of, in our case, marker data. I then remove any markers already shown and recreate the new markers one at a time, attaching the data for that marker to a data field and recording each marker in an array (toadlayers.push). Working through the array allows me to delete the markers when the map changes.

You now need to respond to each marker that is clicked. This means that the click event needs to be responded to. This needs the line

toadmark.on('click',markerClick);

to ensure that any marker that gets clicked is responded to by the markerClick() function. Here the details from the data field are displayed in the side bar.

Throughout the process the message div displays messages about the progress of fetching the marker info and how many toad markers are being displayed.

This may seem a lot of work to display the markers, but we are making two substantial changes to the normal system of displaying markers, firstly we are only displaying some markers. If there were thousands of potential markers spread across a wide area this would make the process manageable by only displaying a few at a time. Secondly instead of just displaying a popup we are making something outside of the map div change, which opens up a wide range of actions.

How does this compare with OpenLayers? Most of it is simpler. OL provides an interface to AJAX which makes the process a bit simpler. It also puts a collection of markers together into a single layer which is easier to manage - Leaflet makes each marker independent so I had to keep track of the markers I had added. I like Leaflet, and for many things it is something I will use.

You can find the code examples on github If you want the database definition or even the data I can supply that too. You can see the working example here.

Saturday 14 May 2011

PSMA and open data don't mix

I've been trying to encourage local authorities to consider using openstreetmap for a while on and off. It seems that Ordnance Survey (OS) recognised the growing competition and have forestalled these efforts. Local authorities use maps so much that they often have a dedicated GIS team. They base most of their activities on OS maps, which is not surprising, OS being the national mapping agency of Great Britain. Until recently the local authorities had to negotiate with OS about what products they wanted to licence. However the UK Government announced that this was to change for the current financial year and a new ten-year Public Sector Mapping Agreement (PSMA) was being introduced.

This agreement allows various public sector bodies access to core geographic datasets from OS, free at the point of use. This sounds very appealing to local authorities since they get to use any of the OS datasets included in the core set as and when they want and they don't have to worry about the cost. What could be wrong with that?

Well actually it is a very shrewd move by OS. The money to fund their licences with public bodies mostly came from central government. Most money that local authorities spend comes from central government. Council tax, for example, only covers about a quarter of local authority spending. The Government have naturally taken away the money they used to give to public bodies for OS licences and put it into the central pot to fund the PSMA. OS receive that cash, so they get the same money as ever, but now have much less effort in renewing the licences each year.

The really smart part is the way OS have tied the hands of local authorities into using OS datasets. When local authorities paid for an OS product, if they found a replacement the brave ones could have chosen not to licence the OS data and use the replacement instead. That would have saved them money - a great incentive. Now they get the OS datasets seemingly for free so why look at anything else?

Why does this matter? Well it matters a great deal if you are interested in open access to data from local authorities and other public bodies. If they want to publish information as text or tables they can do as they please, but as soon as they want to display the data as a graphic overlaid on a map their only option is to use an OS map as the background and suddenly they can't release the data as they choose because they are bound up by the draconian restrictions OS place on their map data. The local authority are not likely to use any other form of map, without these restrictions, because of the effort to move away from the seemingly-free OS datasets.

I am sure that for most of their work OS data suits local authorities very well. I'm not well placed to know if the PSMA is good value for money overall - clearly it is not free as some local authorities have claimed. I want to see a viable way of publishing data from a public body in an open way on maps that is not restricted by the licence of another public body, OS. PSMA has made that less likely to happen and has strengthened the hand of OS as a dominant GIS provider.

Saturday 23 April 2011

Another trip into the wolds

The lovely weather tempted us out into the Wolds again. This time around Millington and Huggate. Last time we were in the area a few days ago we got talking to a guy with an expensive HD camera trying to film Red Kites. Today we saw him again. Last time we had rather too long a conversation which seemed mainly to convey just how money he has and actually convinced me that he has more money than sense. So this time we stopped somewhere else to have a drink and were pleased to see and hear a willow warbler.

The roads in the Wolds have very few name boards, even in the small villages. We found a few new, small residential roads that we had either missed before or were new. One thing I did see, which is worth following up on another visit, is the newly signed National Cycle Network trails. I understand that some of the existing NCN trails are having some sections renumbered where there are optional or alternative routes. This however is a newly designated route which I have yet to follow properly.

One of the roads that show up on the OS Locator tiles was Cobdale Lane. Like most of the roads is the area I wasn't expecting to find an hint of the road name, but I was wrong. The name started at a bend in the road and a pleasant cottage was set on the dip and turn in the road. The name of the cottage was Cobdale Cottage. A few hundred metres futher down the road there was Cobdale Farm too. Maybe the road name is Cobdale Lane as suggested. 

Tuesday 19 April 2011

Lovely day in the wolds

The rolling hills of the Yorkshire Wolds is close to home and is a constant draw when I want to fill my lungs with fresh air. We went out today for a simple ride out, but I couldn't resist checking a few things for OSM. There's a roundabout near Pocklington that is being built - the road now used the roundabout in part at least. There was a small road that has altered in Shiptonthorpe. The main event today though was the scenery bathed in sunshine. It was about 21°C which is certainly warm for mid April.

We looked out for birds at the spot pictured as we enjoyed some tea. We saw distant red kites and buzzards. I heard a green woodpecker a few times, but it remained stubbornly hidden. I think I'll be back, if only to catch a glimpse of that woodpecker.

Saturday 16 April 2011

More villages and lint chasing

A couple of years ago we added what we thought was the last road to OSM within the boundary of Hull. That turned out to be a bit optimistic, but fairly close to the truth. We have taken much longer to move towards completion of the East Riding of Yorkshire. There is a still a long way to go, with towns like Bridlington and Hornsea still requiring a bit of work and a few largish villages still needing attention. The countryside is pleasant and now the spring is arriving with blackthorn hedges bursting into better shows of flower than I remember for many years.

Recently we have surveyed Beeford and Brandsburton. Both were easy to do, but the difference in the ambiance in each village was marked. Beeford was pleasant, Brandsburton was distinctly odd. The list of villages needing their roads and basic facilities mapping in East Yorkshire is now probably about 40, and some are tiny needing only a brief, passing visit. There are still surprises though. We found a secondary road heading out of Beeford which was not mapped. I need to check if there are any more that need adding in the county.

The process of checking roads has been assisted by the release of Ordnance Survey's Locator data. This defines a bounding rectangle and centroid for every named road in the UK. This has proved particularly useful in settlements since it shows up roads that are newly added in an area since it was surveyed. To get this I use the highly useful transparent tiles that ITO World produce. I added them to my own overlay, which now also includes postcodes at high enough zoom levels. The OS Locator data is also available in OS Locator Musical Chairs.

Like all lint tools there are some drawbacks, the main one being that people are tempted to just use the name from the OS Locator dataset rather than going out to check what is really there. On many country roads there are no name boards and OS Locator is usually right, but even so I prefer to go and look before using the OS name. At this time of year the ride out is always worth it.

Lint tool providers have a duty, in my mind, to be as impartial as possible. They can wield quite a lot of power in the way they chose to flag up errors. Many people quickly adopt the scheme as the standard way to tag and there is a flurry of activity to reduce the 'errors' for their area.  Recently there has been some discussion about the way national speed limits are tagged. I strongly believe that tagging schemes should be designed to assist the surveyors and mappers, but some seem to think that the consumers of data should be treated preferentially over mappers. Here's why I don't agree:
  • There are never enough mappers in an area. Attracting them and keeping them is hard. An overly complex tagging scheme means they will quickly get fed up. "All that just to add a speed limit? No thanks."
  • Complex schemes mean mappers make mistakes. These are then likely to be 'fixed' by some bot or mass edit. These are not done by mappers who have visited so surveyed data can be lost or changed.
  • Mappers must use a tag over and over again, each time they encounter the thing that needs tagging. Complex schemes means adding many tags for one object every time you see it which increases the mapper's workload.
  • Programmers write code to recognise and use a tagging scheme once. The map or analysis or whatever that they produce can then be run over and over again with little or no effort, consuming the tags that mappers took hours to add.
  • Retrospectively increasing the complexity of tagging for a commonly encountered object, like a speed limit, means you are asking people to resurvey huge lengths of road. If you don't need a resurvey then the increased complexity is not needed as it can be inferred some other way in a program.
One thing I have noticed and has been commented upon elsewhere is that many of the people suggesting that data consumers need this tag or that tag to make sense of the data have never actually tried to used the data in a program. They have no idea what is really involved and what techniques can be applied to get around any problem.

Friday 25 March 2011

Fairy land

Another string of pronouncements have been made on the Talk mailing list by the ODbL deniers about why OSM should not accept the new licence and contributor terms. Grant despairingly wonders why they should come to these conclusions.

People come to these conclusions because it suits their agenda, not because of the facts. Why let a carefully written licence and all of the discussions that have gone before get in the way of a good conspiracy theory? The world is full of conspiracy theorists and FUD mongers; the fact that OSM attracts them simply shows the project is big enough to attract the cranks as well as the serious.

Some people will always glaze over when confronted with a big blob of words, whether that's a licence agreement or something more important like an explanation of how dangerous radiation is. They then seem to go off into fairy land, inventing fearful scenarios and outcomes that are wrong.  Doing thought-experiments is a great way to test the limits of a licence, but you have to keep checking that you have not exceeded the bounds defined by the licence otherwise you end up in fairy land.

I fully and wholly respect anyone who has an objection to the licence change and who wants to use the CC-BY-SA licence as it is, I just haven't been convinced by their arguments. What I don't respect is the underhand way some people undermine the licence change process by spreading FUD and untruths about what it means and the way the project will work when the change is complete.I support the change to ODbL. I believe it will benefit OSM, but even ODbL is not powerful enough to drag everyone out of fairy land.

Thursday 24 March 2011

HU14

We've been working on addressing close to home. Bing aerial imagery came along and I realised that although Bing is not as good here as elsewhere, it is just about good enough to trace buildings. I traced a few buildings and then took the existing addressing and applied it to buildings. It was much better, so we gathered the remaining addresses for Swanland. I then tried to gather all the addresses for nearby Melton in a day (and didn't quite manage it). Then Blackadder announced he had completed a whole postcode, and I realised that I was more than halfway through HU14, so I just had to polish off North Ferriby. Just! What was I thinking about.

I decided to try Blackadder's scheme of drawing the gardens too. That means working out where the boundaries of private gardens are. This is harder to work out than it may seem. Some are masked by trees, some of the photography is a bit blurry and so on. Then there's the age of the photography - about ten years old. A lot has changed in ten years. I have had to use OS StreetView for a few areas where new housing has been built since the Bing photos were taken. I have tried to represent the differences between hedges, fences and walls as best as I could. I tried to take note of hedges or fences as we wandered around gathering the addresses.  My verdict on drawing gardens: DON'T. It is a pain and makes the difficult job a lot harder.

Gathering the addresses seems a bit easier, it's probably just that we can do it on auto-pilot. I draw the buildings as blanks before we gather the addresses, we can then write on the real addresses as we go round and sketch in any additions. It is still a lot of work. You can see more here.

I'm not doing any more addressing for a while, but I think I will do some more again. The prospect of gathering the addressing for the whole of Hull is overwhelming and would take years.

Friday 18 March 2011

Naming country roads

A lovely day tempted us out, first to do some jobs on the allotment, then trying to resolve a few names on the country roads. We found a couple of small residential roads missed on the first pass. We found two war memorials, a small church, some allotments and a village hall previously missed too.


Before we went out I spotted something we've seen before, or so I thought. There looked like a road name on OS locator that looked like it was really the name of a farm. There is a road called Highgate heading west out of Cherry Burton, which is well signed. OS Locator shows a small section of road called "Dunken Hill Highgate". I expected to find a farm called Dunken Hill and that the same mistake as we have seen before, that is an address has somehow become treated as a road name.

We found the farm called Dunken Hill, exactly as expected, but when I got home I looked at OS StreetView. It shows the road name as Dunken Hill (without the Highgate bit). It shows the farm name as Dunkenhill Farm. There is also a nearby area also labelled as Dunken Hill and the road crosses part of the edge of it. There were no signs, but I decided to accept the OS StreetView name. The OS Locator still seemed muddled and being different from OS StreetView is something that happens. If anyone knows better I will accept their wisdom or knowledge.

I'm not actually too bothered about some of these country road names, the roads are there, there are no houses or businesses on many of them except farms and they have names of their own. Many country roads have no name boards, so using the name to navigate isn't actually possible on the ground. Making sure the missing roads are added is much, much more useful, but hunting down road names can be an excuse for a pleasant ride out.

Monday 7 March 2011

Shared building, two streets

I have been gathering addresses and other detailed information for the villages that make up the postcode area HU14 - more of that soon. One thing I noticed in a few places is where a building that makes up a semi-detached house is on the corner of two streets and each house is on a different street. Having thought about it there is no real reason why both houses in a semi-detached should be on the same street, I just expected that they would be.

The highlighted houses in the picture are especially confusing because their numbers are so close, but one is on Riverview Avenue and the other on Southfield Drive.

Friday 4 March 2011

Who needs approval?

I have seen discussions about what tag to use since I first joined OSM and it's not always easy to understand. Some things seem obvious like tagging a road. It's a highway right, so highway= ... ? Residential is fairly easy (but is it a living_street), but is a substantial road a trunk road or a primary, or even a secondary? Is a small road a service road, a track, an unclassified or even a path?

How do I tag a church? Apparently it's not a church, it's a place or worship, so then it needs an extra religion tag to differentiate churches from temples from synagogues etc. Why not just tag it a church or a temple?

Areas of land are unfathomable. Some get the landuse tag, i.e. how the land is used such as retail or commercial. That makes sense, but then along comes landuse=grass. Not 'used for grazing' or 'used to suppress weeds', just landuse=grass. Hmmm, does that work?

So it seems that somewhere in the past a plan was devised to sort this out: an approval process. New tags would be proposed, discussed and commented upon and then voted on, so tags would be approved for use. Approved tags? What's that all about? There are no approved tags, anyone can use any tag that suits their purpose.  That's at the core of OSM and part of what makes OSM so successful.

Imagine what would happen if I couldn't use any tag I want to. Picture the scene: I come across something that is not in the list of approved tags. I want to add it to the map. I have two choices, 1) add it as something else that is in the approved list but that is not what I have seen or 2) I can walk on by and not add it until it is in the approved list.

Both of these are stupid: option 1) will show up as the wrong thing until the tag is approved. What if the tag is not approved? Will I remember to remove it? How will I feel about removing my hard-won object?

Option 2) is even worse. The object will not get into the map database. So not only will no one looking at the map or database know it is there, no one will know there is such a thing waiting to be added, so when (if) the tag is proposed no one will know how many of them there are waiting for the tag. Will I go back and add it to the database after the tag has been approved? Somehow I doubt it, more likely I would be fed up with the bureaucracy of the process and will have left the project to give my time to someone else.

Then there's the thorny question of who should be the one to approve these new tags? No one owns OSM, so there's no one to appoint an approver. How do I know what it is you are interested in or what you want the map to be about? I don't so doesn't that rule me out from approving your tags? That effectively rules everyone out from being the approver since no one can understand everyone's motivations, so next you need a committee to try to cover more interests. How big will the committee need to be? Who sits on the committee? Am I allowed to vote someone on/off it? Why? What gives me a vote? Do I get a vote because I have been in OSM for six days? Six months? What if I have only changed a single road name in that time? So maybe I get a vote because I've added a thousand nodes. I could just add loads of nodes into existing roads just to get my vote. Can I buy a vote in return for a donation of  ten dollars? What about 10,000 dollars?

So even if some sort of committee gets formed, how often do they meet? Will they need to get advice from other experts? So how long will it take to get a tag approved? When they do decide to turn down a tag will there be some sort of appeal process? What if that is challenged by a rich company that wants a tag approved and who has a large legal team?

How will people be bound by any of this? Clearly they will have to agree to this, so as well as signing up to a licence for the data, they will have to agree to be bound by the Approvals Committee. How many people will agree to that?!?

So the answer is simple. Allow people to use any tag they like and allow a process of natural selection to merge disparate tags together as a consensus develops. Publish a list of tags that are widely used based on statistics. A description of how people have chosen to use it would help other people decide if they want to use it too. A list of the tags supported by some of the main data consumers such as renderers and routers would help too.

If this the right way to do it, then why does the approval process exist? A very good question. I don't think it should not be there at all. Removing the process would remove the right of passage of creating a tag. It would stop the stupidity of someone creating a tag, getting it through the voting process by a few people voting to approve it and then claiming it is an approved tag and demanding it gets rendered even though it has hardly ever been used. It stops people voting for or against a new tag even though they have not used it, have no knowledge about the subject and only voted because someone invited them to do so on a mailing list.

So I do sometimes struggle to find the best tag to use, but I'm relaxed about that. I am not at all happy about the phoney, abused and irrelevant tag-approval process and I would like it to end now.

Saturday 12 February 2011

House or road

More name checks today, the target was the Barmby Moor area near Pocklington. As we have seen before, the OS Locator data is very useful. It showed up a few small new developments in Barmby Moor. It also showed up a number of oddities which we have sorted out.

When I originally surveyed Barmby Moor we did it quite quickly and some street names were missing. Today I wanted to correct that. One area that was a bit awkward was the main road through the village. It only bears a name plate at the north east end, where it says Front Street. According to OS Locator it is called Main Street. We couldn't see any other name boards, so if the name changes it was difficult to see where. I looked at the house numbers for clues, but for once that didn't work - the houses have names and no numbers. I decided to ask the locals.

A very helpful elderly chap was tidying the garden of his bungalow and he explained, at some length, how the name of the street had changed in his lifetime. He explained where Front Street changes to Main Street and how, when the bungalow he lives in was built 30 years ago, the end of the street became West End. Sure enough the few bungalows at the end of the street have a numbering scheme that fits.

The main A1079 that runs by the village is called York Road on OS Locator. There are no signs to that effect that I can see, but it does go to York. Oddly there is a small piece of the road that seem to have a name Gale Hill within the section that is also called York Road. Looking closely I spotted a house called Gales Hill in just the right spot. I have seen this elsewhere with OS data. Near Molescroft there is a road OS think is Constitution Hill, but that is the name of a nearby farm. The road is called Malton Road, I know someone who lives there.

If you are using OS Locator for road names, beware of surveyors who can't tell a house name from a road name.

Friday 11 February 2011

Sympathy

I like the tools that encourage people to get out and look at their surroundings. Some might call it surveying, but often it's just an extra to some other journey or visit. Sadly some people don't get out and look what's there, they just use the tools to blithely add data to the OSM database regardless of how up-to-date it is or how valid it is.

Today we wandered around part of the East Riding looking for names where OSM and OS Locator don't agree. The rain had stopped but the promised brightness didn't appear. I found a new housing development with a few houses on it developed since our last visit (one of the best uses of the OS Locator dataset). We found a couple of roads OS Locator has names for where we couldn't see a name and a few roads where the OS Locator data was wrong. Coppleflat Lane seems to be everywhere and actually it is nowhere. I think the OS surveyor (or whoever they got their data from) must have had a bad day.

I toyed with venturing down a named, very muddy track that we didn't have a trace for and decided that a trace from Bing, well aligned with our GPS track would do. The Bing photos are a bit old, but I estimate the track is hundreds of years old so they will do nicely.

One name I didn't know was Bluestone Bottoms. I can feel some sympathy with the road - this hard chair that I'm on now leaves me feeling a bit like that sometimes.

Friday 4 February 2011

A whole village?

We took a tour around Melton, a small village not far from home. My aim was to add all of the addresses for the place, from collection to editing, in an afternoon and I failed. We wandered around the place in a howling gale, struggling to see some of the addresses. The addresses of some houses which don't display a number can be determined just by looking at the number either side, but on some of the older parts of village streets that doesn't work. Some houses only have names, some are recent additions to the street and the numbers don't fit and sometimes there are gaps in the numbers for no apparent reason.

We completed the whole village quite quickly. Except we didn't - I had forgotten that part of the village lies to the south of the A63. So, addresses for a village in an afternoon? Not quite.

Wednesday 2 February 2011

Binary files

I have downloaded lots of OSM data from the API, XAPI, the Export tab and from other sources such as Geofabrik and Cloudmade. This has usually been XML files which are often compressed and are easy to understand and deal with, if a bit bulky. The ubiquitous XML format means there are XML parsers available for the languages I use and choices in the way of approaching parsing depending on what you are trying to do, though XML parsing remains a fairly slow process especially given that it is rarely an end in itself so the real use of the data cannot start until the XML has been deciphered.

The volume of data for a given area has steadily increased as detail has increased and the one thing XML is not good at is being compact. The process of moving OSM data around involves moving data across networks and this is sometimes a slow process - my own broadband is not very quick and I have no choice of supplier so there is no competitive pressure to improve the performance or the dreadful customer service, but that's another story. One way, potentially, to improve this is to use a binary file format which could be much smaller even than a compressed XML file and might be quicker to encode and decode. Such a format has been created by Scott Crosby based on the Google Protocol Buffers (protobuf). The files are known as .pbf files.

I have not used the protobuf file formats before so I read the Google pages which seemed to make some sense, though the examples are simplistic.  I turned to the OSM wiki page describing how Scott has created the protobuf layouts and it left me baffled, so I looked at the source code of various utilities that now incorporate support for .pbf files.

I wanted to write something using a .pbf file and, having done this kind of thing before, I know that it is easy to copy other people's code and use it without really understanding it; understanding the use of the .pbf files was an important objective for me. Google provide direct support for using protobuf in C, Java and Python, but there were no Python OSM examples to be found, so I decided to start by writing a Python pbf parser so I couldn't just copy someone else's code verbatim.

I downloaded a .pbf file from Geofabrik to work on. Examining the layout of the file was not easy. If you try to use a hex viewer things are not clear because chunks of the data can be compressed and even uncompressed parts are difficult because the protobuf format squashes data, especially numbers, into the minimum number of bytes to save space. Most of the text used in tags are in a string table, so each string only occurs once in each block of data. In the end I simply wrote code to work through the file extracting and examining each part step by step. The OSM wiki page did help in some places, but I got most help by looking at  other code, the protobuf definitions and the Python files the protobuf compiler creates.

I now have a Python script to parse .pbf files, so I can use the data in the same way as I would having parsed it from XML. I have used the Python OSM classes that I created some time ago to store the data so I can write XML as a quick test. If you are interested in seeing the result of my work you can download it from here.

Footnote:
I have tried to parse the downloaded file for England from Geofabrik. It couldn't load all of the nodes in my 3Gb memory and was killed. I removed the code to store the nodes, ways and relations just to let the code run through the whole file. After more than two hours it had run through, but I couldn't have done very much with it as none of the expanded data was saved. It works well for a smaller area, such as a county or a city, but it doesn't handle big files very well at all.

Tuesday 25 January 2011

Surveys

This mapping makes me notice changes that I think would have washed over me before, but I'm sure I would have noticed what has happened to Heron Foods warehouse. They have moved to a shiny new warehouse outside of town and the old one has been demolished, but I'm not sure what the plans are for the site yet.

It will be a while before any of the other maps display the site correctly, for me it is easy, I can just draw the building from the Bing aerials (the one with the mouse cursor on it) and tag it as a brownfield site until I know more. There is a fence around the site, where before there were the walls of the building, so I can add that too.

Rarely does a trip out not reveal something that has been missed or has changed, but not always such a dramatic change as this. Roll on the spring and the chance to do more surveys.

Wednesday 19 January 2011

Using GB postcodes

As mentioned in the last post I have been looking at the Code-Point Open data supplied as part of the Opendata released by Ordnance Survey. The data describes the centroids for each postcode which is given as a list of post codes for England, Scotland and Wales. There are about 1.7 million post codes in the dataset, broken up into CSV files, one for each top level code such as HU. Each file has a lot of fields with no data or dummy data in them, but the postcode and the easting and northing are what I need. 

I wanted to make an overlay displaying the postcode centroids to see if they would be useful in working out the postcode for OSM addresses. Postcodes are part of the jealously-guarded Royal Mail database, sold as the PAF, which makes a couple of million pounds profit each year for Royal Mail. I would have liked to the detailed address data or at least the polygon that a postcode covers but the centroids are the best we have for now.

The process of making an overlay started by converting the OS eastings and northings to longitude and latitude. The original file is pruned to remove the excess and then I used gdaltransform to make the conversion, using the the OSGB datum EPSG:27700, before loading this into a database table.

Next I set about drawing a transparent set of tiles to use. I wanted to draw a dot for the centroid and the postcode as text beneath it. If I retrieve a list of centroids for the tile then any point close to the edge of a tile would have text that extends beyond the edge. The next-door tile, however, wouldn't know about that centroid so the text would be cut off. The simple way to deal with this is to draw each tile a bit bigger retrieving data for a larger area, draw the centroids and text then trim each tile to it's proper size. That way the text for points close to an edge will appear on tiles either side of the edge.

I rendered a set of zoom level 16 tiles for my local area and took a look to see how useful the centroid information is. It varies. For some roads the postcode is obvious from the centroid, some it is not clear where a postcode changes. I think it is of some use, so I pressed on.

I did not want to fill my on-line disk space with tiles that might not get used, so I decided to try creating the tiles on the fly to see if the response would be good enough. It is pretty good and local caching of the tiles helps too. I did have a small problem with the HTTP headers not wanting to work, but now that is sorted out. I can load the tiles as an overlay http://www.raggedred.net/codepoint/ and they work as a layer in JOSM and Potlatch 2 by using http://www.raggedred.net/tiles/codepoint as the place to get the tiles from.

I have only loaded the HU postcode area so far so, I will need to load areas for other areas if people want them. I do hope no one uses this to enter centroids into OSM, which seems a pointless exercise to me, but it might help people work out post codes for addressing.

Monday 17 January 2011

Addresses again

Some time ago I started adding addresses for my local area. It was tedious so I didn't continue. Then, along came the Ordnance Survey Opendata followed by Bing imagery which together mean I could trace buildings. The OS data is deliberately crude so OS still have something better to sell and the locations are a bit flaky. I have found it is not all that up-to-date in parts too, but still much better than nothing.

The Bing imagery is fairly good, but about nine years old. It is more detailed by one zoom level over Hull though but for most buildings it is good enough here. For the recent additions not on Bing, OS sometimes helps, for others a camera, GPS and guestimates are the best I can do for now. Today I went out locally to gather the addresses of a few streets and ended up talking to a pleasant chap who uses maps and is interested in maps, but had not heard of OSM. I gave him a leaflet and promised to send him more information about getting OSM data onto his GPS.

Adding addresses to buildings seems far less arbitrary and is easier to survey if the buildings are traced and printed to take out to write on. I have so far not attained the level of detail that blackadder has achieved. Part of my, more modest attempts are here.

The one set of data that are a struggle are postcodes. When the OS Opendata was released it included postcode centroids. These are the centre of the area a postcode covers. I ignored it, but now I'm working on visualising these centroids in an editor to see if the postcode for an area can be reliably discerned. That is going well, but rendering the tiles may produce far too much data for my web site to handle, so I'm weighing up the options: limited coverage, dynamic tile generation or find somewhere else to host the tiles. First though I'm going to see if they are useful.

Tuesday 11 January 2011

Swords into ploughshare?

Things have been a bit quiet on the mapping front for the past few weeks. Short, dark days are frustrating for surveying, the snow that started to fall at the end of November made getting around more of a problem. Then Christmas came along with visiting relatives being far more interesting than being out in the cold.

I have been fiddling with an idea I have for doing some analysis of OSM files. I have decided to make it browser based, but it is early days yet. I may yet struggle with having only a limited amount of disk space available to me online, but it's fun to try new things.

Heading home from a quick trip into town, we passed the Kingfisher pub that has been up for sale for a while. A new sign caught my eye and I was somewhat surprised to see that the pub had been converted to a church. Now that lots of pubs are closing across the country I have been wondering how the buildings will get reused and now I come to think of it turning it into a church is not as strange as I first thought. It is in the middle of the community, it's a reasonable size, it has rooms that the community might use and it has a car park. I just wonder if they will get more customers than the pub did.