Sunday 16 May 2010

ODbL Deniers

My weekly copy of New Scientist arrived a couple of days ago. Amongst the usual news and articles there is a series of longer articles about denial. You know the sort of thing: climate change is a myth, there was no Holocaust, HIV doesn't cause AIDS, MMR jabs cause autism, evolution is an atheists' lie, smoking doesn't cause cancer and so the list goes on. 

All of these are well known in at least some circles. One at least, Holocaust denial, is illegal in some parts of the world. Some, for example HIV denial, has killed hundreds of thousands of people by denying them treatment.

The very interesting articles describe the similarities between these quite different groups. It outlines the way these denial groups form, spread mis-information, resist and twist the truth and some of the effects they have, including drawing ordinary, sensible people into believing their stories.

As I read these articles, I couldn't help but see the similarities to the process OSM is going through in moving to a different licence. At first there was a lot of discussion about the technicalities, was it the right licence, what might the process be and so on. Once the licence and process had been thrashed out - at great length I might add, the deniers popped up. They had not taken part in the process, they knew nothing about the law behind the licence, they didn't understand the benefits of ODbL or the weakness of the existing CC-BY-SA licence - they just know the change is wrong. So they look for the 'real' reason for change. It must be driven by commercial interests: it must benefit CloudMade or it must be so Google can take over OSM. The wording of the licence is difficult to understand so it must be hiding something and so on and so on.

I know that the OSM licence is not as important as climate change or HIV/AIDS, but it does matter to many people. If you are tempted to grandly delete the bit of the map you have added or send that email accusing someone of hiding commercial bias, ask your yourself this: Am I being sensibly sceptical or stupidly led down a blind alley? Have I studied this or am I reacting without thinking? Do I have real evidence there is a problem or am I making a fool of myself?

I am a natural sceptic. Sceptics question everything, but they let the answers they find lead them on, usually to more questions. Deniers are lead by their agenda and rubbish the parts that don't fit their agenda, however wrong it is, ignoring any contrary evidence. I am a sceptic about ODbL and I think it is the right way to go. CC-BY-SA doesn't fit OSM, it's time to move on. Let's hope the deniers can let go and move on too.

Thursday 13 May 2010

Poles apart

A couple of short trips out today have helped me to understand the way power lines are represented by the Ordnance Survey VectorMap shape files. The shape file gives no clues about the power lines, they are all labelled the same, but there are two types of power line represented.

The first type is lower voltage that is supported by two wooden poles. It does seem that the nodes do represent the actual poles, at least most of the time.

The first photo shows where two lines join with a bit of a crossover and this  to be represented by the ways shown in JOSM.  The crossover seems to be to relieve the strain in the cables. I have followed a couple of lines where they were accessible and checked the poles and they seem to be where the nodes are. This is very far from exhaustive, so I'm only going to add lines that I have checked.  I also followed a short bit of power line on a single pole and that turned out not to be part of the OS data set.

The second kind of power lines have steel pylons. I have checked a couple of lines and the gaps in the ways do seem to be the position of the pylons. I think there are different sizes of pylons. If the gap in the way is proportional to the size of the pylon it might be possible to deduce the type of power line, but again I haven't been able to confirm this yet.

The biggest problem I can see is that it will be a fiddly job to join each short section together with the node in the middle for the tower. I'll see if I can sort out a program to help out ...

Wednesday 12 May 2010

More OS data

I have written up notes about using OS shape files in the wiki here. I hope it's useful for anyone else who wants to use them.

I thought I'd take a look at a few of the other features that might be useful from the VectorMap data set. One thing that I glanced at before was the Settlement_Line. It seems to be electricity transmission lines, at least all the ones I've seen so far only contain this. Loading these power lines could be useful, but in OSM we mark the pylons as well as the lines. 50k OS maps mark the lines but they don't show the locations of the pylons, so you can only be sure there is one at corners and junctions, so I thought these might be the same.

I did the usual re-projection of a shape file for one of the 10x10 km areas close to home. I then modified a python script to extract all of the polylines in a shape file and ran it. When I loaded the osm file in JOSM I realised there are at least two types of power line. One way is in longish sections which are not straight. There is also another type; this is made of single sections between two nodes with a gap of more than ten metres between each section.

I think these are different types of power line. The continuous lines are lower voltage lines, with the gaps marking where the pylons are in a high voltage line. I need to go and look at the lines I have identified locally and see what I find. If I'm right it will take more work to join the sections of line together, since OSM only uses a single node to mark a pylon. It doesn't make the location of power poles on the low voltage clear yet either.

Power lines, especially high voltage power lines, can run large distances from power stations to towns and cities and the 10km squares that the data is supplied in breaks up the lines into annoying sections too.

Thursday 6 May 2010

Sign of the times

My local council (East Riding of Yorkshire) have a rolling programme of improvements that they apply to all of the 280 or so villages in the county. It's called Street Scene and it is a way of fixing the small jobs, like small road repairs, fixing or replacing signs, bins, fences and anything else that the council is responsible for. Like all council schemes it starts out as a good idea then descends into a money-wasting exercise which is basically budget led. In the end jobs get done because there is money that needs to be spent by a certain time or lost and not because the job really needed doing.


All of the street name signs in the village were 'reviewed' and ones that were shabby were replaced. The sign pictured is a new replacement. The sign faces a T junction on one of the main routes into the village. It had an arrow pointing left for West End and an arrow pointing right for Main Street, which is wrong. The break between these two roads is not at this junction and some of West End lies to the right.

I emailed the council explaining that the new sign was misleading, they refused to fix it saying that it was the same as the old sign (not true) and that it was not wrong. I gave up, knowing the real reason was that the money had run out.

Some months later I bumped into Geoff, our local parish council chairman, near the sign. I mentioned that it was wrong; after some thought he agreed and I suggested that all that needed to happen was that the incorrect arrow be painted out. Today I noticed that that is exactly what has happened. I don't know if Geoff did it, an expensive council workman did it or someone else did it who had the same idea as me and had some white enamel paint to hand, but whoever painted out the arrow, thank you.

Wednesday 5 May 2010

Using the VectorMap

I decided to use some of the OS VectorMap data. The stuff that makes the most sense is the natural features: water and woodland. There are some fish farms near Driffield alongside the river Hull that have loads of little lakes and streams, so that looked like a simple target that would add some value.

Having re-projected the VectorMap shape file of the area, I selected a group of lakes and ponds in the shape file using QGIS and saved them as a small shape file. Next a little python scrip was thrown together to extract all of the polygons and change them into an osm file to import into JOSM.

There were a few things to change, the river Hull needed to be sorted into riverbanks, some of the small streams needed to be joined up properly, but really it was very easy, and much better than tracing the ponds.

Sunday 2 May 2010

OS VectorMap District

Using the experience gained with the OS BoundaryLine data I thought I'd take a look at the VectorMap District data that the Ordnance Survey have just released. It is organised into chunks based, not surprisingly, on the OS grid. The grid has 100km squares labelled with letters (you can give a grid ref without the letters, but the VectorMap uses the letters). Within each square there is a simple grid system with two parts, known as northings and eastings. The VectorMap uses a single digit of each to define a 10km square. When you have worked out which square you want and opened the folder for that square there is a variety of shape files in it. My village happens to not only fall across two 10km squares but two 100km squares, so I need SE92 and TA02 to cover the whole village.

The shape files cover a range of features: administrative boundaries, community services, heights, natural features by area and line, railway lines and points, roads as lines, settlements by area and line, text and tidal boundaries. I haven't investigated any of these exhaustively and some only cursorily and my findings so far are mixed.

The administrative boundaries seem to be the some of the same ones in the BoundaryLine dataset, but with the disadvantage that they are spread over many 10km tiles, so using the BoundaryLine ones is easier and has more boundary types.

Community services seem to be schools, I haven't found anything else. I might have expected all public buildings, for example libraries, which get highlighted on the StreetView rasters, but not as far as I can see. They are only shown as a point and with no name or other description.

Heights seem to be spot heights. The feature description is 'Heighted Point', which is neither descriptive nor English.

The natural features is interesting. It seems to contain woodland and water features, both seem useful. It also contains 'custom landform' which I still haven't identified yet. On tidal waters it seems to show the low-tide level, i.e. areas that always have water cover.

Railway lines shows the railway lines! It seems to show at least some sidings and multiple track sections. Some are labelled as multi track railway, some as single track railway.

Railway points shows stations with their names.

Road_line has the roads in the 10km square. They are listed with their type (A road, B road, Minor road, Local street - there may others elsewhere). The A and B roads have numbers, some roads have names, but most do not, including residential and minor roads, which would be the most useful. When I looked closely at the Road_line detail it was clear that some newer roads were missing, including a local estate built about ten years old. Also most private roads are missing too. In contrast, a small estate completed only about 18 months ago is in the Roads_line data. I would not trust this to be complete, and the lack of names limits its value.

Settlements by area seems to attempt to show where buildings are. It is showing this by blocking in areas, but these seem very arbituary. There are gaps between the blocks that don't really make much sense. It is more up-to-date than the Road_line data, since the estates that are missing from the Road_lines are present in the Settlement_area, including leaving the space for the missing roads. It does distinguish some building types, such as glasshouses.

Settlement_line seem to show electricity transmission lines, though not any more detail like their voltage, nor the position of the posts or towers.

Text is a list of points with the text that should be written there. The font size and display angle is listed too. The list of named items includes, places, waterway names, farms, bridges, in fact all sorts of things which may be useful to see on a map. Without the context of the object it seems useless to me.

Tidal_boundaries show the high and low water lines in some detail. I need to compare this to the high water mark in the BoundaryLine data, which I think is somewhat out-of-date. I'll post those results later.

Over all there is some interesting data in here, but, like the rest of the OS data, it is a mixed bag. I think OS have released a lot of data in a fragmented way that makes it hard to use without a lot of work to glue it all together. OSM seems like the perfect platform to put the useful data into, but only with quite some effort and, as always, best tempered with local knowledge.