A quick blog about a big subject.

I was asked today what I thought the optimal number of stakeholders on a moderately sized software development would be. My opinion is somewhere around 7. 7 is often considered the ‘magic number’ as it is about as far as the brain remembers (y’know, the old cognitive psychology bit) – hence why for years it was bad practice to have more than 7 menus on a desktop application; as I’m typing this I’m looking at Firefox with 7 menus on it (File, Edit… you know what they are).

(In fact as I remember, the golden rule was ‘7 plus or minus 2’. Some folk will remember up to 9, others down to 5. But 7 is a nice central number.)

Anyway I can see that there is some kind of analogy with this when it comes to stakeholders. Not only in meetings, but groups you’d meet for requirements gathering as well. Say for instance you’re gathering requirements for a social networking application or new mapping application. Within your room, beyond a certain number of people, you’re going to find people who don’t contribute, or who don’t feel they’re getting heard. Less than this number, and you’re not getting a rounded view of a number of partners. Plus, of course, there’s the inter-relationships between stakeholders, which, as you bring stakeholders together, brings in another dynamic that’s exponentially more complex as you increase the number of participants.

What of Agile? Well as development becomes increasingly Agile – especially so in the current climate where customers are demanding to see progress earlier – it’s more important than ever that there is direct involvement between development team members and customers. After all, they’re all stakeholders together. Making sure that this relationship is fluid is important, so it’s necessary that there aren’t too many people involved to stifle and over-analyse. Otherwise, not only are relationships too complex, but the project is too big.

A little earlier today I posted this very question on Twitter. I got a reply back very quickly saying ‘as few as possible’. Whilst this does make sense, isn’t there a risk that you don’t actually speak to enough people to influence the outcome?

So for all this, ‘7’ seems to be another magic number – stakeholders, be they developers, deliverers, purse-string holders, or users – it seems the right sort of combination. If you go too low, you don’t have enough different opinion. If you have too many, it probably means that your project is too big anyway and needs some kind of decomposing.

Chris Norton mentioned on his blog about the new Facebook Places changes. FB Places will partly revolutionise marketing by the legendary ‘check-in produces offers’ that we always imagined Location Based Services would revolve around.

There’s always the notion that check-in is actually an effort. I use Foursquare and Facebook checkins sometimes – although rarely at the same time, because I have different people on each account. (Facebook is normally friends, whilst 4SQ is mostly other folk interested in geo-things that I know.) But I don’t always remember to check-in; it’s a bit of a faff, so it doesn’t always happen – and then I think ‘I should’ve checked in there….’ long after I’ve left the place.

What I’m surprised about is that geofencing has not been mentioned much with relation to FB Places. (Geofencing is where, rather than you checking in yourself, once you hit a virtual fence, you trigger off an alert with the host.) It’s FB’s logical next step to turn on Geofencing as part of their native mobile apps. Isn’t there some kind of cost model that would have retailers that have a space on FB to create a suitable, standard size polygon around their outlets, that could ping a user when in proximity?

I suppose the question there is, why through Facebook? Why not through Latitude, for instance? Well the obvious reason is that most people are already Facebook users. Even if only a small number opt-in to geofencing, it would still be a large enough proportion for it to make sense. And then of course, the ‘xxx has just checked into yyy coffee shop and can get a free latte’… would pull in more friends, and therefore trade, inevitably.

People have given their trust to Facebook more than other platforms, already. There is more likely to be an adoption of someone knowing where you are, if it’s through Facebook, than through any other website. Far more people will use Facebook Places than Google Latitude, for instance. I’d suggest that enough people have used FB for a sufficient time that they have built up a sense of ‘trust’ with it – whether misguided or not! I would even wager that people don’t care as much that FB is watching you than any other organisation is watching you.

So I’m waiting until FB eventually take the Location Based Services ‘thing’ and do it properly.

I’ve been working with Amazon Web Services for a few months now to get an installation going for a GIS platform for a number of parallel projects. All I can say so far is: it’s simply excellent.

For a long while, there was such a difference between ‘web hosting’ and ‘your own infrastructure’. That is, a small company’s web hosting – maybe with a few apps included, but mostly just a web site, differed massively from a company’s own, large-scale enterprise IT infrastructure. The big point of the cloud is that these two now meet in the middle.

Amazon’s Web Services are a pleasure to use, as long as you’re able to understand the limitations and get your head around the (pretty big) paradigm shift. No longer are you working on something within your own organisation. And that gives you two big changes: you don’t own the infrastructure, but you don’t need to worry about it either. In fact, for most organisations, the big plus is this: you probably have more control over it than if you owned it yourself. Let me say that again – you have more control over it than if you owned it yourself.

What do I mean by this? Well, most organisations have some kind of structure that means that IT changes require (often cumbersome) business processes and gates in place that mean that changes need a large amount of procedure, paperwork, authorisation and (probably) time. When you move to the cloud, you rent the development/testing/prototype/proof of concept platform for as long as you need. So this is the big bonus: you can prove value to the business without needing to go through IT governance to get hold of servers, operating systems, support staff… whatever is apparently required.

We’ve had the Agile ‘thing’ knocking around for a fair few years now, and (hopefully) we know the pros and cons of this approach: stay close to the business, don’t go mad with documentation, know that change is inevitable, present frequently and agree small steps. Now this is where I think the cloud works really well: it’s the ideal platform for a proving ground. And, as requirements are continuously revealed – particularly non-functional – you can expand the platform to fit accordingly.

This is a bit of a high level praise of the concept of cloud computing, but my experience with the Amazon Cloud has been fruitful so far. I’m going to blog some more as my experience continues, and I’m expecting more business benefits and possibilities to become apparent as time goes on.

For now, the path is inevitable and exciting!

Where GIS and BI (Business Intelligence) Mix

What do I mean by this?

And this is where I think GIS will go in the longer term: it will be key in enabling businesses to tune themselves and to make those decisions: where to sell, where to manage, where to concentrate on particular asset types or conditions.

Basic web maps. That’s what many people need from Geographic Information Systems: no more, no less. They want to see some kind of business information in a spatial context, because they need to get some kind of message on location, as well as perhaps proximity, across.

Yet there’s a real difference in quality and appearance of various mapping websites. Take a look here and here and you’ll see what I mean. Both of these use ESRI’s ArcIMS HTML viewers.

Nothing wrong with them per se, only it’s a perhaps a little… well, old. Very 2002. I used ArcIMS for mapping applications around that time, and it worked a treat – web maps online that weren’t just HTML image-maps were great.

Thing is, times have changed. It’s still possible to have an ArcIMS web map service, but the expectation from the end user is something less GIS, and something more map. Or should I say, something more Web 2.0. The standard has been set, and it’s easy to disappoint, unfortunately.

Yet, it shouldn’t be difficult to produce something that’s a little better these days. Since Google Maps completely changed the face of web mapping, there’s no reason that the newer APIs shouldn’t be used – Google Maps API itself being a great place to start. OS OpenSpace is a great place to go, or indeed, to use OpenLayers. Or even choose flash if you have to.

My point being, the role of a map on an organisation’s web site is often to convey information about the location of services. A user is far more likely to get that information using a (newer) intuitive interface, rather than old style, clunky, unwieldy interface that used to be the norm. Again, nothing against the journey we took to get here via the ArcIMS HTML Web Map – it served its purpose very well.

But we’re a long way down that road now, and it’s time to update.

On a number of occasions I have seen that, within the field of GIS, there’s a lack of common terminology around. As an example, the notion of ’site’ or ‘location’. What is a site to a business? Is it the place where some kind of business feature is located? How is the footprint of that activity described: a point (because that’s all you need) or a polygon (because there’s some reason why you need its area – to give an idea of size, or because of proximity to other objects).

This is where ’site’ needs to be understood across the business. And so does ‘location’, if it means something different. So does ‘company’, if it means the owner company or the company who just happens to work there.

Really, when an enterprise wants to start to think about its GIS estate, it needs to ensure that everyone has a common understanding of terminology – otherwise, things get out of hand pretty quickly. ‘Address’ and ’site’ mean many different things, and need to be described (in some kind of ‘business glossary’) in order that people within the same organisation aren’t talking at cross-purposes, and so that you’re not going to need some kind of cleanup or dissemination operation later.

For instance, say a geospatial database contains ‘address’ as a field (or group of fields). There are two aspects here. Firstly, is the address formatted correctly? This is really a syntax issue. Has the data been captured in an ‘intelligent’ format – House number, street, locality, town etc – or has the old ‘Address 1′ ‘Address 2′ format been employed? The second issue is that of semantics: what does the address actually mean. There’s a real difference between a correspondence address, an ownership address, and an address of activity. Any of these are relevant at different times, and it’s important that the business has defined a suitable vocabulary to ensure that everyone – capturers, consumers, decision makers – understand exactly what kind of address is being noted.

So for that reason, it’s imperative that everyone knows the intended meaning of the data in hand. It’s no use people capturing different kinds of information in the same field if there’s a slight difference in meaning, else end users will inevitably make the wrong decision.

Taking the time to make sure we all know ‘what the what is’ is time well spent. And like so many other things in GIS, it comes down to metadata, business process and straightforward availability of information.

Opportunities for GIS in the Public Sector in Troubled Times

The UK government has recently discussed austerity measures, potentially at a scale not seen before. The worst-case scenario here is a 40% reduction in budgets. Now, I’m pretty sure that the government is playing the ‘throw in your worst case deal, then withdraw to the middle ground which looks better’. Probably something around the 25-30% mark is more likely. Having said that, even this kind of figure is a massive amount. In any government organisation that isn’t ringfenced (and that’s everything bar Health and Oversees Aid), that’s going to have a huge impact. Organisations can weather a 5-10% reduction in natural wastage, but 25% is going to be difficult.

Inevitably this is going to result in staff reductions. I myself have been a little unfortunate in all of this, because of the result of a decrease in the amount of work I’m doing. And it’s not nice that these are real people that are going to suffer as a result of it – blame whoever, politics aside, it’s going to hit real folk.

How does GIS fit into this? Well, as the government seeks to reduce overall IT spend, we’ve got to consider ways that GIS can be of benefit. So much GIS work depends upon public sector orders, that if we’re not careful, it will become another inevitable casualty of the cutbacks.

If we’re wise, it could be seen to be one of the areas that provides benefit to the change process. I can see this in a number of ways.

Automation

Sadly, as people are removed from their positions, we’re going to see a dependency on automation of labour. This is where such things as data validation are going to be crucial. Processes, which were performed by manual cast-eyes and Access database manipulation, will need to be replaced by automated tasks based upon minimal human interaction. In GIS world, this is where government needs to start thinking about using Spatial ETL or Data Validation products to ensure that this happens. It’s also about convincing the sector that data quality and automation are worthwhile investments for reduced costs downstream – longer terms costs are driven down. Cost benefit analysis on FTEs vs licences will have to be executed. Again, potentially it’s ‘not nice’, but an inevitability of where we’re at, at the moment.

Hosting & sharing

The government g-Cloud is gaining profile, and for good reason. As there is the move to more centralised hosting and data storage, there are opportunities for cost reduction based on reduction of numbers of IT teams. Again, it’s a regretful place where we’re at, but there are opportunities to rationalise systems into secure multi-hosting of government systems. In the GIS world, this may provide advantage in data sharing, where seemingly disparate government data holders have their data physically close, which should push data sharing even further. It also forces organisations, who themselves have physically disparate hosting, to think about how to consolidate their own infrastructure space. This does give rise to network speed issues (particularly with the transferral of local data), but the overall benefits should outweigh the disadvantages. Equally, non-functional requirements type-risks are moved to the host and away from the government department themselves.

This one will require a mindset shift. IT teams within the public sector are under threat by this, where they will no longer be responsible for ‘their own patch’. Hosting providers may take the ‘best of breed’ IT staff from the public sector to aid them in transition.

Open Source

This one is a threat to the established GIS providers themselves. As there becomes a general drive toward Open Source, where the model is that the government encourages collaboration across the sector to contribute to the better functionality of systems, there will be a reduction in the number of licences of software from the traditional vendors. And that’s where they have to keep one step ahead. In the same spirit as the NHS moving towards bulk buying to get drug prices down, there will no doubt be the same move in the public sector on GIS licensing. Moreover, there will also be the notion that the public sector can shape the direction of Open Source software, as similar functionality requirements across government organisations can be implemented by one body, for the benefit of many.

In many ways the idea that cost reduction should be achieved by a move to Open Source, rather than reducing staff numbers, should be seen as a priority. Where there are Open Source alternatives, they should be considered, rather than reducing staff numbers operating an at-cost solution. Open Source isn’t free, but I’d personally sooner reduce licence costs than have people lose jobs.

Regulation

All these cost-reduction initiatives are occurring at the same time as regulation, particularly from the direction of the EU, gets stepped up. Again, this comes back to the notion that there is a requirement on data quality and consistency in the face of reduced manpower – and hence, the need for better automation. In this case, cost-benefit analysis has the third dimension of mitigating reasons, where the obligatory requirements will need to be fulfilled by the least expensive route.

Re-learning the benefits of GIS

Within the private sector, GIS has been used many years to optimise. How much optimisation has actually been done in government? How much GIS work goes on in the public sector to, for instance, optimise delivery routes of drug deliveries? How much has been done to ensure data capture is as accurate as possible for all, and not just fit-for-purpose for one user? Should the public sector take lessons from the private sector in how GIS can actually be useful to make things easier, better, and cheaper? I certainly think so. It’s my experience that the profit motive can ensure that GIS is used to ensure efficiencies in systems – after all, that’s how many GIS sales in the private sector are made: ‘I’ll show you how my GIS can save you money’. Maybe we need to focus on that: how GIS is an enabler to streamlining, not something that itself should be streamlined.

What about staff?

What about the guys that are no longer employed? Potentially, up to 40% staff cuts is huge. The government is hoping that existing staff re-organise into smaller, private units, that can be called upon to perform the work that is still required – albeit for less, and delivered in a more agile way. Perhaps this will be the legacy of the cull – ‘reluctant entrepreneurs‘ – who have been forced into the position to keep going, but not for the country to lose their knowledge.

In conclusion…

It’s a changing world, and certainly not for the best at the moment. If you’ve been involved with the public sector you’ll know it’s a worrying time – the human cost of the recession has probably not hit yet. The best that GIS can do is to reshape itself into something that provides a more streamlined method of operation, and where – in a world of reduced budgets – costs are reduced by using GIS in a smarter, shared, collaborative way.

Ordnance Survey Central Government conference @ Leeds

I was fortunate enough to attend the Ordnance Survey’s Central Government conference in Leeds yesterday. I’d not attended one before. The event was pretty well attended by groups representing central gov’t from around the north and beyond. Two presentations, in particular, drew my attention.

Can you freestyle?

One was from John Carpenter who gave a great presentation on Product Development of OS data product over the next period. It’s pretty interesting how the OS product set has evolved over time, especially given the Phoenix Project ’stuff’ that’s been going on over the last few years. For the OS to reach ‘product nirvana’ where all products are derived from a single capture-base will be a triumph, and John admitted that they are not there yet. However, it’s an ambitious target given the wealth of historic information and the immense legacy the OS have got, so it can’t be any small task.

The item of interest there surrounded the ability to render information from a single code base, at a variety of scales, but still to be relevant and appropriately accurate, whilst not ‘confuddling’ any data that is to be placed on top of it. As ever, this is the issue: historic paper maps aren’t designed to have anything more than coloured pins inserted on them. Digital maps are designed to be manipulated to have a wealth of information placed on top of them.

How do they do that? Well, VectorMap is a good stab at it. I was interested in the OS’s take on it that they didn’t want to supply style files to consumers in order that they use it: they want the user community to come up with styling themselves, especially if they want to place it below their own business data. In my experience, the business community likes to have styling done for them. In larger organisations, they like the idea that it conforms to a common look-and-feel, and will no doubt ask the OS to provide their own – they won’t want to change it. I suggest that, as much as it would probably pain them, the OS provide a set of different styles for the layer (somewhat like Tom Tom and their mix of rendering styles, I guess).

Web services and Government

Tim Martin gave an excellent presentation on web services. I was interested, though, in the audience reaction to how it’s possible to consume OS data from On Demand or via OpenSpace. The reaction – well, I’d describe it as pensive. I was trying to work out why though – the idea of consuming data from a variety of map sources is great and isn’t really new now. ESRI promoted the GeographyNetwork something like ten years ago – On Demand is clearly a large development from that, but the idea is the same.

Maybe a lot of us are still in the desktop safety zone, with data in close proximity to us. That’s highly understandable, both for political and for ‘warm fuzzy’ reasons as well. Tim’s presentation provided the audience the belief that the total cost of ownership in the new web services, cloud-based world is inevitably cheaper. I’m inclined to very much agree. However, getting this change into a number of public sector organisations is a large shift in culture. It also comes at a time when, unfortunately, public sector budgets are under threat. Although it makes sense to move to centralising data and to consume it direct (rather than hosting yourself, where possible), the cost of change, I think, may be the off-putter.

There’s also the threat that web services themselves take away some of the role of the GIS department in an organisation. Being able to download chunks of data, as opposed to applying for it to your GIS department does indeed knock someone out of a role. However, I’d argue that that person is then freed up to do more analysis and business work – stuff that’s of better value to the business anyway. And potentially less arduous.

So, in summary, it was a good occasion and I’m glad I went to it – I came away with a number of ideas and thoughts as to how this will go, though it may take time to make the move to a service world within central government. We shall see.

Managing Regional Spatial Data

I had a bit of time to think about how regional GIS data was managed, in the context of a national system. That is, how national data sets, that are split between regional teams, were best controlled and potentially, eventually, merged – or at least, conceptualised as a single data set. Some time ago I wrote a requirements document on how a regional data store system could work. There are a couple of bogeymen at work here:

• Some regional data sets are captured to local data standards (i.e. locally decided attributes, locally decided positional accuracy)
• Some regional data sets are captured to national data standards

That’s just the data side. Add to that the presentation picture:

• Sometimes, users want to see regional portions of a data set
• Sometimes, users want to see the whole national picture, and expect seamlessness around regional boundaries.

This does, indeed, cause problems. Clearly, national standards are the way forward with this, but aren’t always available, or haven’t been decided yet, or, more than often the case, regional groups have done it ‘their way’. There are obviously a number of choices here:

• Make all the data sets conform to a nationally agreed standard post-event, which might be a subset of the ‘best bits’ of each
• Start again with the national standard that you really wanted everyone to capture too
• Don’t bother with standards

If you were to choose the latter, it might just be business-reasonable, if it’s a low-value data set. However, this implies that you don’t want to see the data in its seamless entirety, or that you’re happy with a cartesian join across all regions that might just give you a massive (and potentially duplicate-bearing) attribute set.

Don’t forget edge matching

And then of course, there’s the seamless boundary creation. If data has been created by different folk at different regions, you’re bound to get some edge match issues. Of course, you’ve got your regional boundaries as templates, so they’re going to be useful to ensure you clip the edges to where they’re supposed to be. And you might choose to extend certain features to the boundary where required, so as to avoid gaps. Gaps like, where linear features don’t match, but you have to take a strategy on mid-point-join, or go with one region over another.

The practical option is, in this case, the conform-to-best-shot case. After all, that’s what data cleansing is about. I’d probably go as far as saying that 1Spatial’s Radius Studio is probably your best friend on this one, because, if you’ve got more than one copy that you want to do some spatial checking on the way in, you’re probably going to go down that route.

Is it justifiable? Well, again, it depends on the value of the data set. If you’re going for something that needs to be nice and clean (as it should – don’t forget we NEED data quality out there), you should be coming up with a master schema that takes in the best elements of each – perhaps with a bit of massaging- and a bit of edge cleaning.

But voila, you can come up with a great number of data sets that look like one person managed the whole country load in their spare time.

Hyperlocal, Facebook and ‘Like’

I was thinking about hyperlocal a bit more recently and how the real issue hasn’t as yet been solved. That is, why there isn’t a definitive ‘thing’ that people do once they need to find local information, and why we haven’t seen a ‘giant’ taking it on.

I mean, there are a number of sites that do a pretty good job of getting there - patch.com is a good example, but there’s still a whole host of alternatives vying for the local marketplace.

It’s going to be interesting to see how this goes: Facebook’s places and ‘like’. I put a ‘like’ button on this site, but mostly for fun. However, there’s something of a pattern emerging here, and it goes like this:

• Local businesses have a ‘like’ button on their website
• Your friends say they ‘like’  a particular business
• You have a link to a recommended business

How’s this important for location? Well, I’d argue that location is the final piece of this jigsaw. Say I need a plumber, as our boiler has gone up the spout (again). I could turn to yell.com and pick one at random. Or, I could pick one that my friends recommend. That’d be better – businesses that come recommended by friends are slightly more trustworthy.

But I’ve got friends in Australia. How do I make sure that I choose a local one?

I’d suggest that your best bet for a plumber would be one with the most recommendations, who doesn’t live far away from you.

So therefore I’d say plumber choice = (max (recommendations) * min (distance))

Now I’m not sure what Facebook has up its sleeve, but this may be where things are going. Link business to customers via recommendations and distance, and you’ve got trusted hyperlocal.

Are Facebook planning this? I don’t know. But it’s worth a think about.