The problems of measuring eGovernment progress

— Published August 31st, 2005  |  by David Moore

Not as much as you might think. Until very recently, most analyses of the eGovernment landscape assessed sites based on the targets that governments had themselves set.

These mostly covered getting as many services online as possible – the UK and Ireland both set similar goals of full service offering by 2005, for example.

The reports referred to this as 'maturity' or 'online sophistication', and it was relatively easy to measure – either a site was there or it wasn't, either it offered online transactions or it didn't.

Consulting firms Cap Gemini and Accenture both release annual reports of this maturity (Cap Gemini looking solely at European countries, Accenture examining a selection of countries worldwide), and historically Ireland and the UK performed well in these rankings – as early adopters, they had a lot of work done quickly.

Over the last two years, the growth in sophistication in these countries slowed (as there just weren't that many projects left to do), and other nations started to catch up. But the 2005 Cap Gemini report still showed the UK and Ireland joint third in eGovernment online sophistication. Not too shabby.

What's wrong with maturity?

The biggest problem with these maturity calculations is what they don't measure:

1. is anyone actually using any of these services?
2. do the sites deliver a quality user experience (in other words, are they any good?)

It's taken massive expenditure for governments in the UK and Ireland to start asking what value eGovernment sites are delivering to the people who matter most: us, the citizens.

In this year's annual survey, Accenture finally acknowledge that maybe they (and the administrations they advise) weren't looking at the right thing in assessing eGovernment projects: 'governments are making service investment decisions without a clear view of the outcomes they effect. In our view, a fundamental measure of service delivery success is the actual adoption of services and how governments turn that adoption into value.' You think?

Enter, finally, the customer

This year, for the first time, Accenture included a customer service element in their analysis, and seen in this light, the UK and Ireland suddenly don't look so great. On the customer service measurement, Ireland is joint 12th, behind Italy, for example. The UK is joint 14th with Mexico and Belgium, and only just ahead of Malaysia.

No disrespect is intended towards those countries and their eGovernment efforts, but being in a group with Mexico and Malaysia for customer service takes some getting used to after all the progress we've been told we're making.

And where now?

In the UK, the head of eGovernment, Ian Watmore has acknowledged that the race towards maturity is over, and there' a need for new, more meaningful measurements. Speaking to the BBC, he said, " The getting 100% of services online target is something I inherited and the job is pretty much achieved. The real question is 'where do we go from here? Let's make them [online government services] as good as we can and, most importantly, let's move to the point where most people are using them rather than some people are using them".

It remains to be seen if the current government in Ireland grasps the changing focus as clearly.

It's clearly a time of transition in both countries' eGovernment provision, with consolidated first points of contact for citizens' services coming online (DirectGov in the UK, Reach Service in Ireland), and a growing awareness of the importance of delivering real quality to visitors across the provision.

In this environment, iQ Content is releasing its forthcoming eGovernment benchmarking report, analysing and ranking 42 British and Irish sites based on the user experience they deliver. Because to us, that's the only measurement that counts.

iQ Content's eGovernment Benchmarking Report 2005 will be published on Thursday 22nd September. For enquiries, or to pre-book your copy, email Annmarie: annmarie@iqcontent.com.