Do web analytics providers have an obligation to inform customers of the shortcomings in the tools?
Friday, February 19th, 2010Lately I have been coming across a series of small but potentially significant shortcomings in the web analytics tools I have been using. I only unearthed them as I was playing around with different ways of getting the same output, in other words sense checking the data I was looking at.
The trouble is if I hadn’t done that I might have taken the first answer as gospel and moved on, unaware that the insight I was inferring was based on incorrect data. I’m talking about the kinds of things that are genuinely obscure and might not impact that many users but for those that do they could make a big difference.
Given that the industry’s mantra is broadly ‘rubbish data = zero insight and worse, possibly damaging action taken as a result’, it seems as though the suppliers of web analytics tools have a responsibility to document and make freely available the shortcoming of their products especially where those shortcomings could have a serious impact. Of course the trouble that would be suicide.
It’s a catch 22, keep quiet and hope that you don’t do too much damage to your client’s site’s or take a more responsible standpoint, ’fess up and deal with implications that come with it.
