Change tracking
Analytics is not a means to an end in its own right. Even insight that comes from analytics isn’t a means to an end in its own right. Looking at analytics data provides some initial insight but often (in my experience) it has thrown up almost as many questions as it has answers. That’s not a bad thing. The ultimate goal is action that leads to positive results.
Analytics can’t answer all questions so directly asking visitors via usability studies and/or surveys is a good way to find answers to questions that analytics data falls short on. There is also another source of information which often seems overlooked.
Many sites undergo frequent if minor tweaks and updates. Improvement often works better and is less risky as an iterative process rather than as a big re-design. In addition, if a number of issues have been identified, by picking them off one by one it’s easier to assess the relative impact as solutions are implemented.
There have been many occasions when I have been to see clients to go through the findings of an analysis and I’ve had questions as well as answers. I’ve not always had the benefit of usability studies or site surveys to help unearth answers to my questions so I’ve simply had to ask my client. On occasions like this I’ve been met with blank faces or furrowed brows as memories are searched for activity that took place around the time (shown on the trended chart) where change is apparent. Quite often an explanation is clear but this raises another issue, that of change tracking. It’s amazing how many organisations appear not to log this (adequately).
There are literally dozens of events that can have an impact on site performance, ranging from a complete site re-design to a sunny day or public holiday. I’m not suggesting that weather conditions should be factored into every site analysis, although some might consider it important enough, but changes to marketing activity on and off line will have an impact. Additionally, back end technical issues will have an impact; changing the position of a link or even just the destination page will have an impact; changing some wording; tweaking a business rule, the list goes on. Remember also that a relatively innocuous change can’t have a disproportionately significant impact.
With all the minor and not so minor tweaks that continually take place it makes sense to log these in a central data base. This could be as simple as a collection of spreadsheets for each department to a central resource that can be access by all relevant stakeholders. Each change is logged by date and ideally time of day. The benefit of being able to refer back to such a resource and accurately identify what took place around the time of a change in performance can’t be underestimated. In addition the cumulative learning that can be built up if performance tracking were included in such a database would be invaluable. If you’re reading this and you’re not doing it, give it a shot.
