Blog

30
May

Why are Analytics so Difficult?

Almost every company I talk to wants to know which data table, which numbers, which analytics package will provide answers. The unifying question is “What should we be looking at?” I restate the question to say what they really want to know: “How can we get the answers we need?”

Avinash Kaushik writes a brilliant analysis of what to do with that expensive analytics package. Summary: toss it and hire someone who knows what they are doing. Put the money in a person, not in an expensive product that produces hundreds of meaningless reports.

Why is this sound advice? Laws of information.

I keep trying to find simpler ways of explaining this, but it is difficult. So, bear with me and feel free to comment with any questions.

Information is bound to the same natural laws as gravity and physics. Information has to come from a creative source that can develop the information, encode it and transmit it. In the web world, we understand that as site development, design and coding. However, just as critical as the creative source to develop and transmit the information is a receiver that can understand and “decode” the information we send. In web marketing, we have multiple receivers, human and machine, and the decoding is different for each, based on the purpose of the information.

Building a website requires an understanding of search engines, as search engines are one of the primary methods of acquiring visitors and customers. However, once the search engine’s information requirements are met, the site has to be able to communicate a different message to the human visitor. Thus, our sending a message is two-fold. The message of our site must have results, and the expected results are different based on the two receivers.

The machine receiver, the search engine, runs algorithmic rules to determine the importance of a site and where to rank it for certain queries. To the human receiver, the site must provide compelling information and draw the visitor to take an action. The action can be to stay on the site longer, download information, make a purchase, register, or ask for more information.

This leads us to the ultimate goal of sending information, the results. What was the value of sending our information and was the purpose fulfilled? If the purpose of a website is only to gain rankings and the site ranks well, then the purpose is served. If the purpose of the site is to gain rankings and gain sales or leads, then the result needs to be measured in order to determine if the message is effective. In this case, the human is the only measure of value; a machine cannot “appreciate” a message, such as Homer’s Iliad. A machine cannot “appreciate” the combination of words. This is why search engines can be fooled to rank sites with low informational value for humans, but high informational value for machines. Yes kids, that’s called Spam.

This leads us now to why understanding website statistics is so difficult. Website statistics are created from log files. Either server created or JavaScript created. Despite the method, the data is the same. It is aggregated by a machine by how a machine “sees” humans using the site. Because of this, we see reports, graphs and charts, all generated by a machine’s view of the human visitors to the site.

It is easier for the machine to report on “bot” visits to the site, as it is machine reporting on a machine. However, to report on how human visits to the site are arranged is difficult. John Marshall explained this best when he commented on the uselessness of path analysis. “Your users are not cattle!” he exclaimed, “They do not travel your site in a pre-determined path.” Yet we believe a path analysis can provide insight, yet it fails dramatically, as there is no “typical” user to a website.

The key is this: Questions.

Questioning is the ultimate human ability. Only in asking questions can one uncover the data that is happening in website statistics. Drilling down and segmenting information, is a result of asking questions. Simply browsing machine generated reports will not provide insight, but asking questions of the data to find the mind of your users will uncover a gold mine of information.

Socrates would be proud, as this method of analytics fits his method of questioning, that questions lead to more questions. Except in this case, the dialogue is not between to people, but between the two parties to whom have the most at stake in the transmitted information; the human and the machine. Asking questions is an active process, forcing critical thinking, which is necessary to understanding website statistics. The Socratic Method helps us to release our preconceived notions by realizing answers may not be what we believe they are. In doing so, it opens us to effective communication; it enables us to be teachable, which is naturally human.

About Matt Bailey
Matt is the owner and founder of SiteLogic and has over 15 years in the internet marketing industry. He focuses on consulting and training to help companies take control of their websites and marketing strategies. You can find out more by reading his book: Internet Marketing: An Hour a Day

1 Comment for this entry

Clint
June 2nd, 2006 on 2:56 am

I have found that it is needful to back up one more question from “How can we get the answers we need?”

It has been my experience that business owners often cannot define what success is for their websites. So, they can’t even frame ‘need’ correctly.

Often you have to turn the questions back to them and start with “How will you know if you are winning?” If the business owner can (with or without your help) articulate what determines success, then you can move to how they can get what they need.

This is one of the most important reasons why investing in smart people is so critical for analytics, the analyst need to be able to take business success (and failure) criteria and translate those to the available quantitative and qualitative measures available via survey, customer service, and web analytics data.

A software package, no matter how robust just can’t do this.











+ 7 = nine