Retailers need to beware of emotionally analysing data and skewing the results. Good data analysis can help a company but bad data analysis can ruin it.
Data is not just another part of the retail mix; it’s quickly becoming the most important part. Profit and loss, foot traffic, staff performance, marketing success — the returns, or lack of them, can all be proven through thorough analysis of your company’s or third-party data. But data analysis can be as dangerous as it can be decisive. Done wrong, it can harm your retail business.
I see the emotional impact of data analysis far too often, and it’s not something exclusive to those new to data. Data is often thought of as black and white. Data is made up of numbers, and many people carry the preconceived idea that numbers never lie, that the numbers themselves should be able to tell the story, and a very accurate story, but it’s actually the analyser that often ends up telling the story using the numbers as an aid for a desired outcome.
Here is a common occurrence: two people from the same company come together to analyse a data set. They go through the figures and end up coming to completely different conclusions. People bring baggage to the analysis. They have opinions on what they want the figures to prove, and it’s common for them to, subconsciously or otherwise, twist those figures to try and achieve a desired outcome. It’s called cognitive bias. This doesn’t mean tampering with the data, but prioritising certain information that lends itself to strengthening a point of view formed long before seeing the data.
This puts retailers in an extremely dangerous position. Future planning becomes compromised as staff begin to make decisions as a result of cognitive bias disguised as decisions backed up by hard data. Trading hours, certain products to stock, e-commerce strategies — the issues may seem small to begin with, but slightly unstable foundations have a habit of creating terribly unstable buildings.
The solution is part acknowledgement, part discipline. Now that you are aware of your potential cognitive bias, ask yourself: have I explored every possible outcome? Have I used all the small details available to paint the bigger picture? Am I confusing correlation with causation? Is my data statistically significant?
In the most basic sense, statistical significance indicates there is a low probability that your hypothesis is not true by mathematically assessing the difference between group averages, sample size and standard deviations of a group. If you are making business decisions before your data is statistically significant there is a good probability that your decision is wrong, which can cause serious harm to your organisation. Make sure you’ve collected enough data with simple calculators like this one from Kissmetrics. It will help you decide if your data is statistically significant and give you mathematical confidence that you’re acting on a valid outcome.
But just because your data is statistically significant doesn’t mean you’ve explored every option. When analysing data, it’s imperative that you go in with an open mind. If you are attached to a certain outcome, you may find yourself grabbing at bits and pieces of the data that partially prove your point. Remember, correlation does not imply causation. Sherlock Holmes would never close a case with only one clue; neither should you.
An example: your web traffic decreases 30 percent week-on-week (WoW) even though you have the same spend. You ‘think’ it must be because the new creative the design team forced on you is not working. You check the click-through-rate (CTR) and sure enough this is down 25 percent WoW. It seems obvious that it is the creative, but if you explore deeper you might find your budget has shifted from retargeting campaigns to prospecting campaigns. These typically have a lower CTR and thus decrease CTR as a percentage of spend.
Good data analysis can help a company but bad data analysis can ruin it. Make a commitment to yourself and your company that you will go into an analysis with an open mind and explore all available outcomes. For a good analyst, this is the only option and will ensure the decisions you are making are sound for the company.