I once saw Clayton Christensen talk about the growing obsession with data and the decisions made by the collection of more and more data. He was lamenting the rise of big data and the march towards blindly following what the data says. In fact, he went as far as saying that (and I’m paraphrasing slightly) ‘Data should go to Hell. Why? Because it lies!”
Previously, I wrote a post detailing the way Google decided on the shade of blue they use on their search result pages to show links. It’s a great example of how data can be used to hone in on a decision. But, I’ve heard people in other great product companies say that this isn’t an approach they would always endorse. They would rather watch and observe the reactions of users. In fact, it seems they are looking for feedback from experiments and hypotheses, but they’re more rooted in observational data rather than raw numbers. The way Apple describe their approach suggest a more intuitive, human-centered approach rather than one based predominantly on number based data. In fact, it’s described as part of the magic in their development process.
The reality is that both are important. There is a qualitative and quantitative approach needed. Collecting large quantities of data without being clear on the question being asked or the expected response is next to useless. But so is also relying on one or two interactions and assuming that it speaks for all. Today, there is a trend being discussed called Thick Data – it’s the idea that identifying a smaller set of users who are extreme or core and matching what the numbers say with how the users think, feel and behave is the optimal approach.
What type of evidence gathering will work best to combine the best of intuition and data for your work?