Good Data Intentions
Not too long ago, Verizon, Sprint, and T-Mobile battled it out in their commercials using colorful balls to represent data, telling us why they were bigger, better, faster and more reliable than their competitors. (If you haven’t seen the commercials you can check them out here.) It was humorous watching them try to outdo each other with big, eye-catching numbers. Now, phone companies aren’t the only ones using data. In a world with easy access to real-time data where we can track everything – from users download speeds to the number of steps we take in a day, companies are using data to help make informed business decisions. The problem is, although informed, the knowledge gained isn’t always being used to make the optimal decisions. If used incorrectly, data can be as tragic as the classic story of Romeo & Juliet.
Just as Romeo and Juliet’s love was full of good intentions, most companies’ love for data started there too. Many companies will collect lots of quantitative data using analytic tools and continually watch the numbers. If a number is too high, low, or changes dramatically, they will quickly implement a new solution to make the numbers look better. Although admirable, this approach has huge implications if companies don’t take the time to look at the bigger picture. As we cipher through all of the analytics, we need to ask ourselves:
- Does this data tell me what I think it’s telling me?
- Why are users doing what they’re doing?
Let’s think about a commonly collected analytic like time spent on a site for a blogging company. The numbers show users are spending lots of time on your site. You think, great, people are enjoying our blog, and we are going to rake in a huge profit!’ However, a few months down the road you’re scratching your head as the profits don’t seem to be rolling in. The data wasn’t wrong, how you interpreted it was. You assumed because users were spending more time on your site because they were enjoying it, when in fact, they were spending more time because users couldn’t find the content they wanted.
You think, “No problem!” We will add a new searching component and filtering system so users can quickly narrow down the content. Then, measure how little time users on our site because it won’t take as long to find what they want. You see as the months go by your numbers drop, hurrah! But again, the profit line isn’t changing. The data must be lying, you say! No, the data isn’t lying, it is telling you a story. The problem is, this is only PART of the story. If you had asked the question WHY are users not finding the content they wanted, you would have discovered it wasn’t because users were struggling to search for content. Rather, it was that your content wasn’t fitting the user’s needs. So, instead of spending money on new searching and filtering capabilities, you should have been spending money on new content that would be relevant to their needs.
Uncovering the ‘Why’
So, why are users doing what they are doing? There are multiple ways to interact with users, whether through Guerrilla research, A/B testings, surveys or other numerous tactics. Just as we can fall into misleading quantitative data, we must also be wary of not falling into the trap of creating misleading qualitative data. How can users be wrong, you might ask. Take a look at the following questions you may see on a survey.
1. What type of vehicle do you own?
2. How much more space do you think a van has compared to an SUV?
3. Do you always have your oil changed regularly?
At first glance, the questions don’t seem bad. However, the way they are written may alter the user’s perspective and/or unintentionally force them to answer a question inaccurately. For example, take a look at the first question. We are assuming our user owns a car AND that it is one of the four choices. What do they select if they own a Convertible or an Electric Car? The second question has a leading bias. The wording already suggesting vans have more space compared to SUVs. What if the user didn’t know that vans had more space compared to SUVs? Finally, the third question leaves the user in question. What is regularly? Most of the time I have my oil changed every three months, but I sometimes forget…should I answer yes? All of these questions could lead us to gather misleading or misinformed information. Thus, it’s important to not only to ask questions but to ensure they are unbiased.
Where to go from here?
Data can be fickle, but don’t throw in the towel! Data is still a powerful tool. The key is, it can’t be our only tool for gaining knowledge and making decisions. As Erika Hall so eloquently put it,
“No matter how good your data, you will need to ask the right questions, interpret the answers, and determine the implications for what you’re designing.”
Find out more
If you are interested in this, or anything else Universal Mind has to say, sign up for our monthly newsletter.