Research suggest that the dashboards and visualization tools we use for decision making may actually hinder us from gaining value from data. Here's why making data harder to consume can increase the actionable insights we can gain from it.
I'm surrounded by data. On one screen I have a marketing/sales dashboard, on a different one I have operations data on all of our client related activity - costs, activities, revenue, categories, resource, etc. The data is represented in raw tables and graphs while some of it is organized in easy to view dashboards and charts. It's a lot of data and much of it is valuable. Yet, regardless of whether the data is presented raw and uncut, or if it's presented as an easy to digest visual, I'm simply not turning as much of it as I should into usable knowledge or action. And I'm assuming I'm not alone in this issue.
Charles Duhigg highlights a similar dilemma in the last chapter of his book, Smarter Better Faster. South Avondale Elementary School, in 2007, was ranked as one of the worst schools in Cincinnati. Located in a low income neighborhood with high unemployment and crime, the school had been declared an academic emergency. However, the story departs from the typical narrative because resources were not an issue.
Thanks to benefactors such as Procter & Gamble, the City of Cincinnati had been investing heavily in the downtrodden community's public schools - almost 3 times more than affluent schools in nearby areas. In addition to educational resources for the students, Cincinnati public schools invested in software and hardware. These tools were used to track each student's attendance, homework, test scores and participation which could be viewed by parents and faculty on cutting edge visualization dashboards. Memos and spreadsheets detailing this data would show weekly, monthly and yearly status of all students. However, after 6 years of having the online dashboards available, schools such as South Avondale were no better in terms of academic improvement. In fact, 90% of the teachers admitted they did not even look at the dashboards or use the data sent to them by the district.
Frustrated by these results and their lack of growth in analytic maturity, in 2008 the school system decided to change tactics by developing the Elementary Initiative (EI). As it stood, the program stakeholders did not come to the conclusion that data was the problem. They felt data was transformative - but it could only be used if it was understood. Through this new initiative, the principals and teachers within these schools were taught how to look past the dashboards and visualization software and were educated in how to interpret the data being collected manually. From these insight they were then able to form plans and ultimately actions.
Within two years of starting EI, students at South Avondale went from only 37% meeting standards to over 80% hitting the mark. They went from academic emergency to model school. The reason for the improvement was attributed to the data and the use of this data to help students and teachers improve.
The irony of the story is that the EI program made use of the data but eschewed the online dashboards and visualization tools from the district. Instead, they mandated that teachers use a "data room" where they would meet to transcribe data onto index cards and draw graphs on butcher paper by hand. Based on the data they would then plan and execute experiments in the classroom. These experiments involved smaller learning groups, teachers trading classrooms, and other similar changes which were all monitored and manually mapped to see if results were positive. Engaging with the data rather than passively absorbing it was the difference maker.
This shift from passive to manual interaction with information is an example of cognitive disfluency. In general, people like things to be fluent - easy to process, more familiar and comfortable. While there have been studies that revealed lawyers with easier to pronounce names ascend more quickly, and stocks with less complicated names initially do better, new studies are revealing that making it a little more difficult to process information may have some benefits. This effect is called cognitive dis-fluency. Adam Alter, a researcher at NYU believes disfluency may prompt people to process information more carefully, deeply and abstractly.
Disfluency is kind of like an alert system that triggers when we are taken away from our comfort zone. Like the principals and teachers in the Elementary Initiative, this shift to something less comfortable makes us pay better attention to what we're doing. A direct example of this is when we encounter a font that is more difficult to read or interpret. Digesting it forces us to focus on the wording more closely to understand it.
This study was repeated with different variables, but each time the results indicated that those with hand written note scored higher. It seems as though the disfluency of handwriting compared to type written notes required more concentrated effort in both writing and interpreting/studying after class, thus leading to better decision making during testing, ultimately leading to better outcomes in performance.
On a personal level, and much more anecdotal than anything else, I find myself looking at dashboards waiting for them to tell me the answer. Sometimes, like when a gauge goes into the red, the information provided is absolute and immediately actionable, but most of the time it isn't. Data can be transformative and having it presented into easy-to-view formats can be tremendously beneficial, but only if we actively engage with it.
As the Cincinnati School District and other examples show, our brain is wired so that sometimes, making it a little more difficult to process information, makes us more likely to understand that information. The value of data is best when we understand it and use it to drive further action. Technology is moving forward where it can not only provide data, but also tell us the action to take and perhaps even take the action itself. But those advancements are still in their infancy. For most of us, to derive the real benefits from data in our professional and personal lives, we're required to make initial interpretations of the presented information. It's then that we can formulate actions intended to push continued measurements of the data in the desired direction.
In IT, when it comes to data, our roles are focused on the collection and presentation of information. But, perhaps we shouldn't stop there and instead ask, 'what happens next?' If it is not being turned into knowledge by other parts of the the business, what can be done? Can IT help increase engagement by the users? Would increasing disfluency of the data prompt users to process it more deeply? These are not the typical areas that IT wades into, but if companies are going to realize real value of data, maybe we should.
For now, the Intelligence in BI is still you and me.