This post is part of a series on how the 9 Laws of Data Mining from Tom Khabaza can be applied to analytics. You can find previous posts here.
Law #6: “Insight Law” – Data mining [analytics] amplifies perception in the business domain
Data mining tools are “perception amplifiers,” increasing our understanding of the business world. Tom Khabaza uses the term “intelligence amplifiers” from the field of Artificial Intelligence. The process of data mining or analytics reveals patterns, context, and connections that we would not normally be able to identify by simply looking at the raw data. The tools allow us to discover outliers, unknown associations, correlations, clusters, complementary processes, new classifications, and even in some cases, an estimate or prediction of why the pattern is happening or what results the pattern might produce.
Khabaza’s 6th Law of Data Mining is called the “Insight Law.” While data mining and analytics tools can’t produce insight on their own, they can lead us to an effective insight. What do we mean by this? Insight, according to Merriam Webster is, 1) “the power or act of seeing into a situation,” or 2) “the act or result of apprehending the inner nature of things or of seeing intuitively.” For the purposes of data mining and analytics, we can define it this way: “Insight is the act or result of perceiving the value of the information for a business.” An effective insight would be when the insight can be turned into action in order to improve a business situation.
Essentially, the data mining process turns data into information, but this is not enough. We need insight to solve business problems, and this is done by creative people, not by data mining algorithms, GIS, or statistics software. In order to understand how we develop insight, I recommend using Bloom’s Taxonomy which is used in the education field to classify different levels of learning and thinking. The taxonomy is represented as a pyramid with the most basic levels of thinking at the bottom of the pyramid. Thinking skills and learning advance as you move up the pyramid. There are currently two versions of Bloom’s Taxonomy. The original version used nouns. The revised version uses verbs and made some slight changes to the connotations of the levels. Both versions have some merit, so they are shown side by side below.
Data mining tools are essentially helping us with the lower levels of the pyramid, but we need to progress to the higher levels of the pyramid to gain insight. In fact, the data mining process leads us through the entire taxonomy of thinking skills. First, we feed the model data (Knowledge – facts, things that we know) and these are “remembered” in order to produce information or understanding. Using our analytics tools we can enhance our understanding of the data to apply the results of the models to our hypothesis. From there we continue to apply and analyze the results until we have a model that best represents the data and the business context.
However, at this point, our computer tools still have some need of improvement. In the evaluating level we make judgments about the utility of the model for the business purpose. I also like the word “synthesis” at this level because insight is about making connections. Computer systems can only make connections within the data that we provide. On the other hand, our minds have a beautiful capacity to make external connections and these become the true meaning of insight. Finally, if we can determine a positive value for the analytics (the insight), we can then create the plan of action.
This is not to say that a business that does not employ data mining and analytics will not be successful; only that a business that follows the data mining process will have the capability to identify patterns and act upon them before others in the market. A startup business can be very successful without any formal analytics, but the longer the business exists, the more likely that it will need to change in order to continue its success. Knowing how to change and making decisions that will be the most beneficial to the business are the result of the data mining and market analytics processes being used by creative minds.
Of course, the existence of the data mining and analytics will not translate into a guarantee of business success on their own. The results will not be useful unless the human problem solver translates the results into a recommendation for an improved business process. The insight must be interpreted in the context of the business value and then actions need to be taken to ensure that the proper change to the business process occurs.
