Deriving insight and intuition from business data: Review by example

‘Insight’ combines several ideas. It includes ‘classic’ areas, such as knowing who consumers are, what they do, where they are, what they buy, what they would like to buy, what media they are exposed to and what media they choose to view, listen to or read. It also includes more psychological areas – what consumers think and feel, what their objectives and strategies are, and how these influence the way they behave. The list of attributes to examine is long and it should remain in the hands of the researcher or data analyst which variables are retained in the model and which are excluded. Hopefully, these decisions will be based on a thorough knowledge of the phenomenon being modelled. We will often be in the position of either building or interpreting a regression model that has been constructed from a large database consisting of many variables. Having gone through the process of building such a model, we will have a better sense of what sorts of subjective choices are made along the way. Statistical models are an assistant, not a master and this article gives an introduction to the subject by reviewing some of the widely available algorithms and comparing their capabilities, strengths, and weakness in three business area examples: direct marketing, predicting financial indicators, and market mix modelling. In doing so, the article creates ‘common language’ between analytics personnel (Statisticians, data miners, and computer scientists) and management (personnel with marketing and financial expertise) and narrow the gap that exists between algorithmic reasoning (theory) and practical application such as Return of Investment (ROI) for business decision making.