Problem solving framework for analytics

In my previous blogs, I covered the several types of analytics (descriptive, predictive, and prescriptive), including Big Data Analytics.

By now, we know that, the primary objective of analytics is to aid decision making. In the process of enabling decisions, problems will have to be solved. How does one solve a problem using analytics?

The typical framework for solving an analytic problem would be:

1) Defining the objective/problem

The problem definition could be anywhere between straightforward process change (e.g., reduce the time for KYC approvals, to increase customer satisfaction) to complex strategic initiatives (e.g., identifying a specific segment of customers, that will buy more goods).

This is probably the most critical step in the analytical framework. Many analytics projects fail and/or don’t yield the right results, as the end objective is not clearly defined.

2) Knowing the problem category

Once the problem definition is clear, the next step is to know the category of problem. Broadly there are 5 types of categories.

Many simple problems get convoluted with the wrong choice – why need a gun to kill an ant? Be wary of choosing the appropriate ones!

3) Selecting the appropriate analytical techniques

Once the type of problem to be solved, and the category is clear, then the focus is on choosing the appropriate technique. A technique defines the “algorithm” for solving the problem.

This is usually a confusing step, as there are very many options. By and large, businesses deploy few well-known and popular techniques, to address a majority of the problem.  A beginner would do well to start with the popular techniques and then continuously expand the knowledge base. Some of the commonly used techniques are:

A couple of key aspects to bear in mind:

1. Choosing the technique is at times a trial and error approach – try out multiple techniques and choose the one that produces the best outcomes
2. A technique is not necessarily tied to the methodology. For example, Logistic Regression is also used for solving classification problems
3. Many a times, a combinatorial/hybrid approach will be required – combining multiple techniques to arrive at the desired outcome

4) Executing

This is the stage where everything chosen in the previous steps gets applied to solve the problem. This is an involved and an iterative process, comprising of:

1. Data gathering: The data required to derive insights from are gathered in this step
2. Data preparation: Preparing the collected data to be used in the analytical models. This involves formatting, cleaning, transforming etc
3. Analytics model building: Irrespective of simple summarisation or complex algorithms, this step involves building the right model. A model is a combination of features (or variables or predictors) that represent the input data, statistical computation to be applied, and the presentation of the output. For example:
   1. Descriptive Analytics: If you are building a dashboard to understand monthly revenue from product lines, then input features would be the historical data on “Month, Revenue, Product Lines”, and the computation to be applied is “summarisation by month & product line”
   1. Predictive Analytics: If you are building a model to predict weather, then the input features would be the historical data on “Day, Rained? Temperature”, and the statistical technique to be applied is “finding the probability for rainfall, given a certain temperature”
4. Validating: In this step, the results from the model are validated. The output is analysed for accuracy & correctness
5. Fine-tuning: Any analytics program is an iterative approach, which involves experimenting with various features and models, until the best results are achieved

The Big Picture!

A MIT Sloan study, found that “data is not the biggest obstacle” to adopting analytics, despite popular opinion. It is the “lack of understanding of how to use analytics to improve the business” which is the biggest obstacle.

A proper understanding of the analytics framework, will help overcome the biggest obstacle.

In the next blog, let’s get a grip on Machine Learning and Artificial Intelligence, and how it ties up to the analytics framework.