Logistic Regression: The Discrete Beauty!

“Why was your recent credit card application rejected? How did your telecom service provider figure out you were unhappy? How did your friend predict your favourite team losing in the world cup?”

It’s logistic regression at work.

How many times have we wondered, if life could be a simple yes or no? One-day, logistic regression, might take us there. Let’s know more about it.

What is Logistic Regression?

Logistic regression is predictive, like other regression analysis techniques. The key different is that the output is discrete. In other words, the outcome of logistic regression, is the probability of an event happening or not. Basis, the probability, and the types of output required, several types of logistic regression used are:

Type Output of dependent variable
Binomial Logistic Regression Only 2 possible outcomes (0,1). For e.g., Yes/No, Win/Loss, Fraud/Not-Fraud
Multinomial Logistic Regression 3 or more possible outcomes, but discrete. For e.g., Yes/No/Maybe, High/Medium/Low
Ordinal Logistic Regression Discrete possible outcomes, but ordered. For e.g., Excellent, Good, Average, Poor

Therefore, logistic regression is widely used for classification problems.

What is under the hood?

At the heart of the logistic regression models, is the S-Curve (sigmoid curve). The Sigmoid curve can take any real-value, and produce an output with value between the range 0 and 1.

Source: https://en.wikipedia.org/wiki/Sigmoid_function

Therefore, this finds favours when classification problems need to be solved. Take an example, where you want to predict the outcome, as loan-approved (1) or loan-denied (0). There could be many input variables (demographics, salary, credit score, etc), basis which the outcome is decided. You might build a business process, like the one below:

Output (0 – 1 Range) Business Outcome
<0.5 Loan – Denied
>0.5 and <0.75 Loan – Hold (perform secondary verification)
> 0.75 Loan – Approved

Further reading & learning

For the ones interested in getting into the depths of the subject, these are the topics that will come handy.

Maximum Likelihood Estimate https://en.wikipedia.org/wiki/Maximum_likelihood
Sensitivity & Specificity https://en.wikipedia.org/wiki/Sensitivity_and_specificity
ROC Curve https://en.wikipedia.org/wiki/Receiver_operating_characteristic

This brings us to an end of the commonly used regression tools. In the upcoming blog, let’s learn about the decision trees.

2 thoughts on “Logistic Regression: The Discrete Beauty!

Comments are closed.