What is Confusion Matrix?

An N x N matrix called a confusion matrix is used to assess the effectiveness of a classification model, where N is the total number of target classes. In the matrix, the actual goal values are contrasted with those that the machine learning model anticipated. This provides us with a comprehensive understanding of the classification model's performance and the types of mistakes it is making.

A 2 x 2 matrix with 4 values is what we would have for a binary classification problem:

Now let's interpret the matrix:

• Positive or negative values can be assigned to the target variable.

The target variable's actual values are shown in the columns, while its anticipated values are shown in the rows.

True Positive (TP): When the model's predicted value matches the actual value and the actual value was positive.

True Negative (TN): When the model's prediction and the observed value coincide. When the observed value was negative and the model had anticipated a negative value.

Type 1 error: False Positive (FP)

Also known as the Type 1 mistake, this error occurs when the anticipated value is incorrectly forecasted, the actual value is negative, while the model projected a positive value.

Type 2 error: False Negative (FN)

Also known as the Type 2 error, these conditions include: the predicted value was incorrectly forecasted; the actual value was positive although the model projected a negative value; and

To help you grasp this better, let's use an example. Consider a classification dataset that contained 10000 data points. We apply a classifier to it and obtain the confusion matrix shown below:

The Confusion matrix's various values would be as follows:

True Positive (TP) = 6500, indicating that the model accurately categorised 6500 positive class data points.

True Negative (TN) = 2300, indicating that the model properly identified 2300 data points in the negative class.

False Positive (FP) = 700, which means that the model misclassified 700 negative class data points as being in the positive class.

500 positive class data points were mistakenly assigned to the negative class by the model, resulting in 500 false negatives (FN)

Evaluation parameters

1. Accuracy
2. Precision
3. Recall
4. F1-Score

The following are the evaluation parameters considered:

Accuracy: The number of all accurate predictions divided by the overall dataset size yields accuracy (ACC). The accuracy ranges from 0.0 to 1.0, with 1.0 being the best. You can alternatively calculate it by using 1 - ERR.

Technically, Accuracy is calculated as the total number of two correct predictions (TP + TN) divided by the total number of a dataset (P + N).

Accuracy=(TP+TN)/(TP+FP+FN+TN)

Precision: Precision is an evaluation metric that combines relevant and successfully retrieved items over all of the results that were successfully obtained. When the likelihood of a false-positive prediction is large, it is mostly employed.

Precision (TNR) = TP/(TP+FP)

Recall: Recall is a measure when a False negative is considered.

Recall (Sensitivity or TPR) = TP/(TP+FN)

F1-Score: F1-Score is an evaluation technique that maintains a balance between precision and recall.

F1-Score = 2 * (Precision*Recall)/(Precision+Recall)

What is the Purpose of a Confusion Matrix?

Let's consider a classification issue before we respond to this query.

Consider the scenario where you want to segregate those who are infected with an infectious virus from the healthy population before they begin to exhibit symptoms. Our goal variable would have the following two values: Sick and Not Sick.

You're probably thinking why we need a confusion matrix when we already have Accuracy, our go-to companion in any situation. Let's see where accuracy falls short.

Let's take an example of an unbalanced dataset. The negative class has 947 data points, while the positive class has only three. We'll calculate the accuracy as follows: