Roles of Confusion Matrix in IDS in the Cyber Security world.

What is Confusion Matrix?

As the name suggests the confusion matrix might confuse you in understanding what it showcases:), just kidding!!

Let me tell you with an example what really a confusion matrix is. As you all know that nowadays Machine or Artificial Intelligence has taken over much of the tasks, and Machine Learning is mainly used for analyzing the data and predicting the result. So, let’s take an example of ML that in a hospital there is a dataset that contains different blood tests taken by the different patients and so the hospital uses Machine Learning to predict that how many patients are non-diabetic based upon the dataset of the patient i.e different blood tests that patient has taken, etc. Now let’s say our Machine Learning model predicts that out of 100 patients 55 are non-diabetic and obviously then 45 will be diabetic i.e our Machine Learning gives 55 “yes” and 45 “no”. So to check the accuracy of our Machine learning model we take the help of Confusion Matrix which gives us this type of result(acc. to example)

So, in the above output, that confusion matrix gives us, the first cell (i.e (1,1)) represent TF (True Positive), 2nd cell (i.e (1,2)) represents FN (False Negative), 3rd cell (i.e (2,1)) represents FP (False Positive) and last cell (i.e (2,2)) represents TN (True Negative). So what is these all TF, FN,TN and FP so lets understand it from the diagram:

• True Positive(1st cell): It means that if we compare our machine’s prediction with the actual value then we can see that the Machine has predicted the result right i.e it has given us the correct output and since according to the matrix it shows 40 patients are True Positive, so it means that for 40 patients our machine learning has predicted right i.e it gave a “yes” as output and as it predicted that they are not diabetic so it is positive for us i.e good news for us so we say it TRUE POSITIVE( True for machine prediction and positive for us)
• True Negative(4th cell): It means that if we compare our machine’s prediction with the actual value then we can see that the Machine has predicted the result right i.e it has given us the correct output and since according to the matrix it shows 20 patients are True Negative, so it means that for 20 patients our machine learning has predicted right i.e it gave a “no” as output and actually those patients were not non-diabetic. So, as it predicted that they are diabetic (which is true) so it is negative for us i.e bad news for us you can say, so we say it TRUE NEGATIVE( True for machine prediction and negative for us)
• False Negative(2nd cell): It means that if we compare our machine’s prediction with the actual value then we can see that the Machine has predicted the result wrong i.e it has given us the incorrect output and since according to the matrix it shows 30 patients are False Negative, so it means that for 30 patients our machine learning has predicted wrong i.e it gave a “no” as output and actually those patients were non-diabetic. So, as it predicted that they are diabetic (which is false) so it is negative for us i.e bad news for us you can say, so we say it FALSE NEGATIVE( False for machine prediction and negative for us)
• False Positive(3rd cell): It means that if we compare our machine’s prediction with the actual value then we can see that the Machine has predicted the result wrong i.e it has given us the incorrect output and since according to the matrix it shows 10 patients are False Positive, so it means that for 10 patients our machine learning has predicted wrong i.e it gave a “yes” as output and actually those patients were not non-diabetic. So, as it predicted that they are not diabetic so it is positive for us i.e good news for us so we say it FALSE POSITIVE( False for machine prediction and positive for us).

Now after you know what is a confusion matrix let’s see how it finds the accuracy. So in the above picture of a confusion matrix diagram if we add the 1st and 4th cell values then it will give us the total no of correct answers given by our machine and if we divide it by total no of values in cells than that is the accuracy of our model. Eg. in the following picture,

Accuracy is [(40+20)/(40+30+10+20)]x 100= 60% accuracy of our model.

This example might clear your thoughts or any confusion about the confusion matrix. Now lets see how confusion matrix helps in the cybersecurity world

Confusion matrix in Cyber Security

In cyber security we have many IDS ( Intrusion detection System ) which work on Machine Learning i.e these IDS run ML model behind the scene to analyse the network packet and see if the coming packet is used by the attckers to get into the system. So, the SecOps team i.e Security Operations team of any company that use IDS uses confusion matrix to find the accuracy of the model i.e the security team checks the no of False Positives from the confusion matrix that the model is giving and accordingly make the model better.

The False Positives of the confusion matrix for a ML model is very dangerous for the company as false positives in the IDS means that if the IDS analyze the packet and says that this packet does not seem to be of attackers then it means that though it is positive( as we are relieved that no one is attacking us) but it is false i.e that packet is sent by the attacker. So these false positives of the model are being reduced as far as possible by the SecOps team of a company by using the confusion matrix.

In most of the cases it is observed that the cyber attacks that happen onto a website or company is due to the negligence of the false positives by the security teams.

How to deal with false positives?

When checking for false positives, we have to make sure that they are indeed false. By nature, we humans tend to start ignoring false positives very quickly. For example, suppose a web application security scanner detects 100 SQL Injection vulnerabilities. So, if the first 20 variants are false positives, then penetration tester assumes that all the others are false positives and ignore all the rest. By doing so, there are chances that real web application vulnerabilities are left undetected. This is why it is crucial to check every vulnerability and deal with each false positive separately to ensure false positives.