Logging and monitoring are different processes with different missions. Logging is the process of storing data related to a system. This information can be used later to troubleshoot a problem or investigate an incident, including security incidents. Monitoring is a different beast. We monitor to have meaningful alerts when something specific happens. In order to monitor, we need logs. That’s the cause of the confusion, but they are different.
Sometimes I see that someone says: “the company had logs but they only reviewed them after the incident”. That’s ok. You review the logs only when you need to review them. You are not supposed to review the logs at regular intervals to see if there is something weird in them.
The mission of the logging process is to have the information you need in case you need to investigate a problem or an incident.
This mission has some implications on what kind of information we log. Logging tends to be massive. You log data that could be useful, but without a clear objective because you cannot anticipate the problem you will have.
It’s important to have the capacity to search over the logs and retrieve relevant information. That can sound trivial but some organizations generate such a large amount of logs that when they need to search through them, they cannot do it with normal resources, and during an incident it is difficult for them to access the necessary ones.
In order to monitor, we have to know what we want to know and when. For example, if we want to monitor if someone has disabled the MFA option in their profile, we need the information when it happens or soon after it happens. We need to have in some place the information about someone disabling their MFA option in the authentication system and we need a mechanism to inform who needs to know this information. This monitoring can catch an attacker trying to disable the MFA of an account in order to access it.
To monitor, we don’t need to store logs massively. We can even discard the information about someone disabling its MFA after some time or after the data being parsed and being stored in another system, for example a SIEM. We can discard the data that doesn’t have any information that is useful for monitoring.
We will talk about this in another article, but not having a list of what we want to monitor is one of the main problems for not implementing a really useful monitoring problem. We try to do too many things from the beginning, we are flooded with meaningless information and false positives, and we don’t catch the important events when they happen. We should avoid monitoring information that is not meaningful and that no one can or would want to act upon it. If we want to have that information stored just in case we need it, that’s logging.
Should we monitor that someone is port scanning us and the firewall in the perimeter is blocking them? I don’t think so. The firewall is doing its work and everything is working as expected. We don’t want to monitor that event. However, I think we should monitor indicators of partially successful attacks (IPSAs). IPSAs are events that indicate that a potential attacker has reached further than expected. For example, think about the email case. We may have a secure email gateway in the perimeter, but sometimes a malicious email reaches the endpoint where the anti-malware software detects and deletes it. The email should have been blocked in the secure email gateway but it wasn’t. That was a partial success for the attacker as it avoided the secure email gateway. We should investigate why the email was not detected as malicious in the secure email gateway and maybe fine tune it. As we are going to act upon this event, I think it is useful to monitor it.
Anyway, we must prioritize monitoring what we can really be able to analyze and not more. If any monitoring is going to increase the workload in a way that it won’t be possible for us to be on top of the alerts, we have to reduce the number of monitoring that we do and maybe redirect some of that information to the logging process and facilities.
In my opinion, monitoring is much more useful to security than logging as it can help actively detect and prevent attacks. If you have to start with something, start with monitoring. On the other hand, logging is more useful for other parts of the company so if you can move a project of logging better because you have the support of other parts of the organization, it can also be a good place to start and then use that logging information to get what you need and build a monitoring process.
