Logging Solution For Kubernetes

October 3, 2018

Kubernetes Logging

Follow Best Practices

Logging on Kubernetes is split into two main components the first is your Container monitoring and the second is your host monitoring. The techniques for logging on containers are not fundamentally different from logging best practices outside of containers. The problem is most people when switching to a container environment don’t follow best practices and are forced to retrofit. If logging is properly externalized within an application than it should be simple to pull logs within a containerized environment.

Aggregating Logs

The larger problem comes from the increased amount of moving parts in a container environment. This is why centralized logging and monitoring become even more paramount. This phenomenon only increases with the transition to microservices. There is a fairly standard architecture when it comes to how to get logs out of containers and Kubenernetes, but there is no standard for where to put those logs. This becomes an organizational choice. Most of the time it is best to just use the same logging system you are already familiar with.

Logging Vs. Monitoring

These become two parts of the same coin. Logging is important but is only one part of the equation. What you do with the logs is much more important, otherwise, they just create noise. Monitoring is where the value of logs come from. Monitoring is where this conversation can switch from a science to an art. Yes, there are some generic things you should look for such as CPU, Memory, errors, etc. Knowing when to create an alert though is the art portion of this. After all the alerting, and being able to figure out what went wrong is where these systems shine. They should be able to effectively give you a better signal to noise ratio on your infrastructure.

So what are the tools?

Sending the Logs

Getting the logs out of containers is done with FluentD and Fluent Bit. Fluent Bit runs as a sidecar container that connects back to fluentD. FluentD then sends the logs off to a central location.

Honorable Mentioned

Log Aggregation

The second part is where the logs get sent to and aggregated. This one is dependent on your current setup, and is a matter of preference.

Comercial Solutions

Open Source Solutions

Monitoring

The last step is what to do with the data. These tools enable monitoring and alerting for a container environment.

Comercial Solutions

Open Source Solutions


comments powered by Disqus