Linux containers have taken over the enterprise, and we hear so much about Docker and Kubernetes that we forget the equally important tasks of monitoring and logging. Docker continues to grow, and with it comes the growth of the ecosystem of services built around it. Now, though deploying containers predominantly involves running a single app or service inside each container, as the deployments get larger it becomes more and more important to get visibility into the status and health of the environment, not just at the OS or application level, but at the container level as well.
The landscape of monitoring and logging tools is divided into real-time databases, metric gatherers, visualizers, pollers, loggers, and more. There’s been an explosion of open-source products and commercial monitoring applications over the past few years and here’s a look at 10 of the most popular ones.
Prometheus is a leading open-source cloud monitoring solution that deals with time series monitoring, among other things. It’s also one of the nine projects backed by the CNCF, and the second project after Kubernetes to be hosted by the CNCF. This makes it a top choice for monitoring your cloud applications, purely because of the multivendor backing that includes the likes of Google, CoreOS, RedHat, IBM, and others. Prometheus was originally developed by music hosting site SoundCloud, and has since become an integral art of the Docker ecosystem. Prometheus is also actively being used by the CoreOS team to improve the performance of Kubernetes.
Prometheus has developed into an advanced system alerting and monitoring toolkit that’s capable of monitoring a broad range of resources including containers, Hadoop clusters, language runtimes, and application stacks like Python or Go. A lot of people compare it to Graphite (see below), although it adds a lot of functionality including alerting. In short, Prometheus is an open-source monitoring system, time-series database, and a framework for implementing your code. Prometheus doesn’t generate metrics, however, and you have to make the metrics available to it over http in a language it will understand.
Graphite is a popular open-source tool used to build web-based dashboards for monitoring and analysis. Though it’s been around for a while and was originally built and designed by Chris Davis at Orbitz over a decade ago to monitor and graph the performance of computer systems, it’s still used in production by numerous large companies like GitHub, Salesforce, and Reddit. Though Graphite does not collect metrics or come bundled with a metrics gatherer, it’s not hard to find a common metric gatherer to feed your metrics into Graphite. Graphite is made up of three parts: Carbon, which is a daemon that listens for time-series data, a simple database called Whisper (similar to RRDtool), and a webapp called Graphite-web that renders graphics on demand. Graphite is still quite popular in the enterprise, and one of the reasons could be a level of automation where all new data points are automatically created in the database.
The modern landscape of enterprise tools can be especially confusing since there are ecosystems being developed around ecosystems. What makes matters more complicated is tools like Graphite, essentially three separate tools that are coupled together that can also be used separately and integrated with other tools. Here’s an image of the cloud-native landscape today, where you see monitoring and analysis in the boxes on the right.