Part-2: Setup Prometheus, Kube State metrics and Integrate Grafana with Kubernetes

Yash Gupta
DevOps.dev
Published in
4 min readJun 17, 2022

In the previous blog, we set up Prometheus in our local Kubernetes cluster along with ConfigMap which included the Prometheus configurations. We were able to access the Prometheus dashboard on the localhost

Overview

In this blog, we continue from where we left off.

  • A brief look at what is Kube State Metrics
  • Setting up Kube State Metrics in our cluster to monitor all the API objects
  • Integrate Grafana on Kubernetes with Prometheus

Prerequisites

  • Kubernetes Cluster setup
  • Followed all the steps in Part 1- Refer to this

Kube State Metrics

Kube State Metrics is a service that communicates with the Kubernetes API server to obtain information about all API objects such as deployments, pods, and DaemonSets. It essentially gives Kubernetes API object metrics that aren’t available via native Kubernetes monitoring components.

Few key objects you can monitor with Kube State Metrics:

  • Monitor node status, node capacity (CPU and memory)
  • Monitor replica-set compliance (desired/available/unavailable/updated status of replicas per deployment)
  • Monitor pod status (waiting, running, ready, etc)
  • Monitor the resource requests and limits.

Setting Up Kube State Metrics

Step 1: Firstly, We create a Service Account

Creates a ServiceAccount for Kube-State-Metrics

Apply the changes by running the following command:

kubectl apply -f service-account.yaml

Step 2: Next, We Create Cluster Role for Kube-State-Metrics to access all the Kubernetes API objects.

Creating ClusterRole for Kube-State-Metrics

Apply the changes by running the following command:

kubectl apply -f cluster-role.yaml

Step 3: To bind the Cluster Role with the Service Account, We will use Cluster Role Binding.

Creating ClusterRoleBinding for Kube-State-Metrics

Apply the changes by running the following command:

kubectl apply -f cluster-role-binding.yaml

Step 4: Now, We create a Deployment using the Kube-State-Metrics docker image

Deployment of Kube-State-Metrics

Apply the changes by running the following command:

kubectl apply -f deployment.yaml

To view the metrics locally we can use port-forwarding. For that, we get the name of the deployment pod with kubectl get pods -n kube-system the command

Then use the pod name for port forwarding

kubectl port-forward <pod-name> 8080:8080 -n monitoring

If we go to localhost:8080, we see something like this

Kube-state-metrics on localhost

Setup Grafana on Kubernetes

Grafana is a free and open-source dashboard application. It may connect to a variety of data sources, including Prometheus, Elastic Search, AWS CloudWatch, Google Cloud’s operations, and others. Through Prometheus, we can construct dashboards on Grafana for all Kubernetes metrics.

Step 1: Create Config Map containing the data source configuration for Prometheus. We can add more data sources for Grafana in this file if necessary

Apply the changes by running the following command:

kubectl apply -f grafana-datasource-config.yaml

Step 2: Now, We Create a Deployment using the Grafana docker image

Apply the changes by running the following command:

kubectl apply -f grafana-deployment.yaml

We can access the Grafana on localhost using port-forwarding

First, get the name of the pod with kubectl get pods -n monitoring

Then use the grafana pod name for port forwarding like below

kubectl port-forward <grafana-pod-name> 3000:3000 -n monitoring

Now We can access the dashboard on localhpst:3000.

Grafana login

The default credentials are listed below:

Grafana Credentials
User: admin
Password: admin

We can now access Grafana and use it to import or build dashboards and visualize the metrics data.

Conclusion

In this part, we learned about Kube-State-Metrics & Grafana, how to deploy them on Kubernetes, and access it on our localhost. I’ll also set up a Grafana Dashboard & view the metrics in the form of visualizations in my next post.

Connect with me on LinkedIn | Twitter

Free

Distraction-free reading. No ads.

Organize your knowledge with lists and highlights.

Tell your story. Find your audience.

Membership

Read member-only stories

Support writers you read most

Earn money for your writing

Listen to audio narrations

Read offline with the Medium app

--

--

Published in DevOps.dev

Devops.dev is a community of DevOps enthusiasts sharing insight, stories, and the latest development in the field.

Written by Yash Gupta

ReactJS | DevOps | Kubernetes | SDE @HBK

No responses yet

What are your thoughts?