Part-2: Setup Prometheus, Kube State metrics and Integrate Grafana with Kubernetes

In the previous blog, we set up Prometheus in our local Kubernetes cluster along with ConfigMap which included the Prometheus configurations. We were able to access the Prometheus dashboard on the localhost
Overview
In this blog, we continue from where we left off.
- A brief look at what is Kube State Metrics
- Setting up Kube State Metrics in our cluster to monitor all the API objects
- Integrate Grafana on Kubernetes with Prometheus
Prerequisites
- Kubernetes Cluster setup
- Followed all the steps in Part 1- Refer to this
Kube State Metrics
Kube State Metrics is a service that communicates with the Kubernetes API server to obtain information about all API objects such as deployments, pods, and DaemonSets. It essentially gives Kubernetes API object metrics that aren’t available via native Kubernetes monitoring components.
Few key objects you can monitor with Kube State Metrics:
- Monitor node status, node capacity (CPU and memory)
- Monitor replica-set compliance (desired/available/unavailable/updated status of replicas per deployment)
- Monitor pod status (waiting, running, ready, etc)
- Monitor the resource requests and limits.
Setting Up Kube State Metrics
Step 1: Firstly, We create a Service Account
Apply the changes by running the following command:
kubectl apply -f service-account.yaml
Step 2: Next, We Create Cluster Role for Kube-State-Metrics to access all the Kubernetes API objects.
Apply the changes by running the following command:
kubectl apply -f cluster-role.yaml
Step 3: To bind the Cluster Role with the Service Account, We will use Cluster Role Binding.
Apply the changes by running the following command:
kubectl apply -f cluster-role-binding.yaml
Step 4: Now, We create a Deployment using the Kube-State-Metrics docker image
Apply the changes by running the following command:
kubectl apply -f deployment.yaml
To view the metrics locally we can use port-forwarding. For that, we get the name of the deployment pod with kubectl get pods -n kube-system
the command

Then use the pod name for port forwarding
kubectl port-forward <pod-name> 8080:8080 -n monitoring

If we go to localhost:8080
, we see something like this

Setup Grafana on Kubernetes
Grafana is a free and open-source dashboard application. It may connect to a variety of data sources, including Prometheus, Elastic Search, AWS CloudWatch, Google Cloud’s operations, and others. Through Prometheus, we can construct dashboards on Grafana for all Kubernetes metrics.
Step 1: Create Config Map containing the data source configuration for Prometheus. We can add more data sources for Grafana in this file if necessary
Apply the changes by running the following command:
kubectl apply -f grafana-datasource-config.yaml
Step 2: Now, We Create a Deployment using the Grafana docker image
Apply the changes by running the following command:
kubectl apply -f grafana-deployment.yaml
We can access the Grafana on localhost using port-forwarding
First, get the name of the pod with kubectl get pods -n monitoring

Then use the grafana
pod name for port forwarding like below
kubectl port-forward <grafana-pod-name> 3000:3000 -n monitoring

Now We can access the dashboard on localhpst:3000
.

The default credentials are listed below:
Grafana Credentials
User: admin
Password: admin
We can now access Grafana and use it to import or build dashboards and visualize the metrics data.
Conclusion
In this part, we learned about Kube-State-Metrics & Grafana, how to deploy them on Kubernetes, and access it on our localhost. I’ll also set up a Grafana Dashboard & view the metrics in the form of visualizations in my next post.