Kube.NETes Prometheus
簡(jiǎn)介
在k8s平臺(tái)上部署Prometheus監(jiān)控有幾種方式
- 老老實(shí)實(shí)寫yaml部署腳本，這種方式部署太麻煩，細(xì)節(jié)太多，不建議
- 使用開源項(xiàng)目prometheus-operator部署
- 使用開源項(xiàng)目kube-prometheus部署
prometheus-operator只包含一個(gè)operator，該operator管理和操作Prometheus和alertmanager集群，項(xiàng)目地址：https://github.com/prometheus-operator/prometheus-operator
kube Prometheus以Prometheus Operator和一系列manifests文件為基礎(chǔ)，以幫助你快速在kubernetes集群中部署Prometheus監(jiān)控系統(tǒng)，項(xiàng)目地址：https://github.com/prometheus-operator/kube-prometheus
這里我選用的是kube Prometheus去部署監(jiān)控
下載Kube-Prometheus項(xiàng)目#我用的版本是 release-0.11https://github.com/prometheus-operator/kube-prometheus/tree/release-0.11安裝 Kube-Prometheus#GitHub上已經(jīng)給出了安裝方法[root@master kube-prometheus-release-0.11]# kubectl Apply --server-side -f manifests/setupcustomresourcedefinition.apiextensions.k8s.io/alertmanagerconfigs.monitoring.coreos.com serverside-appliedcustomresourcedefinition.apiextensions.k8s.io/alertmanagers.monitoring.coreos.com serverside-appliedcustomresourcedefinition.apiextensions.k8s.io/podmonitors.monitoring.coreos.com serverside-appliedcustomresourcedefinition.apiextensions.k8s.io/probes.monitoring.coreos.com serverside-appliedcustomresourcedefinition.apiextensions.k8s.io/prometheuses.monitoring.coreos.com serverside-appliedcustomresourcedefinition.apiextensions.k8s.io/prometheusrules.monitoring.coreos.com serverside-appliedcustomresourcedefinition.apiextensions.k8s.io/servicemonitors.monitoring.coreos.com serverside-appliedcustomresourcedefinition.apiextensions.k8s.io/thanosrulers.monitoring.coreos.com serverside-appliednamespace/monitoring serverside-applied[root@master kube-prometheus-release-0.11]# until kubectl get servicemonitors --all-namespaces ; do date; sleep 1; echo ""; doneNo resources foundkubectl apply -f manifests/安裝完成，看下資源情況[root@master kube-prometheus-release-0.11]# kubectl get all -n monitoringNAME READY STATUS RESTARTS AGEpod/alertmanager-main-0 2/2 Running 0 65spod/alertmanager-main-1 2/2 Running 0 65spod/alertmanager-main-2 2/2 Running 0 63spod/blackbox-exporter-559db48fd-4c6rf 3/3 Running 0 2m40spod/grafana-546559f668-ft5zs 1/1 Running 0 2m15spod/kube-state-metrics-576b75c6f7-dx8vs 3/3 Running 0 2m9spod/node-exporter-fzwzs 2/2 Running 0 2mpod/node-exporter-qstbq 2/2 Running 0 2mpod/node-exporter-r9w26 2/2 Running 0 2m1spod/prometheus-adapter-5f68766c85-hvvhn 1/1 Running 0 86spod/prometheus-adapter-5f68766c85-vkh7l 1/1 Running 0 86spod/prometheus-k8s-0 2/2 Running 0 49spod/prometheus-k8s-1 0/2 PodInitializing 0 49spod/prometheus-operator-68845dfbbf-ldvvz 2/2 Running 0 81sNAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGEservice/alertmanager-main ClusterIP 10.0.0.120 9093/TCP,8080/TCP 2m46sservice/alertmanager-operated ClusterIP None 9093/TCP,9094/TCP,9094/UDP 66sservice/blackbox-exporter ClusterIP 10.0.0.164 9115/TCP,19115/TCP 2m41sservice/grafana ClusterIP 10.0.0.80 3000/TCP 2m18sservice/kube-state-metrics ClusterIP None 8443/TCP,9443/TCP 2m10sservice/node-exporter ClusterIP None 9100/TCP 2m2sservice/prometheus-adapter ClusterIP 10.0.0.213 443/TCP 91sservice/prometheus-k8s ClusterIP 10.0.0.28 9090/TCP,8080/TCP 100sservice/prometheus-operated ClusterIP None 9090/TCP 51sservice/prometheus-operator ClusterIP None 8443/TCP 84sNAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGEdaemonset.apps/node-exporter 3 3 3 3 3 kubernetes.io/os=linux 2m4sNAME READY UP-TO-DATE AVAILABLE AGEdeployment.apps/blackbox-exporter 1/1 1 1 2m49sdeployment.apps/grafana 1/1 1 1 2m25sdeployment.apps/kube-state-metrics 1/1 1 1 2m19sdeployment.apps/prometheus-adapter 2/2 2 2 100sdeployment.apps/prometheus-operator 1/1 1 1 93sNAME DESIRED CURRENT READY AGEreplicaset.apps/blackbox-exporter-559db48fd 1 1 1 2m51sreplicaset.apps/grafana-546559f668 1 1 1 2m27sreplicaset.apps/kube-state-metrics-576b75c6f7 1 1 1 2m20sreplicaset.apps/prometheus-adapter-5f68766c85 2 2 2 102sreplicaset.apps/prometheus-operator-68845dfbbf 1 1 1 95sNAME READY AGEstatefulset.apps/alertmanager-main 2/3 75sstatefulset.apps/prometheus-k8s 1/2 59s
在上面可以看到自動(dòng)創(chuàng)建了一個(gè)monitoring的NameSpace，Pod也都創(chuàng)建好了。
如何訪問Grafana[root@master kube-prometheus-release-0.11]# kubectl get svc -n monitoringNAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGEalertmanager-main ClusterIP 10.0.0.120 9093/TCP,8080/TCP 4m59salertmanager-operated ClusterIP None 9093/TCP,9094/TCP,9094/UDP 3m19sblackbox-exporter ClusterIP 10.0.0.164 9115/TCP,19115/TCP 4m54sgrafana ClusterIP 10.0.0.80 3000/TCP 4m31skube-state-metrics ClusterIP None 8443/TCP,9443/TCP 4m23snode-exporter ClusterIP None 9100/TCP 4m15sprometheus-adapter ClusterIP 10.0.0.213 443/TCP 3m44sprometheus-k8s ClusterIP 10.0.0.28 9090/TCP,8080/TCP 3m53sprometheus-operated ClusterIP None 9090/TCP 3m4sprometheus-operator ClusterIP None 8443/TCP 3m37s
默認(rèn)情況下，服務(wù)的網(wǎng)絡(luò)類型都是ClusterIP，無(wú)法在外面訪問，這里最好的方法是使用ingress配置對(duì)外提供服務(wù)，由于我的集群里沒有安裝ingress，現(xiàn)在我就修改成NodePort方式對(duì)外提供服務(wù)
vim manifests/grafana-service.yamlspec:ports:- name: httpport: 3000targetPort: httptype: NodePortmanifests/alertmanager-service.yamlmanifests/prometheus-service.yaml#grafana、alertmanager、prometheus都配置成type: NodePortkubectl apply -f manifests/grafana-service.yamlkubectl apply -f manifests/alertmanager-service.yamlkubectl apply -f manifests/prometheus-service.yaml
再查看一下Service信息
alertmanager-main NodePort 10.0.0.120 9093:47927/TCP,8080:31539/TCP 10malertmanager-operated ClusterIP None 9093/TCP,9094/TCP,9094/UDP 9m12sblackbox-exporter ClusterIP 10.0.0.164 9115/TCP,19115/TCP 10mgrafana NodePort 10.0.0.80 3000:37010/TCP 10mkube-state-metrics ClusterIP None 8443/TCP,9443/TCP 10mnode-exporter ClusterIP None 9100/TCP 10mprometheus-adapter ClusterIP 10.0.0.213 443/TCP 9m37sprometheus-k8s NodePort 10.0.0.28 9090:40124/TCP,8080:42004/TCP 9m46sprometheus-operated ClusterIP None 9090/TCP 8m57sprometheus-operator ClusterIP None 8443/TCP 9m30s
這里就可以看到 Grafana/Prometheus/Alertmanager都變成了NodePort
我們挑一個(gè)服務(wù)IP地址訪問一下
Grafana
K8s Grafana
Prometheus
K8s Prometheus
Alertmanager
K8s Alertmanager
卸載Prometheus方式kubectl delete --ignore-not-found=true -f manifests/ -f manifests/setup
這樣k8s的資源就可以監(jiān)控起來(lái)了，這中間還有一個(gè)問題是，我安裝Kube-Promethues的時(shí)候有很多鏡像下載不了，下篇我說(shuō)下怎樣下載k8s.gcr.io的鏡像。
