• 安装部署监控日志

  • 【已解决】prometheus-k8s-0容器一直处于SchedulerError状态，无法使用监控功能

操作系统信息
例如：虚拟机，Ubuntu18.04，6C/8G

Kubernetes版本信息

Client Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.3", GitCommit:"1e11e4a2108024935ecfcb2912226cedeafd99df", GitTreeState:"clean", BuildDate:"2020-10-14T12:50:19Z", GoVersion:"go1.15.2", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.4", GitCommit:"e87da0bd6e03ec3fea7933c4b5263d151aafd07c", GitTreeState:"clean", BuildDate:"2021-02-18T16:03:00Z", GoVersion:"go1.15.8", Compiler:"gc", Platform:"linux/amd64"}

容器运行时

Client: Docker Engine - Community
 Version:           20.10.7
 API version:       1.41
 Go version:        go1.13.15
 Git commit:        f0df350
 Built:             Wed Jun  2 11:56:40 2021
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.7
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       b0f5bc3
  Built:            Wed Jun  2 11:54:48 2021
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.4.8
  GitCommit:        7eba5930496d9bbe375fdf71603e610ad737d2b2
 runc:
  Version:          1.0.0
  GitCommit:        v1.0.0-0-g84113ee
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

KubeSphere版本信息
版本：v3.1.0。使用kk以 All-in-One 模式安装。

问题是什么

无监控数据：

prometheus-k8s-0提示running PreBind plugin "VolumeBinding": binding volumes: timed out waiting for the condition

prometheus-k8s-db-prometheus-k8s-0的状态：

看下kubesphere-monitoring-system命名空间下pvc是否创建了

    showjason 没有PVC

    ~# kubectl -n kubesphere-monitoring-system get pvc
NAME                                 STATUS    VOLUME   CAPACITY   ACCESS MODES   STORAGECLASS   AGE
prometheus-k8s-db-prometheus-k8s-0   Pending                                      local          4h41m

        Mattie 分别查看一下 这个pvc的 event， 还有 openebs-localpv-provisioner的logs。

        kubectl -n kube-system logs openebs-localpv-provisioner-xx
kubectl -n kubesphere-monitoring-system describe pvc xxx

            RolandMa1986

            没有`openebs-localpv-provisioner`的相关资源，都没有创建

            # kubectl get pod -n kube-system
NAME                                      READY   STATUS    RESTARTS   AGE
calico-kube-controllers-8f59968d4-7jqxt   1/1     Running   1618       175d
calico-node-tntr5                         1/1     Running   19         175d
coredns-65944cbcb8-8r7r8                  1/1     Running   21         175d
coredns-65944cbcb8-pl4dv                  1/1     Running   18         175d
kube-apiserver-ubuntu                     1/1     Running   12         175d
kube-controller-manager-ubuntu            1/1     Running   21         175d
kube-proxy-lrgnv                          1/1     Running   11         175d
kube-scheduler-ubuntu                     1/1     Running   22         175d
metrics-server-666566b66d-lq97w           1/1     Running   2          23h
nodelocaldns-hdpsx                        1/1     Running   12         175d
snapshot-controller-0                     1/1     Running   11         174d

root@kubesphere2:~# kubectl get all -n kube-system | grep openebs
root@kubesphere2:~# 
            # kubectl -n kubesphere-monitoring-system describe pvc prometheus-k8s-db-prometheus-k8s-0
Name:          prometheus-k8s-db-prometheus-k8s-0
Namespace:     kubesphere-monitoring-system
StorageClass:  local
Status:        Pending
Volume:        
Labels:        app=prometheus
               prometheus=k8s
Annotations:   volume.beta.kubernetes.io/storage-provisioner: openebs.io/local
               volume.kubernetes.io/selected-node: ubuntu
Finalizers:    [kubernetes.io/pvc-protection]
Capacity:      
Access Modes:  
VolumeMode:    Filesystem
Used By:       prometheus-k8s-0
Events:
  Type    Reason                Age                     From                         Message
  ----    ------                ----                    ----                         -------
  Normal  ExternalProvisioning  17s (x1384 over 5h45m)  persistentvolume-controller  waiting for a volume to be created, either by external provisioner "openebs.io/local" or manually created by system administrator

                Mattie 有没有可能被意外删除了。 或是查看一下deployment 是否存在。都不再的话，可以手动装一个。

                    RolandMa1986 应该不是被意外删除的，我安装后监控页面一开始就没有数据的。那请问是要手动安装deployment吗？有没有该 deployment 的 yaml file 可以参考一下呢，谢谢。

                        Mattie 更改标题为「【已解决】prometheus-k8s-0容器一直处于SchedulerError状态，无法使用监控功能」
                        1 个月 后

                        Mattie

                        问下如果存在pvc情况，kubectl -n kubesphere-monitoring-system get pvc对应的prometheus-k8s-db-prometheus-k8s 处于Bound状态，我该如何处理？

                        没有找到思路

                            6 天 后

                            liujie1008cn 不好意思我对这些也不是很了解，你可以寻求其他大佬的帮助。我之前出问题时prometheus-k8s-0容器有提示VolumeBinding的异常信息，你可以从异常信息那边看看是哪个方面的原因。

                                1 年 后

                                Mattie 你好，可以把最后的解决步骤写一下吗，我也遇到这个问题了 😀