在其中一台master节点上重启了docker服务,然后 所有的docker容器都没了。

另一台正常的master节点上容器是好好的 如图:

CONTAINER ID        IMAGE                                                        COMMAND                  CREATED                  STATUS                              PORTS          
     NAMES                                                                                                                                                                            
eddf3ac3e3f3        bb10f0e78768                                                 "kube-apiserver --ad…"   Less than a second ago   Up Less than a second                              
     k8s_kube-apiserver_kube-apiserver-master2_kube-system_6b67e5622eea9fe3f7608527509f9e96_41                                                                                        
6969ba2071e1        d568210f4ccc                                                 "docker-entrypoint.s…"   2 months ago             Up 2 months                                        
     k8s_ks-console_ks-console-55bc574876-bhr49_kubesphere-system_dbdf9f54-719f-43eb-89bd-d8e427c6bef7_0                                                                              
4373700868cd        harbor.edpglobal.cn/mirrorgooglecontainers/pause-amd64:3.1   "/pause"                 2 months ago             Up 2 months                                        
     k8s_POD_ks-console-55bc574876-bhr49_kubesphere-system_dbdf9f54-719f-43eb-89bd-d8e427c6bef7_0                                                                                     
faa4db5b9e7a        b4eb6195d563                                                 "ks-iam --logtostder…"   2 months ago             Up 2 months                                        
     k8s_ks-account_ks-account-64b7dc4cb4-vkvxm_kubesphere-system_747b8418-a1e0-40d4-8046-e77a99f16293_0                                                                              
1dc381e506a9        9fa8f431a6fd                                                 "controller-manager …"   2 months ago             Up 2 months                                        
     k8s_ks-controller-manager_ks-controller-manager-7bd878c7b7-tbsnh_kubesphere-system_cd524a07-1bf8-4bbe-8b54-b3f7cc8a288d_0                                                        
fdf20517e20d        harbor.edpglobal.cn/mirrorgooglecontainers/pause-amd64:3.1   "/pause"                 2 months ago             Up 2 months                                        
     k8s_POD_ks-controller-manager-7bd878c7b7-tbsnh_kubesphere-system_cd524a07-1bf8-4bbe-8b54-b3f7cc8a288d_0                                                                          
d63693b7140b        harbor.edpglobal.cn/mirrorgooglecontainers/pause-amd64:3.1   "/pause"                 2 months ago             Up 2 months                                        
     k8s_POD_ks-account-64b7dc4cb4-vkvxm_kubesphere-system_747b8418-a1e0-40d4-8046-e77a99f16293_0                                                                                     
5042dc2f86d2        e705063b9c6c                                                 "ks-apiserver --logt…"   2 months ago             Up 2 months                                        
     k8s_ks-apiserver_ks-apiserver-745c98d5f6-pdql6_kubesphere-system_6eb6f256-7640-43c1-8a99-db0fd25ac372_0                                                                          
a398958a2937        harbor.edpglobal.cn/mirrorgooglecontainers/pause-amd64:3.1   "/pause"                 2 months ago             Up 2 months                                        
     k8s_POD_ks-apiserver-745c98d5f6-pdql6_kubesphere-system_6eb6f256-7640-43c1-8a99-db0fd25ac372_0                                                                                   
d4b9f5806c28        88392e2cfe5a                                                 "/bin/sh -c 'export …"   2 months ago             Up 2 months                                        
     k8s_ks-apigateway_ks-apigateway-74fdc5f4b5-w4t4j_kubesphere-system_2aaee442-aafc-4305-8d4d-6d31296f6487_0        
 
而另一个我上面说的那个master机器里这些容器都没了,怎么起来了。
请问有人知道不                              

异常的节点 看看docker、kubelet的服务日志

Feb 23 19:12:45 master1 kubelet[64245]: I0223 19:12:45.723303 64245 flags.go:33] FLAG: –rotate-certificates=“false”
Feb 23 19:12:45 master1 kubelet[64245]: I0223 19:12:45.723305 64245 flags.go:33] FLAG: –rotate-server-certificates=“false”
Feb 23 19:12:45 master1 kubelet[64245]: I0223 19:12:45.723308 64245 flags.go:33] FLAG: –runonce=“false”
Feb 23 19:12:45 master1 systemd[1]: kubelet.service failed.
Feb 23 19:12:45 master1 kubelet[64245]: I0223 19:12:45.723311 64245 flags.go:33] FLAG: –runtime-cgroups=“/systemd/system.slice”
Feb 23 19:12:45 master1 kubelet[64245]: I0223 19:12:45.723314 64245 flags.go:33] FLAG: –runtime-request-timeout=“2m0s”
Feb 23 19:12:45 master1 kubelet[64245]: I0223 19:12:45.723317 64245 flags.go:33] FLAG: –seccomp-profile-root=“/var/lib/kubelet/seccomp”
Feb 23 19:12:45 master1 kubelet[64245]: I0223 19:12:45.723320 64245 flags.go:33] FLAG: –serialize-image-pulls=“true”
Feb 23 19:12:45 master1 kubelet[64245]: I0223 19:12:45.723322 64245 flags.go:33] FLAG: –skip-headers=“false”
Feb 23 19:12:45 master1 kubelet[64245]: I0223 19:12:45.723325 64245 flags.go:33] FLAG: –skip-log-headers=“false”
Feb 23 19:12:45 master1 kubelet[64245]: I0223 19:12:45.723328 64245 flags.go:33] FLAG: –stderrthreshold=“2”
Feb 23 19:12:45 master1 kubelet[64245]: I0223 19:12:45.723331 64245 flags.go:33] FLAG: –storage-driver-buffer-duration=“1m0s”
Feb 23 19:12:45 master1 kubelet[64245]: I0223 19:12:45.723334 64245 flags.go:33] FLAG: –storage-driver-db=“cadvisor”
Feb 23 19:12:45 master1 kubelet[64245]: I0223 19:12:45.723337 64245 flags.go:33] FLAG: –storage-driver-host=“localhost:8086”
Feb 23 19:12:45 master1 kubelet[64245]: I0223 19:12:45.723339 64245 flags.go:33] FLAG: –storage-driver-password=“root”
Feb 23 19:12:45 master1 kubelet[64245]: I0223 19:12:45.723342 64245 flags.go:33] FLAG: –storage-driver-secure=“false”
Feb 23 19:12:45 master1 kubelet[64245]: I0223 19:12:45.723345 64245 flags.go:33] FLAG: –storage-driver-table=“stats”
Feb 23 19:12:45 master1 kubelet[64245]: I0223 19:12:45.723347 64245 flags.go:33] FLAG: –storage-driver-user=“root”
Feb 23 19:12:45 master1 kubelet[64245]: I0223 19:12:45.723350 64245 flags.go:33] FLAG: –streaming-connection-idle-timeout=“4h0m0s”
Feb 23 19:12:45 master1 kubelet[64245]: I0223 19:12:45.723353 64245 flags.go:33] FLAG: –sync-frequency=“1m0s”
Feb 23 19:12:45 master1 kubelet[64245]: I0223 19:12:45.723356 64245 flags.go:33] FLAG: –system-cgroups=""
Feb 23 19:12:45 master1 kubelet[64245]: I0223 19:12:45.723359 64245 flags.go:33] FLAG: –system-reserved=""
Feb 23 19:12:45 master1 kubelet[64245]: I0223 19:12:45.723361 64245 flags.go:33] FLAG: –system-reserved-cgroup=""
Feb 23 19:12:45 master1 kubelet[64245]: I0223 19:12:45.723364 64245 flags.go:33] FLAG: –tls-cert-file=""
Feb 23 19:12:45 master1 kubelet[64245]: I0223 19:12:45.723368 64245 flags.go:33] FLAG: –tls-cipher-suites=“[]”
Feb 23 19:12:45 master1 kubelet[64245]: I0223 19:12:45.723372 64245 flags.go:33] FLAG: –tls-min-version=""
Feb 23 19:12:45 master1 kubelet[64245]: I0223 19:12:45.723374 64245 flags.go:33] FLAG: –tls-private-key-file=""
Feb 23 19:12:45 master1 kubelet[64245]: I0223 19:12:45.723377 64245 flags.go:33] FLAG: –v=“2”
Feb 23 19:12:45 master1 kubelet[64245]: I0223 19:12:45.723379 64245 flags.go:33] FLAG: –version=“false”
Feb 23 19:12:45 master1 kubelet[64245]: I0223 19:12:45.723384 64245 flags.go:33] FLAG: –vmodule=""
Feb 23 19:12:45 master1 kubelet[64245]: I0223 19:12:45.723387 64245 flags.go:33] FLAG: –volume-plugin-dir=“/usr/libexec/kubernetes/kubelet-plugins/volume/exec/”
Feb 23 19:12:45 master1 kubelet[64245]: I0223 19:12:45.723391 64245 flags.go:33] FLAG: –volume-stats-agg-period=“1m0s”
Feb 23 19:12:45 master1 kubelet[64245]: W0223 19:12:45.723418 64245 feature_gate.go:208] Setting GA feature gate KubeletPluginsWatcher=true. It will be removed in a future release.
Feb 23 19:12:45 master1 kubelet[64245]: I0223 19:12:45.723424 64245 feature_gate.go:216] feature gates: &{map[CSINodeInfo:true ExpandCSIVolumes:true KubeletPluginsWatcher:true Volu
meSnapshotDataSource:true]}
Feb 23 19:12:45 master1 kubelet[64245]: Flag –feature-gates has been deprecated, This parameter should be set via the config file specified by the Kubelet’s –config flag. See https
://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Feb 23 19:12:45 master1 kubelet[64245]: W0223 19:12:45.724579 64245 feature_gate.go:208] Setting GA feature gate KubeletPluginsWatcher=true. It will be removed in a future release.
Feb 23 19:12:45 master1 kubelet[64245]: I0223 19:12:45.724586 64245 feature_gate.go:216] feature gates: &{map[CSINodeInfo:true ExpandCSIVolumes:true KubeletPluginsWatcher:true Volu
meSnapshotDataSource:true]}
Feb 23 19:12:45 master1 kubelet[64245]: W0223 19:12:45.724626 64245 feature_gate.go:208] Setting GA feature gate KubeletPluginsWatcher=true. It will be removed in a future release.
Feb 23 19:12:45 master1 kubelet[64245]: I0223 19:12:45.724631 64245 feature_gate.go:216] feature gates: &{map[CSINodeInfo:true ExpandCSIVolumes:true KubeletPluginsWatcher:true Volu
meSnapshotDataSource:true]}
Feb 23 19:12:45 master1 kubelet[64245]: I0223 19:12:45.728661 64245 mount_linux.go:175] Detected OS with systemd
Feb 23 19:12:45 master1 kubelet[64245]: I0223 19:12:45.728763 64245 server.go:425] Version: v1.15.5
Feb 23 19:12:45 master1 kubelet[64245]: W0223 19:12:45.728800 64245 feature_gate.go:208] Setting GA feature gate KubeletPluginsWatcher=true. It will be removed in a future release.
Feb 23 19:12:45 master1 kubelet[64245]: I0223 19:12:45.728806 64245 feature_gate.go:216] feature gates: &{map[CSINodeInfo:true ExpandCSIVolumes:true KubeletPluginsWatcher:true Volu
meSnapshotDataSource:true]}
Feb 23 19:12:45 master1 kubelet[64245]: W0223 19:12:45.728846 64245 feature_gate.go:208] Setting GA feature gate KubeletPluginsWatcher=true. It will be removed in a future release.
Feb 23 19:12:45 master1 kubelet[64245]: I0223 19:12:45.728851 64245 feature_gate.go:216] feature gates: &{map[CSINodeInfo:true ExpandCSIVolumes:true KubeletPluginsWatcher:true Volu
meSnapshotDataSource:true]}
Feb 23 19:12:45 master1 kubelet[64245]: I0223 19:12:45.728906 64245 plugins.go:103] No cloud provider specified.
Feb 23 19:12:45 master1 kubelet[64245]: I0223 19:12:45.728920 64245 server.go:541] No cloud provider specified: "“ from the config file: ”"
Feb 23 19:12:45 master1 kubelet[64245]: E0223 19:12:45.730321 64245 bootstrap.go:263] Part of the existing bootstrap client certificate is expired: 2021-02-13 04:04:25 +0000 UTC
Feb 23 19:12:45 master1 kubelet[64245]: I0223 19:12:45.730332 64245 bootstrap.go:117] Using bootstrap kubeconfig to generate TLS client cert, key and kubeconfig file
Feb 23 19:12:45 master1 kubelet[64245]: F0223 19:12:45.730346 64245 server.go:273] failed to run Kubelet: unable to load bootstrap kubeconfig: stat /etc/kubernetes/bootstrap-kubele
t.conf: no such file or directory