系统版本 centos7.4,16c64g+500g,kubesphere-2.1.0
最初报错信息如下:
TASK [plugins/NFS-Client : NFS-Client | Deploy NFS-Client] ******************************************************************************************************************
Tuesday 26 November 2019 15:30:03 +0800 (0:00:00.058) 0:01:02.364 ******
fatal: [master]: FAILED! => {
"changed": true,
"cmd": "/usr/local/bin/helm upgrade --install nfs-client /etc/kubernetes/addons/nfs-client-provisioner --set nfs.server=****u****5.cn-****.nas.aliyuncs.com --set nfs.path=True --set storageClass.defaultClass=True --set storageClass.archiveOnDelete=false --namespace kube-system",
"delta": "0:00:00.162227",
"end": "2019-11-26 15:30:03.289167",
"rc": 1,
"start": "2019-11-26 15:30:03.126940"
}
STDOUT:
UPGRADE FAILED
Error: "nfs-client" has no deployed releases
STDERR:
Error: UPGRADE FAILED: "nfs-client" has no deployed releases
MSG:
non-zero return code
PLAY RECAP ******************************************************************************************************************************************************************
master : ok=216 changed=8 unreachable=0 failed=1
node1 : ok=208 changed=4 unreachable=0 failed=0
node2 : ok=208 changed=4 unreachable=0 failed=0
Tuesday 26 November 2019 15:30:03 +0800 (0:00:00.264) 0:01:02.629 ******
===============================================================================
prepare/nodes : pip | Installing pip --------------------------------------------------------------------------------------------------------------------------------- 2.49s
plugins/NFS-Client : NFS-Client | Getting NFS-Client files ----------------------------------------------------------------------------------------------------------- 2.14s
download : Download items -------------------------------------------------------------------------------------------------------------------------------------------- 1.79s
prepare/nodes : Labeling worker nodes -------------------------------------------------------------------------------------------------------------------------------- 1.31s
download : Sync container -------------------------------------------------------------------------------------------------------------------------------------------- 0.91s
prepare/nodes : Compare host kubectl with hyperkube container -------------------------------------------------------------------------------------------------------- 0.79s
download : Sync container -------------------------------------------------------------------------------------------------------------------------------------------- 0.67s
download : Download items -------------------------------------------------------------------------------------------------------------------------------------------- 0.66s
download : Download items -------------------------------------------------------------------------------------------------------------------------------------------- 0.65s
download : Sync container -------------------------------------------------------------------------------------------------------------------------------------------- 0.62s
prepare/nodes : Installing epel-release (YUM) ------------------------------------------------------------------------------------------------------------------------ 0.49s
prepare/nodes : GlusterFS | Installing glusterfs-client (YUM) -------------------------------------------------------------------------------------------------------- 0.48s
prepare/nodes : Copy admin kubeconfig to root user home -------------------------------------------------------------------------------------------------------------- 0.46s
prepare/nodes : Annotating nodes ------------------------------------------------------------------------------------------------------------------------------------- 0.46s
plugins/LocalVolume : openebs | Creating manifests ------------------------------------------------------------------------------------------------------------------- 0.41s
prepare/nodes : Copy get-pip.py -------------------------------------------------------------------------------------------------------------------------------------- 0.37s
download : Register docker images info ------------------------------------------------------------------------------------------------------------------------------- 0.37s
prepare/nodes : Copy pip.conf ---------------------------------------------------------------------------------------------------------------------------------------- 0.36s
download : Register docker images info ------------------------------------------------------------------------------------------------------------------------------- 0.35s
download : Register docker images info ------------------------------------------------------------------------------------------------------------------------------- 0.35s
failed!
**********************************
please refer to https://kubesphere.io/docs/v2.1/zh-CN/faq/faq-install/
**********************************
`
### kubectl get pods --all-namespaces 结果如下
`
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system calico-kube-controllers-6787ff6fbb-6xllf 1/1 Running 0 163m
kube-system calico-node-4g489 1/1 Running 0 163m
kube-system calico-node-6l9rc 1/1 Running 1 163m
kube-system calico-node-kmr56 1/1 Running 0 163m
kube-system coredns-647d4f64c4-6d6nx 1/1 Running 0 33m
kube-system dns-autoscaler-77486c4479-rfts5 1/1 Running 0 163m
kube-system kube-apiserver-master 1/1 Running 0 164m
kube-system kube-controller-manager-master 1/1 Running 0 164m
kube-system kube-proxy-7rd79 1/1 Running 0 163m
kube-system kube-proxy-b6kv8 1/1 Running 0 163m
kube-system kube-proxy-wbzwd 1/1 Running 0 164m
kube-system kube-scheduler-master 1/1 Running 0 164m
kube-system nodelocaldns-6gxkd 1/1 Running 0 163m
kube-system nodelocaldns-jdfzl 1/1 Running 0 163m
kube-system nodelocaldns-lb5zx 1/1 Running 0 163m
kube-system openebs-localpv-provisioner-7b55587dbd-wtjlq 1/1 Running 0 63m
kube-system openebs-ndm-7fmlv 1/1 Running 0 63m
kube-system openebs-ndm-g8bx4 1/1 Running 0 63m
kube-system openebs-ndm-operator-69cbc86975-s9b24 1/1 Running 1 63m
kube-system tiller-deploy-55fc49c595-vcw5m 1/1 Running 0 163m
helm版本
Client: &version.Version{SemVer:"v2.14.3", GitCommit:"0e7f3b6637f7af8fcfddb3d2941fcc7cbebb0085", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.14.3", GitCommit:"0e7f3b6637f7af8fcfddb3d2941fcc7cbebb0085", GitTreeState:"clean"}
起初一直以为是服务本身问题,也曾怀疑过是阿里的文件存储有访问策略,手动在三台节点上mount挂载,挂载成功,没有报错
检查conf/common.yaml发现将nfs_path: /nfs 项目误配置成 nfs_path: true,修改后执行uninstall.sh卸载,再执行install.sh重新安装后成功跳过
紧接着又重复出现了昨天的报错:minio启动报错
用kubectl get pods –all-namespaces 查看pods运行情况,发现nfs-client-nfs-client-provisioner-685f7fdf45-4xbzj这个pod启动失败
用kubectl describe pod nfs-client-nfs-client-provisioner-685f7fdf45-4xbzj -n kube-system查看发现是nfs
nfs挂载失败导致的,失败命令:mount -t nfs …. ***.aliyun.com:/nfs
问题来了,检查阿里的文件存储服务发现路径是 / ,/nfs是我自己挂载到服务器目录,而common.yaml里nfs_path: 的配置项是要求填写nfs服务端的路径,所以,在common.yaml里改成nfs_path: /,再次执行install.sh集群安装成功!
总结本次部署失败原因在于错误理解了nfs_path配置项的含义,希望后面的小伙伴以此为戒,检查好nfs路径。