check the storage configuration and storage server

huochai

k8s版本 v1.15.0
# kubectl version Client Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.0", GitCommit:"e8462b5b5dc2584fdcd18e6bcfe9f1e4d970a529", GitTreeState:"clean", BuildDate:"2019-06-19T16:40:16Z", GoVersion:"go1.12.5", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.0", GitCommit:"e8462b5b5dc2584fdcd18e6bcfe9f1e4d970a529", GitTreeState:"clean", BuildDate:"2019-06-19T16:32:14Z", GoVersion:"go1.12.5", Compiler:"gc", Platform:"linux/amd64"}

网络类型为 flannel

helm版本 v2.15.2
# helm version Client: &version.Version{SemVer:"v2.15.2", GitCommit:"8dce272473e5f2a7bf58ce79bb5c3691db54c96b", GitTreeState:"clean"} Server: &version.Version{SemVer:"v2.15.2", GitCommit:"8dce272473e5f2a7bf58ce79bb5c3691db54c96b", GitTreeState:"clean"}

自己搭建的ceph集群，通过测试pod可以绑定对应ceph的

在部署安装kubesphere的时候，好像被跳过了

最后安装结束后如下

有没有大佬告知下哪里出了问题啊？

huochai

helm del --purge ks-minio && kubectl delete job -n kubesphere-system ks-minio-make-bucket-job
删除后重新部署几次都是一样的

Forest-L

huochai
报错之后，kubectl get pvc -n kubesphere-system /kubectl get pv -n kubesphere-system 有pv pvc生成吗？

huochai

Forest-L 有生成 minio 的pv 和 pvc，其他的没有

Forest-L

huochai 看你在master0挂载的没问题，安装的不是在master0机器吧，客户端安装了没， /etc/resolv.conf 里面的ip都是通的是吧，mysql的pvc也没有吗？ kubectl get pvc –all-namespaces有哪些呢？

huochai

Forest-L 客户端都测试过了，可以绑定pvc，IP也没问题。MySQL/redis 的pvc都没有

Forest-L

huochai 中午/晚上方便远程你的环境看下吗？

huochai

Forest-L 可以远程啊，感谢了

huochai

Forest-L 我加下你微信吧，帮忙看看，感谢了。

YSH

Forest-L 我也是一样的问题，到部署minio报了五个错，解决两三天了，没办法解决。能远程解决下吗大佬，有偿

Forest-L

huochai 已加，微信通过下，且把该信息删除吧

john

Forest-L

微信多少啊，分享一下吧

Forest-L

存储报错信息：

Events:
  Type     Reason                  Age                 From                     Message
  ----     ------                  ----                ----                     -------
  Warning  FailedScheduling        18m                 default-scheduler        pod has unbound immediate PersistentVolumeClaims (repeated 6 times)
  Normal   Scheduled               18m                 default-scheduler        Successfully assigned kubesphere-system/minio-8cd46c8d9-xzhkh to 192.168.30.103
  Normal   SuccessfulAttachVolume  18m                 attachdetach-controller  AttachVolume.Attach succeeded for volume "pvc-010eae1a-5213-4633-9ce3-ec8d1897278a"
  Warning  FailedMount             98s (x16 over 18m)  kubelet, 192.168.30.103  MountVolume.NewMounter initialization failed for volume "pvc-010eae1a-5213-4633-9ce3-ec8d1897278a" : Couldn't get secret kubesphere-system/ceph-secret err: secrets "ceph-secret" not found
  Warning  FailedMount             24s (x8 over 16m)   kubelet, 192.168.30.103  Unable to mount volumes for pod "minio-8cd46c8d9-xzhkh_kubesphere-system(7d021cd2-b9ca-4aa1-9463-c28435206df9)": timeout expired waiting for volumes to attach or mount for pod "kubesphere-system"/"minio-8cd46c8d9-xzhkh". list of unmounted volumes=[export]. list of unattached volumes=[export minio-user minio-config-dir ks-minio-token-zdwk2]

huochai

Forest-L 感谢大佬的排查，这边已经解决。看日志是ceph存储的配置问题，非常感谢

vipmachao

Forest-L 大佬我也遇到相同的问题可以帮忙协助看一下吗多谢

TASK [common : Kubesphere | Deploy minio] **************************************
fatal: [localhost]: FAILED! => {“changed”: true, “cmd”: “/usr/local/bin/helm upgrade –install ks-minio /kubesphere/kubesphere/minio-ha -f /kubesphere/kubesphere/custom-values-minio.yaml –set fullnameOverride=minio –namespace kubesphere-system –wait –timeout 1800s\n”, “delta”: “0:30:12.536292”, “end”: “2021-05-31 10:30:04.335264”, “msg”: “non-zero return code”, “rc”: 1, “start”: “2021-05-31 09:59:51.798972”, “stderr”: “Error: failed post-install: timed out waiting for the condition”, “stderr_lines”: [“Error: failed post-install: timed out waiting for the condition”], “stdout”: "Release \“ks-minio\” does not exist. Installing it now.", “stdout_lines”: ["Release \“ks-minio\” does not exist. Installing it now."]}
…ignoring

TASK [common : debug] **********************************************************
ok: [localhost] => {
“msg”: [
“1. check the storage configuration and storage server”,
“2. make sure the DNS address in /etc/resolv.conf is available”,
“3. execute ‘kubectl logs -n kubesphere-system -l job-name=minio-make-bucket-job’ to watch logs”,
“4. execute ‘helm -n kubesphere-system uninstall ks-minio && kubectl -n kubesphere-system delete job minio-make-bucket-job’”,
“5. Restart the installer pod in kubesphere-system namespace”
]

john

huochai

怎么排查的，可以说一下嘛

easy033

具体怎么解决的遇到了一样的问题

Small-market

vipmachao minio这个问题是网络或者储存出来问题,你可以在部署之前设置下kubernetes的仓库

cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=http://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=0
repo_gpgcheck=0
gpgkey=http://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg
       http://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF
yum makecache fast

Small-market

vipmachao 然后在进行安装,应该就好了,我就是这么弄,就弄好了

phoenix_s

@Forest-L 大佬，我也遇到同样的问题的，方便解决下吗？微信：itlzjava

yuswift

YSH 您可以参考一下工单服务
https://kubesphere.cloud/ticket/

misteruly

mysq-pvc没有生成呢，设置了默认存储了吗？kubectl get sc 看一看