TASK [common : KubeSphere | Creating Storage ProvisionerCapability] ************
 \"ProvisionerCapability\" in version \"storage.kubesphere.io/v1alpha1\"\nunable to recognize \"/kubesphere/kubesphere/ks-crds/provisonercapability/rbd.yaml\": no matches for kind \"ProvisionerCapability\" in version \"storage.kubesphere.io/v1alpha1\"", "stderr_lines": ["unable to recognize \"/kubesphere/kubesphere/ks-crds/provisonercapability/aws-ebs.yaml\": no matches for kind \"ProvisionerCapability\" in version \"storage.kubesphere.io/v1alpha1\"", "unable to recognize \"/kubesphere/kubesphere/ks-crds/provisonercapability/azure-disk.yaml\": no matches for kind \"ProvisionerCapability\" in version \"storage.kubesphere.io/v1alpha1\"", "unable to recognize \"/kubesphere/kubesphere/ks-crds/provisonercapability/azure-file.yaml\": no matches for kind \"ProvisionerCapability\" in version \"storage.kubesphere.io/v1alpha1\"", "unable to recognize \"/kubesphere/kubesphere/ks-crds/provisonercapability/cinder.yaml\": no matches for kind \"ProvisionerCapability\" in version \"storage.kubesphere.io/v1alpha1\"", "unable to recognize \"/kubesphere/kubesphere/ks-crds/provisonercapability/gce-pd.yaml\": no matches for kind \"ProvisionerCapability\" in version \"storage.kubesphere.io/v1alpha1\"", "unable to recognize \"/kubesphere/kubesphere/ks-crds/provisonercapability/glusterfs.yaml\": no matches for kind \"ProvisionerCapability\" in version \"storage.kubesphere.io/v1alpha1\"", "unable to recognize \"/kubesphere/kubesphere/ks-crds/provisonercapability/portworx-volume.yaml\": no matches for kind \"ProvisionerCapability\" in version \"storage.kubesphere.io/v1alpha1\"", "unable to recognize \"/kubesphere/kubesphere/ks-crds/provisonercapability/rbd-csi-ceph.yaml\": no matches for kind \"ProvisionerCapability\" in version \"storage.kubesphere.io/v1alpha1\"", "unable to recognize \"/kubesphere/kubesphere/ks-crds/provisonercapability/rbd-external.yaml\": no matches for kind \"ProvisionerCapability\" in version \"storage.kubesphere.io/v1alpha1\"", "unable to recognize \"/kubesphere/kubesfatal: [localhost]: FAILED! => {"changed": true, "cmd": "/usr/local/bin/kubectl apply -R -f /kubesphere/kubesphere/ks-crds/provisonercapability", "delta": "0:00:01.651819", "end": "2021-05-29 14:41:24.769446", "failed_when_result": true, "msg": "non-zero return code", "rc": 1, "start": "2021-05-29 14:41:23.117627", "stderr": "unable to recognize \"/kubesphere/kubesphere/ks-crds/provisonercapability/aws-ebs.yaml\": no matches for kind \"ProvisionerCapability\" in version \"storage.kubesphere.io/v1alpha1\"\nunable to recognize \"/kubesphere/kubesphere/ks-crds/provisonercapability/azure-disk.yaml\": no matches for kind \"ProvisionerCapability\" in version \"storage.kubesphere.io/v1alpha1\"\nunable to recognize \"/kubesphere/kubesphere/ks-crds/provisonercapability/azure-file.yaml\": no matches for kind \"ProvisionerCapability\" in version \"storage.kubesphere.io/v1alpha1\"\nunable to recognize \"/kubesphere/kubesphere/ks-crds/provisonercapability/cinder.yaml\": no matches for kind \"ProvisionerCapability\" in version \"storage.kubesphere.io/v1alpha1\"\nunable to recognize \"/kubesphere/kubesphere/ks-crds/provisonercapability/gce-pd.yaml\": no matches for kind \"ProvisionerCapability\" in version \"storage.kubesphere.io/v1alpha1\"\nunable to recognize \"/kubesphere/kubesphere/ks-crds/provisonercapability/glusterfs.yaml\": no matches for kind \"ProvisionerCapability\" in version \"storage.kubesphere.io/v1alpha1\"\nunable to recognize \"/kubesphere/kubesphere/ks-crds/provisonercapability/portworx-volume.yaml\": no matches for kind \"ProvisionerCapability\" in version \"storage.kubesphere.io/v1alpha1\"\nunable to recognize \"/kubesphere/kubesphere/ks-crds/provisonercapability/rbd-csi-ceph.yaml\": no matches for kind \"ProvisionerCapability\" in version \"storage.kubesphere.io/v1alpha1\"\nunable to recognize \"/kubesphere/kubesphere/ks-crds/provisonercapability/rbd-external.yaml\": no matches for kind \"ProvisionerCapability\" in version \"storage.kubesphere.io/v1alpha1\"\nunable to recognize \"/kubesphere/kubesphere/ks-crds/provisonercapability/rbd.yaml\": no matches for kind \"ProvisionerCapability\" in version \"storage.kubesphere.io/v1alpha1\"", "stderr_lines": ["unable to recognize \"/kubesphere/kubesphere/ks-crds/provisonercapability/aws-ebs.yaml\": no matches for kind \"ProvisionerCapability\" in version \"storage.kubesphere.io/v1alpha1\"", "unable to recognize \"/kubesphere/kubesphere/ks-crds/provisonercapability/azure-disk.yaml\": no matches for kind \"ProvisionerCapability\" in version \"storage.kubesphere.io/v1alpha1\"", "unable to recognize \"/kubesphere/kubesphere/ks-crds/provisonercapability/azure-file.yaml\": no matches for kind \"ProvisionerCapability\" in version \"storage.kubesphere.io/v1alpha1\"", "unable to recognize \"/kubesphere/kubesphere/ks-crds/provisonercapability/cinder.yaml\": no matches for kind \"ProvisionerCapability\" in version \"storage.kubesphere.io/v1alpha1\"", "unable to recognize \"/kubesphere/kubesphere/ks-crds/provisonercapability/gce-pd.yaml\": no matches for kind \"ProvisionerCapability\" in version \"storage.kubesphere.io/v1alpha1\"", "unable to recognize \"/kubesphere/kubesphere/ks-crds/provisonercapability/glusterfs.yaml\": no matches for kind \"ProvisionerCapability\" in version \"storage.kubesphere.io/v1alpha1\"", "unable to recognize \"/kubesphere/kubesphere/ks-crds/provisonercapability/portworx-volume.yaml\": no matches for kind \"ProvisionerCapability\" in version \"storage.kubesphere.io/v1alpha1\"", "unable to recognize \"/kubesphere/kubesphere/ks-crds/provisonercapability/rbd-csi-ceph.yaml\": no matches for kind \"ProvisionerCapability\" in version \"storage.kubesphere.io/v1alpha1\"", "unable to recognize \"/kubesphere/kubesphere/ks-crds/provisonercapability/rbd-external.yaml\": no matches for kind \"ProvisionerCapability\" in version \"storage.kubesphere.io/v1alpha1\"", "unable to recognize \"/kubesphere/kubesphere/ks-crds/provisonercapability/rbd.yaml\": no matches for kind \"ProvisionerCapability\" in version \"storage.kubesphere.io/v1alpha1\""], "stdout": "", "stdout_lines": []}

PLAY RECAP *********************************************************************
localhost                  : ok=13   changed=8    unreachable=0    failed=1    skipped=4    rescued=0    ignored=0   
[upl-image-preview url=https://kubesphere.com.cn/forum/assets/files/2021-05-29/1622270887-498780-16222708651.png]
  • zhangwenhuai 回复了此帖

  • zhangwenhuai 将ks-installer对应pod删除即会重新触发安装：

    kubectl delete -n kubesphere-system pod $(kubectl get pod -n kubesphere-system -l app=ks-install -o jsonpath='{.items[0].metadata.name}')

      看起来是ProvisionerCapability这个CRD没有安装成功
      这个CRD是探测存储类的快照/克隆/扩容等能力，并在console中启用对应功能所需的，对你在CLI中以及console中使用其他功能不会有影响
      可以试试手动安装此CRD，看下是哪里出问题了：

      kubectl -n kubesphere-system exec -it $(kubectl get pod -n kubesphere-system -l app=ks-install -o jsonpath='{.items[0].metadata.name}') -- kubectl apply -f kubesphere/ks-crds/storage.kubesphere.io_provisionercapabilities.yaml

        zhangwenhuai
        看来这个CRD又被创建成功了，看看是什么时候创建的呢：

        kubectl get crds provisionercapabilities.storage.kubesphere.io

        以及，既然CRD已存在了，试试看安装之前失败的那些CR：

        kubectl -n kubesphere-system exec -it $(kubectl get pod -n kubesphere-system -l app=ks-install -o jsonpath='{.items[0].metadata.name}') -- kubectl apply -R -f kubesphere/ks-crds/provisonercapability

            手动模拟创建这部门 也没有 报错，但是 执行部署安装的时候报错，是为什么呢？

              zhangwenhuai 在你初次安装（14:41）时，provisionercapabilities.storage.kubesphere.io这个CRD由于某些原因没有创建成功，而红圈中的执行步骤依赖于此CRD，因此同样执行失败。
              而该CRD在之后（14:57）被创建成功，因此你手动执行红圈中的步骤也能成功了。

                  zhangwenhuai 将ks-installer对应pod删除即会重新触发安装：

                  kubectl delete -n kubesphere-system pod $(kubectl get pod -n kubesphere-system -l app=ks-install -o jsonpath='{.items[0].metadata.name}')

                      kevendeng 现在触发重新安装了， 有什么办法 可以复盘一下 当时为什么没有 安装上
                      provisionercapabilities.storage.kubesphere.io CRD 对象呢？

                          zhangwenhuai 从ks-installer的日志来看，kubectl apply CRD的操作确实是执行成功了，但api内CRD直到10分钟后才创建成功，这很不正常，因为这个过程虽然是异步的，但一般会立刻完成。
                          建议检查下你初次安装的时间段内，kube-apiserver处理该CRD请求的日志，看看是否有什么异常，导致这个过程卡住了。
                          refer：https://stackoverflow.com/questions/57115602/how-to-kubectl-wait-for-crd-creation

                            6 个月 后