kubesphere 在 k8s上最小化安装之后正常启动进入，然后按照https://kubesphere.com.cn/docs/pluggable-components/app-store/的方法启用devops,openpitrix组件之后，mysql 和 etcd始终处于异常状态，显示容器创建中。尝试过全部删除后重新安装依旧如此，请问这是什么原因？

    hongming kubesphere我重装之后mysql正常了但是ectd还是容器创建中

    kubectl get pod -n openpitrix-system

    NAME READY STATUS RESTARTS AGE
    hyperpitrix-generate-kubeconfig-jfzn5 0/2 Init:0/1 0 6m40s
    hyperpitrix-release-app-job-c479m 0/1 Init:0/2 0 6m41s
    openpitrix-hyperpitrix-deployment-5b585565cb-cbp5n 0/1 Init:1/11 0 6m43s

    kubectl get svc –all-namespaces

    NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
    default kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 27h
    kube-system etcd ClusterIP None <none> 2379/TCP 4h29m
    kube-system kube-controller-manager-svc ClusterIP None <none> 10252/TCP 23h
    kube-system kube-dns ClusterIP 10.96.0.10 <none> 53/UDP,53/TCP,9153/TCP 27h
    kube-system kube-scheduler-svc ClusterIP None <none> 10251/TCP 23h
    kube-system kubelet ClusterIP None <none> 10250/TCP,10255/TCP,4194/TCP 23h
    kubesphere-controls-system default-http-backend ClusterIP 10.103.59.109 <none> 80/TCP 17m
    kubesphere-devops-system ks-jenkins NodePort 10.104.133.232 <none> 80:30180/TCP 16m
    kubesphere-devops-system ks-jenkins-agent ClusterIP 10.111.218.118 <none> 50000/TCP 16m
    kubesphere-devops-system s2ioperator ClusterIP 10.104.107.144 <none> 443/TCP 16m
    kubesphere-devops-system s2ioperator-metrics-service ClusterIP 10.108.192.140 <none> 8080/TCP 16m
    kubesphere-devops-system uc-jenkins-update-center ClusterIP 10.103.40.109 <none> 80/TCP 17m
    kubesphere-devops-system webhook-server-service ClusterIP 10.109.68.201 <none> 443/TCP 16m
    kubesphere-monitoring-system alertmanager-main ClusterIP 10.105.136.173 <none> 9093/TCP 16m
    kubesphere-monitoring-system alertmanager-operated ClusterIP None <none> 9093/TCP,9094/TCP,9094/UDP 16m
    kubesphere-monitoring-system kube-state-metrics ClusterIP None <none> 8443/TCP,9443/TCP 16m
    kubesphere-monitoring-system node-exporter ClusterIP None <none> 9100/TCP 16m
    kubesphere-monitoring-system notification-manager-controller-metrics ClusterIP 10.102.209.130 <none> 8443/TCP 16m
    kubesphere-monitoring-system notification-manager-svc ClusterIP 10.101.107.204 <none> 19093/TCP 16m
    kubesphere-monitoring-system prometheus-k8s ClusterIP 10.107.172.143 <none> 9090/TCP 16m
    kubesphere-monitoring-system prometheus-operated ClusterIP None <none> 9090/TCP 16m
    kubesphere-monitoring-system prometheus-operator ClusterIP None <none> 8443/TCP 16m
    kubesphere-system etcd ClusterIP 10.105.41.185 <none> 2379/TCP 22h
    kubesphere-system ks-apiserver ClusterIP 10.109.121.27 <none> 80/TCP 23h
    kubesphere-system ks-console NodePort 10.105.10.11 <none> 80:30880/TCP 23h
    kubesphere-system ks-controller-manager ClusterIP 10.104.239.36 <none> 443/TCP 23h
    kubesphere-system minio ClusterIP 10.99.158.212 <none> 9000/TCP 18m
    kubesphere-system mysql ClusterIP 10.105.167.128 <none> 3306/TCP 22h
    kubesphere-system openldap ClusterIP None <none> 389/TCP 18m
    kubesphere-system redis ClusterIP 10.111.138.160 <none> 6379/TCP 18m
    kubesphere-system redis-ha ClusterIP None <none> 6379/TCP,26379/TCP 18m
    kubesphere-system redis-ha-announce-0 ClusterIP 10.96.13.107 <none> 6379/TCP,26379/TCP 18m
    kubesphere-system redis-ha-announce-1 ClusterIP 10.105.191.97 <none> 6379/TCP,26379/TCP 18m
    kubesphere-system redis-ha-announce-2 ClusterIP 10.105.121.73 <none> 6379/TCP,26379/TCP 18m
    longhorn-system csi-attacher ClusterIP 10.104.125.24 <none> 12345/TCP 27h
    longhorn-system csi-provisioner ClusterIP 10.107.232.166 <none> 12345/TCP 27h
    longhorn-system csi-resizer ClusterIP 10.102.7.84 <none> 12345/TCP 27h
    longhorn-system longhorn-backend ClusterIP 10.99.46.21 <none> 9500/TCP 27h
    longhorn-system longhorn-frontend NodePort 10.100.102.220 <none> 80:30890/TCP 27h
    openpitrix-system hyperpitrix ClusterIP None <none> 9123/TCP,9121/TCP,9100/TCP,9102/TCP,9113/TCP,9122/TCP,9104/TCP,9118/TCP,9106/TCP,9101/TCP,9108/TCP,9103/TCP,9107/TCP 12m

    kubectl get pod –all-namespaces

    NAMESPACE NAME READY STATUS RESTARTS AGE
    kube-system calico-kube-controllers-84445dd79f-fr7ql 1/1 Running 2 27h
    kube-system calico-node-8hr79 1/1 Running 1 27h
    kube-system calico-node-kmvwh 1/1 Running 0 27h
    kube-system calico-node-mk2jm 1/1 Running 0 27h
    kube-system calico-node-p8p87 1/1 Running 0 27h
    kube-system calico-node-vz77k 1/1 Running 0 27h
    kube-system calico-node-zpszl 1/1 Running 0 27h
    kube-system coredns-66bff467f8-rcw4v 1/1 Running 1 27h
    kube-system coredns-66bff467f8-wc9b8 1/1 Running 1 27h
    kube-system etcd-k8s-master1 1/1 Running 1 27h
    kube-system etcd-k8s-master2 1/1 Running 0 27h
    kube-system etcd-k8s-master3 1/1 Running 0 27h
    kube-system kube-apiserver-k8s-master1 1/1 Running 1 27h
    kube-system kube-apiserver-k8s-master2 1/1 Running 0 27h
    kube-system kube-apiserver-k8s-master3 1/1 Running 0 27h
    kube-system kube-controller-manager-k8s-master1 1/1 Running 2 27h
    kube-system kube-controller-manager-k8s-master2 1/1 Running 0 27h
    kube-system kube-controller-manager-k8s-master3 1/1 Running 0 27h
    kube-system kube-proxy-658dc 1/1 Running 0 27h
    kube-system kube-proxy-7c8fh 1/1 Running 0 27h
    kube-system kube-proxy-bgmwq 1/1 Running 0 27h
    kube-system kube-proxy-bh8s9 1/1 Running 0 27h
    kube-system kube-proxy-jxz22 1/1 Running 0 27h
    kube-system kube-proxy-wf4vx 1/1 Running 1 27h
    kube-system kube-scheduler-k8s-master1 1/1 Running 2 27h
    kube-system kube-scheduler-k8s-master2 1/1 Running 0 27h
    kube-system kube-scheduler-k8s-master3 1/1 Running 0 27h
    kube-system kube-sealyun-lvscare-k8s-work1 1/1 Running 0 27h
    kube-system kube-sealyun-lvscare-k8s-work2 1/1 Running 0 27h
    kube-system kube-sealyun-lvscare-k8s-work3 1/1 Running 0 27h
    kube-system snapshot-controller-0 1/1 Running 0 19m
    kubesphere-controls-system default-http-backend-857d7b6856-l26l7 1/1 Running 0 18m
    kubesphere-controls-system kubectl-admin-58f985d8f6-5lgwk 1/1 Running 0 17m
    kubesphere-devops-system ks-jenkins-54455f5db8-p8klx 1/1 Running 0 16m
    kubesphere-devops-system s2ioperator-0 1/1 Running 1 13m
    kubesphere-devops-system uc-jenkins-update-center-cd9464fff-xrtfv 1/1 Running 0 17m
    kubesphere-monitoring-system alertmanager-main-0 2/2 Running 0 16m
    kubesphere-monitoring-system alertmanager-main-1 2/2 Running 0 16m
    kubesphere-monitoring-system alertmanager-main-2 2/2 Running 0 16m
    kubesphere-monitoring-system kube-state-metrics-95c974544-5fgrv 3/3 Running 0 17m
    kubesphere-monitoring-system node-exporter-4ltrl 2/2 Running 0 17m
    kubesphere-monitoring-system node-exporter-bnls5 2/2 Running 0 17m
    kubesphere-monitoring-system node-exporter-lb69x 2/2 Running 0 17m
    kubesphere-monitoring-system node-exporter-mbd8c 2/2 Running 0 17m
    kubesphere-monitoring-system node-exporter-mpmbn 2/2 Running 0 17m
    kubesphere-monitoring-system node-exporter-nbjpk 2/2 Running 0 17m
    kubesphere-monitoring-system notification-manager-deployment-7c8df68d94-8l4j6 1/1 Running 0 16m
    kubesphere-monitoring-system notification-manager-deployment-7c8df68d94-l9zs9 1/1 Running 0 16m
    kubesphere-monitoring-system notification-manager-operator-6958786cd6-px467 2/2 Running 0 16m
    kubesphere-monitoring-system prometheus-k8s-0 3/3 Running 1 16m
    kubesphere-monitoring-system prometheus-k8s-1 3/3 Running 1 16m
    kubesphere-monitoring-system prometheus-operator-84d58bf775-4mcwp 2/2 Running 0 17m
    kubesphere-system etcd-65796969c7-48f4g 0/1 ContainerCreating 0 13m
    kubesphere-system ks-apiserver-54cbbf6995-8nfqt 1/1 Running 0 12m
    kubesphere-system ks-apiserver-54cbbf6995-ngqrg 1/1 Running 0 12m
    kubesphere-system ks-apiserver-54cbbf6995-x7×85 1/1 Running 0 12m
    kubesphere-system ks-console-786b9846d4-7269h 1/1 Running 0 17m
    kubesphere-system ks-console-786b9846d4-94hsr 1/1 Running 0 17m
    kubesphere-system ks-console-786b9846d4-tt69v 1/1 Running 0 17m
    kubesphere-system ks-controller-manager-75766f9676-2ckxw 1/1 Running 0 12m
    kubesphere-system ks-controller-manager-75766f9676-44cdr 1/1 Running 0 12m
    kubesphere-system ks-controller-manager-75766f9676-h8rjk 1/1 Running 0 12m
    kubesphere-system ks-installer-7cb866bd-gcbnt 1/1 Running 0 20m
    kubesphere-system minio-7bfdb5968b-pkzk4 1/1 Running 0 19m
    kubesphere-system mysql-7f64d9f584-m4mh6 1/1 Running 0 18m
    kubesphere-system openldap-0 1/1 Running 0 19m
    kubesphere-system openldap-1 1/1 Running 0 18m
    kubesphere-system redis-ha-haproxy-5c6559d588-4ndn6 1/1 Running 0 19m
    kubesphere-system redis-ha-haproxy-5c6559d588-lkndt 1/1 Running 0 19m
    kubesphere-system redis-ha-haproxy-5c6559d588-xs64t 1/1 Running 0 19m
    kubesphere-system redis-ha-server-0 2/2 Running 0 19m
    kubesphere-system redis-ha-server-1 2/2 Running 0 18m
    kubesphere-system redis-ha-server-2 2/2 Running 0 18m
    longhorn-system csi-attacher-5b4745c5f7-gxpbs 1/1 Running 0 27h
    longhorn-system csi-attacher-5b4745c5f7-pzscm 1/1 Running 0 27h
    longhorn-system csi-attacher-5b4745c5f7-xdghd 1/1 Running 0 27h
    longhorn-system csi-provisioner-57d6dbf5f4-cfpw7 1/1 Running 0 27h
    longhorn-system csi-provisioner-57d6dbf5f4-hxkvn 1/1 Running 0 27h
    longhorn-system csi-provisioner-57d6dbf5f4-z2nwq 1/1 Running 0 27h
    longhorn-system csi-resizer-75ff56bc48-dx9qs 1/1 Running 0 27h
    longhorn-system csi-resizer-75ff56bc48-hfvkf 1/1 Running 0 27h
    longhorn-system csi-resizer-75ff56bc48-kzsdx 1/1 Running 0 27h
    longhorn-system engine-image-ei-ee18f965-2qghf 1/1 Running 0 27h
    longhorn-system engine-image-ei-ee18f965-ghv6q 1/1 Running 0 27h
    longhorn-system engine-image-ei-ee18f965-hbg7x 1/1 Running 0 27h
    longhorn-system engine-image-ei-ee18f965-kk6bh 1/1 Running 0 27h
    longhorn-system engine-image-ei-ee18f965-lcm6g 1/1 Running 1 27h
    longhorn-system engine-image-ei-ee18f965-zq8rn 1/1 Running 0 27h
    longhorn-system instance-manager-e-311a6336 1/1 Running 0 27h
    longhorn-system instance-manager-e-5f198dc4 1/1 Running 0 27h
    longhorn-system instance-manager-e-a6b473f8 1/1 Running 0 27h
    longhorn-system instance-manager-e-c6c60496 1/1 Running 0 9h
    longhorn-system instance-manager-e-cd642c0f 1/1 Running 0 27h
    longhorn-system instance-manager-e-e7b78410 1/1 Running 0 27h
    longhorn-system instance-manager-r-274c768f 1/1 Running 0 27h
    longhorn-system instance-manager-r-57346b70 1/1 Running 0 27h
    longhorn-system instance-manager-r-62baae1f 1/1 Running 0 27h
    longhorn-system instance-manager-r-8e1e2f94 1/1 Running 0 27h
    longhorn-system instance-manager-r-b7612d59 1/1 Running 0 27h
    longhorn-system instance-manager-r-dec0896a 1/1 Running 0 9h
    longhorn-system longhorn-csi-plugin-5hvt8 2/2 Running 3 27h
    longhorn-system longhorn-csi-plugin-5xfr4 2/2 Running 0 27h
    longhorn-system longhorn-csi-plugin-b5mh2 2/2 Running 0 27h
    longhorn-system longhorn-csi-plugin-ktpvw 2/2 Running 0 27h
    longhorn-system longhorn-csi-plugin-lq7cj 2/2 Running 0 27h
    longhorn-system longhorn-csi-plugin-vn5j9 2/2 Running 0 27h
    longhorn-system longhorn-driver-deployer-6756bb8fd6-s4gfh 1/1 Running 0 27h
    longhorn-system longhorn-manager-4nmqw 1/1 Running 1 27h
    longhorn-system longhorn-manager-5qpf6 1/1 Running 2 27h
    longhorn-system longhorn-manager-9wjbn 1/1 Running 1 27h
    longhorn-system longhorn-manager-n5g5z 1/1 Running 2 27h
    longhorn-system longhorn-manager-qr2vn 1/1 Running 2 27h
    longhorn-system longhorn-manager-slvr9 1/1 Running 1 27h
    longhorn-system longhorn-ui-6fb889895f-hjfhb 1/1 Running 0 27h
    openpitrix-system hyperpitrix-generate-kubeconfig-jfzn5 0/2 Init:0/1 0 13m
    openpitrix-system hyperpitrix-release-app-job-c479m 0/1 Init:0/2 0 13m
    openpitrix-system openpitrix-hyperpitrix-deployment-5b585565cb-cbp5n 0/1 Init:1/11 0 13m

        ladyishenlong

        kubectl -n kubesphere-system describe deploy etcd
kubectl -n kubesphere-system get pvc

            hongming
            `

            kubectl -n kubesphere-system describe deploy etcd

            Name: etcd
            Namespace: kubesphere-system
            CreationTimestamp: Fri, 20 Nov 2020 13:59:58 +0800
            Labels: app=kubesphere
            tier=etcd
            Annotations: deployment.kubernetes.io/revision: 1
            Selector: app=kubesphere,tier=etcd
            Replicas: 1 desired | 1 updated | 1 total | 0 available | 1 unavailable
            StrategyType: Recreate
            MinReadySeconds: 0
            Pod Template:
            Labels: app=kubesphere
            tier=etcd
            Containers:
            etcd:
            Image: kubesphere/etcd:v3.2.18
            Port: 2379/TCP
            Host Port: 0/TCP
            Command:
            etcd
            –data-dir=/data
            –listen-client-urls=http://0.0.0.0:2379
            –advertise-client-urls=http://etcd.kubesphere-system.svc:2379
            –max-snapshots=5
            –max-wals=5
            –auto-compaction-retention=168
            Environment: <none>
            Mounts:
            /data from etcd-persistent-storage (rw)
            Volumes:
            etcd-persistent-storage:
            Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
            ClaimName: etcd-pvc
            ReadOnly: false
            Conditions:
            Type Status Reason

            Available False MinimumReplicasUnavailable
            Progressing False ProgressDeadlineExceeded
            OldReplicaSets: <none>
            NewReplicaSet: etcd-65796969c7 (1/1 replicas created)
            Events:
            Type Reason Age From Message

            Normal ScalingReplicaSet 22m deployment-controller Scaled up replica set etcd-65796969c7 to 1
            `

            kubectl -n kubesphere-system get pvc
            NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
            data-redis-ha-server-0 Bound pvc-56a499c9-a555-452f-b1f4-fa1ca7704adc 2Gi RWO longhorn 28m
            data-redis-ha-server-1 Bound pvc-1374503b-cfdc-4d33-b1e1-662a28155e4a 2Gi RWO longhorn 27m
            data-redis-ha-server-2 Bound pvc-58c011fc-040a-41b8-8072-08cbfebdd462 2Gi RWO longhorn 27m
            etcd-pvc Bound pvc-3739bbe0-2380-4f0d-8a19-8b2f65e0d961 20Gi RWO longhorn 23m
            minio Bound pvc-274cce95-af24-47ac-936e-0e7f9b5e7079 20Gi RWO longhorn 28m
            mysql-pvc Bound pvc-1e7baaba-4fe1-4c99-acf5-77f84672fdeb 20Gi RWO longhorn 27m
            openldap-pvc-openldap-0 Bound pvc-8c10c9d1-7296-46b3-ae67-25622a2f789d 2Gi RWO longhorn 28m
            openldap-pvc-openldap-1 Bound pvc-7ab2da20-d24a-4cb0-ab71-5f49c7ad667e 2Gi RWO longhorn 27m

                ladyishenlong

                kubectl -n kubesphere-system describe po etcd-65796969c7-48f4g

                    hongming

                    kubectl -n kubesphere-system describe po etcd-65796969c7-48f4g

                    Name: etcd-65796969c7-48f4g
                    Namespace: kubesphere-system
                    Priority: 0
                    Node: k8s-work3/192.168.1.209
                    Start Time: Fri, 20 Nov 2020 14:00:02 +0800
                    Labels: app=kubesphere
                    pod-template-hash=65796969c7
                    tier=etcd
                    Annotations: <none>
                    Status: Pending
                    IP:
                    IPs: <none>
                    Controlled By: ReplicaSet/etcd-65796969c7
                    Containers:
                    etcd:
                    Container ID:
                    Image: kubesphere/etcd:v3.2.18
                    Image ID:
                    Port: 2379/TCP
                    Host Port: 0/TCP
                    Command:
                    etcd
                    –data-dir=/data
                    –listen-client-urls=http://0.0.0.0:2379
                    –advertise-client-urls=http://etcd.kubesphere-system.svc:2379
                    –max-snapshots=5
                    –max-wals=5
                    –auto-compaction-retention=168
                    State: Waiting
                    Reason: ContainerCreating
                    Ready: False
                    Restart Count: 0
                    Environment: <none>
                    Mounts:
                    /data from etcd-persistent-storage (rw)
                    /var/run/secrets/kubernetes.io/serviceaccount from default-token-qhtjl (ro)
                    Conditions:
                    Type Status
                    Initialized True
                    Ready False
                    ContainersReady False
                    PodScheduled True
                    Volumes:
                    etcd-persistent-storage:
                    Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
                    ClaimName: etcd-pvc
                    ReadOnly: false
                    default-token-qhtjl:
                    Type: Secret (a volume populated by a Secret)
                    SecretName: default-token-qhtjl
                    Optional: false
                    QoS Class: BestEffort
                    Node-Selectors: <none>
                    Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
                    node.kubernetes.io/unreachable:NoExecute for 300s
                    Events:
                    Type Reason Age From Message

                    Warning FailedScheduling 28m (x3 over 28m) default-scheduler running “VolumeBinding” filter plugin for pod “etcd-65796969c7-48f4g”: pod has unbound immediate PersistentVolumeClaims
                    Normal Scheduled 28m default-scheduler Successfully assigned kubesphere-system/etcd-65796969c7-48f4g to k8s-work3
                    Warning FailedAttachVolume 28m (x2 over 28m) attachdetach-controller AttachVolume.Attach failed for volume “pvc-3739bbe0-2380-4f0d-8a19-8b2f65e0d961” : rpc error: code = Internal desc = Bad response statusCode [500]. Status [500 Internal Server Error]. Body: [message=unable to attach volume pvc-3739bbe0-2380-4f0d-8a19-8b2f65e0d961 to k8s-work3: volume pvc-3739bbe0-2380-4f0d-8a19-8b2f65e0d961 not scheduled, code=Server Error, detail=] from [http://longhorn-backend:9500/v1/volumes/pvc-3739bbe0-2380-4f0d-8a19-8b2f65e0d961?action=attach]
                    Warning FailedAttachVolume 18m (x11 over 28m) attachdetach-controller AttachVolume.Attach failed for volume “pvc-3739bbe0-2380-4f0d-8a19-8b2f65e0d961” : rpc error: code = Internal desc = Bad response statusCode [500]. Status [500 Internal Server Error]. Body:

                     from [http://longhorn-backend:9500/v1/volumes/pvc-3739bbe0-2380-4f0d-8a19-8b2f65e0d961?action=attach]
  Warning  FailedMount         6m7s (x10 over 26m)  kubelet, k8s-work3       Unable to attach or mount volumes: unmounted volumes=[etcd-persistent-storage], unattached volumes=[etcd-persistent-storage default-token-qhtjl]: timed out waiting for the condition
  Warning  FailedMount         97s (x2 over 3m53s)  kubelet, k8s-work3       Unable to attach or mount volumes: unmounted volumes=[etcd-persistent-storage], unattached volumes=[default-token-qhtjl etcd-persistent-storage]: timed out waiting for the condition
  Warning  FailedAttachVolume  2s (x9 over 16m)     attachdetach-controller  AttachVolume.Attach failed for volume "pvc-3739bbe0-2380-4f0d-8a19-8b2f65e0d961" : rpc error: code = Internal desc = Bad response statusCode [500]. Status [500 Internal Server Error]. 
Body: [detail=, message=unable to attach volume pvc-3739bbe0-2380-4f0d-8a19-8b2f65e0d961 to k8s-work3: volume pvc-3739bbe0-2380-4f0d-8a19-8b2f65e0d961 not scheduled, code=Server Error] from [http://longhorn-backend:9500/v1/volumes/pvc-3739bbe0-2380-4f0d-8a19-8b2f65e0d961?action=attach]

                        hongming
                        也就是说是Longhorn的问题是吧，是这个吗

                        It seems that there are only 2 worker nodes for the Longhorn system.

                        You can choose one of the following solutions to fix this issue:

                        1. Enable the setting Replica Node Level Soft Anti-Affinity.

                        2. Create one more StorageClass by copying the default StorageClass but decreasing the parameter numberOfReplicas value.
                          The parameters of the existing StrogeClass are immutable after the deployment hence you cannot directly modify the default StorageClass.
                          The parameter numberOfReplicas cannot be greater than the worker node number when the setting Replica Node Level Soft Anti-Affinity is disabled