Feynman
我认为先决条件都已经设置好了啊。都是干净的Centos,防火墙也全部关闭了。该提前安装的也都安装了。

先./kk delete cluster -f <config yaml>清理一下环境再安装试试

    rayzhou2017 好的,我重试一下,vsphere 部署文档有歧义,准备虚拟机应该是8台,VIP不应计算为一台虚拟机。

    rayzhou2017
    [master-21 10.10.10.21] MSG:
    [reset] Reading configuration from the cluster…
    [reset] FYI: You can look at this config file with ‘kubectl -n kube-system get cm kubeadm-config -oyaml’
    W0906 12:58:47.629895 40112 reset.go:99] [reset] Unable to fetch the kubeadm-config ConfigMap from cluster: failed to get config map: Get https://lb.kubesphere.local:6443/api/v1/namespaces/kube-system/configmaps/kubeadm-config?timeout=10s: dial tcp 10.10.10.20:6443: connect: connection refused
    [preflight] Running pre-flight checks
    W0906 12:58:47.630260 40112 removeetcdmember.go:79] [reset] No kubeadm config, using etcd pod spec to get data directory
    [reset] No etcd config found. Assuming external etcd
    [reset] Please, manually reset etcd to prevent further issues
    [reset] Stopping the kubelet service
    [reset] Unmounting mounted directories in “/var/lib/kubelet”
    [reset] Deleting contents of config directories: [/etc/kubernetes/manifests /etc/kubernetes/pki]
    [reset] Deleting files: [/etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/bootstrap-kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf]
    [reset] Deleting contents of stateful directories: [/var/lib/kubelet /var/lib/dockershim /var/run/kubernetes /var/lib/cni]

    The reset process does not clean CNI configuration. To do so, you must remove /etc/cni/net.d

    The reset process does not reset or clean up iptables rules or IPVS tables.
    If you wish to reset iptables, you must do so manually by using the “iptables” command.

    If your cluster was setup to utilize IPVS, run ipvsadm –clear (or similar)
    to reset your system’s IPVS tables.

    The reset process does not clean your kubeconfig files and you must remove them manually.
    Please, check the contents of the $HOME/.kube/config file.

    rayzhou2017 我换成v1.17.9 安装仍然有问题。
    配置文件
    `

    apiVersion: kubekey.kubesphere.io/v1alpha1
    kind: Cluster
    metadata:
      name: config-sample
    spec:
      hosts:
      - {name: master-21, address: 10.10.10.21, internalAddress: 10.10.10.21, password: an@123@#!}
      - {name: master-22, address: 10.10.10.22, internalAddress: 10.10.10.22, password: an@123@#!}
      - {name: master-23, address: 10.10.10.23, internalAddress: 10.10.10.23, password: an@123@#!}
      - {name: node-24, address: 10.10.10.24, internalAddress: 10.10.10.24, password: an@123@#!}
      - {name: node-25, address: 10.10.10.25, internalAddress: 10.10.10.25, password: an@123@#!}
      - {name: node-26, address: 10.10.10.26, internalAddress: 10.10.10.26, password: an@123@#!}
      - {name: node-27, address: 10.10.10.27, internalAddress: 10.10.10.27, password: an@123@#!}
      - {name: node-28, address: 10.10.10.28, internalAddress: 10.10.10.28, password: an@123@#!}
      roleGroups:
        etcd:
        - master-21
        - master-22
        - master-23
        master:
        - master-21
        - master-22
        - master-23
        worker:
        - node-24
        - node-25
        - node-26
        - node-27
        - node-28
      controlPlaneEndpoint:
        domain: lb.kubesphere.local
        # vip
        address: "10.10.10.20"
        port: "6443"
      kubernetes:
        version: v1.17.9
        imageRepo: kubesphere
        clusterName: cluster.local
        masqueradeAll: false  # masqueradeAll tells kube-proxy to SNAT everything if using the pure iptables proxy mode. [Default: false]
        maxPods: 110  # maxPods is the number of pods that can run on this Kubelet. [Default: 110]
        nodeCidrMaskSize: 24  # internal network node size allocation. This is the size allocated to each node on your network. [Default: 24]
        proxyMode: ipvs  # mode specifies which proxy mode to use. [Default: ipvs]
      network:
        plugin: calico
        calico:
          ipipMode: Always  # IPIP Mode to use for the IPv4 POOL created at start up. If set to a value other than Never, vxlanMode should be set to "Never". [Always | CrossSubnet | Never] [Default: Always]
          vxlanMode: Never  # VXLAN Mode to use for the IPv4 POOL created at start up. If set to a value other than Never, ipipMode should be set to "Never". [Always | CrossSubnet | Never] [Default: Never]
          vethMTU: 1440  # The maximum transmission unit (MTU) setting determines the largest packet size that can be transmitted through your network. [Default: 1440]
        kubePodsCIDR: 10.233.64.0/18
        kubeServiceCIDR: 10.233.0.0/18
      registry:
        registryMirrors: []
        insecureRegistries: []
      addons: [] # add your persistent storage and LoadBalancer plugin configuration here if you have, see https://kubesphere.io/docs/installing-on-linux/introduction/storage-configuration

    zackzhang 就是上面的错误,感觉是lb通信问题。但是我PING了是通的。
    [master-21 10.10.10.21] MSG:
    [reset] Reading configuration from the cluster…
    [reset] FYI: You can look at this config file with ‘kubectl -n kube-system get cm kubeadm-config -oyaml’
    W0906 12:58:47.629895 40112 reset.go:99] [reset] Unable to fetch the kubeadm-config ConfigMap from cluster: failed to get config map: Get https://lb.kubesphere.local:6443/api/v1/namespaces/kube-system/configmaps/kubeadm-config?timeout=10s: dial tcp 10.10.10.20:6443: connect: connection refused
    [preflight] Running pre-flight checks
    W0906 12:58:47.630260 40112 removeetcdmember.go:79] [reset] No kubeadm config, using etcd pod spec to get data directory
    [reset] No etcd config found. Assuming external etcd
    [reset] Please, manually reset etcd to prevent further issues
    [reset] Stopping the kubelet service
    [reset] Unmounting mounted directories in “/var/lib/kubelet”
    [reset] Deleting contents of config directories: [/etc/kubernetes/manifests /etc/kubernetes/pki]
    [reset] Deleting files: [/etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/bootstrap-kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf]
    [reset] Deleting contents of stateful directories: [/var/lib/kubelet /var/lib/dockershim /var/run/kubernetes /var/lib/cni]

    The reset process does not clean CNI configuration. To do so, you must remove /etc/cni/net.d

    The reset process does not reset or clean up iptables rules or IPVS tables.
    If you wish to reset iptables, you must do so manually by using the “iptables” command.

    If your cluster was setup to utilize IPVS, run ipvsadm –clear (or similar)
    to reset your system’s IPVS tables.

    The reset process does not clean your kubeconfig files and you must remove them manually.
    Please, check the contents of the $HOME/.kube/config file.

      leejor

      Storing the certificates in Secret “kubeadm-certs” in the “kube-system” Namespace
      error execution phase upload-certs: error uploading certs: error creating token: timed out waiting for the condition

      连接到kube-apiserver超时,能ping通vip并不代表vip的端口能通,要保证vip设置的kub-apiserver端口能通。

        Cauchy

        lb是自己搭的?还是用的云lb?lb上是不是挂了防火墙或者安全组

          leejor
          那有可能是lb搞得有问题,curl -k https://{masterip}:6443 curl -k https://{vip}:{lbport} 对比下返回结果,不一样的话,就是lb搞得有问题了。

            12 天 后

            Mark一下,我今天也遇到相同的问题@leejor 最后解决了吗

            Cauchy 我的确实是lb的问题,lb配置用之前的测试主机,防火墙是关了,但是发现唯独这台有配置其它业务的iptables

            5 个月 后

            我也遇到了,
            yum install -y socat conntrack 解决

            10 天 后

            我这边也遇到该错误,不过我不是连接超时,不清楚怎么解决