INFO[12:49:12 CST] Generating etcd certs
INFO[12:49:14 CST] Synchronizing etcd certs
INFO[12:49:14 CST] Creating etcd service
[k8s-master3 10.0.0.8] MSG:
etcd will be installed
[k8s-master1 10.0.0.6] MSG:
etcd will be installed
[k8s-master2 10.0.0.7] MSG:
etcd will be installed
INFO[12:49:19 CST] Starting etcd cluster
[k8s-master1 10.0.0.6] MSG:
Configuration file will be created
[k8s-master2 10.0.0.7] MSG:
Configuration file will be created
[k8s-master3 10.0.0.8] MSG:
Configuration file will be created
INFO[12:49:19 CST] Refreshing etcd configuration
[k8s-master3 10.0.0.8] MSG:
Created symlink from /etc/systemd/system/multi-user.target.wants/etcd.service to /etc/systemd/system/etcd.service.
[k8s-master1 10.0.0.6] MSG:
Created symlink from /etc/systemd/system/multi-user.target.wants/etcd.service to /etc/systemd/system/etcd.service.
Waiting for etcd to start
Waiting for etcd to start
[k8s-master2 10.0.0.7] MSG:
Created symlink from /etc/systemd/system/multi-user.target.wants/etcd.service to /etc/systemd/system/etcd.service.
Waiting for etcd to start
INFO[12:49:25 CST] Backup etcd data regularly
INFO[12:49:31 CST] Get cluster status
[k8s-master1 10.0.0.6] MSG:
Cluster will be created.
INFO[12:49:31 CST] Installing kube binaries
Push /root/kubekey/v1.20.4/amd64/kubeadm to 10.0.0.7:/tmp/kubekey/kubeadm Done
Push /root/kubekey/v1.20.4/amd64/kubeadm to 10.0.0.4:/tmp/kubekey/kubeadm Done
Push /root/kubekey/v1.20.4/amd64/kubeadm to 10.0.0.2:/tmp/kubekey/kubeadm Done
Push /root/kubekey/v1.20.4/amd64/kubeadm to 10.0.0.8:/tmp/kubekey/kubeadm Done
Push /root/kubekey/v1.20.4/amd64/kubeadm to 10.0.0.5:/tmp/kubekey/kubeadm Done
Push /root/kubekey/v1.20.4/amd64/kubeadm to 10.0.0.6:/tmp/kubekey/kubeadm Done
Push /root/kubekey/v1.20.4/amd64/kubelet to 10.0.0.6:/tmp/kubekey/kubelet Done
Push /root/kubekey/v1.20.4/amd64/kubelet to 10.0.0.7:/tmp/kubekey/kubelet Done
Push /root/kubekey/v1.20.4/amd64/kubectl to 10.0.0.6:/tmp/kubekey/kubectl Done
Push /root/kubekey/v1.20.4/amd64/kubelet to 10.0.0.5:/tmp/kubekey/kubelet Done
Push /root/kubekey/v1.20.4/amd64/kubelet to 10.0.0.2:/tmp/kubekey/kubelet Done
Push /root/kubekey/v1.20.4/amd64/kubelet to 10.0.0.4:/tmp/kubekey/kubelet Done
Push /root/kubekey/v1.20.4/amd64/helm to 10.0.0.6:/tmp/kubekey/helm Done
Push /root/kubekey/v1.20.4/amd64/kubelet to 10.0.0.8:/tmp/kubekey/kubelet Done
Push /root/kubekey/v1.20.4/amd64/kubectl to 10.0.0.4:/tmp/kubekey/kubectl Done
Push /root/kubekey/v1.20.4/amd64/cni-plugins-linux-amd64-v0.8.6.tgz to 10.0.0.6:/tmp/kubekey/cni-plugins-linux-amd64-v0.8.6.tgz Done
Push /root/kubekey/v1.20.4/amd64/kubectl to 10.0.0.2:/tmp/kubekey/kubectl Done
Push /root/kubekey/v1.20.4/amd64/kubectl to 10.0.0.8:/tmp/kubekey/kubectl Done
Push /root/kubekey/v1.20.4/amd64/kubectl to 10.0.0.7:/tmp/kubekey/kubectl Done
Push /root/kubekey/v1.20.4/amd64/helm to 10.0.0.2:/tmp/kubekey/helm Done
Push /root/kubekey/v1.20.4/amd64/helm to 10.0.0.4:/tmp/kubekey/helm Done
Push /root/kubekey/v1.20.4/amd64/cni-plugins-linux-amd64-v0.8.6.tgz to 10.0.0.2:/tmp/kubekey/cni-plugins-linux-amd64-v0.8.6.tgz Done
Push /root/kubekey/v1.20.4/amd64/helm to 10.0.0.7:/tmp/kubekey/helm Done
Push /root/kubekey/v1.20.4/amd64/helm to 10.0.0.8:/tmp/kubekey/helm Done
Push /root/kubekey/v1.20.4/amd64/cni-plugins-linux-amd64-v0.8.6.tgz to 10.0.0.4:/tmp/kubekey/cni-plugins-linux-amd64-v0.8.6.tgz Done
Push /root/kubekey/v1.20.4/amd64/kubectl to 10.0.0.5:/tmp/kubekey/kubectl Done
Push /root/kubekey/v1.20.4/amd64/cni-plugins-linux-amd64-v0.8.6.tgz to 10.0.0.7:/tmp/kubekey/cni-plugins-linux-amd64-v0.8.6.tgz Done
Push /root/kubekey/v1.20.4/amd64/cni-plugins-linux-amd64-v0.8.6.tgz to 10.0.0.8:/tmp/kubekey/cni-plugins-linux-amd64-v0.8.6.tgz Done
Push /root/kubekey/v1.20.4/amd64/helm to 10.0.0.5:/tmp/kubekey/helm Done
Push /root/kubekey/v1.20.4/amd64/cni-plugins-linux-amd64-v0.8.6.tgz to 10.0.0.5:/tmp/kubekey/cni-plugins-linux-amd64-v0.8.6.tgz Done
INFO[12:49:40 CST] Initializing kubernetes cluster
[k8s-master1 10.0.0.6] MSG:
[reset] Reading configuration from the cluster…
[reset] FYI: You can look at this config file with ‘kubectl -n kube-system get cm kubeadm-config -o yaml’
W0524 12:54:42.673144 8862 reset.go:99] [reset] Unable to fetch the kubeadm-config ConfigMap from cluster: failed to get config map: Get “https://lb.kubesphere.local:6443/api/v1/namespaces/kube-system/configmaps/kubeadm-config?timeout=10s”: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
[preflight] Running pre-flight checks
W0524 12:54:42.673276 8862 removeetcdmember.go:79] [reset] No kubeadm config, using etcd pod spec to get data directory
[reset] No etcd config found. Assuming external etcd
[reset] Please, manually reset etcd to prevent further issues
[reset] Stopping the kubelet service
[reset] Unmounting mounted directories in “/var/lib/kubelet”
[reset] Deleting contents of config directories: [/etc/kubernetes/manifests /etc/kubernetes/pki]
[reset] Deleting files: [/etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/bootstrap-kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf]
[reset] Deleting contents of stateful directories: [/var/lib/kubelet /var/lib/dockershim /var/run/kubernetes /var/lib/cni]
The reset process does not clean CNI configuration. To do so, you must remove /etc/cni/net.d
The reset process does not reset or clean up iptables rules or IPVS tables.
If you wish to reset iptables, you must do so manually by using the “iptables” command.
If your cluster was setup to utilize IPVS, run ipvsadm –clear (or similar)
to reset your system’s IPVS tables.
The reset process does not clean your kubeconfig files and you must remove them manually.
Please, check the contents of the $HOME/.kube/config file.
[k8s-master1 10.0.0.6] MSG:
[reset] Reading configuration from the cluster…
[reset] FYI: You can look at this config file with ‘kubectl -n kube-system get cm kubeadm-config -o yaml’
W0524 12:59:47.472015 11713 reset.go:99] [reset] Unable to fetch the kubeadm-config ConfigMap from cluster: failed to get config map: Get “https://lb.kubesphere.local:6443/api/v1/namespaces/kube-system/configmaps/kubeadm-config?timeout=10s”: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
[preflight] Running pre-flight checks
W0524 12:59:47.472111 11713 removeetcdmember.go:79] [reset] No kubeadm config, using etcd pod spec to get data directory
[reset] No etcd config found. Assuming external etcd
[reset] Please, manually reset etcd to prevent further issues
[reset] Stopping the kubelet service
[reset] Unmounting mounted directories in “/var/lib/kubelet”
[reset] Deleting contents of config directories: [/etc/kubernetes/manifests /etc/kubernetes/pki]
[reset] Deleting files: [/etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/bootstrap-kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf]
[reset] Deleting contents of stateful directories: [/var/lib/kubelet /var/lib/dockershim /var/run/kubernetes /var/lib/cni]
The reset process does not clean CNI configuration. To do so, you must remove /etc/cni/net.d
The reset process does not reset or clean up iptables rules or IPVS tables.
If you wish to reset iptables, you must do so manually by using the “iptables” command.
If your cluster was setup to utilize IPVS, run ipvsadm –clear (or similar)
to reset your system’s IPVS tables.
The reset process does not clean your kubeconfig files and you must remove them manually.
Please, check the contents of the $HOME/.kube/config file.
ERRO[13:04:11 CST] Failed to init kubernetes cluster: Failed to exec command: sudo env PATH=$PATH /bin/sh -c “/usr/local/bin/kubeadm init –config=/etc/kubernetes/kubeadm-config.yaml –ignore-preflight-errors=FileExisting-crictl”
W0524 12:59:49.113044 12180 utils.go:69] The recommended value for “clusterDNS” in “KubeletConfiguration” is: [10.233.0.10]; the provided value is: [169.254.25.10]
[init] Using Kubernetes version: v1.20.4
[preflight] Running pre-flight checks
[WARNING IsDockerSystemdCheck]: detected “cgroupfs” as the Docker cgroup driver. The recommended driver is “systemd”. Please follow the guide at https://kubernetes.io/docs/setup/cri/
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using ‘kubeadm config images pull’
[certs] Using certificateDir folder “/etc/kubernetes/pki”
[certs] Generating “ca” certificate and key
[certs] Generating “apiserver” certificate and key
[certs] apiserver serving cert is signed for DNS names [k8s-master1 k8s-master1.cluster.local k8s-master2 k8s-master2.cluster.local k8s-master3 k8s-master3.cluster.local k8s-node1 k8s-node1.cluster.local k8s-node2 k8s-node2.cluster.local k8s-node3 k8s-node3.cluster.local kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local lb.kubesphere.local localhost] and IPs [10.233.0.1 10.0.0.6 127.0.0.1 10.0.0.9 10.0.0.7 10.0.0.8 10.0.0.2 10.0.0.5 10.0.0.4]
[certs] Generating “apiserver-kubelet-client” certificate and key
[certs] Generating “front-proxy-ca” certificate and key
[certs] Generating “front-proxy-client” certificate and key
[certs] External etcd mode: Skipping etcd/ca certificate authority generation
[certs] External etcd mode: Skipping etcd/server certificate generation
[certs] External etcd mode: Skipping etcd/peer certificate generation
[certs] External etcd mode: Skipping etcd/healthcheck-client certificate generation
[certs] External etcd mode: Skipping apiserver-etcd-client certificate generation
[certs] Generating “sa” key and public key
[kubeconfig] Using kubeconfig folder “/etc/kubernetes”
[kubeconfig] Writing “admin.conf” kubeconfig file
[kubeconfig] Writing “kubelet.conf” kubeconfig file
[kubeconfig] Writing “controller-manager.conf” kubeconfig file
[kubeconfig] Writing “scheduler.conf” kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file “/var/lib/kubelet/kubeadm-flags.env”
[kubelet-start] Writing kubelet configuration to file “/var/lib/kubelet/config.yaml”
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder “/etc/kubernetes/manifests”
[control-plane] Creating static Pod manifest for “kube-apiserver”
[control-plane] Creating static Pod manifest for “kube-controller-manager”
[control-plane] Creating static Pod manifest for “kube-scheduler”
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory “/etc/kubernetes/manifests”. This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
error execution phase wait-control-plane: couldn’t initialize a Kubernetes cluster
To see the stack trace of this error execute with –v=5 or higher: Process exited with status 1 node=10.0.0.6
WARN[13:04:11 CST] Task failed …
WARN[13:04:11 CST] error: interrupted by error
Error: Failed to init kubernetes cluster: interrupted by error
Usage:
kk create cluster [flags]
Flags:
–download-cmd string The user defined command to download the necessary binary files. The first param ‘%s’ is output path, the second param ‘%s’, is the URL (default “curl -L -o %s %s”)
-f, –filename string Path to a configuration file
-h, –help help for cluster
–skip-pull-images Skip pre pull images
–with-kubernetes string Specify a supported version of kubernetes (default “v1.19.8”)
–with-kubesphere Deploy a specific version of kubesphere (default v3.1.0)
–with-local-storage Deploy a local PV provisioner
-y, –yes Skip pre-check of the installation
Global Flags:
–debug Print detailed information (default true)
–in-cluster Running inside the cluster
Failed to init kubernetes cluster: interrupted by error
