腾讯云CentOS 7.8 安装失败 不知道什么原因
[reset] Deleting files: [/etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/bootstrap-kubelet.conf /etc/kubernetes/controller-manager.conf /et c/kubernetes/scheduler.conf]
[reset] Deleting contents of stateful directories: [/var/lib/kubelet /var/lib/dockershim /var/run/kubernetes /var/lib/cni]
The reset process does not clean CNI configuration. To do so, you must remove /etc/cni/net.d
The reset process does not reset or clean up iptables rules or IPVS tables.
If you wish to reset iptables, you must do so manually by using the “iptables” command.
If your cluster was setup to utilize IPVS, run ipvsadm –clear (or similar)
to reset your system’s IPVS tables.
The reset process does not clean your kubeconfig files and you must remove them manually.
Please, check the contents of the $HOME/.kube/config file.
ERRO[19:19:58 CST] Failed to init kubernetes cluster: Failed to exec command: sudo -E /bin/sh -c “/usr/local/bin/kubeadm init –config=/etc/kubernetes/kubeadm-conf ig.yaml”
W1228 19:15:28.146005 13471 utils.go:26] The recommended value for “clusterDNS” in “KubeletConfiguration” is: [10.233.0.10]; the provided value is: [169.254.25.1 0]
W1228 19:15:28.146106 13471 configset.go:202] WARNING: kubeadm cannot validate component configs for API groups [kubelet.config.k8s.io kubeproxy.config.k8s.io]
[init] Using Kubernetes version: v1.18.6
[preflight] Running pre-flight checks
[WARNING Service-Docker]: docker service is not enabled, please run ‘systemctl enable docker.service’
[WARNING IsDockerSystemdCheck]: detected “cgroupfs” as the Docker cgroup driver. The recommended driver is “systemd”. Please follow the guide at https://ku bernetes.io/docs/setup/cri/
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 20.10.1. Latest validated version: 19.03
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using ‘kubeadm config images pull’
[kubelet-start] Writing kubelet environment file with flags to file “/var/lib/kubelet/kubeadm-flags.env”
[kubelet-start] Writing kubelet configuration to file “/var/lib/kubelet/config.yaml”
[kubelet-start] Starting the kubelet
[certs] Using certificateDir folder “/etc/kubernetes/pki”
[certs] Generating “ca” certificate and key
[certs] Generating “apiserver” certificate and key
[certs] apiserver serving cert is signed for DNS names [master1 kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local lb.kubesp here.local kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local localhost lb.kubesphere.local master1 master1.cluster.local ma ster2 master2.cluster.local master3 master3.cluster.local node1 node1.cluster.local node2 node2.cluster.local node3 node3.cluster.local] and IPs [10.233.0.1 172.16 .1.9 127.0.0.1 172.16.1.0 172.16.1.9 172.16.1.8 172.16.1.7 172.16.1.5 172.16.1.4 172.16.1.3 10.233.0.1]
[certs] Generating “apiserver-kubelet-client” certificate and key
[certs] Generating “front-proxy-ca” certificate and key
[certs] Generating “front-proxy-client” certificate and key
[certs] External etcd mode: Skipping etcd/ca certificate authority generation
[certs] External etcd mode: Skipping etcd/server certificate generation
[certs] External etcd mode: Skipping etcd/peer certificate generation
[certs] External etcd mode: Skipping etcd/healthcheck-client certificate generation
[certs] External etcd mode: Skipping apiserver-etcd-client certificate generation
[certs] Generating “sa” key and public key
[kubeconfig] Using kubeconfig folder “/etc/kubernetes”
[kubeconfig] Writing “admin.conf” kubeconfig file
[kubeconfig] Writing “kubelet.conf” kubeconfig file
[kubeconfig] Writing “controller-manager.conf” kubeconfig file
[kubeconfig] Writing “scheduler.conf” kubeconfig file
[control-plane] Using manifest folder “/etc/kubernetes/manifests”
[control-plane] Creating static Pod manifest for “kube-apiserver”
W1228 19:15:30.231227 13471 manifests.go:225] the default kube-apiserver authorization-mode is “Node,RBAC”; using “Node,RBAC”
[control-plane] Creating static Pod manifest for “kube-controller-manager”
W1228 19:15:30.237004 13471 manifests.go:225] the default kube-apiserver authorization-mode is “Node,RBAC”; using “Node,RBAC”
[control-plane] Creating static Pod manifest for “kube-scheduler”
W1228 19:15:30.237799 13471 manifests.go:225] the default kube-apiserver authorization-mode is “Node,RBAC”; using “Node,RBAC”
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory “/etc/kubernetes/manifests”. This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'error execution phase wait-control-plane: couldn’t initialize a Kubernetes cluster
To see the stack trace of this error execute with –v=5 or higher: Process exited with status 1 node=172.16.1.9
WARN[19:19:58 CST] Task failed …
WARN[19:19:58 CST] error: interrupted by error
Error: Failed to init kubernetes cluster: interrupted by error
Usage:
kk create cluster [flags]
Flags:
-f, –filename string Path to a configuration file
-h, –help help for cluster
–skip-pull-images Skip pre pull images
–with-kubernetes string Specify a supported version of kubernetes
–with-kubesphere Deploy a specific version of kubesphere (default v3.0.0)
-y, –yes Skip pre-check of the installation
Global Flags:
–debug Print detailed information (default true)
Failed to init kubernetes cluster: interrupted by error
[root@master1 ~]# ./kk create cluster -f config-sample.yaml
+———+——+——+———+———-+——-+——-+———–+——–+————+————-+——————+————–+
| name | sudo | curl | openssl | ebtables | socat | ipset | conntrack | docker | nfs client | ceph client | glusterfs client | time |
+———+——+——+———+———-+——-+——-+———–+——–+————+————-+——————+————–+
| node2 | y | y | y | y | y | y | y | y | | | | CST 19:22:47 |
| node1 | y | y | y | y | y | y | y | y | | | | CST 19:22:46 |
| master1 | y | y | y | y | y | y | y | y | | | | CST 19:22:46 |
| node3 | y | y | y | y | y | y | y | y | | | | CST 19:22:46 |
| master2 | y | y | y | y | y | y | y | y | | | | CST 19:22:47 |
| master3 | y | y | y | y | y | y | y | y | | | | CST 19:22:47 |
+———+——+——+———+———-+——-+——-+———–+——–+————+————-+——————+————–+
This is a simple check of your environment.
Before installation, you should ensure that your machines meet all requirements specified at
https://github.com/kubesphere/kubekey#requirements-and-recommendations
Continue this installation? [yes/no]: yes
INFO[19:22:49 CST] Downloading Installation Files
INFO[19:22:49 CST] Downloading kubeadm …
INFO[19:22:49 CST] Downloading kubelet …
INFO[19:22:49 CST] Downloading kubectl …
INFO[19:22:49 CST] Downloading helm …
INFO[19:22:49 CST] Downloading kubecni …
INFO[19:22:49 CST] Configurating operating system …
[master2 172.16.1.8] MSG:
net.ipv4.ip_forward = 1
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.default.accept_source_route = 0
kernel.core_uses_pid = 1
net.ipv4.tcp_syncookies = 1
kernel.msgmnb = 65536
kernel.msgmax = 65536
net.ipv4.conf.all.promote_secondaries = 1
net.ipv4.conf.default.promote_secondaries = 1
net.ipv6.neigh.default.gc_thresh3 = 4096
net.ipv4.neigh.default.gc_thresh3 = 4096
kernel.softlockup_panic = 1
kernel.sysrq = 1
net.ipv6.conf.all.disable_ipv6 = 0
net.ipv6.conf.default.disable_ipv6 = 0
net.ipv6.conf.lo.disable_ipv6 = 0
kernel.numa_balancing = 0
kernel.shmmax = 68719476736
kernel.printk = 5
net.bridge.bridge-nf-call-arptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_local_reserved_ports = 30000-32767
[node3 172.16.1.3] MSG:
net.ipv4.ip_forward = 1
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.default.accept_source_route = 0
kernel.core_uses_pid = 1
net.ipv4.tcp_syncookies = 1
kernel.msgmnb = 65536
kernel.msgmax = 65536
net.ipv4.conf.all.promote_secondaries = 1
net.ipv4.conf.default.promote_secondaries = 1
net.ipv6.neigh.default.gc_thresh3 = 4096
net.ipv4.neigh.default.gc_thresh3 = 4096
kernel.softlockup_panic = 1
net.ipv6.conf.all.disable_ipv6 = 0
net.ipv6.conf.default.disable_ipv6 = 0
net.ipv6.conf.lo.disable_ipv6 = 0
kernel.numa_balancing = 0
kernel.shmmax = 68719476736
kernel.printk = 5
kernel.sysrq = 1
net.bridge.bridge-nf-call-arptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_local_reserved_ports = 30000-32767
[node1 172.16.1.5] MSG:
net.ipv4.ip_forward = 1
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.default.accept_source_route = 0
kernel.core_uses_pid = 1
net.ipv4.tcp_syncookies = 1
kernel.msgmnb = 65536
kernel.msgmax = 65536
net.ipv4.conf.all.promote_secondaries = 1
net.ipv4.conf.default.promote_secondaries = 1
net.ipv6.neigh.default.gc_thresh3 = 4096
net.ipv4.neigh.default.gc_thresh3 = 4096
kernel.softlockup_panic = 1
net.ipv6.conf.all.disable_ipv6 = 0
net.ipv6.conf.default.disable_ipv6 = 0
net.ipv6.conf.lo.disable_ipv6 = 0
kernel.numa_balancing = 0
kernel.shmmax = 68719476736
kernel.printk = 5
kernel.sysrq = 1
net.bridge.bridge-nf-call-arptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_local_reserved_ports = 30000-32767
[node2 172.16.1.4] MSG:
net.ipv4.ip_forward = 1
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.default.accept_source_route = 0
kernel.core_uses_pid = 1
net.ipv4.tcp_syncookies = 1
kernel.msgmnb = 65536
kernel.msgmax = 65536
net.ipv4.conf.all.promote_secondaries = 1
net.ipv4.conf.default.promote_secondaries = 1
net.ipv6.neigh.default.gc_thresh3 = 4096
net.ipv4.neigh.default.gc_thresh3 = 4096
kernel.softlockup_panic = 1
net.ipv6.conf.all.disable_ipv6 = 0
net.ipv6.conf.default.disable_ipv6 = 0
net.ipv6.conf.lo.disable_ipv6 = 0
kernel.numa_balancing = 0
kernel.shmmax = 68719476736
kernel.printk = 5
kernel.sysrq = 1
net.bridge.bridge-nf-call-arptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_local_reserved_ports = 30000-32767
[master1 172.16.1.9] MSG:
net.ipv4.ip_forward = 1
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.default.accept_source_route = 0
kernel.core_uses_pid = 1
net.ipv4.tcp_syncookies = 1
kernel.msgmnb = 65536
kernel.msgmax = 65536
net.ipv4.conf.all.promote_secondaries = 1
net.ipv4.conf.default.promote_secondaries = 1
net.ipv6.neigh.default.gc_thresh3 = 4096
net.ipv4.neigh.default.gc_thresh3 = 4096
kernel.softlockup_panic = 1
net.ipv6.conf.all.disable_ipv6 = 0
net.ipv6.conf.default.disable_ipv6 = 0
net.ipv6.conf.lo.disable_ipv6 = 0
kernel.numa_balancing = 0
kernel.shmmax = 68719476736
kernel.printk = 5
kernel.sysrq = 1
net.bridge.bridge-nf-call-arptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_local_reserved_ports = 30000-32767
[master3 172.16.1.7] MSG:
net.ipv4.ip_forward = 1
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.default.accept_source_route = 0
kernel.core_uses_pid = 1
net.ipv4.tcp_syncookies = 1
kernel.msgmnb = 65536
kernel.msgmax = 65536
net.ipv4.conf.all.promote_secondaries = 1
net.ipv4.conf.default.promote_secondaries = 1
net.ipv6.neigh.default.gc_thresh3 = 4096
net.ipv4.neigh.default.gc_thresh3 = 4096
kernel.softlockup_panic = 1
net.ipv6.conf.all.disable_ipv6 = 0
net.ipv6.conf.default.disable_ipv6 = 0
net.ipv6.conf.lo.disable_ipv6 = 0
kernel.numa_balancing = 0
kernel.shmmax = 68719476736
kernel.printk = 5
kernel.sysrq = 1
net.bridge.bridge-nf-call-arptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_local_reserved_ports = 30000-32767
INFO[19:22:50 CST] Installing docker …
INFO[19:22:51 CST] Start to download images on all nodes
[node3] Downloading image: kubesphere/pause:3.2
[master2] Downloading image: kubesphere/etcd:v3.3.12
[node1] Downloading image: kubesphere/pause:3.2
[master1] Downloading image: kubesphere/etcd:v3.3.12
[node2] Downloading image: kubesphere/pause:3.2
[master3] Downloading image: kubesphere/etcd:v3.3.12
[master2] Downloading image: kubesphere/pause:3.2
[node1] Downloading image: kubesphere/kube-proxy:v1.18.6
[master1] Downloading image: kubesphere/pause:3.2
[master3] Downloading image: kubesphere/pause:3.2
[node3] Downloading image: kubesphere/kube-proxy:v1.18.6
[node2] Downloading image: kubesphere/kube-proxy:v1.18.6
[master2] Downloading image: kubesphere/kube-apiserver:v1.18.6
[master1] Downloading image: kubesphere/kube-apiserver:v1.18.6
[node1] Downloading image: coredns/coredns:1.6.9
[master3] Downloading image: kubesphere/kube-apiserver:v1.18.6
[node1] Downloading image: kubesphere/k8s-dns-node-cache:1.15.12
[master3] Downloading image: kubesphere/kube-controller-manager:v1.18.6
[node1] Downloading image: calico/kube-controllers:v3.15.1
[node2] Downloading image: coredns/coredns:1.6.9
[node3] Downloading image: coredns/coredns:1.6.9
[master2] Downloading image: kubesphere/kube-controller-manager:v1.18.6
[master1] Downloading image: kubesphere/kube-controller-manager:v1.18.6
[master3] Downloading image: kubesphere/kube-scheduler:v1.18.6
[node1] Downloading image: calico/cni:v3.15.1
[master2] Downloading image: kubesphere/kube-scheduler:v1.18.6
[node3] Downloading image: kubesphere/k8s-dns-node-cache:1.15.12
[node2] Downloading image: kubesphere/k8s-dns-node-cache:1.15.12
[master1] Downloading image: kubesphere/kube-scheduler:v1.18.6
[master3] Downloading image: kubesphere/kube-proxy:v1.18.6
[node1] Downloading image: calico/node:v3.15.1
[node3] Downloading image: calico/kube-controllers:v3.15.1
[master1] Downloading image: kubesphere/kube-proxy:v1.18.6
[master2] Downloading image: kubesphere/kube-proxy:v1.18.6
[master3] Downloading image: coredns/coredns:1.6.9
[node2] Downloading image: calico/kube-controllers:v3.15.1
[node1] Downloading image: calico/pod2daemon-flexvol:v3.15.1
[master1] Downloading image: coredns/coredns:1.6.9
[master3] Downloading image: kubesphere/k8s-dns-node-cache:1.15.12
[master1] Downloading image: kubesphere/k8s-dns-node-cache:1.15.12
[master3] Downloading image: calico/kube-controllers:v3.15.1
[master3] Downloading image: calico/cni:v3.15.1
[node3] Downloading image: calico/cni:v3.15.1
[master1] Downloading image: calico/kube-controllers:v3.15.1
[master2] Downloading image: coredns/coredns:1.6.9
[node2] Downloading image: calico/cni:v3.15.1
[master3] Downloading image: calico/node:v3.15.1
[master2] Downloading image: kubesphere/k8s-dns-node-cache:1.15.12
[node2] Downloading image: calico/node:v3.15.1
[master3] Downloading image: calico/pod2daemon-flexvol:v3.15.1
[master1] Downloading image: calico/cni:v3.15.1
[node3] Downloading image: calico/node:v3.15.1
[master2] Downloading image: calico/kube-controllers:v3.15.1
[node2] Downloading image: calico/pod2daemon-flexvol:v3.15.1
[node3] Downloading image: calico/pod2daemon-flexvol:v3.15.1
[master1] Downloading image: calico/node:v3.15.1
[master2] Downloading image: calico/cni:v3.15.1
[master1] Downloading image: calico/pod2daemon-flexvol:v3.15.1
[master2] Downloading image: calico/node:v3.15.1
[master2] Downloading image: calico/pod2daemon-flexvol:v3.15.1
INFO[19:24:35 CST] Generating etcd certs
INFO[19:24:37 CST] Synchronizing etcd certs
INFO[19:24:37 CST] Creating etcd service
INFO[19:24:48 CST] Starting etcd cluster
[master1 172.16.1.9] MSG:
Configuration file already exists
Waiting for etcd to start
[master2 172.16.1.8] MSG:
Configuration file already exists
[master3 172.16.1.7] MSG:
Configuration file already exists
INFO[19:24:55 CST] Refreshing etcd configuration
INFO[19:24:56 CST] Backup etcd data regularly
INFO[19:24:56 CST] Get cluster status
[master1 172.16.1.9] MSG:
Cluster already exists.
[master1 172.16.1.9] MSG:
v1.18.6
WARN[19:39:17 CST] Task failed …
WARN[19:39:17 CST] error: Failed to upload kubeadm certs: Failed to exec command: sudo -E /bin/sh -c “/usr/local/bin/kubeadm init phase upload-certs –upload-certs”
W1228 19:36:57.074360 23527 version.go:102] could not fetch a Kubernetes version from the internet: unable to get URL “https://dl.k8s.io/release/stable-1.txt”: Get https://storage.googleapis.com/kubernetes-release/release/stable-1.txt: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
W1228 19:36:57.074407 23527 version.go:103] falling back to the local client version: v1.18.6
W1228 19:36:57.074532 23527 configset.go:202] WARNING: kubeadm cannot validate component configs for API groups [kubelet.config.k8s.io kubeproxy.config.k8s.io]
[upload-certs] Storing the certificates in Secret “kubeadm-certs” in the “kube-system” Namespace
error execution phase upload-certs: error uploading certs: error creating token: timed out waiting for the condition
To see the stack trace of this error execute with –v=5 or higher: Process exited with status 1
Error: Failed to get cluster status: Failed to upload kubeadm certs: Failed to exec command: sudo -E /bin/sh -c “/usr/local/bin/kubeadm init phase upload-certs –upload-certs”
W1228 19:36:57.074360 23527 version.go:102] could not fetch a Kubernetes version from the internet: unable to get URL “https://dl.k8s.io/release/stable-1.txt”: Get https://storage.googleapis.com/kubernetes-release/release/stable-1.txt: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
W1228 19:36:57.074407 23527 version.go:103] falling back to the local client version: v1.18.6
W1228 19:36:57.074532 23527 configset.go:202] WARNING: kubeadm cannot validate component configs for API groups [kubelet.config.k8s.io kubeproxy.config.k8s.io]
[upload-certs] Storing the certificates in Secret “kubeadm-certs” in the “kube-system” Namespace
error execution phase upload-certs: error uploading certs: error creating token: timed out waiting for the condition
To see the stack trace of this error execute with –v=5 or higher: Process exited with status 1
Usage:
kk create cluster [flags]
Flags:
-f, –filename string Path to a configuration file
-h, –help help for cluster
–skip-pull-images Skip pre pull images
–with-kubernetes string Specify a supported version of kubernetes
–with-kubesphere Deploy a specific version of kubesphere (default v3.0.0)
-y, –yes Skip pre-check of the installation
Global Flags:
–debug Print detailed information (default true)
Failed to get cluster status: Failed to upload kubeadm certs: Failed to exec command: sudo -E /bin/sh -c “/usr/local/bin/kubeadm init phase upload-certs –upload-certs”
W1228 19:36:57.074360 23527 version.go:102] could not fetch a Kubernetes version from the internet: unable to get URL “https://dl.k8s.io/release/stable-1.txt”: Get https://storage.googleapis.com/kubernetes-release/release/stable-1.txt: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
W1228 19:36:57.074407 23527 version.go:103] falling back to the local client version: v1.18.6
W1228 19:36:57.074532 23527 configset.go:202] WARNING: kubeadm cannot validate component configs for API groups [kubelet.config.k8s.io kubeproxy.config.k8s.io]
[upload-certs] Storing the certificates in Secret “kubeadm-certs” in the “kube-system” Namespace
error execution phase upload-certs: error uploading certs: error creating token: timed out waiting for the condition
To see the stack trace of this error execute with –v=5 or higher: Process exited with status 1
