Linux上的多合一安装失败

itlidaye

./kk create cluster –with-kubernetes v1.18.6 –with-kubesphere v3.0.0

Configuration file already exists
Waiting for etcd to start
INFO[16:10:46 CST] Refreshing etcd configuration
INFO[16:10:47 CST] Get cluster status
[shiku183 192.168.0.120] MSG:
Cluster will be created.
INFO[16:10:48 CST] Installing kube binaries
Push /opt/kubekey/v1.18.6/amd64/kubeadm to 192.168.0.120:/tmp/kubekey/kubeadm Done
Push /opt/kubekey/v1.18.6/amd64/kubelet to 192.168.0.120:/tmp/kubekey/kubelet Done
Push /opt/kubekey/v1.18.6/amd64/kubectl to 192.168.0.120:/tmp/kubekey/kubectl Done
Push /opt/kubekey/v1.18.6/amd64/helm to 192.168.0.120:/tmp/kubekey/helm Done
Push /opt/kubekey/v1.18.6/amd64/cni-plugins-linux-amd64-v0.8.6.tgz to 192.168.0.120:/tmp/kubekey/cni-plugins-linux-amd64-v0.8.6.tgz Done
ERRO[16:11:30 CST] Failed to enable kubelet service: Failed to exec command: sudo -E /bin/sh -c “systemctl enable kubelet && ln -snf /usr/local/bin/kubelet /usr/bin/kubelet”
Failed to execute operation: File exists: Process exited with status 1 node=192.168.0.120
WARN[16:11:30 CST] Task failed …
WARN[16:11:30 CST] error: interrupted by error
Error: Failed to install kube binaries: interrupted by error
Usage:
kk create cluster [flags]

Flags:
-f, –filename string Path to a configuration file
-h, –help help for cluster
–skip-pull-images Skip pre pull images
–with-kubernetes string Specify a supported version of kubernetes
–with-kubesphere Deploy a specific version of kubesphere (default v3.0.0)
-y, –yes Skip pre-check of the installation

Global Flags:
–debug Print detailed information (default true)

Failed to install kube binaries: interrupted by error

ruiyaoOps

可以查看下kubelet的二进制文件和kubelet的服务日志
可以参考：https://asciinema.org/a/364816
还有就是检查二进制文件到底发没发到192.168.0.120节点上

itlidaye

kubelet的服务日志怎么查看

yuswift

itlidaye
journalctl | grep kubelet

itlidaye

cni-plugins-linux-amd64-v0.8.6.tgz 文件是存在的

ruiyaoOps

itlidaye kubelet的日志有吗

itlidaye

failed to load Kubelet config file /var/lib/kubelet/config.yaml, error failed to readkubelet config file “/var/lib/kubelet/config.yaml”, error: open /var/lib/kubelet/config.yaml: no such file or directory
9月 14 18:08:02 shiku183 systemd[1]: kubelet.service: main process exited, code=exited, status=255/n/a
9月 14 18:08:03 shiku183 systemd[1]: Unit kubelet.service entered failed state.
9月 14 18:08:03 shiku183 systemd[1]: kubelet.service failed.
9月 14 18:08:13 shiku183 systemd[1]: kubelet.service holdoff time over, scheduling restart.
9月 14 18:08:13 shiku183 systemd[1]: Stopped kubelet: The Kubernetes Node Agent.
9月 14 18:08:13 shiku183 systemd[1]: Started kubelet: The Kubernetes Node Agent.
9月 14 18:08:13 shiku183 kubelet[26945]: F0914 18:08:13.291755 26945 server.go:198] failed to load Kubelet config file /var/lib/kubelet/config.yaml, error failed to readkubelet config file “/var/lib/kubelet/config.yaml”, error: open /var/lib/kubelet/config.yaml: no such file or directory
9月 14 18:08:13 shiku183 systemd[1]: kubelet.service: main process exited, code=exited, status=255/n/a
9月 14 18:08:13 shiku183 systemd[1]: Unit kubelet.service entered failed state.
9月 14 18:08:13 shiku183 systemd[1]: kubelet.service failed.
9月 14 18:08:23 shiku183 systemd[1]: kubelet.service holdoff time over, scheduling restart.
9月 14 18:08:23 shiku183 systemd[1]: Stopped kubelet: The Kubernetes Node Agent.
9月 14 18:08:23 shiku183 systemd[1]: Started kubelet: The Kubernetes Node Agent.
9月 14 18:08:23 shiku183 kubelet[27009]: F0914 18:08:23.531154 27009 server.go:198] failed to load Kubelet config file /var/lib/kubelet/config.yaml, error failed to readkubelet config file “/var/lib/kubelet/config.yaml”, error: open /var/lib/kubelet/config.yaml: no such file or directory
9月 14 18:08:23 shiku183 systemd[1]: kubelet.service: main process exited, code=exited, status=255/n/a
9月 14 18:08:23 shiku183 systemd[1]: Unit kubelet.service entered failed state.
9月 14 18:08:23 shiku183 systemd[1]: kubelet.service failed.
9月 14 18:08:33 shiku183 systemd[1]: kubelet.service holdoff time over, scheduling restart.
9月 14 18:08:33 shiku183 systemd[1]: Stopped kubelet: The Kubernetes Node Agent.
9月 14 18:08:33 shiku183 systemd[1]: Started kubelet: The Kubernetes Node Agent.
9月 14 18:08:33 shiku183 kubelet[27065]: F0914 18:08:33.794065 27065 server.go:198] failed to load Kubelet config file /var/lib/kubelet/config.yaml, error failed to readkubelet config file “/var/lib/kubelet/config.yaml”, error: open /var/lib/kubelet/config.yaml: no such file or directory

ruiyaoOps

itlidaye 你是不是之前用kubekey安装过192.168.0.120这个节点?

ruiyaoOps

itlidaye 你这情况我之前遇到过，我是因为用kubekey安装后自己手动清理节点没清理干净
你可以参考：https://github.com/kubesphere/kubekey#delete-nodes
把你的节点清理一下，然后再安装试试

itlidaye

之前应该是没有安装过，只有这次安装 kubesphere 一直失败，之前只装过docker

itlidaye

./kk delete node
[root@shiku183 opt]# ./kk delete node
[root@shiku183 opt]# ./kk delete node
[root@shiku183 opt]# ./kk delete cluster
Are you sure to delete this cluster? [yes/no]: yes
INFO[17:32:58 CST] Resetting kubernetes cluster …
[shiku183 192.168.0.120] MSG:
[preflight] Running pre-flight checks
W1012 17:32:59.549709 22773 removeetcdmember.go:79] [reset] No kubeadm config, using etcd pod spec to get data directory
[reset] No etcd config found. Assuming external etcd
[reset] Please, manually reset etcd to prevent further issues
[reset] Stopping the kubelet service
[reset] Unmounting mounted directories in “/var/lib/kubelet”
[reset] Deleting contents of config directories: [/etc/kubernetes/manifests /etc/kubernetes/pki]
[reset] Deleting files: [/etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/bootstrap-kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf]
[reset] Deleting contents of stateful directories: [/var/lib/kubelet /var/lib/dockershim /var/run/kubernetes /var/lib/cni]

The reset process does not clean CNI configuration. To do so, you must remove /etc/cni/net.d

The reset process does not reset or clean up iptables rules or IPVS tables.
If you wish to reset iptables, you must do so manually by using the “iptables” command.

If your cluster was setup to utilize IPVS, run ipvsadm –clear (or similar)
to reset your system’s IPVS tables.

The reset process does not clean your kubeconfig files and you must remove them manually.
Please, check the contents of the $HOME/.kube/config file.
iptables -F && iptables -X && iptables -F -t nat && iptables -X -t nat && ip link del kube-ipvs0 && ip link del nodelocaldns
INFO[17:33:14 CST] Successful.

itlidaye

find / -name kubelet

没有找到结果了

执行安装

./kk create cluster –with-kubernetes v1.17.9 –with-kubesphere v3.0.0
+———-+——+——+———+———-+——-+——-+———–+——–+————+————-+——————+————–+
| name | sudo | curl | openssl | ebtables | socat | ipset | conntrack | docker | nfs client | ceph client | glusterfs client | time |
+———-+——+——+———+———-+——-+——-+———–+——–+————+————-+——————+————–+
| shiku183 | y | y | y | y | y | y | y | y | | | | CST 17:40:35 |
+———-+——+——+———+———-+——-+——-+———–+——–+————+————-+——————+————–+

This is a simple check of your environment.
Before installation, you should ensure that your machines meet all requirements specified at
https://github.com/kubesphere/kubekey#requirements-and-recommendations

Continue this installation? [yes/no]: yes
INFO[17:40:46 CST] Downloading Installation Files
INFO[17:40:46 CST] Downloading kubeadm …
INFO[17:41:23 CST] Downloading kubelet …

itlidaye

结果还是一样

Continue this installation? [yes/no]: yes
INFO[17:40:46 CST] Downloading Installation Files
INFO[17:40:46 CST] Downloading kubeadm …
INFO[17:41:23 CST] Downloading kubelet …
INFO[17:43:14 CST] Downloading kubectl …
INFO[17:43:53 CST] Downloading kubecni …
INFO[17:44:27 CST] Downloading helm …
INFO[17:45:20 CST] Configurating operating system …
[shiku183 192.168.0.120] MSG:
net.ipv4.tcp_retries1 = 0
net.ipv4.tcp_retries2 = 0
net.ipv4.tcp_tw_recycle = 0
net.ipv4.tcp_timestamps = 0
fs.file-max = 360000
net.ipv4.ip_local_port_range = 1024 65000
net.ipv4.tcp_keepalive_time = 60
net.ipv4.tcp_keepalive_probes = 2
net.ipv4.tcp_keepalive_intvl = 3
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
vm.max_map_count = 655360
net.ipv4.ip_forward = 1
net.bridge.bridge-nf-call-arptables = 1
net.ipv4.ip_local_reserved_ports = 30000-32767
vm.swappiness = 0
INFO[17:45:46 CST] Installing docker …
INFO[17:45:47 CST] Start to download images on all nodes
[shiku183] Downloading image: kubesphere/etcd:v3.3.12
[shiku183] Downloading image: kubesphere/pause:3.1
[shiku183] Downloading image: kubesphere/kube-apiserver:v1.17.9
[shiku183] Downloading image: kubesphere/kube-controller-manager:v1.17.9
[shiku183] Downloading image: kubesphere/kube-scheduler:v1.17.9
[shiku183] Downloading image: kubesphere/kube-proxy:v1.17.9
[shiku183] Downloading image: coredns/coredns:1.6.9
[shiku183] Downloading image: kubesphere/k8s-dns-node-cache:1.15.12
[shiku183] Downloading image: calico/kube-controllers:v3.15.1
[shiku183] Downloading image: calico/cni:v3.15.1
[shiku183] Downloading image: calico/node:v3.15.1
[shiku183] Downloading image: calico/pod2daemon-flexvol:v3.15.1
INFO[17:48:44 CST] Generating etcd certs
INFO[17:48:50 CST] Synchronizing etcd certs
INFO[17:48:50 CST] Creating etcd service
INFO[17:48:55 CST] Starting etcd cluster
[shiku183 192.168.0.120] MSG:
Configuration file will be created
INFO[17:48:56 CST] Refreshing etcd configuration
INFO[17:49:01 CST] Get cluster status
[shiku183 192.168.0.120] MSG:
Cluster will be created.
INFO[17:49:02 CST] Installing kube binaries
Push /opt/kubekey/v1.17.9/amd64/kubeadm to 192.168.0.120:/tmp/kubekey/kubeadm Done
Push /opt/kubekey/v1.17.9/amd64/kubelet to 192.168.0.120:/tmp/kubekey/kubelet Done
Push /opt/kubekey/v1.17.9/amd64/kubectl to 192.168.0.120:/tmp/kubekey/kubectl Done
Push /opt/kubekey/v1.17.9/amd64/helm to 192.168.0.120:/tmp/kubekey/helm Done
Push /opt/kubekey/v1.17.9/amd64/cni-plugins-linux-amd64-v0.8.6.tgz to 192.168.0.120:/tmp/kubekey/cni-plugins-linux-amd64-v0.8.6.tgz Done
ERRO[17:49:42 CST] Failed to enable kubelet service: Failed to exec command: sudo -E /bin/sh -c “systemctl enable kubelet && ln -snf /usr/local/bin/kubelet /usr/bin/kubelet”
Failed to execute operation: File exists: Process exited with status 1 node=192.168.0.120
WARN[17:49:42 CST] Task failed …
WARN[17:49:42 CST] error: interrupted by error
Error: Failed to install kube binaries: interrupted by error
Usage:
kk create cluster [flags]

Global Flags:
–debug Print detailed information (default true)

Failed to install kube binaries: interrupted by error

ruiyaoOps

itlidaye 能让我远程到你的机器上瞅瞅吗

itlidaye

Failed to install kube binaries: interrupted by error
[root@shiku183 opt]# sudo -E /bin/sh -c “systemctl enable kubelet && ln -snf /usr/local/bin/kubelet /usr/bin/kubelet”
Failed to execute operation: File exists
[root@shiku183 opt]#

ruiyaoOps

itlidaye Failed to execute operation: File exists 这个报错基本可以认为是你的环境没有清理干净

ruiyaoOps

itlidaye

Ls569689934

你好，我也遇到这个歌问题，请问解决了吗

Forest-L

Ls569689934 可以重新建一个，报错信息贴下。