创建部署问题时，请参考下面模板，你提供的信息越多，越容易及时获得解答。如果未按模板创建问题，管理员有权关闭问题。
确保帖子格式清晰易读，用 markdown code block 语法格式化代码块。
你只花一分钟创建的问题，不能指望别人花上半个小时给你解答。

操作系统信息
PVE虚拟机，Ubuntu2204，2C/4G

Kubernetes版本信息
将 kubectl version 命令执行结果贴在下方

Client Version: v1.28.12

Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3

The connection to the server localhost:8080 was refused - did you specify the right host or port?

容器运行时
将 docker version / crictl version / nerdctl version 结果贴在下方

Version: 0.1.0

RuntimeName: containerd

RuntimeVersion: v1.7.13

RuntimeApiVersion: v1

KubeSphere版本信息
例如：v2.1.1/v3.0.0。离线安装还是在线安装。在已有K8s上安装还是使用kk安装。

kk version: &version.Info{Major:“3”, Minor:“1”, GitVersion:“v3.1.7”, GitCommit:“da475c670813fc8a4dd3b1312aaa36e96ff01a1f”, GitTreeState:“clean”, BuildDate:“2024-10-30T09:41:20Z”, GoVersion:“go1.19.2”, Compiler:“gc”, Platform:“linux/amd64”}

使用kk在线安装

问题是什么
报错日志是什么，最好有截图。

07:37:34 UTC [GreetingsModule] Greetings

07:37:34 UTC message: [sealos-node2]

Greetings, KubeKey!

07:37:34 UTC message: [sealos-master]

Greetings, KubeKey!

07:37:35 UTC message: [sealos-node1]

Greetings, KubeKey!

07:37:35 UTC success: [sealos-node2]

07:37:35 UTC success: [sealos-master]

07:37:35 UTC success: [sealos-node1]

07:37:35 UTC [NodePreCheckModule] A pre-check on nodes

07:37:35 UTC success: [sealos-master]

07:37:35 UTC success: [sealos-node1]

07:37:35 UTC success: [sealos-node2]

07:37:35 UTC [ConfirmModule] Display confirmation form

+—————+——+——+———+———-+——-+——-+———+———–+——–+——–+————+————+————-+——————+————–+

| name | sudo | curl | openssl | ebtables | socat | ipset | ipvsadm | conntrack | chrony | docker | containerd | nfs client | ceph client | glusterfs client | time |

+—————+——+——+———+———-+——-+——-+———+———–+——–+——–+————+————+————-+——————+————–+

| sealos-master | y | y | y | y | y | y | | y | | | v1.7.13 | | | | UTC 07:37:35 |

| sealos-node1 | y | y | y | y | y | y | | y | | | v1.7.13 | | | | UTC 07:37:35 |

| sealos-node2 | y | y | y | y | y | y | | y | | | v1.7.13 | | | | UTC 07:37:35 |

+—————+——+——+———+———-+——-+——-+———+———–+——–+——–+————+————+————-+——————+————–+

This is a simple check of your environment.

Before installation, ensure that your machines meet all requirements specified at

https://github.com/kubesphere/kubekey#requirements-and-recommendations

Install k8s with specify version: v1.28.12

Continue this installation? [yes/no]: yes

07:37:39 UTC success: [LocalHost]

07:37:39 UTC [NodeBinariesModule] Download installation binaries

07:37:39 UTC message: [localhost]

downloading amd64 kubeadm v1.28.12 …

07:37:39 UTC message: [localhost]

kubeadm exists

07:37:39 UTC message: [localhost]

downloading amd64 kubelet v1.28.12 …

07:37:39 UTC message: [localhost]

kubelet exists

07:37:39 UTC message: [localhost]

downloading amd64 kubectl v1.28.12 …

07:37:39 UTC message: [localhost]

kubectl exists

07:37:39 UTC message: [localhost]

downloading amd64 helm v3.14.3 …

07:37:39 UTC message: [localhost]

helm exists

07:37:39 UTC message: [localhost]

downloading amd64 kubecni v1.2.0 …

07:37:39 UTC message: [localhost]

kubecni exists

07:37:39 UTC message: [localhost]

downloading amd64 crictl v1.29.0 …

07:37:39 UTC message: [localhost]

crictl exists

07:37:39 UTC message: [localhost]

downloading amd64 etcd v3.5.13 …

07:37:39 UTC message: [localhost]

etcd exists

07:37:39 UTC message: [localhost]

downloading amd64 containerd 1.7.13 …

07:37:39 UTC message: [localhost]

containerd exists

07:37:39 UTC message: [localhost]

downloading amd64 runc v1.1.12 …

07:37:39 UTC message: [localhost]

runc exists

07:37:39 UTC message: [localhost]

downloading amd64 calicoctl v3.27.4 …

07:37:40 UTC message: [localhost]

calicoctl exists

07:37:40 UTC success: [LocalHost]

07:37:40 UTC [ConfigureOSModule] Get OS release

07:37:40 UTC success: [sealos-master]

07:37:40 UTC success: [sealos-node1]

07:37:40 UTC success: [sealos-node2]

07:37:40 UTC [ConfigureOSModule] Prepare to init OS

07:37:40 UTC success: [sealos-node1]

07:37:40 UTC success: [sealos-node2]

07:37:40 UTC success: [sealos-master]

07:37:40 UTC [ConfigureOSModule] Generate init os script

07:37:40 UTC success: [sealos-master]

07:37:40 UTC success: [sealos-node1]

07:37:40 UTC success: [sealos-node2]

07:37:40 UTC [ConfigureOSModule] Exec init os script

07:37:41 UTC stdout: [sealos-master]

swapoff: Not superuser.

modprobe: FATAL: Module ip_vs not found in directory /lib/modules/5.19.7-2-pve

modprobe: FATAL: Module ip_vs_rr not found in directory /lib/modules/5.19.7-2-pve

modprobe: FATAL: Module ip_vs_wrr not found in directory /lib/modules/5.19.7-2-pve

modprobe: FATAL: Module ip_vs_sh not found in directory /lib/modules/5.19.7-2-pve

modprobe: FATAL: Module nf_conntrack not found in directory /lib/modules/5.19.7-2-pve

net.ipv4.conf.default.rp_filter = 0

net.ipv4.conf.all.rp_filter = 0

net.ipv4.ip_forward = 1

net.bridge.bridge-nf-call-arptables = 1

net.bridge.bridge-nf-call-ip6tables = 1

net.bridge.bridge-nf-call-iptables = 1

net.ipv4.ip_local_reserved_ports = 30000-32767

sysctl: cannot stat /proc/sys/net/core/netdev_max_backlog: No such file or directory

sysctl: cannot stat /proc/sys/net/core/rmem_max: No such file or directory

sysctl: cannot stat /proc/sys/net/core/wmem_max: No such file or directory

sysctl: cannot stat /proc/sys/net/core/somaxconn: No such file or directory

net.ipv4.tcp_max_syn_backlog = 1048576

sysctl: cannot stat /proc/sys/net/ipv4/neigh/default/gc_thresh1: No such file or directory

sysctl: cannot stat /proc/sys/net/ipv4/neigh/default/gc_thresh2: No such file or directory

sysctl: cannot stat /proc/sys/net/ipv4/neigh/default/gc_thresh3: No such file or directory

net.ipv4.tcp_retries2 = 15

net.ipv4.tcp_max_tw_buckets = 1048576

sysctl: cannot stat /proc/sys/net/ipv4/tcp_max_orphans: No such file or directory

net.ipv4.tcp_keepalive_time = 600

net.ipv4.tcp_keepalive_intvl = 30

net.ipv4.tcp_keepalive_probes = 10

net.ipv4.udp_rmem_min = 131072

net.ipv4.udp_wmem_min = 131072

net.ipv4.conf.all.arp_accept = 1

net.ipv4.conf.default.arp_accept = 1

net.ipv4.conf.all.arp_ignore = 1

net.ipv4.conf.default.arp_ignore = 1

sysctl: setting key “vm.max_map_count”, ignoring: Read-only file system

sysctl: setting key “vm.swappiness”, ignoring: Read-only file system

sysctl: setting key “vm.overcommit_memory”, ignoring: Read-only file system

sysctl: setting key “fs.inotify.max_user_instances”, ignoring: Read-only file system

sysctl: setting key “fs.inotify.max_user_watches”, ignoring: Read-only file system

sysctl: setting key “fs.pipe-max-size”, ignoring: Read-only file system

sysctl: setting key “fs.aio-max-nr”, ignoring: Read-only file system

sysctl: setting key “kernel.pid_max”, ignoring: Read-only file system

sysctl: setting key “kernel.watchdog_thresh”, ignoring: Read-only file system

sysctl: setting key “kernel.hung_task_timeout_secs”, ignoring: Read-only file system

net.ipv6.conf.all.disable_ipv6 = 0

net.ipv6.conf.default.disable_ipv6 = 0

net.ipv6.conf.lo.disable_ipv6 = 0

net.ipv6.conf.all.forwarding = 1

/usr/local/bin/kube-scripts/initOS.sh: line 191: /proc/sys/vm/drop_caches: Read-only file system

07:37:41 UTC stdout: [sealos-node2]

swapoff: Not superuser.

modprobe: FATAL: Module ip_vs not found in directory /lib/modules/5.19.7-2-pve

modprobe: FATAL: Module ip_vs_rr not found in directory /lib/modules/5.19.7-2-pve

modprobe: FATAL: Module ip_vs_wrr not found in directory /lib/modules/5.19.7-2-pve

modprobe: FATAL: Module ip_vs_sh not found in directory /lib/modules/5.19.7-2-pve

modprobe: FATAL: Module nf_conntrack not found in directory /lib/modules/5.19.7-2-pve

net.ipv4.conf.default.rp_filter = 0

net.ipv4.conf.all.rp_filter = 0

net.ipv4.ip_forward = 1

net.bridge.bridge-nf-call-arptables = 1

net.bridge.bridge-nf-call-ip6tables = 1

net.bridge.bridge-nf-call-iptables = 1

net.ipv4.ip_local_reserved_ports = 30000-32767

sysctl: cannot stat /proc/sys/net/core/netdev_max_backlog: No such file or directory

sysctl: cannot stat /proc/sys/net/core/rmem_max: No such file or directory

sysctl: cannot stat /proc/sys/net/core/wmem_max: No such file or directory

sysctl: cannot stat /proc/sys/net/core/somaxconn: No such file or directory

net.ipv4.tcp_max_syn_backlog = 1048576

sysctl: cannot stat /proc/sys/net/ipv4/neigh/default/gc_thresh1: No such file or directory

sysctl: cannot stat /proc/sys/net/ipv4/neigh/default/gc_thresh2: No such file or directory

sysctl: cannot stat /proc/sys/net/ipv4/neigh/default/gc_thresh3: No such file or directory

net.ipv4.tcp_retries2 = 15

net.ipv4.tcp_max_tw_buckets = 1048576

sysctl: cannot stat /proc/sys/net/ipv4/tcp_max_orphans: No such file or directory

net.ipv4.tcp_keepalive_time = 600

net.ipv4.tcp_keepalive_intvl = 30

net.ipv4.tcp_keepalive_probes = 10

net.ipv4.udp_rmem_min = 131072

net.ipv4.udp_wmem_min = 131072

net.ipv4.conf.all.arp_accept = 1

net.ipv4.conf.default.arp_accept = 1

net.ipv4.conf.all.arp_ignore = 1

net.ipv4.conf.default.arp_ignore = 1

sysctl: setting key “vm.max_map_count”, ignoring: Read-only file system

sysctl: setting key “vm.swappiness”, ignoring: Read-only file system

sysctl: setting key “vm.overcommit_memory”, ignoring: Read-only file system

sysctl: setting key “fs.inotify.max_user_instances”, ignoring: Read-only file system

sysctl: setting key “fs.inotify.max_user_watches”, ignoring: Read-only file system

sysctl: setting key “fs.pipe-max-size”, ignoring: Read-only file system

sysctl: setting key “fs.aio-max-nr”, ignoring: Read-only file system

sysctl: setting key “kernel.pid_max”, ignoring: Read-only file system

sysctl: setting key “kernel.watchdog_thresh”, ignoring: Read-only file system

sysctl: setting key “kernel.hung_task_timeout_secs”, ignoring: Read-only file system

net.ipv6.conf.all.disable_ipv6 = 0

net.ipv6.conf.default.disable_ipv6 = 0

net.ipv6.conf.lo.disable_ipv6 = 0

net.ipv6.conf.all.forwarding = 1

/usr/local/bin/kube-scripts/initOS.sh: line 191: /proc/sys/vm/drop_caches: Read-only file system

07:37:41 UTC stdout: [sealos-node1]

swapoff: Not superuser.

modprobe: FATAL: Module ip_vs not found in directory /lib/modules/5.19.7-2-pve

modprobe: FATAL: Module ip_vs_rr not found in directory /lib/modules/5.19.7-2-pve

modprobe: FATAL: Module ip_vs_wrr not found in directory /lib/modules/5.19.7-2-pve

modprobe: FATAL: Module ip_vs_sh not found in directory /lib/modules/5.19.7-2-pve

modprobe: FATAL: Module nf_conntrack not found in directory /lib/modules/5.19.7-2-pve

net.ipv4.conf.default.rp_filter = 0

net.ipv4.conf.all.rp_filter = 0

net.ipv4.ip_forward = 1

net.bridge.bridge-nf-call-arptables = 1

net.bridge.bridge-nf-call-ip6tables = 1

net.bridge.bridge-nf-call-iptables = 1

net.ipv4.ip_local_reserved_ports = 30000-32767

sysctl: cannot stat /proc/sys/net/core/netdev_max_backlog: No such file or directory

sysctl: cannot stat /proc/sys/net/core/rmem_max: No such file or directory

sysctl: cannot stat /proc/sys/net/core/wmem_max: No such file or directory

sysctl: cannot stat /proc/sys/net/core/somaxconn: No such file or directory

net.ipv4.tcp_max_syn_backlog = 1048576

sysctl: cannot stat /proc/sys/net/ipv4/neigh/default/gc_thresh1: No such file or directory

sysctl: cannot stat /proc/sys/net/ipv4/neigh/default/gc_thresh2: No such file or directory

sysctl: cannot stat /proc/sys/net/ipv4/neigh/default/gc_thresh3: No such file or directory

net.ipv4.tcp_retries2 = 15

net.ipv4.tcp_max_tw_buckets = 1048576

sysctl: cannot stat /proc/sys/net/ipv4/tcp_max_orphans: No such file or directory

net.ipv4.tcp_keepalive_time = 600

net.ipv4.tcp_keepalive_intvl = 30

net.ipv4.tcp_keepalive_probes = 10

net.ipv4.udp_rmem_min = 131072

net.ipv4.udp_wmem_min = 131072

net.ipv4.conf.all.arp_accept = 1

net.ipv4.conf.default.arp_accept = 1

net.ipv4.conf.all.arp_ignore = 1

net.ipv4.conf.default.arp_ignore = 1

sysctl: setting key “vm.max_map_count”, ignoring: Read-only file system

sysctl: setting key “vm.swappiness”, ignoring: Read-only file system

sysctl: setting key “vm.overcommit_memory”, ignoring: Read-only file system

sysctl: setting key “fs.inotify.max_user_instances”, ignoring: Read-only file system

sysctl: setting key “fs.inotify.max_user_watches”, ignoring: Read-only file system

sysctl: setting key “fs.pipe-max-size”, ignoring: Read-only file system

sysctl: setting key “fs.aio-max-nr”, ignoring: Read-only file system

sysctl: setting key “kernel.pid_max”, ignoring: Read-only file system

sysctl: setting key “kernel.watchdog_thresh”, ignoring: Read-only file system

sysctl: setting key “kernel.hung_task_timeout_secs”, ignoring: Read-only file system

net.ipv6.conf.all.disable_ipv6 = 0

net.ipv6.conf.default.disable_ipv6 = 0

net.ipv6.conf.lo.disable_ipv6 = 0

net.ipv6.conf.all.forwarding = 1

/usr/local/bin/kube-scripts/initOS.sh: line 191: /proc/sys/vm/drop_caches: Read-only file system

07:37:41 UTC success: [sealos-master]

07:37:41 UTC success: [sealos-node2]

07:37:41 UTC success: [sealos-node1]

07:37:41 UTC [ConfigureOSModule] configure the ntp server for each node

07:37:41 UTC skipped: [sealos-node2]

07:37:41 UTC skipped: [sealos-master]

07:37:41 UTC skipped: [sealos-node1]

07:37:41 UTC [KubernetesStatusModule] Get kubernetes cluster status

07:37:41 UTC success: [sealos-master]

07:37:41 UTC [InstallContainerModule] Sync containerd binaries

07:37:41 UTC skipped: [sealos-master]

07:37:41 UTC skipped: [sealos-node1]

07:37:41 UTC skipped: [sealos-node2]

07:37:41 UTC [InstallContainerModule] Generate containerd service

07:37:41 UTC skipped: [sealos-master]

07:37:41 UTC skipped: [sealos-node1]

07:37:41 UTC skipped: [sealos-node2]

07:37:41 UTC [InstallContainerModule] Generate containerd config

07:37:41 UTC skipped: [sealos-master]

07:37:41 UTC skipped: [sealos-node1]

07:37:41 UTC skipped: [sealos-node2]

07:37:41 UTC [InstallContainerModule] Enable containerd

07:37:41 UTC skipped: [sealos-master]

07:37:41 UTC skipped: [sealos-node2]

07:37:41 UTC skipped: [sealos-node1]

07:37:41 UTC [InstallContainerModule] Sync crictl binaries

07:37:41 UTC skipped: [sealos-master]

07:37:41 UTC skipped: [sealos-node1]

07:37:41 UTC skipped: [sealos-node2]

07:37:41 UTC [InstallContainerModule] Generate crictl config

07:37:41 UTC success: [sealos-master]

07:37:41 UTC success: [sealos-node2]

07:37:41 UTC success: [sealos-node1]

07:37:41 UTC [PullModule] Start to pull images on all nodes

07:37:41 UTC message: [sealos-master]

downloading image: registry.cn-beijing.aliyuncs.com/kubesphereio/pause:3.9

07:37:41 UTC message: [sealos-node2]

downloading image: registry.cn-beijing.aliyuncs.com/kubesphereio/pause:3.9

07:37:41 UTC message: [sealos-node1]

downloading image: registry.cn-beijing.aliyuncs.com/kubesphereio/pause:3.9

07:37:41 UTC message: [sealos-master]

downloading image: registry.cn-beijing.aliyuncs.com/kubesphereio/kube-apiserver:v1.28.12

07:37:41 UTC message: [sealos-node1]

downloading image: registry.cn-beijing.aliyuncs.com/kubesphereio/kube-proxy:v1.28.12

07:37:41 UTC message: [sealos-node2]

downloading image: registry.cn-beijing.aliyuncs.com/kubesphereio/kube-proxy:v1.28.12

07:37:41 UTC message: [sealos-master]

downloading image: registry.cn-beijing.aliyuncs.com/kubesphereio/kube-controller-manager:v1.28.12

07:37:41 UTC message: [sealos-node1]

downloading image: registry.cn-beijing.aliyuncs.com/kubesphereio/coredns:1.9.3

07:37:41 UTC message: [sealos-node2]

downloading image: registry.cn-beijing.aliyuncs.com/kubesphereio/coredns:1.9.3

07:37:41 UTC message: [sealos-master]

downloading image: registry.cn-beijing.aliyuncs.com/kubesphereio/kube-scheduler:v1.28.12

07:37:41 UTC message: [sealos-node1]

downloading image: registry.cn-beijing.aliyuncs.com/kubesphereio/k8s-dns-node-cache:1.22.20

07:37:41 UTC message: [sealos-node2]

downloading image: registry.cn-beijing.aliyuncs.com/kubesphereio/k8s-dns-node-cache:1.22.20

07:37:41 UTC message: [sealos-master]

downloading image: registry.cn-beijing.aliyuncs.com/kubesphereio/kube-proxy:v1.28.12

07:37:41 UTC message: [sealos-node1]

downloading image: registry.cn-beijing.aliyuncs.com/kubesphereio/kube-controllers:v3.27.4

07:37:41 UTC message: [sealos-node2]

downloading image: registry.cn-beijing.aliyuncs.com/kubesphereio/kube-controllers:v3.27.4

07:37:41 UTC message: [sealos-master]

downloading image: registry.cn-beijing.aliyuncs.com/kubesphereio/coredns:1.9.3

07:37:41 UTC message: [sealos-node1]

downloading image: registry.cn-beijing.aliyuncs.com/kubesphereio/cni:v3.27.4

07:37:41 UTC message: [sealos-node2]

downloading image: registry.cn-beijing.aliyuncs.com/kubesphereio/cni:v3.27.4

07:37:41 UTC message: [sealos-master]

downloading image: registry.cn-beijing.aliyuncs.com/kubesphereio/k8s-dns-node-cache:1.22.20

07:37:41 UTC message: [sealos-node1]

downloading image: registry.cn-beijing.aliyuncs.com/kubesphereio/node:v3.27.4

07:37:41 UTC message: [sealos-node2]

downloading image: registry.cn-beijing.aliyuncs.com/kubesphereio/node:v3.27.4

07:37:41 UTC message: [sealos-master]

downloading image: registry.cn-beijing.aliyuncs.com/kubesphereio/kube-controllers:v3.27.4

07:37:41 UTC message: [sealos-node1]

downloading image: registry.cn-beijing.aliyuncs.com/kubesphereio/pod2daemon-flexvol:v3.27.4

07:37:41 UTC message: [sealos-node2]

downloading image: registry.cn-beijing.aliyuncs.com/kubesphereio/pod2daemon-flexvol:v3.27.4

07:37:41 UTC message: [sealos-master]

downloading image: registry.cn-beijing.aliyuncs.com/kubesphereio/cni:v3.27.4

07:37:41 UTC message: [sealos-master]

downloading image: registry.cn-beijing.aliyuncs.com/kubesphereio/node:v3.27.4

07:37:41 UTC message: [sealos-master]

downloading image: registry.cn-beijing.aliyuncs.com/kubesphereio/pod2daemon-flexvol:v3.27.4

07:37:41 UTC success: [sealos-node2]

07:37:41 UTC success: [sealos-node1]

07:37:41 UTC success: [sealos-master]

07:37:41 UTC [ETCDPreCheckModule] Get etcd status

07:37:41 UTC stdout: [sealos-master]

ETCD_NAME=etcd-sealos-master

07:37:41 UTC success: [sealos-master]

07:37:41 UTC [CertsModule] Fetch etcd certs

07:37:41 UTC success: [sealos-master]

07:37:41 UTC [CertsModule] Generate etcd Certs

[certs] Using existing ca certificate authority

[certs] Using existing admin-sealos-master certificate and key on disk

[certs] Using existing member-sealos-master certificate and key on disk

[certs] Using existing node-sealos-master certificate and key on disk

07:37:41 UTC success: [LocalHost]

07:37:41 UTC [CertsModule] Synchronize certs file

07:37:42 UTC success: [sealos-master]

07:37:42 UTC [CertsModule] Synchronize certs file to master

07:37:42 UTC skipped: [sealos-master]

07:37:42 UTC [InstallETCDBinaryModule] Install etcd using binary

07:37:42 UTC skipped: [sealos-master]

07:37:42 UTC [InstallETCDBinaryModule] Generate etcd service

07:37:42 UTC skipped: [sealos-master]

07:37:42 UTC [InstallETCDBinaryModule] Generate access address

07:37:42 UTC success: [sealos-master]

07:37:42 UTC [ETCDConfigureModule] Health check on exist etcd

07:37:42 UTC message: [sealos-master]

etcd health check failed: Failed to exec command: sudo -E /bin/bash -c “export ETCDCTL_API=2;export ETCDCTL_CERT_FILE=‘/etc/ssl/etcd/ssl/admin-sealos-master.pem’;export ETCDCTL_KEY_FILE=‘/etc/ssl/etcd/ssl/admin-sealos-master-key.pem’;export ETCDCTL_CA_FILE=‘/etc/ssl/etcd/ssl/ca.pem’;/usr/local/bin/etcdctl –endpoints=https://192.168.110.12:2379 cluster-health | grep -q ‘cluster is healthy’”

Error: client: etcd cluster is unavailable or misconfigured; error #0: dial tcp 192.168.110.12:2379: connect: connection refused

error #0: dial tcp 192.168.110.12:2379: connect: connection refused: Process exited with status 1

07:37:42 UTC retry: [sealos-master]

07:37:47 UTC message: [sealos-master]

etcd health check failed: Failed to exec command: sudo -E /bin/bash -c “export ETCDCTL_API=2;export ETCDCTL_CERT_FILE=‘/etc/ssl/etcd/ssl/admin-sealos-master.pem’;export ETCDCTL_KEY_FILE=‘/etc/ssl/etcd/ssl/admin-sealos-master-key.pem’;export ETCDCTL_CA_FILE=‘/etc/ssl/etcd/ssl/ca.pem’;/usr/local/bin/etcdctl –endpoints=https://192.168.110.12:2379 cluster-health | grep -q ‘cluster is healthy’”

Error: client: etcd cluster is unavailable or misconfigured; error #0: dial tcp 192.168.110.12:2379: connect: connection refused

error #0: dial tcp 192.168.110.12:2379: connect: connection refused: Process exited with status 1

07:37:47 UTC retry: [sealos-master]

07:37:52 UTC message: [sealos-master]

etcd health check failed: Failed to exec command: sudo -E /bin/bash -c “export ETCDCTL_API=2;export ETCDCTL_CERT_FILE=‘/etc/ssl/etcd/ssl/admin-sealos-master.pem’;export ETCDCTL_KEY_FILE=‘/etc/ssl/etcd/ssl/admin-sealos-master-key.pem’;export ETCDCTL_CA_FILE=‘/etc/ssl/etcd/ssl/ca.pem’;/usr/local/bin/etcdctl –endpoints=https://192.168.110.12:2379 cluster-health | grep -q ‘cluster is healthy’”

Error: client: etcd cluster is unavailable or misconfigured; error #0: dial tcp 192.168.110.12:2379: connect: connection refused

error #0: dial tcp 192.168.110.12:2379: connect: connection refused: Process exited with status 1

07:37:52 UTC retry: [sealos-master]

07:37:57 UTC message: [sealos-master]

etcd health check failed: Failed to exec command: sudo -E /bin/bash -c “export ETCDCTL_API=2;export ETCDCTL_CERT_FILE=‘/etc/ssl/etcd/ssl/admin-sealos-master.pem’;export ETCDCTL_KEY_FILE=‘/etc/ssl/etcd/ssl/admin-sealos-master-key.pem’;export ETCDCTL_CA_FILE=‘/etc/ssl/etcd/ssl/ca.pem’;/usr/local/bin/etcdctl –endpoints=https://192.168.110.12:2379 cluster-health | grep -q ‘cluster is healthy’”

Error: client: etcd cluster is unavailable or misconfigured; error #0: dial tcp 192.168.110.12:2379: connect: connection refused

error #0: dial tcp 192.168.110.12:2379: connect: connection refused: Process exited with status 1

07:37:57 UTC retry: [sealos-master]

07:38:02 UTC message: [sealos-master]

etcd health check failed: Failed to exec command: sudo -E /bin/bash -c “export ETCDCTL_API=2;export ETCDCTL_CERT_FILE=‘/etc/ssl/etcd/ssl/admin-sealos-master.pem’;export ETCDCTL_KEY_FILE=‘/etc/ssl/etcd/ssl/admin-sealos-master-key.pem’;export ETCDCTL_CA_FILE=‘/etc/ssl/etcd/ssl/ca.pem’;/usr/local/bin/etcdctl –endpoints=https://192.168.110.12:2379 cluster-health | grep -q ‘cluster is healthy’”

Error: client: etcd cluster is unavailable or misconfigured; error #0: dial tcp 192.168.110.12:2379: connect: connection refused

error #0: dial tcp 192.168.110.12:2379: connect: connection refused: Process exited with status 1

07:38:02 UTC retry: [sealos-master]

  • cici 回复了此帖

    @小k 看起来是etcd启动失败了，你来分析回答一下吧，如果要看etcd日志的话，可以通过 journalctl -f -u etcd 命令查看。

      根据您提供的信息，etcd 健康检查失败，提示连接被拒绝。这通常意味着客户端（在这种情况下是 Kubernetes 控制平面组件）无法连接到 etcd 服务器。以下是一些可能的解决步骤：

      1. 检查 etcd 服务状态：
        使用以下命令检查 etcd 服务是否正在运行：

           sudo systemctl status etcd

        如果服务没有运行，尝试启动它：

           sudo systemctl start etcd

      2. 检查网络连接：
        确保 Kubernetes 控制平面组件和 etcd 服务器之间的网络连接没有问题。您可以使用 telnet 或 nc（netcat）来测试连接：

           telnet 192.168.110.12 2379

        或者

           nc -zv 192.168.110.12 2379

        如果连接失败，检查防火墙规则或网络配置。

      3. 查看 etcd 日志：
        使用以下命令查看 etcd 的日志，以获取更多错误信息：

           journalctl -f -u etcd

        查找任何与连接失败或配置错误相关的错误消息。

      4. 检查证书和密钥：
        如果您的 etcd 集群使用 TLS，请确保证书和密钥文件正确，并且没有权限问题。

      如果问题仍然存在，请提供更多的日志信息，特别是 etcd 和 Kubernetes 控制平面组件的日志，以便进行更深入的分析。

      Cauchy

      真的是etcd启动失败了，但是是使用kk启动的，为什么会出现这种情况呢

      etcd日志

      Nov 13 01:15:53 sealos-master systemd[1]: etcd.service: Main process exited, code=exited, status=1/FAILURE

      Nov 13 01:15:53 sealos-master systemd[1]: etcd.service: Failed with result ‘exit-code’.

      Nov 13 01:15:53 sealos-master systemd[1]: Failed to start etcd.

      Nov 13 01:16:03 sealos-master systemd[1]: etcd.service: Scheduled restart job, restart counter is at 6135.

      Nov 13 01:16:03 sealos-master systemd[1]: Stopped etcd.

      Nov 13 01:16:03 sealos-master systemd[1]: Starting etcd…

      Nov 13 01:16:03 sealos-master etcd[35990]: {“level”:“info”,“ts”:“2024-11-13T01:16:03.343323Z”,“caller”:“flags/flag.go:113”,“msg”:“recognized and used environment variable”,“variable-name”:“ETCD_ADVERTISE_CLIENT_URLS”,“variable-value”:“https://192.168.110.12:2379”}

      Nov 13 01:16:03 sealos-master etcd[35990]: {“level”:“info”,“ts”:“2024-11-13T01:16:03.343374Z”,“caller”:“flags/flag.go:113”,“msg”:“recognized and used environment variable”,“variable-name”:“ETCD_AUTO_COMPACTION_RETENTION”,“variable-value”:“8”}

      Nov 13 01:16:03 sealos-master etcd[35990]: {“level”:“info”,“ts”:“2024-11-13T01:16:03.343381Z”,“caller”:“flags/flag.go:113”,“msg”:“recognized and used environment variable”,“variable-name”:“ETCD_CERT_FILE”,“variable-value”:“/etc/ssl/etcd/ssl/member-sealos-master.pem”}

      Nov 13 01:16:03 sealos-master etcd[35990]: {“level”:“info”,“ts”:“2024-11-13T01:16:03.343387Z”,“caller”:“flags/flag.go:113”,“msg”:“recognized and used environment variable”,“variable-name”:“ETCD_CLIENT_CERT_AUTH”,“variable-value”:“true”}

      Nov 13 01:16:03 sealos-master etcd[35990]: {“level”:“info”,“ts”:“2024-11-13T01:16:03.343395Z”,“caller”:“flags/flag.go:113”,“msg”:“recognized and used environment variable”,“variable-name”:“ETCD_DATA_DIR”,“variable-value”:“/var/lib/etcd”}

      Nov 13 01:16:03 sealos-master etcd[35990]: {“level”:“info”,“ts”:“2024-11-13T01:16:03.343401Z”,“caller”:“flags/flag.go:113”,“msg”:“recognized and used environment variable”,“variable-name”:“ETCD_ELECTION_TIMEOUT”,“variable-value”:“5000”}

      Nov 13 01:16:03 sealos-master etcd[35990]: {“level”:“info”,“ts”:“2024-11-13T01:16:03.343406Z”,“caller”:“flags/flag.go:113”,“msg”:“recognized and used environment variable”,“variable-name”:“ETCD_ENABLE_V2”,“variable-value”:“true”}

      Nov 13 01:16:03 sealos-master etcd[35990]: {“level”:“info”,“ts”:“2024-11-13T01:16:03.343418Z”,“caller”:“flags/flag.go:113”,“msg”:“recognized and used environment variable”,“variable-name”:“ETCD_HEARTBEAT_INTERVAL”,“variable-value”:“250”}

      Nov 13 01:16:03 sealos-master etcd[35990]: {“level”:“info”,“ts”:“2024-11-13T01:16:03.343426Z”,“caller”:“flags/flag.go:113”,“msg”:“recognized and used environment variable”,“variable-name”:“ETCD_INITIAL_ADVERTISE_PEER_URLS”,“variable-value”:“https://192.168.110.12:2380”}

      Nov 13 01:16:03 sealos-master etcd[35990]: {“level”:“info”,“ts”:“2024-11-13T01:16:03.343431Z”,“caller”:“flags/flag.go:113”,“msg”:“recognized and used environment variable”,“variable-name”:“ETCD_INITIAL_CLUSTER”,“variable-value”:“etcd-sealos-master=https://192.168.110.12:2380”}

      Nov 13 01:16:03 sealos-master etcd[35990]: {“level”:“info”,“ts”:“2024-11-13T01:16:03.343435Z”,“caller”:“flags/flag.go:113”,“msg”:“recognized and used environment variable”,“variable-name”:“ETCD_INITIAL_CLUSTER_STATE”,“variable-value”:“existing”}

      Nov 13 01:16:03 sealos-master etcd[35990]: {“level”:“info”,“ts”:“2024-11-13T01:16:03.343439Z”,“caller”:“flags/flag.go:113”,“msg”:“recognized and used environment variable”,“variable-name”:“ETCD_INITIAL_CLUSTER_TOKEN”,“variable-value”:“k8s_etcd”}

      Nov 13 01:16:03 sealos-master etcd[35990]: {“level”:“info”,“ts”:“2024-11-13T01:16:03.343445Z”,“caller”:“flags/flag.go:113”,“msg”:“recognized and used environment variable”,“variable-name”:“ETCD_KEY_FILE”,“variable-value”:“/etc/ssl/etcd/ssl/member-sealos-master-key.pem”}

      Nov 13 01:16:03 sealos-master etcd[35990]: {“level”:“info”,“ts”:“2024-11-13T01:16:03.343452Z”,“caller”:“flags/flag.go:113”,“msg”:“recognized and used environment variable”,“variable-name”:“ETCD_LISTEN_CLIENT_URLS”,“variable-value”:“https://192.168.110.12:2379,https://127.0.0.1:2379”}

      Nov 13 01:16:03 sealos-master etcd[35990]: {“level”:“info”,“ts”:“2024-11-13T01:16:03.343458Z”,“caller”:“flags/flag.go:113”,“msg”:“recognized and used environment variable”,“variable-name”:“ETCD_LISTEN_PEER_URLS”,“variable-value”:“https://192.168.110.12:2380”}

      Nov 13 01:16:03 sealos-master etcd[35990]: {“level”:“info”,“ts”:“2024-11-13T01:16:03.343464Z”,“caller”:“flags/flag.go:113”,“msg”:“recognized and used environment variable”,“variable-name”:“ETCD_METRICS”,“variable-value”:“basic”}

      Nov 13 01:16:03 sealos-master etcd[35990]: {“level”:“info”,“ts”:“2024-11-13T01:16:03.343468Z”,“caller”:“flags/flag.go:113”,“msg”:“recognized and used environment variable”,“variable-name”:“ETCD_NAME”,“variable-value”:“etcd-sealos-master”}

      Nov 13 01:16:03 sealos-master etcd[35990]: {“level”:“info”,“ts”:“2024-11-13T01:16:03.343474Z”,“caller”:“flags/flag.go:113”,“msg”:“recognized and used environment variable”,“variable-name”:“ETCD_PEER_CERT_FILE”,“variable-value”:“/etc/ssl/etcd/ssl/member-sealos-master.pem”}

      Nov 13 01:16:03 sealos-master etcd[35990]: {“level”:“info”,“ts”:“2024-11-13T01:16:03.343479Z”,“caller”:“flags/flag.go:113”,“msg”:“recognized and used environment variable”,“variable-name”:“ETCD_PEER_CLIENT_CERT_AUTH”,“variable-value”:“true”}

      Nov 13 01:16:03 sealos-master etcd[35990]: {“level”:“info”,“ts”:“2024-11-13T01:16:03.343487Z”,“caller”:“flags/flag.go:113”,“msg”:“recognized and used environment variable”,“variable-name”:“ETCD_PEER_KEY_FILE”,“variable-value”:“/etc/ssl/etcd/ssl/member-sealos-master-key.pem”}

      Nov 13 01:16:03 sealos-master etcd[35990]: {“level”:“info”,“ts”:“2024-11-13T01:16:03.343492Z”,“caller”:“flags/flag.go:113”,“msg”:“recognized and used environment variable”,“variable-name”:“ETCD_PEER_TRUSTED_CA_FILE”,“variable-value”:“/etc/ssl/etcd/ssl/ca.pem”}

      Nov 13 01:16:03 sealos-master etcd[35990]: {“level”:“info”,“ts”:“2024-11-13T01:16:03.343496Z”,“caller”:“flags/flag.go:113”,“msg”:“recognized and used environment variable”,“variable-name”:“ETCD_PROXY”,“variable-value”:“off”}

      Nov 13 01:16:03 sealos-master etcd[35990]: {“level”:“info”,“ts”:“2024-11-13T01:16:03.343508Z”,“caller”:“flags/flag.go:113”,“msg”:“recognized and used environment variable”,“variable-name”:“ETCD_SNAPSHOT_COUNT”,“variable-value”:“10000”}

      Nov 13 01:16:03 sealos-master etcd[35990]: {“level”:“info”,“ts”:“2024-11-13T01:16:03.343513Z”,“caller”:“flags/flag.go:113”,“msg”:“recognized and used environment variable”,“variable-name”:“ETCD_TRUSTED_CA_FILE”,“variable-value”:“/etc/ssl/etcd/ssl/ca.pem”}

      Nov 13 01:16:03 sealos-master etcd[35990]: {“level”:“warn”,“ts”:“2024-11-13T01:16:03.343544Z”,“caller”:“embed/config.go:679”,“msg”:“Running http and grpc server on single port. This is not recommended for production.”}

      Nov 13 01:16:03 sealos-master etcd[35990]: {“level”:“info”,“ts”:“2024-11-13T01:16:03.343564Z”,“caller”:“etcdmain/etcd.go:73”,“msg”:“Running: ”,“args”:[“/usr/local/bin/etcd”]}

      Nov 13 01:16:03 sealos-master etcd[35990]: {“level”:“info”,“ts”:“2024-11-13T01:16:03.343601Z”,“caller”:“etcdmain/etcd.go:116”,“msg”:“server has been already initialized”,“data-dir”:“/var/lib/etcd”,“dir-type”:“member”}

      Nov 13 01:16:03 sealos-master etcd[35990]: {“level”:“warn”,“ts”:“2024-11-13T01:16:03.343614Z”,“caller”:“embed/config.go:679”,“msg”:“Running http and grpc server on single port. This is not recommended for production.”}

      Nov 13 01:16:03 sealos-master etcd[35990]: {“level”:“info”,“ts”:“2024-11-13T01:16:03.343623Z”,“caller”:“embed/etcd.go:127”,“msg”:“configuring peer listeners”,“listen-peer-urls”:[“https://192.168.110.12:2380”]}

      Nov 13 01:16:03 sealos-master etcd[35990]: {“level”:“info”,“ts”:“2024-11-13T01:16:03.343638Z”,“caller”:“embed/etcd.go:494”,“msg”:“starting with peer TLS”,“tls-info”:“cert = /etc/ssl/etcd/ssl/member-sealos-master.pem, key = /etc/ssl/etcd/ssl/member-sealos-master-key.pem, client-cert=, client-key=, trusted-ca = /etc/ssl/etcd/ssl/ca.pem, client-cert-auth = true, crl-file = ”,“cipher-suites”:[]}

      Nov 13 01:16:03 sealos-master etcd[35990]: {“level”:“info”,“ts”:“2024-11-13T01:16:03.344006Z”,“caller”:“embed/etcd.go:135”,“msg”:“configuring client listeners”,“listen-client-urls”:[“https://127.0.0.1:2379”,“https://192.168.110.12:2379”]}

      Nov 13 01:16:03 sealos-master etcd[35990]: {“level”:“info”,“ts”:“2024-11-13T01:16:03.344061Z”,“caller”:“embed/etcd.go:308”,“msg”:“starting an etcd server”,“etcd-version”:“3.5.13”,“git-sha”:“c9063a0dc”,“go-version”:“go1.21.8”,“go-os”:“linux”,“go-arch”:“amd64”,“max-cpu-set”:2,“max-cpu-available”:2,“member-initialized”:false,“name”:“etcd-sealos-master”,“data-dir”:“/var/lib/etcd”,“wal-dir”:"",“wal-dir-dedicated”:"",“member-dir”:“/var/lib/etcd/member”,“force-new-cluster”:false,“heartbeat-interval”:“250ms”,“election-timeout”:“5s”,“initial-election-tick-advance”:true,“snapshot-count”:10000,“max-wals”:5,“max-snapshots”:5,“snapshot-catchup-entries”:5000,“initial-advertise-peer-urls”:[“https://192.168.110.12:2380”],“listen-peer-urls”:[“https://192.168.110.12:2380”],“advertise-client-urls”:[“https://192.168.110.12:2379”],“listen-client-urls”:[“https://127.0.0.1:2379”,“https://192.168.110.12:2379”],“listen-metrics-urls”:[],“cors”:[“*”],“host-whitelist”:[“*”],“initial-cluster”:“etcd-sealos-master=https://192.168.110.12:2380”,“initial-cluster-state”:“existing”,“initial-cluster-token”:“k8s_etcd”,“quota-backend-bytes”:2147483648,“max-request-bytes”:1572864,“max-concurrent-streams”:4294967295,“pre-vote”:true,“initial-corrupt-check”:false,“corrupt-check-time-interval”:“0s”,“compact-check-time-enabled”:false,“compact-check-time-interval”:“1m0s”,“auto-compaction-mode”:“periodic”,“auto-compaction-retention”:“8h0m0s”,“auto-compaction-interval”:“8h0m0s”,“discovery-url”:"",“discovery-proxy”:"",“downgrade-check-interval”:“5s”}

      Nov 13 01:16:03 sealos-master etcd[35990]: {“level”:“warn”,“ts”:“2024-11-13T01:16:03.344098Z”,“caller”:“fileutil/fileutil.go:53”,“msg”:“check file permission”,“error”:“directory \”/var/lib/etcd\" exist, but the permission is \“drwxr-xr-x\”. The recommended permission is \“-rwx——\” to prevent possible unprivileged access to the data"}

      Nov 13 01:16:03 sealos-master etcd[35990]: {“level”:“info”,“ts”:“2024-11-13T01:16:03.345254Z”,“caller”:“etcdserver/backend.go:81”,“msg”:“opened backend db”,“path”:“/var/lib/etcd/member/snap/db”,“took”:“1.048608ms”}

      Nov 13 01:16:03 sealos-master etcd[35990]: {“level”:“info”,“ts”:“2024-11-13T01:16:03.347275Z”,“caller”:“embed/etcd.go:375”,“msg”:“closing etcd server”,“name”:“etcd-sealos-master”,“data-dir”:“/var/lib/etcd”,“advertise-peer-urls”:[“https://192.168.110.12:2380”],“advertise-client-urls”:[“https://192.168.110.12:2379”]}

      Nov 13 01:16:03 sealos-master etcd[35990]: {“level”:“info”,“ts”:“2024-11-13T01:16:03.347308Z”,“caller”:“embed/etcd.go:377”,“msg”:“closed etcd server”,“name”:“etcd-sealos-master”,“data-dir”:“/var/lib/etcd”,“advertise-peer-urls”:[“https://192.168.110.12:2380”],“advertise-client-urls”:[“https://192.168.110.12:2379”]}

      Nov 13 01:16:03 sealos-master etcd[35990]: {“level”:“fatal”,“ts”:“2024-11-13T01:16:03.347319Z”,“caller”:“etcdmain/etcd.go:204”,“msg”:“discovery failed”,“error”:“cannot fetch cluster info from peer urls: could not retrieve cluster information from the given URLs”,“stacktrace”:“go.etcd.io/etcd/server/v3/etcdmain.startEtcdOrProxyV2\n\tgo.etcd.io/etcd/server/v3/etcdmain/etcd.go:204\ngo.etcd.io/etcd/server/v3/etcdmain.Main\n\tgo.etcd.io/etcd/server/v3/etcdmain/main.go:40\nmain.main\n\tgo.etcd.io/etcd/server/v3/main.go:31\nruntime.main\n\truntime/proc.go:267”}

      Nov 13 01:16:03 sealos-master systemd[1]: etcd.service: Main process exited, code=exited, status=1/FAILURE

      Nov 13 01:16:03 sealos-master systemd[1]: etcd.service: Failed with result ‘exit-code’.

      Nov 13 01:16:03 sealos-master systemd[1]: Failed to start etcd.

        Cauchy 将防火墙关闭后重新安装出现如下报错

        KERNEL_VERSION: 5.19.7-2-pve

        OS: Linux

        CGROUPS_CPU: enabled

        CGROUPS_CPUSET: enabled

        CGROUPS_DEVICES: enabled

        CGROUPS_FREEZER: enabled

        CGROUPS_MEMORY: enabled

        CGROUPS_PIDS: enabled

        CGROUPS_HUGETLB: enabled

        CGROUPS_IO: enabled

        error execution phase preflight: [preflight] Some fatal errors occurred:

        [ERROR SystemVerification]: failed to parse kernel config: unable to load kernel module: "configs", output: "modprobe: FATAL: Module configs not found in directory /lib/modules/5.19.7-2-pve\\n", err: exit status 1

        [preflight] If you know what you are doing, you can make a check non-fatal with `–ignore-preflight-errors=…`

        To see the stack trace of this error execute with –v=5 or higher: Process exited with status 1

        02:25:53 UTC failed: [sealos-master]

        error: Pipeline[CreateClusterPipeline] execute failed: Module[InitKubernetesModule] exec failed:

        failed: [sealos-master] [KubeadmInit] exec failed after 3 retries: init kubernetes cluster failed: Failed to exec command: sudo -E /bin/bash -c “/usr/local/bin/kubeadm init –config=/etc/kubernetes/kubeadm-config.yaml –ignore-preflight-errors=FileExisting-crictl,ImagePull”

        W1113 02:25:53.322755 48399 utils.go:69] The recommended value for “clusterDNS” in “KubeletConfiguration” is: [10.233.0.10]; the provided value is: [169.254.25.10]

        [init] Using Kubernetes version: v1.28.12

        [preflight] Running pre-flight checks

        [WARNING Swap]: swap is enabled; production deployments should disable swap unless testing the NodeSwap feature gate of the kubelet

[WARNING FileExisting-ethtool]: ethtool not found in system path

        [preflight] The system verification failed. Printing the output from the verification:

        KERNEL_VERSION: 5.19.7-2-pve

        OS: Linux

        CGROUPS_CPU: enabled

        CGROUPS_CPUSET: enabled

        CGROUPS_DEVICES: enabled

        CGROUPS_FREEZER: enabled

        CGROUPS_MEMORY: enabled

        CGROUPS_PIDS: enabled

        CGROUPS_HUGETLB: enabled

        CGROUPS_IO: enabled

        error execution phase preflight: [preflight] Some fatal errors occurred:

        [ERROR SystemVerification]: failed to parse kernel config: unable to load kernel module: "configs", output: "modprobe: FATAL: Module configs not found in directory /lib/modules/5.19.7-2-pve\\n", err: exit status 1

        [preflight] If you know what you are doing, you can make a check non-fatal with `–ignore-preflight-errors=…`

        To see the stack trace of this error execute with –v=5 or higher: Process exited with status 1

            6 天 后

            我也碰见这个问题了现在解决了吗

            hubertshelley
            看起来是操作系统内核有问题，报错是 kubeadm 对系统检查的问题

            [ERROR SystemVerification]: failed to parse kernel config: unable to load kernel module: "configs", output: "modprobe: FATAL: Module configs not found in directory /lib/modules/5.19.7-2-pve\\n", err: exit status 1
              1 个月 后