kubeedge部署不成功
系统环境及软件版本
基础环境-1
| 系统及版本信息 | 角色 | IP | DNS | 架构 | Docker版本 |
|:-😐:-😐:-😐:-😐:-😐:-😐
|Ubuntu 23.10|Master|—-|168.95.1.1|x86_64|v20.10
|Ubuntu 23.10|Node1|—-|168.95.1.1|x86_64|v20.10
|Ubuntu 23.10|Node2|—-|168.95.1.1|x86_64|v.20.10
|Ubuntu 23.10|edge|—-|168.95.1.1|x86_64|v26.0.0
|Ubuntu 22.04|edge|—-|168.95.1.1|arm64|
|Ubuntu 22.04|edge|—-|168.95.1.1|arm64|
kubeedge
最初在瑞芯微RK3588开发板上部署kubeedge，一直无法成功，以为是开发板的系统内核是被裁切过。后来陆续在x86的Ubuntu虚拟机和树莓派3B上部署，发现都遇到同样的问题。
基础环境-2
|软件名称|版本|
|:-😐:-😐-😐
KubeKey|v3.0.7|
Kubesphere|v3.3.0|
Kubernetes|v1.21.1|
kubeedge|v1.9|
安装遇到的问题
1. cgroup驱动不匹配，经查询得知，Docker默认的cgroup驱动为systemd，kubeedge配置文件指定驱动为cgroupfs，直接单独安装完docker，接着安装kubeedge会报错，提示驱动不匹配，手动在/etc/docker路径下添加daemon.json文件，配置cgroup驱动为cgroupfs，与kubeedge配置文件一致，容器组正常开始配置
2. 部署kubeedge时会自动配置三个容器组，其中 kube-proxy能正常初始化，另两个容器组nodelocaldns与calico-node无法正常初始化。
3. 在kubesphere的Web端查看边缘节点的状态时，无法使用日志功能，提示10350端口无法连接。
无法初始化容器组的日志
nodelocaldns
# 通过portainer查看容器状态，日志如下：
2024/04/15 01:36:55 [INFO] Using Corefile /etc/coredns/Corefile
2024/04/15 01:36:55 [FATAL] Failed to obtain CacheApp instance, err Unable to lookup IP address of Upstream service coredns, env $COREDNS_SERVICE_HOST \`\`
容器状态中有一条命令
CMD -localip 169.254.25.10 -conf /etc/coredns/Corefile -upstreamsvc coredns
在kubeedge边缘设备上无法ping通这个IP，回到Master节点可以ping通这个IP
calico-node
# 使用 journalctl -u edgecore.service 查看运行日志如下：
Apr 12 09:21:27 k8s-kubeedge1 edgecore[2441]: E0412 09:21:27.160519 2441 metaclient.go:112] send sync message kube-system/configmap/kubernetes-services-endpoint failed, error:timeout to get response, retries: 2
Apr 12 09:21:27 k8s-kubeedge1 edgecore[2441]: I0412 09:21:27.160615 2441 record.go:19] Warning Failed Error: get configmap from metaManager failed, err: timed out waiting for the condition
Apr 12 09:21:27 k8s-kubeedge1 edgecore[2441]: I0412 09:21:27.160497 2441 process.go:282] ########## process get: req[{Header:{ID:3d549606-6469-4e62-a397-43bcde5ba1e3 ParentID: Timestamp:1712913567149 ResourceVersion: Sync:true MessageType:} Router:{Source:edged Destination: Group:meta Operation:query Resource:kube-system/configmap/kubernetes-services-endpoint} Content:<nil>}], resp[{Header:{ID: ParentID: Timestamp:0 ResourceVersion: Sync:false MessageType:} Router:{Source: Destination: Group: Operation: Resource:} Content:<nil>}], err[timeout to get response]
Apr 12 09:21:27 k8s-kubeedge1 edgecore[2441]: E0412 09:21:27.160730 2441 process.go:284] remote query failed: timeout to get response
Apr 12 09:21:27 k8s-kubeedge1 edgecore[2441]: E0412 09:21:27.160772 2441 kuberuntime_manager.go:864] init container &Container{Name:upgrade-ipam,Image:calico/cni:v3.23.2,Command:[/opt/cni/bin/calico-ipam -upgrade],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{EnvVar{Name:KUBERNETES_NODE_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:spec.nodeName,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:CALICO_NETWORKING_BACKEND,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:calico-config,},Key:calico_backend,Optional:nil,},SecretKeyRef:nil,},},},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:host-local-net-dir,ReadOnly:false,MountPath:/var/lib/cni/networks,SubPath:,MountPropagation:nil,SubPathExpr:,},VolumeMount{Name:cni-bin-dir,ReadOnly:false,MountPath:/host/opt/cni/bin,SubPath:,MountPropagation:nil,SubPathExpr:,},VolumeMount{Name:kube-api-access-g5ghn,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:&SecurityContext{Capabilities:nil,Privileged:\*true,SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,ReadOnlyRootFilesystem:nil,AllowPrivilegeEscalation:nil,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{EnvFromSource{Prefix:,ConfigMapRef:&ConfigMapEnvSource{LocalObjectReference:LocalObjectReference{Name:kubernetes-services-endpoint,},Optional:\*true,},SecretRef:nil,},},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,} start failed in pod calico-node-xrkwm_kube-system(49983c1d-6c81-44ce-a5a9-9a45286a1052): CreateContainerConfigError: get configmap from metaManager failed, err: timed out waiting for the condition
Apr 12 09:21:27 k8s-kubeedge1 edgecore[2441]: I0412 09:21:27.160845 2441 edged.go:990] worker [2] get pod addition item [calico-node-xrkwm]
Apr 12 09:21:27 k8s-kubeedge1 edgecore[2441]: E0412 09:21:27.160898 2441 edged.go:993] consume pod addition backoff: Back-off consume pod [calico-node-xrkwm] addition error, backoff: [5m0s]
Apr 12 09:21:27 k8s-kubeedge1 edgecore[2441]: I0412 09:21:27.160938 2441 edged.go:995] worker [2] backoff pod addition item [calico-node-xrkwm] failed, re-add to queue
Apr 12 09:21:27 k8s-kubeedge1 edgecore[2441]: E0412 09:21:27.160945 2441 edged.go:1009] worker [2] handle pod addition item [calico-node-xrkwm] failed: failed to "StartContainer" for "upgrade-ipam" with CreateContainerConfigError: "get configmap from metaManager failed, err: timed out waiting for the condition", re-add to queue
Apr 12 09:21:37 k8s-kubeedge1 edgecore[2441]: I0412 09:21:37.161965 2441 edged.go:990] worker [0] get pod addition item [calico-node-xrkwm]
Apr 12 09:21:37 k8s-kubeedge1 edgecore[2441]: E0412 09:21:37.162024 2441 edged.go:993] consume pod addition backoff: Back-off consume pod [calico-node-xrkwm] addition error, backoff: [5m0s]
Apr 12 09:21:37 k8s-kubeedge1 edgecore[2441]: I0412 09:21:37.162070 2441 edged.go:995] worker [0] backoff pod addition item [calico-node-xrkwm] failed, re-add to queue
Apr 12 09:22:03 k8s-kubeedge1 edgecore[2441]: I0412 09:22:03.907300 2441 process.go:329] start to sync pod status in edge-store to cloud
# 这里说找不到指定的configmap，不知道是不是跟calico-node有关。
# 日志中提到的找不到configmap，以下是在Master上的查询记录
root@k8s-master01:/etc# kubectl get configmap -n kube-system
NAME DATA AGE
calico-config 4 73d
coredns 1 73d
extension-apiserver-authentication 6 73d
kube-proxy 2 73d
kube-root-ca.crt 1 73d
kubeadm-config 2 73d
kubelet-config-1.21 1 73d
nodelocaldns 1 73d
root@k8s-master01:/etc# kubectl get configmap kubernetes-services-endpoint -n kube-system
Error from server (NotFound): configmaps "kubernetes-services-endpoint" not found
root@k8s-master01:/etc#
