v4.1.2 KubeEdge边缘节点监控数据不显示

Hhyiyang · 4 1月

创建部署问题时，请参考下面模板，你提供的信息越多，越容易及时获得解答。如果未按模板创建问题，管理员有权关闭问题。
确保帖子格式清晰易读，用 markdown code block 语法格式化代码块。
你只花一分钟创建的问题，不能指望别人花上半个小时给你解答。

操作系统信息

集群使用kk命令部署。node1和node2是x86 Ubuntu22，node3是arm Kylin v10。

k get node -o wide                                                                                                                                
NAME    STATUS   ROLES                  AGE     VERSION                     INTERNAL-IP    EXTERNAL-IP   OS-IMAGE                          KERNEL-VERSION                     CONTAINER-RUNTIME
node1   Ready    control-plane,worker   6d15h   v1.28.12                    192.168.3.45   <none>        Ubuntu 22.04.5 LTS                6.8.0-49-generic                   containerd://1.7.13
node2   Ready    worker                 6d15h   v1.28.12                    192.168.3.21   <none>        Ubuntu 22.04.5 LTS                6.8.0-49-generic                   containerd://1.7.13
node3   Ready    agent,edge             5d20h   v1.23.15-kubeedge-v1.13.1   192.168.3.58   <none>        Kylin Linux Advanced Server V10   4.19.90-25.10.v2101.ky10.aarch64   containerd://1.7.23

Kubernetes版本信息

k version                                                                                                                                         
Client Version: v1.28.12
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.28.12

容器运行时

# node1 node2
containerd -v                                                                                                                                     
containerd github.com/containerd/containerd v1.7.13 7c3aca7a610df76212171d200ca3811ff6096eb8
# node3
containerd github.com/containerd/containerd v1.7.23 57f17b0a6295a39009d861b89e3b3b87b005ca27

问题是什么
x86节点上的监控信息可以正确显示，arm节点上的监控信息不显示:

命令行能获取到cpu memory信息：

kubectl get --raw /apis/metrics.k8s.io/v1beta1/nodes
{"kind":"NodeMetricsList","apiVersion":"metrics.k8s.io/v1beta1","metadata":{},"items":[{"metadata":{"name":"node1","creationTimestamp":"2025-01-04T00:47:42Z","labels"
:{"beta.kubernetes.io/arch":"amd64","beta.kubernetes.io/os":"linux","kubernetes.io/arch":"amd64","kubernetes.io/hostname":"node1","kubernetes.io/os":"linux","node-rol
e.kubernetes.io/control-plane":"","node-role.kubernetes.io/worker":"","node.kubernetes.io/exclude-from-external-load-balancers":""}},"timestamp":"2025-01-04T00:47:30Z
","window":"10.035s","usage":{"cpu":"346786248n","memory":"8704288Ki"}},{"metadata":{"name":"node2","creationTimestamp":"2025-01-04T00:47:42Z","labels":{"beta.kuberne
tes.io/arch":"amd64","beta.kubernetes.io/os":"linux","kubernetes.io/arch":"amd64","kubernetes.io/hostname":"node2","kubernetes.io/os":"linux","node-role.kubernetes.io
/worker":""}},"timestamp":"2025-01-04T00:47:33Z","window":"10.031s","usage":{"cpu":"139966105n","memory":"3049932Ki"}},{"metadata":{"name":"node3","creationTimestamp"
:"2025-01-04T00:47:42Z","labels":{"beta.kubernetes.io/arch":"arm64","beta.kubernetes.io/os":"linux","kubeedge.io/internal-ip":"192.168.3.58","kubernetes.io/arch":"arm
64","kubernetes.io/hostname":"node3","kubernetes.io/os":"linux","node-role.kubernetes.io/agent":"","node-role.kubernetes.io/edge":""}},"timestamp":"2025-01-04T00:47:3
6Z","window":"20.118s","usage":{"cpu":"894901934n","memory":"2056512Ki"}}]}

k get --raw "/apis/metrics.k8s.io/v1beta1/namespaces/default/pods/sleep" 
{"kind":"PodMetrics","apiVersion":"metrics.k8s.io/v1beta1","metadata":{"name":"sleep","namespace":"default","creationTimestamp":"2025-01-04T01:05:53Z"},"timestamp":"2
025-01-04T01:05:39Z","window":"14.947s","containers":[{"name":"sleep","usage":{"cpu":"0","memory":"704Ki"}}]}

k get pod -A -o wide                                                                                                                              
NAMESPACE                      NAME                                                          READY   STATUS      RESTARTS   AGE     IP             NODE    NOMINATED NODE   READINESS GATES
default                        sleep                                                         1/1     Running     0          54m     10.88.0.9      node3   <none>           <none>
extension-metrics-server       helm-executor-install-metrics-server-agent-vsxgvd-6kmgl       0/1     Completed   0          15h     10.233.96.56   node2   <none>           <none>
extension-metrics-server       helm-executor-install-metrics-server-n6l6w4-htctg             0/1     Completed   0          15h     10.233.96.55   node2   <none>           <none>
extension-metrics-server       metrics-server-855d5579c-tgs87                                1/1     Running     0          15h     10.233.96.57   node2   <none>           <none>
extension-openpitrix           helm-executor-install-openpitrix-ckvq9h-bkvv6                 0/1     Completed   0          6d14h   10.233.96.3    node2   <none>           <none>
extension-openpitrix           openpitrix-apps-6589b545-vfxg9                                1/1     Running     0          6d14h   10.233.96.4    node2   <none>           <none>
extension-whizard-telemetry    helm-executor-install-whizard-telemetry-4cmcmq-tg8gk          0/1     Completed   0          15h     10.233.96.58   node2   <none>           <none>
extension-whizard-telemetry    whizard-telemetry-apiserver-7b799c59fd-7cw76                  1/1     Running     0          15h     10.233.96.59   node2   <none>           <none>
kube-system                    calico-kube-controllers-568d4f5458-855hv                      1/1     Running     0          6d15h   10.233.90.2    node1   <none>           <none>
kube-system                    calico-node-2k5xn                                             1/1     Running     0          5d20h   192.168.3.21   node2   <none>           <none>
kube-system                    calico-node-pxqpg                                             1/1     Running     0          5d20h   192.168.3.45   node1   <none>           <none>
kube-system                    coredns-549d49586-dfjjj                                       1/1     Running     0          6d15h   10.233.90.1    node1   <none>           <none>
kube-system                    coredns-549d49586-xzngm                                       1/1     Running     0          6d15h   10.233.90.3    node1   <none>           <none>
kube-system                    kube-apiserver-node1                                          1/1     Running     0          6d15h   192.168.3.45   node1   <none>           <none>
kube-system                    kube-controller-manager-node1                                 1/1     Running     0          6d15h   192.168.3.45   node1   <none>           <none>
kube-system                    kube-proxy-2g8pj                                              1/1     Running     0          5d20h   192.168.3.45   node1   <none>           <none>
kube-system                    kube-proxy-b4lcp                                              1/1     Running     0          5d20h   192.168.3.21   node2   <none>           <none>
kube-system                    kube-scheduler-node1                                          1/1     Running     0          6d15h   192.168.3.45   node1   <none>           <none>
kube-system                    nodelocaldns-cdp5r                                            1/1     Running     0          5d20h   192.168.3.21   node2   <none>           <none>
kube-system                    nodelocaldns-fczlw                                            1/1     Running     0          5d20h   192.168.3.45   node1   <none>           <none>
kubeedge                       cloud-iptables-manager-9gpmf                                  1/1     Running     0          5d20h   192.168.3.21   node2   <none>           <none>
kubeedge                       cloud-iptables-manager-hhmhj                                  1/1     Running     0          5d20h   192.168.3.45   node1   <none>           <none>
kubeedge                       cloudcore-58cb9867ff-snjcn                                    1/1     Running     0          5d20h   10.233.96.42   node2   <none>           <none>
kubeedge                       helm-executor-install-kubeedge-jvwr4s-kphn2                   0/1     Completed   0          5d20h   10.233.96.40   node2   <none>           <none>
kubeedge                       kubeedge-proxy-6dbf475855-5qlf6                               1/1     Running     0          5d20h   10.233.96.41   node2   <none>           <none>
kubesphere-monitoring-system   helm-executor-install-whizard-monitoring-agent-mbcnjn-pf92k   0/1     Completed   0          15h     10.233.96.61   node2   <none>           <none>
kubesphere-monitoring-system   helm-executor-install-whizard-monitoring-bj525j-8qpns         0/1     Completed   0          15h     10.233.96.60   node2   <none>           <none>
kubesphere-monitoring-system   kube-state-metrics-7765d4c747-56dcs                           3/3     Running     0          15h     10.233.90.7    node1   <none>           <none>
kubesphere-monitoring-system   node-exporter-cvrgq                                           2/2     Running     0          15h     192.168.3.45   node1   <none>           <none>
kubesphere-monitoring-system   node-exporter-fn7f7                                           2/2     Running     0          15h     192.168.3.21   node2   <none>           <none>
kubesphere-monitoring-system   prometheus-k8s-0                                              2/2     Running     0          14h     10.233.90.13   node1   <none>           <none>
kubesphere-monitoring-system   prometheus-operator-78ddfd6994-mvdwj                          1/1     Running     0          15h     10.233.96.63   node2   <none>           <none>
kubesphere-system              extensions-museum-6c94cb4f5b-ktsm4                            1/1     Running     0          9h      10.233.96.79   node2   <none>           <none>
kubesphere-system              ks-apiserver-5c4f498b4c-jkcx6                                 1/1     Running     0          6d15h   10.233.90.4    node1   <none>           <none>
kubesphere-system              ks-console-6d44547684-rwwpn                                   1/1     Running     0          6d15h   10.233.90.5    node1   <none>           <none>
kubesphere-system              ks-controller-manager-7f69955949-6xmm8                        1/1     Running     0          6d15h   10.233.90.6    node1   <none>           <none>
kubesphere-system              restart-extensions-museum-28929120-m9tqk                      0/1     Completed   0          2d9h    10.233.96.51   node2   <none>           <none>
kubesphere-system              restart-extensions-museum-28930560-s2kjp                      0/1     Completed   0          33h     10.233.96.53   node2   <none>           <none>
kubesphere-system              restart-extensions-museum-28932000-ncfhs                      0/1     Completed   0          9h      10.233.96.78   node2   <none>           <none>
openebs                        openebs-localpv-provisioner-55bd776445-6nfwl                  1/1     Running     0          14h     10.233.96.77   node2   <none>           <none>
openebs                        openebs-ndm-2bmk9                                             1/1     Running     0          14h     192.168.3.21   node2   <none>           <none>
openebs                        openebs-ndm-7rnr2                                             1/1     Running     0          14h     192.168.3.45   node1   <none>           <none>
openebs                        openebs-ndm-cluster-exporter-5d9cd45dc6-2hz9z                 1/1     Running     0          14h     10.233.96.75   node2   <none>           <none>
openebs                        openebs-ndm-node-exporter-7866v                               1/1     Running     0          14h     10.233.96.76   node2   <none>           <none>
openebs                        openebs-ndm-node-exporter-v7xqm                               1/1     Running     0          14h     10.233.90.11   node1   <none>           <none>
openebs                        openebs-ndm-operator-76b7fcf846-42xb5                         1/1     Running     0          14h     10.233.96.74   node2   <none>           <none>

Hhyiyang · 4 1月

另外使用以下脚本避免一些ds再边缘节点运行

#!/bin/bash

NoShedulePatchJson='{"spec":{"template":{"spec":{"affinity":{"nodeAffinity":{"requiredDuringSchedulingIgnoredDuringExecution":{"nodeSelectorTerms":[{"matchExpressions":[{"key":"node-role.kubernetes.io/edge","operator":"DoesNotExist"}]}]}}}}}}}'
ns="kube-system"
DaemonSets=("nodelocaldns" "kube-proxy" "calico-node")
length=${#DaemonSets[@]}
for((i=0;i<length;i++));  
do
    ds=${DaemonSets[$i]}
    echo "Patching resources:DaemonSet/${ds}" in ns:"$ns",
    kubectl -n $ns patch DaemonSet/${ds} --type merge --patch "$NoShedulePatchJson"
    sleep 1
done

Hhyiyang · 4 1月

补充：边缘节点上有监控信息

curl -i http://127.0.0.1:10350/stats/summary
HTTP/1.1 200 OK
Content-Type: application/json
Date: Sat, 04 Jan 2025 03:47:14 GMT
Transfer-Encoding: chunked

{
 "node": {
  "nodeName": "node3",
...
  {
   "podRef": {
    "name": "sleep",
    "namespace": "default",
    "uid": "5788c239-4ccc-4669-a4ed-b2ca21106d87"
   },
   "startTime": "2025-01-04T00:53:23Z",
   "containers": [
    {
     "name": "sleep",
     "startTime": "2025-01-04T00:53:31Z",
     "cpu": {
      "time": "2025-01-04T03:47:14Z",
      "usageNanoCores": 0,
      "usageCoreNanoSeconds": 34868340
     },
     "memory": {
      "time": "2025-01-04T03:47:14Z",
      "workingSetBytes": 720896
     },

Zzuoxuesong · 10 1月

看pod列表，感觉扩展组件安装是不是不太齐全，麻烦贴一下您这个环境中安装了哪些扩展组件以及组件配置吧