WhizardTelemetry 日志和WhizardTelemetry 事件扩展报错

MMakura · 25 4月

创建部署问题时，请参考下面模板，你提供的信息越多，越容易及时获得解答。如果未按模板创建问题，管理员有权关闭问题。
确保帖子格式清晰易读，用 markdown code block 语法格式化代码块。
你只花一分钟创建的问题，不能指望别人花上半个小时给你解答。

操作系统信息
例如：虚拟机，Centos7.6，4C/16G

Kubernetes版本信息
将 kubectl version 命令执行结果贴在下方

Client Version: v1.26.12

Kustomize Version: v4.5.7

Server Version: v1.26.12

容器运行时
将 docker version / crictl version / nerdctl version 结果贴在下方

Version: 0.1.0

RuntimeName: containerd

RuntimeVersion: v1.7.13

RuntimeApiVersion: v1

KubeSphere版本信息
例如：v2.1.1/v3.0.0。离线安装还是在线安装。在已有K8s上安装还是使用kk安装。

KubeSphere 版本 : v4.1.3，kk离线部署k8s集群和kubesphpere

问题信息

刚接触kubesphere，两个扩展的配置全都是kubesphere默认的，请帮你忙查看一下是什么原因

NullFox · 25 4月

看一下whzard平台服务的配置以及whizard-apiserver的日志

MMakura · 25 4月

api日志

2025-04-25T11:20:56.590291490+08:00 I0425 11:20:56.589917       1 apiserver.go:89] Register /kapis/monitoring.kubesphere.io/v1beta1

2025-04-25T11:20:56.590337420+08:00 I0425 11:20:56.590005       1 apiserver.go:91]     GET /kapis/monitoring.kubesphere.io/v1beta1/cluster_metrics

2025-04-25T11:20:56.590344741+08:00 I0425 11:20:56.590014       1 apiserver.go:91]     GET /kapis/monitoring.kubesphere.io/v1beta1/node_metrics

2025-04-25T11:20:56.590349006+08:00 I0425 11:20:56.590021       1 apiserver.go:91]     GET /kapis/monitoring.kubesphere.io/v1beta1/workspace_metrics

2025-04-25T11:20:56.590352692+08:00 I0425 11:20:56.590027       1 apiserver.go:91]     GET /kapis/monitoring.kubesphere.io/v1beta1/namespace_metrics

2025-04-25T11:20:56.590355916+08:00 I0425 11:20:56.590035       1 apiserver.go:91]     GET /kapis/monitoring.kubesphere.io/v1beta1/workload_metrics

2025-04-25T11:20:56.590359388+08:00 I0425 11:20:56.590042       1 apiserver.go:91]     GET /kapis/monitoring.kubesphere.io/v1beta1/pod_metrics

2025-04-25T11:20:56.590362556+08:00 I0425 11:20:56.590049       1 apiserver.go:91]     GET /kapis/monitoring.kubesphere.io/v1beta1/container_metrics

2025-04-25T11:20:56.590365893+08:00 I0425 11:20:56.590057       1 apiserver.go:91]     GET /kapis/monitoring.kubesphere.io/v1beta1/persistentvolumeclaim_metrics

2025-04-25T11:20:56.590369353+08:00 I0425 11:20:56.590064       1 apiserver.go:91]     GET /kapis/monitoring.kubesphere.io/v1beta1/component_metrics

2025-04-25T11:20:56.590372679+08:00 I0425 11:20:56.590070       1 apiserver.go:91]     GET /kapis/monitoring.kubesphere.io/v1beta1/targets/query

2025-04-25T11:20:56.590375941+08:00 I0425 11:20:56.590076       1 apiserver.go:91]     GET /kapis/monitoring.kubesphere.io/v1beta1/targets/metadata

2025-04-25T11:20:56.590379070+08:00 I0425 11:20:56.590083       1 apiserver.go:91]     GET /kapis/monitoring.kubesphere.io/v1beta1/targets/labelvalues

2025-04-25T11:20:56.590382748+08:00 I0425 11:20:56.590089       1 apiserver.go:91]     GET /kapis/monitoring.kubesphere.io/v1beta1/targets/labelsets

2025-04-25T11:20:56.590387075+08:00 I0425 11:20:56.590096       1 apiserver.go:89] Register /kapis/logging.kubesphere.io/v1alpha2

2025-04-25T11:20:56.590391695+08:00 I0425 11:20:56.590103       1 apiserver.go:91]     GET /kapis/logging.kubesphere.io/v1alpha2/logs

2025-04-25T11:20:56.590395702+08:00 I0425 11:20:56.590109       1 apiserver.go:91]     GET /kapis/logging.kubesphere.io/v1alpha2/events

2025-04-25T11:20:56.590399638+08:00 I0425 11:20:56.590115       1 apiserver.go:89] Register /kapis/notification.kubesphere.io/v2beta2

2025-04-25T11:20:56.590402834+08:00 I0425 11:20:56.590122       1 apiserver.go:91]     POST /kapis/notification.kubesphere.io/v2beta2/verification

2025-04-25T11:20:56.590406124+08:00 I0425 11:20:56.590129       1 apiserver.go:91]     POST /kapis/notification.kubesphere.io/v2beta2/users/{user}/verification

2025-04-25T11:20:56.590409306+08:00 I0425 11:20:56.590137       1 apiserver.go:91]     GET /kapis/notification.kubesphere.io/v2beta2/{resources}

2025-04-25T11:20:56.590412464+08:00 I0425 11:20:56.590143       1 apiserver.go:91]     GET /kapis/notification.kubesphere.io/v2beta2/{resources}/{name}

2025-04-25T11:20:56.590415747+08:00 I0425 11:20:56.590149       1 apiserver.go:91]     POST /kapis/notification.kubesphere.io/v2beta2/{resources}

2025-04-25T11:20:56.590418904+08:00 I0425 11:20:56.590165       1 apiserver.go:91]     PUT /kapis/notification.kubesphere.io/v2beta2/{resources}/{name}

2025-04-25T11:20:56.590421999+08:00 I0425 11:20:56.590173       1 apiserver.go:91]     PATCH /kapis/notification.kubesphere.io/v2beta2/{resources}/{name}

2025-04-25T11:20:56.590425086+08:00 I0425 11:20:56.590180       1 apiserver.go:91]     DELETE /kapis/notification.kubesphere.io/v2beta2/{resources}/{name}

2025-04-25T11:20:56.590428271+08:00 I0425 11:20:56.590187       1 apiserver.go:91]     GET /kapis/notification.kubesphere.io/v2beta2/users/{user}/{resources}

2025-04-25T11:20:56.590435108+08:00 I0425 11:20:56.590194       1 apiserver.go:91]     GET /kapis/notification.kubesphere.io/v2beta2/users/{user}/{resources}/{name}

2025-04-25T11:20:56.590438635+08:00 I0425 11:20:56.590202       1 apiserver.go:91]     POST /kapis/notification.kubesphere.io/v2beta2/users/{user}/{resources}

2025-04-25T11:20:56.590441808+08:00 I0425 11:20:56.590209       1 apiserver.go:91]     PUT /kapis/notification.kubesphere.io/v2beta2/users/{user}/{resources}/{name}

2025-04-25T11:20:56.590445136+08:00 I0425 11:20:56.590215       1 apiserver.go:91]     PATCH /kapis/notification.kubesphere.io/v2beta2/users/{user}/{resources}/{name}

2025-04-25T11:20:56.590448637+08:00 I0425 11:20:56.590222       1 apiserver.go:91]     DELETE /kapis/notification.kubesphere.io/v2beta2/users/{user}/{resources}/{name}

2025-04-25T11:20:56.590497274+08:00 I0425 11:20:56.590290       1 apiserver.go:140] Start listening on :9090

2025-04-25T11:23:52.742595195+08:00 E0425 11:23:52.742453       1 handler.go:67] invalid character 'O' looking for beginning of value

2025-04-25T11:23:52.742663452+08:00 E0425 11:23:52.742503       1 utils.go:50] /workspace/pkg/kapis/logging/v1alpha2/handler.go:68 invalid character 'O' looking for beginning of value

2025-04-25T11:23:52.742668238+08:00 I0425 11:23:52.742545       1 apiserver.go:181] 10.233.106.31 - "GET /kapis/logging.kubesphere.io/v1alpha2/logs?cluster=host&end_time=1745555853&interval=30m&operation=histogram&start_time=1745512653 HTTP/1.1" 500 53 76ms

2025-04-25T11:23:53.037643592+08:00 E0425 11:23:53.037516       1 handler.go:67] invalid character 'O' looking for beginning of value

2025-04-25T11:23:53.037667238+08:00 E0425 11:23:53.037558       1 utils.go:50] /workspace/pkg/kapis/logging/v1alpha2/handler.go:68 invalid character 'O' looking for beginning of value

2025-04-25T11:23:53.037671578+08:00 I0425 11:23:53.037580       1 apiserver.go:181] 10.233.82.59 - "GET /kapis/logging.kubesphere.io/v1alpha2/logs?cluster=host&end_time=1745555853&operation=statistics&start_time=1745510400 HTTP/1.1" 500 53 1570ms

2025-04-25T11:33:06.879069055+08:00 E0425 11:33:06.878912       1 handler.go:67] invalid character 'O' looking for beginning of value

2025-04-25T11:33:06.879110012+08:00 E0425 11:33:06.878939       1 utils.go:50] /workspace/pkg/kapis/logging/v1alpha2/handler.go:68 invalid character 'O' looking for beginning of value

2025-04-25T11:33:06.879115597+08:00 I0425 11:33:06.878963       1 apiserver.go:181] 10.233.82.59 - "GET /kapis/logging.kubesphere.io/v1alpha2/logs?cluster=host&end_time=1745556408&interval=30m&operation=histogram&start_time=1745513208 HTTP/1.1" 500 53 59ms

2025-04-25T11:33:06.887829597+08:00 E0425 11:33:06.887712       1 handler.go:67] invalid character 'O' looking for beginning of value

2025-04-25T11:33:06.887854443+08:00 E0425 11:33:06.887733       1 utils.go:50] /workspace/pkg/kapis/logging/v1alpha2/handler.go:68 invalid character 'O' looking for beginning of value

2025-04-25T11:33:06.887859198+08:00 I0425 11:33:06.887766       1 apiserver.go:181] 10.233.106.31 - "GET /kapis/logging.kubesphere.io/v1alpha2/logs?cluster=host&end_time=1745556409&operation=statistics&start_time=1745510400 HTTP/1.1" 500 53 23ms

2025-04-25T12:56:48.954854520+08:00 E0425 12:56:48.954573       1 handler.go:67] invalid character 'O' looking for beginning of value

2025-04-25T12:56:48.954874629+08:00 E0425 12:56:48.954597       1 utils.go:50] /workspace/pkg/kapis/logging/v1alpha2/handler.go:68 invalid character 'O' looking for beginning of value

2025-04-25T12:56:48.954878926+08:00 I0425 12:56:48.954621       1 apiserver.go:181] 10.233.82.54 - "GET /kapis/logging.kubesphere.io/v1alpha2/logs?cluster=host&end_time=1745561431&interval=30m&operation=histogram&start_time=1745518231 HTTP/1.1" 500 53 23ms

2025-04-25T12:56:48.954882782+08:00 E0425 12:56:48.954641       1 handler.go:67] invalid character 'O' looking for beginning of value

2025-04-25T12:56:48.954897057+08:00 E0425 12:56:48.954662       1 utils.go:50] /workspace/pkg/kapis/logging/v1alpha2/handler.go:68 invalid character 'O' looking for beginning of value

2025-04-25T12:56:48.954900984+08:00 I0425 12:56:48.954686       1 apiserver.go:181] 10.233.106.31 - "GET /kapis/logging.kubesphere.io/v1alpha2/logs?cluster=host&end_time=1745561431&operation=statistics&start_time=1745510400 HTTP/1.1" 500 53 23ms

2025-04-25T12:58:26.394383549+08:00 E0425 12:58:26.394240       1 handler.go:67] invalid character 'O' looking for beginning of value

2025-04-25T12:58:26.394410340+08:00 E0425 12:58:26.394262       1 utils.go:50] /workspace/pkg/kapis/logging/v1alpha2/handler.go:68 invalid character 'O' looking for beginning of value

2025-04-25T12:58:26.394415083+08:00 I0425 12:58:26.394283       1 apiserver.go:181] 10.233.106.31 - "GET /kapis/logging.kubesphere.io/v1alpha2/logs?cluster=host&end_time=1745561528&interval=30m&operation=histogram&start_time=1745518328 HTTP/1.1" 500 53 26ms

2025-04-25T12:58:27.384214087+08:00 E0425 12:58:27.384093       1 handler.go:67] invalid character 'O' looking for beginning of value

2025-04-25T12:58:27.384244673+08:00 E0425 12:58:27.384113       1 utils.go:50] /workspace/pkg/kapis/logging/v1alpha2/handler.go:68 invalid character 'O' looking for beginning of value

2025-04-25T12:58:27.384249322+08:00 I0425 12:58:27.384132       1 apiserver.go:181] 10.233.82.59 - "GET /kapis/logging.kubesphere.io/v1alpha2/logs?cluster=host&end_time=1745561528&operation=statistics&start_time=1745510400 HTTP/1.1" 500 53 10ms

2025-04-25T13:03:17.877609941+08:00 E0425 13:03:17.877446       1 handler.go:67] invalid character 'O' looking for beginning of value

2025-04-25T13:03:17.877643151+08:00 E0425 13:03:17.877477       1 utils.go:50] /workspace/pkg/kapis/logging/v1alpha2/handler.go:68 invalid character 'O' looking for beginning of value

2025-04-25T13:03:17.877647360+08:00 I0425 13:03:17.877498       1 apiserver.go:181] 10.233.82.54 - "GET /kapis/logging.kubesphere.io/v1alpha2/logs?cluster=host&end_time=1745561820&operation=statistics&start_time=1745510400 HTTP/1.1" 500 53 23ms

2025-04-25T13:03:17.960381815+08:00 E0425 13:03:17.960258       1 handler.go:67] invalid character 'O' looking for beginning of value

2025-04-25T13:03:17.960412555+08:00 E0425 13:03:17.960299       1 utils.go:50] /workspace/pkg/kapis/logging/v1alpha2/handler.go:68 invalid character 'O' looking for beginning of value

2025-04-25T13:03:17.960464399+08:00 I0425 13:03:17.960331       1 apiserver.go:181] 10.233.82.59 - "GET /kapis/logging.kubesphere.io/v1alpha2/logs?cluster=host&end_time=1745561820&interval=30m&operation=histogram&start_time=1745518620 HTTP/1.1" 500 53 11ms

2025-04-25T13:04:36.942655831+08:00 E0425 13:04:36.942275       1 handler.go:67] invalid character 'O' looking for beginning of value

2025-04-25T13:04:36.942679623+08:00 E0425 13:04:36.942293       1 utils.go:50] /workspace/pkg/kapis/logging/v1alpha2/handler.go:68 invalid character 'O' looking for beginning of value

2025-04-25T13:04:36.942684065+08:00 I0425 13:04:36.942310       1 apiserver.go:181] 10.233.106.31 - "GET /kapis/logging.kubesphere.io/v1alpha2/logs?cluster=host&end_time=1745561898&interval=30m&operation=histogram&start_time=1745518698 HTTP/1.1" 500 53 14ms

2025-04-25T13:04:36.943560025+08:00 E0425 13:04:36.943413       1 handler.go:67] invalid character 'O' looking for beginning of value

2025-04-25T13:04:36.943589073+08:00 E0425 13:04:36.943426       1 utils.go:50] /workspace/pkg/kapis/logging/v1alpha2/handler.go:68 invalid character 'O' looking for beginning of value

2025-04-25T13:04:36.943592995+08:00 I0425 13:04:36.943441       1 apiserver.go:181] 10.233.82.54 - "GET /kapis/logging.kubesphere.io/v1alpha2/logs?cluster=host&end_time=1745561899&operation=statistics&start_time=1745510400 HTTP/1.1" 500 53 16ms

2025-04-25T13:06:06.968872764+08:00 E0425 13:06:06.968736       1 handler.go:67] invalid character 'O' looking for beginning of value

2025-04-25T13:06:06.968903495+08:00 E0425 13:06:06.968768       1 utils.go:50] /workspace/pkg/kapis/logging/v1alpha2/handler.go:68 invalid character 'O' looking for beginning of value

2025-04-25T13:06:06.968908276+08:00 I0425 13:06:06.968794       1 apiserver.go:181] 10.233.106.31 - "GET /kapis/logging.kubesphere.io/v1alpha2/logs?cluster=host&end_time=1745561986&operation=statistics&start_time=1745510400 HTTP/1.1" 500 53 3026ms

2025-04-25T13:06:48.632849858+08:00 E0425 13:06:48.632682       1 handler.go:67] invalid character 'O' looking for beginning of value

2025-04-25T13:06:48.632875889+08:00 E0425 13:06:48.632702       1 utils.go:50] /workspace/pkg/kapis/logging/v1alpha2/handler.go:68 invalid character 'O' looking for beginning of value

2025-04-25T13:06:48.632880078+08:00 I0425 13:06:48.632720       1 apiserver.go:181] 10.233.82.59 - "GET /kapis/logging.kubesphere.io/v1alpha2/logs?cluster=host&end_time=1745562030&interval=30m&operation=histogram&start_time=1745518830 HTTP/1.1" 500 53 12ms

2025-04-25T13:06:48.635987820+08:00 E0425 13:06:48.635904       1 handler.go:67] invalid character 'O' looking for beginning of value

2025-04-25T13:06:48.636006745+08:00 E0425 13:06:48.635926       1 utils.go:50] /workspace/pkg/kapis/logging/v1alpha2/handler.go:68 invalid character 'O' looking for beginning of value

2025-04-25T13:06:48.636020939+08:00 I0425 13:06:48.635944       1 apiserver.go:181] 10.233.106.31 - "GET /kapis/logging.kubesphere.io/v1alpha2/logs?cluster=host&end_time=1745562031&operation=statistics&start_time=1745510400 HTTP/1.1" 500 53 12ms

2025-04-25T13:19:16.778036700+08:00 E0425 13:19:16.777909       1 handler.go:123] invalid character 'O' looking for beginning of value

2025-04-25T13:19:16.778064151+08:00 E0425 13:19:16.777937       1 utils.go:50] /workspace/pkg/kapis/logging/v1alpha2/handler.go:124 invalid character 'O' looking for beginning of value

2025-04-25T13:19:16.778068329+08:00 I0425 13:19:16.777964       1 apiserver.go:181] 10.233.82.59 - "GET /kapis/logging.kubesphere.io/v1alpha2/events?cluster=host&end_time=1745562780&interval=30m&operation=statistics&start_time=1745510400 HTTP/1.1" 500 53 33ms

2025-04-25T13:19:16.813589076+08:00 E0425 13:19:16.812331       1 handler.go:123] invalid character 'O' looking for beginning of value

2025-04-25T13:19:16.813606319+08:00 E0425 13:19:16.812388       1 utils.go:50] /workspace/pkg/kapis/logging/v1alpha2/handler.go:124 invalid character 'O' looking for beginning of value

2025-04-25T13:19:16.813610825+08:00 I0425 13:19:16.812413       1 apiserver.go:181] 10.233.82.54 - "GET /kapis/logging.kubesphere.io/v1alpha2/events?cluster=host&end_time=1745562780&interval=30m&operation=histogram&start_time=1745519580 HTTP/1.1" 500 53 18ms

2025-04-25T13:21:35.383856092+08:00 E0425 13:21:35.383724       1 handler.go:67] invalid character 'O' looking for beginning of value

2025-04-25T13:21:35.383888047+08:00 E0425 13:21:35.383759       1 utils.go:50] /workspace/pkg/kapis/logging/v1alpha2/handler.go:68 invalid character 'O' looking for beginning of value

2025-04-25T13:21:35.383892285+08:00 I0425 13:21:35.383785       1 apiserver.go:181] 10.233.82.54 - "GET /kapis/logging.kubesphere.io/v1alpha2/logs?cluster=host&end_time=1745562916&interval=30m&operation=histogram&start_time=1745519716 HTTP/1.1" 500 53 255ms

2025-04-25T13:21:38.043126404+08:00 E0425 13:21:38.042971       1 handler.go:67] invalid character 'O' looking for beginning of value

2025-04-25T13:21:38.043168110+08:00 E0425 13:21:38.042999       1 utils.go:50] /workspace/pkg/kapis/logging/v1alpha2/handler.go:68 invalid character 'O' looking for beginning of value

2025-04-25T13:21:38.043173828+08:00 I0425 13:21:38.043037       1 apiserver.go:181] 10.233.106.31 - "GET /kapis/logging.kubesphere.io/v1alpha2/logs?cluster=host&end_time=1745562917&operation=statistics&start_time=1745510400 HTTP/1.1" 500 53 22ms

2025-04-25T13:23:09.261200161+08:00 E0425 13:23:09.261044       1 handler.go:123] invalid character 'O' looking for beginning of value

2025-04-25T13:23:09.261241519+08:00 E0425 13:23:09.261072       1 utils.go:50] /workspace/pkg/kapis/logging/v1alpha2/handler.go:124 invalid character 'O' looking for beginning of value

2025-04-25T13:23:09.261246059+08:00 I0425 13:23:09.261103       1 apiserver.go:181] 10.233.106.31 - "GET /kapis/logging.kubesphere.io/v1alpha2/events?cluster=host&end_time=1745563012&interval=30m&operation=histogram&start_time=1745519812 HTTP/1.1" 500 53 32ms

2025-04-25T13:23:09.346432432+08:00 E0425 13:23:09.346270       1 handler.go:123] invalid character 'O' looking for beginning of value

2025-04-25T13:23:09.346475944+08:00 E0425 13:23:09.346301       1 utils.go:50] /workspace/pkg/kapis/logging/v1alpha2/handler.go:124 invalid character 'O' looking for beginning of value

2025-04-25T13:23:09.346480588+08:00 I0425 13:23:09.346327       1 apiserver.go:181] 10.233.82.59 - "GET /kapis/logging.kubesphere.io/v1alpha2/events?cluster=host&end_time=1745563012&interval=30m&operation=statistics&start_time=1745510400 HTTP/1.1" 500 53 172ms

2025-04-25T13:24:19.297564734+08:00 E0425 13:24:19.297384       1 handler.go:123] invalid character 'O' looking for beginning of value

2025-04-25T13:24:19.297599805+08:00 E0425 13:24:19.297410       1 utils.go:50] /workspace/pkg/kapis/logging/v1alpha2/handler.go:124 invalid character 'O' looking for beginning of value

2025-04-25T13:24:19.297604108+08:00 I0425 13:24:19.297436       1 apiserver.go:181] 10.233.82.54 - "GET /kapis/logging.kubesphere.io/v1alpha2/events?cluster=host&end_time=1745563012&interval=30m&operation=histogram&start_time=1745519812 HTTP/1.1" 500 53 26ms

平台服务配置

global:
  ## Global image registry to use if it needs to be overriden for some specific use cases (e.g local registries, custom images, ...)
  ##
  imageRegistry: ""

  ## Reference to one or more secrets to be used when pulling images
  ## ref: https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/
  ##
  imagePullSecrets: []
  # - name: "image-pull-secret"
  # or
  # - "image-pull-secret"
  nodeSelector: {}

whizard-telemetry:
  config:
    monitoring:
      enabled: true
      kind: 0
      endpoint: http://prometheus-k8s.kubesphere-monitoring-system.svc:9090
    notification:
      endpoint: http://notification-manager-svc.kubesphere-monitoring-system.svc:19093
    events:
      enable: true
      servers:
        - elasticsearch:
            endpoints:
              - https://opensearch-cluster-data.kubesphere-logging-system:9200
            version: opensearchv2
            indexPrefix: "{{ .cluster }}-events"
            timestring: "%Y.%m.%d"
            basicAuth: true
            username: admin
            password: admin
    logging:
      enable: true
      servers:
        - elasticsearch:
            endpoints:
              - https://opensearch-cluster-data.kubesphere-logging-system:9200
            version: opensearchv2
            indexPrefix: "{{ .cluster }}-logs"
            timestring: "%Y.%m.%d"
            basicAuth: true
            username: admin
            password: admin

  apiserver:
    image:
      repository: kubesphere/whizard-telemetry-apiserver
      pullPolicy: IfNotPresent
      # Overrides the image tag whose default is the chart appVersion.
      tag: "v1.2.2"

    nodeSelector: {}

    tolerations: []

    affinity: {}

MMakura · 25 4月

NullFox 日志和配置文件贴在楼下了

NullFox · 25 4月

opensearch正常吗？日志看一下

MMakura · 25 4月

NullFox 好像问题在opensearch

MMakura · 25 4月

NullFox 请教一下，我尝试重新安装opensearch结果安装失败

2025-04-25T15:28:13.809373737+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled
2025-04-25T15:28:15.810739675+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled
2025-04-25T15:28:17.808883759+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled
2025-04-25T15:28:19.809292086+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled
2025-04-25T15:28:21.811504797+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled
2025-04-25T15:28:23.810236822+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled
2025-04-25T15:28:25.809939942+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled
2025-04-25T15:28:27.808771885+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled
2025-04-25T15:28:29.808870780+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled
2025-04-25T15:28:31.808531355+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled
2025-04-25T15:28:33.810082556+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled
2025-04-25T15:28:35.810096353+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled
2025-04-25T15:28:37.811041245+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled
2025-04-25T15:28:39.808540154+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled
2025-04-25T15:28:41.811042702+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled
2025-04-25T15:28:43.810537852+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled
2025-04-25T15:28:45.810087192+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled
2025-04-25T15:28:47.809284674+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled
2025-04-25T15:28:49.812400539+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled
2025-04-25T15:28:51.809305359+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled
2025-04-25T15:28:53.811136643+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled
2025-04-25T15:28:55.809270366+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled
2025-04-25T15:28:57.810590479+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled
2025-04-25T15:28:59.810090247+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled
2025-04-25T15:29:01.809285927+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled
2025-04-25T15:29:03.810522444+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled
2025-04-25T15:29:05.809322866+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled
2025-04-25T15:29:07.809153102+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled
2025-04-25T15:29:09.813379598+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled
2025-04-25T15:29:11.809248740+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled
2025-04-25T15:29:13.810023635+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled
2025-04-25T15:29:15.810388613+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled
2025-04-25T15:29:17.809356115+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled
2025-04-25T15:29:19.810139552+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled
2025-04-25T15:29:21.808399447+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled
2025-04-25T15:29:23.808911834+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled
2025-04-25T15:29:25.808947030+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled
2025-04-25T15:29:27.808895344+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled
2025-04-25T15:29:29.812005069+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled
2025-04-25T15:29:31.809112887+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled
2025-04-25T15:29:33.810863500+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled
2025-04-25T15:29:35.809021044+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled
2025-04-25T15:29:37.811441425+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled
2025-04-25T15:29:39.809687758+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled
2025-04-25T15:29:41.809753795+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled
2025-04-25T15:29:43.808805608+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled
2025-04-25T15:29:45.810527671+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled
2025-04-25T15:29:47.810332469+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled
2025-04-25T15:29:49.813276541+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled
2025-04-25T15:29:51.809304330+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled
2025-04-25T15:29:53.810537623+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled
2025-04-25T15:29:55.809515210+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled
2025-04-25T15:29:57.813938012+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled
2025-04-25T15:29:59.810708819+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled
2025-04-25T15:30:01.810410600+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled
2025-04-25T15:30:03.810704184+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled
2025-04-25T15:30:05.815336968+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled
2025-04-25T15:30:07.809454623+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled
2025-04-25T15:30:09.809976303+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled
2025-04-25T15:30:11.808231680+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled
2025-04-25T15:30:13.807836433+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled
2025-04-25T15:30:15.813280965+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled
2025-04-25T15:30:17.810428207+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled
2025-04-25T15:30:19.809896130+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled
2025-04-25T15:30:21.810019205+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled
2025-04-25T15:30:23.810779667+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled
2025-04-25T15:30:25.814089458+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled
2025-04-25T15:30:27.813491988+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled
2025-04-25T15:30:29.810017730+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled
2025-04-25T15:30:31.815886078+08:00 ready.go:420: [debug] StatefulSet is not ready: kubesphere-logging-system/opensearch-cluster-data. 0 out of 3 expected pods have been scheduled

配置为扩展默认配置，请问该如何排查