ks3.1 如何卸载某一已安装的组件，比如logging

csz711

Cauchy

改成false之后，已经运行的相关工作负载要自己删除下。

yinmin2020

https://kubesphere.com.cn/docs/pluggable-components/logging/
关闭试试？true改成false

csz711

Cauchy
怎么重新安装呢，删除再重启installer 不起作用
（1）删除ns

kubectl delete ns kubesphere-logging-system

（2）logging开关为true
（3）重启ks-installer

kubectl rollout restart deploy ks-installer -n kubesphere-system

kubesphere-logging-system下的pod没有被创建

ssslkj123

csz711
如果删除了某个组件想重新安装的话，需要将 kubesphere-system 命名空间下的clusterconfigurations.installer.kubesphere.io ks-installer配置文件status 有关改服务的status 字段一并删除掉，再次更新clusterconfigurations 就会新安装了。

csz711

@ssslkj123 @Cauchy 删除status重启ks-installer重新安装后，感觉缺少部分pod，查看对应operator日志

bglab@master:~$ kubectl -n kubesphere-logging-system get pod
NAME                                            READY   STATUS    RESTARTS   AGE
elasticsearch-logging-data-0                    1/1     Running   0          42m
elasticsearch-logging-data-1                    1/1     Running   0          41m
elasticsearch-logging-discovery-0               1/1     Running   0          42m
fluentbit-operator-5c8c98d45d-v88hm             1/1     Running   0          6m51s
ks-events-operator-6b7998ddc9-726p9             1/1     Running   0          40m
ks-events-ruler-7b56865f66-c6bm5                2/2     Running   0          39m
ks-events-ruler-7b56865f66-lvsqb                2/2     Running   0          39m
kube-auditing-operator-755b9b8f86-lt2bh         1/1     Running   0          40m
kube-auditing-webhook-deploy-6598d55679-f676z   1/1     Running   0          40m
kube-auditing-webhook-deploy-6598d55679-z7p7w   1/1     Running   0          40m
logsidecar-injector-deploy-649c64cd5-58bjp      2/2     Running   0          49m
logsidecar-injector-deploy-649c64cd5-pwn9h      2/2     Running   0          49m

查看fluentbit-operator日志(缺少fluent-bit的daemonset)
kubectl -n kubesphere-logging-system logs fluentbit-operator-5c8c98d45d-v88hm


2021-06-01T06:25:46.229Z        DEBUG   controller-runtime.controller   Successfully Reconciled {"controller": "fluentbitconfig", "request": "kubesphere-logging-system/fluent-bit-config"}
2021-06-01T06:25:46.232Z        DEBUG   controller-runtime.controller   Successfully Reconciled {"controller": "fluentbitconfig", "request": "kubesphere-logging-system/tail-auditing"}
2021-06-01T06:25:46.233Z        DEBUG   controller-runtime.controller   Successfully Reconciled {"controller": "fluentbitconfig", "request": "kubesphere-logging-system/tail-events"}
2021-06-01T06:25:46.235Z        DEBUG   controller-runtime.controller   Successfully Reconciled {"controller": "fluentbitconfig", "request": "kubesphere-logging-system/tail"}
2021-06-01T06:25:46.241Z        ERROR   controller-runtime.controller   Reconciler error        {"controller": "fluentbitconfig", "request": "kubesphere-logging-system/docker", "error": "Operation cannot be fulfilled on secrets \"fluent-bit-config\": the object has been modified; please apply your changes to the latest version and try again"}
github.com/go-logr/zapr.(*zapLogger).Error
        /go/pkg/mod/github.com/go-logr/zapr@v0.1.0/zapr.go:128
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
        /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.5.2/pkg/internal/controller/controller.go:258
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
        /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.5.2/pkg/internal/controller/controller.go:232
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker
        /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.5.2/pkg/internal/controller/controller.go:211
k8s.io/apimachinery/pkg/util/wait.JitterUntil.func1
        /go/pkg/mod/k8s.io/apimachinery@v0.17.2/pkg/util/wait/wait.go:152
k8s.io/apimachinery/pkg/util/wait.JitterUntil
        /go/pkg/mod/k8s.io/apimachinery@v0.17.2/pkg/util/wait/wait.go:153
k8s.io/apimachinery/pkg/util/wait.Until
        /go/pkg/mod/k8s.io/apimachinery@v0.17.2/pkg/util/wait/wait.go:88

查看kube-auditing-operator日志 kubectl -n kubesphere-logging-system logs kube-auditing-operator-755b9b8f86-lt2bh

2021-06-01T14:21:17.766+0800    ERROR   controller-runtime.controller   Reconciler error        {"controller": "webhook", "request": "/kube-auditing-webhook", "error": "Operation cannot be fulfilled on deployments.apps \"kube-auditing-webhook-deploy\": the object has been modified; please apply your changes to the latest version and try again"}
github.com/go-logr/zapr.(*zapLogger).Error
        /go/pkg/mod/github.com/go-logr/zapr@v0.1.0/zapr.go:128
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
        /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.5.0/pkg/internal/controller/controller.go:258
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
        /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.5.0/pkg/internal/controller/controller.go:232
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker
        /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.5.0/pkg/internal/controller/controller.go:211
k8s.io/apimachinery/pkg/util/wait.JitterUntil.func1
        /go/pkg/mod/k8s.io/apimachinery@v0.17.2/pkg/util/wait/wait.go:152
k8s.io/apimachinery/pkg/util/wait.JitterUntil
        /go/pkg/mod/k8s.io/apimachinery@v0.17.2/pkg/util/wait/wait.go:153
k8s.io/apimachinery/pkg/util/wait.Until
        /go/pkg/mod/k8s.io/apimachinery@v0.17.2/pkg/util/wait/wait.go:88

ERROR controller-runtime.controller Reconciler error {"controller": "webhook", "request": "/kube-auditing-webhook", "error": "Operation cannot be fulfilled on deployments.apps \"kube-auditing-webhook-deploy\": the object has been modified; please apply your changes to the latest version and try again"}这种应该怎么修复

csz711

目前
缺少event-exportor

查看ks-events-operator日志也没看到明显报错

bglab@master:~$ kubectl -n kubesphere-logging-system logs ks-events-operator-6b7998ddc9-pfldf
2021-06-01T16:56:12.486+0800    INFO    controller-runtime.metrics      metrics server is starting to listen    {"addr": ":8080"}
2021-06-01T16:56:12.486+0800    INFO    controller-runtime.builder      Registering a mutating webhook  {"GVK": "events.kubesphere.io/v1alpha1, Kind=Rule", "path": "/mutate-events-kubesphere-io-v1alpha1-rule"}
2021-06-01T16:56:12.486+0800    INFO    controller-runtime.webhook      registering webhook     {"path": "/mutate-events-kubesphere-io-v1alpha1-rule"}
2021-06-01T16:56:12.486+0800    INFO    controller-runtime.builder      Registering a validating webhook        {"GVK": "events.kubesphere.io/v1alpha1, Kind=Rule", "path": "/validate-events-kubesphere-io-v1alpha1-rule"}
2021-06-01T16:56:12.486+0800    INFO    controller-runtime.webhook      registering webhook     {"path": "/validate-events-kubesphere-io-v1alpha1-rule"}
2021-06-01T16:56:12.486+0800    INFO    setup   starting manager
2021-06-01T16:56:12.487+0800    INFO    controller-runtime.webhook.webhooks     starting webhook server
2021-06-01T16:56:12.487+0800    INFO    controller-runtime.manager      starting metrics server {"path": "/metrics"}
2021-06-01T16:56:12.487+0800    INFO    controller-runtime.controller   Starting EventSource    {"controller": "ruler", "source": "kind source: /, Kind="}
2021-06-01T16:56:12.487+0800    INFO    controller-runtime.controller   Starting EventSource    {"controller": "exporter", "source": "kind source: /, Kind="}
2021-06-01T16:56:12.487+0800    INFO    controller-runtime.controller   Starting EventSource    {"controller": "rule", "source": "kind source: /, Kind="}
2021-06-01T16:56:12.487+0800    INFO    controller-runtime.certwatcher  Updated current TLS certificate
2021-06-01T16:56:12.487+0800    INFO    controller-runtime.webhook      serving webhook server  {"host": "", "port": 9443}
2021-06-01T16:56:12.487+0800    INFO    controller-runtime.certwatcher  Starting certificate watcher
2021-06-01T16:56:12.587+0800    INFO    controller-runtime.controller   Starting EventSource    {"controller": "ruler", "source": "kind source: /, Kind="}
2021-06-01T16:56:12.587+0800    INFO    controller-runtime.controller   Starting Controller     {"controller": "rule"}
2021-06-01T16:56:12.588+0800    INFO    controller-runtime.controller   Starting EventSource    {"controller": "exporter", "source": "kind source: /, Kind="}
2021-06-01T16:56:12.687+0800    INFO    controller-runtime.controller   Starting workers        {"controller": "rule", "worker count": 1}
2021-06-01T16:56:12.687+0800    INFO    controller-runtime.controller   Starting EventSource    {"controller": "ruler", "source": "kind source: /, Kind="}
2021-06-01T16:56:12.729+0800    INFO    controller-runtime.controller   Starting EventSource    {"controller": "exporter", "source": "kind source: /, Kind="}
2021-06-01T16:56:12.830+0800    INFO    controller-runtime.controller   Starting EventSource    {"controller": "ruler", "source": "kind source: /, Kind="}
2021-06-01T16:56:13.130+0800    INFO    controller-runtime.controller   Starting EventSource    {"controller": "exporter", "source": "kind source: /, Kind="}
2021-06-01T16:56:13.131+0800    INFO    controller-runtime.controller   Starting EventSource    {"controller": "ruler", "source": "kind source: /, Kind="}
2021-06-01T16:56:13.231+0800    INFO    controller-runtime.controller   Starting EventSource    {"controller": "exporter", "source": "kind source: /, Kind="}
2021-06-01T16:56:13.231+0800    INFO    controller-runtime.controller   Starting EventSource    {"controller": "ruler", "source": "kind source: /, Kind="}
2021-06-01T16:56:13.331+0800    INFO    controller-runtime.controller   Starting EventSource    {"controller": "ruler", "source": "kind source: /, Kind="}
2021-06-01T16:56:13.331+0800    INFO    controller-runtime.controller   Starting EventSource    {"controller": "exporter", "source": "kind source: /, Kind="}
2021-06-01T16:56:13.432+0800    INFO    controller-runtime.controller   Starting Controller     {"controller": "ruler"}
2021-06-01T16:56:13.432+0800    INFO    controller-runtime.controller   Starting workers        {"controller": "ruler", "worker count": 1}
2021-06-01T16:56:13.432+0800    INFO    controller-runtime.controller   Starting Controller     {"controller": "exporter"}
2021-06-01T16:56:13.432+0800    INFO    controller-runtime.controller   Starting workers        {"controller": "exporter", "worker count": 1}
2021-06-01T16:56:13.440+0800    DEBUG   controller-runtime.controller   Successfully Reconciled {"controller": "ruler", "request": "kubesphere-logging-system/ks-events-ruler"}
bglab@csz711#25534 缺少event-exportor

查看ks-events-operator日志也没看到明显报错

wanjunlei

kubectl get fluentbits.logging.kubesphere.io -n kubesphere-logging-system fluent-bit -oyaml
执行这个看看

csz711

wanjunlei 现在fluent-bit组件已经好了，但是ks-event-exportor还没好,日志见上

bglab@master:~$ kubectl -n kubesphere-logging-system get pod
NAME                                                              READY   STATUS      RESTARTS   AGE
elasticsearch-logging-curator-elasticsearch-curator-162256j4v6p   0/1     Completed   0          12h
elasticsearch-logging-data-0                                      1/1     Running     0          19h
elasticsearch-logging-data-1                                      1/1     Running     0          19h
elasticsearch-logging-discovery-0                                 1/1     Running     0          19h
fluent-bit-btvhp                                                  1/1     Running     0          20h
fluent-bit-cpflv                                                  1/1     Running     0          20h
fluent-bit-j4nz7                                                  1/1     Running     0          20h
fluent-bit-nbz58                                                  1/1     Running     0          20h
fluentbit-operator-5c8c98d45d-5hjzn                               1/1     Running     0          21h
ks-events-operator-6b7998ddc9-pfldf                               1/1     Running     0          20h
ks-events-ruler-7b56865f66-c6bm5                                  2/2     Running     0          22h
ks-events-ruler-7b56865f66-lvsqb                                  2/2     Running     0          22h
kube-auditing-operator-755b9b8f86-cmfh2                           1/1     Running     0          20h
kube-auditing-webhook-deploy-7d94d46f77-5dt7q                     1/1     Running     0          20h
kube-auditing-webhook-deploy-7d94d46f77-nzdwg                     1/1     Running     0          20h
logsidecar-injector-deploy-649c64cd5-58bjp                        2/2     Running     0          23h
logsidecar-injector-deploy-649c64cd5-pwn9h                        2/2     Running     0          23h

wanjunlei

执行这个看看
kubectl get exporters.events.kubesphere.io -n kubesphere-logging-system ks-events-exporter -oyaml

csz711

wanjunlei 没有找到
kubectl get exporters.events.kubesphere.io -n kubesphere-logging-system
No resources found in kubesphere-logging-system namespace.

kubectl get exporters.events.kubesphere.io -n kubesphere-logging-system ks-events-exporter -oyaml
Error from server (NotFound): exporters.events.kubesphere.io “ks-events-exporter” not found

bglab@master:~$ kubectl get crd exporters.events.kubesphere.io -o yaml
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  annotations:
    controller-gen.kubebuilder.io/version: v0.2.5
  creationTimestamp: "2020-12-23T06:27:01Z"
  generation: 1
  name: exporters.events.kubesphere.io
  resourceVersion: "11036"
  selfLink: /apis/apiextensions.k8s.io/v1/customresourcedefinitions/exporters.events.kubesphere.io
  uid: bd3d7322-7f11-45a7-a2b6-0648497973df
spec:
  conversion:
    strategy: None
  group: events.kubesphere.io
  names:
    kind: Exporter
    listKind: ExporterList
    plural: exporters
    shortNames:
    - ex
    singular: exporter
  preserveUnknownFields: true
  scope: Namespaced
  versions:
  - name: v1alpha1
    schema:
      openAPIV3Schema:
        description: Exporter is the Schema for the exporter API
        properties:
          apiVersion:
            description: 'APIVersion defines the versioned schema of this representation
              of an object. Servers should convert recognized schemas to the latest
              internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
            type: string
          kind:
            description: 'Kind is a string value representing the REST resource this
              object represents. Servers may infer this from the endpoint the client
              submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
            type: string
          metadata:
            type: object
          spec:
            description: Spec defines the specification of the desired behavior of
              the Exporter.
            properties:
              image:
                description: Docker image of events exporter
                type: string
              imagePullPolicy:
                description: Image pull policy. One of Always, Never, IfNotPresent.
                type: string
              resources:
                description: Resources defines resources requests and limits for single
                  Pod.
                properties:
                  limits:
                    additionalProperties:
                      anyOf:
                      - type: integer
                      - type: string
                      pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                      x-kubernetes-int-or-string: true
                    description: 'Limits describes the maximum amount of compute resources
                      allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
                    type: object
                  requests:
                    additionalProperties:
                      anyOf:
                      - type: integer
                      - type: string
                      pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                      x-kubernetes-int-or-string: true
                    description: 'Requests describes the minimum amount of compute
                      resources required. If Requests is omitted for a container,
                      it defaults to Limits if that is explicitly specified, otherwise
                      to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/'
                    type: object
                type: object
              sinks:
                description: Sinks defines details of events sinks
                properties:
                  stdout:
                    description: Stdout represents whether to write events to stdout.
                      Output when configure an empty struct `{}`, but do nothing when
                      no configuration
                    type: object
                  webhooks:
                    description: Webhooks is a list of ExporterWebhookSink
                    items:
                      description: ExporterWebhookSink defines parameters for webhook
                        sink of Events Exporter.
                      properties:
                        service:
                          description: '`service` is a reference to the service for
                            this webhook. Either `service` or `url` must be specified.
                            If the webhook is running within the cluster, then you
                            should use `service`.'
                          properties:
                            name:
                              description: '`name` is the name of the service.'
                              type: string
                            namespace:
                              description: '`namespace` is the namespace of the service.'
                              type: string
                            path:
                              description: '`path` is an optional URL path which will
                                be sent in any request to this service.'
                              type: string
                            port:
                              description: '`port` is the port on the service and
                                should be a valid port number (1-65535, inclusive).'
                              type: integer
                          required:
                          - name
                          - namespace
                          type: object
                        url:
                          description: '`url` gives the location of the webhook, in
                            standard URL form (`scheme://host:port/path`). Exactly
                            one of `url` or `service` must be specified.'
                          type: string
                      type: object
                    type: array
                type: object
            required:
            - image
            type: object
          status:
            description: ExporterStatus defines the observed state of Exporter
            type: object
        required:
        - spec
        type: object
    served: true
    storage: true
status:
  acceptedNames:
    kind: Exporter
    listKind: ExporterList
    plural: exporters
    shortNames:
    - ex
    singular: exporter
  conditions:
  - lastTransitionTime: "2020-12-23T06:27:01Z"
    message: no conflicts found
    reason: NoConflicts
    status: "True"
    type: NamesAccepted
  - lastTransitionTime: "2020-12-23T06:27:01Z"
    message: the initial names have been accepted
    reason: InitialNamesAccepted
    status: "True"
    type: Established
  storedVersions:
  - v1alpha1
bglab@master:~$

csz711

wanjunlei 知道问题了，是上面CRD太旧了吧，删除CRD重建后成功了。多谢！

以下是重建步骤，若有不对不完善的地方，请指正
1、卸载event组件

bglab@master:~$ helm -n kubesphere-logging-system list
NAME                            NAMESPACE                       REVISION        UPDATED                                 STATUS          CHART
elasticsearch-logging           kubesphere-logging-system       1               2021-06-01 17:19:07.46387536 +0800 CST  deployed        elasticsearch-1.22.1
elasticsearch-logging-curator   kubesphere-logging-system       1               2021-06-01 17:19:10.617829084 +0800 CST deployed        elasticsearch-curator
ks-events                       kubesphere-logging-system       2               2021-06-01 14:21:45.655861263 +0800 CST deployed        kube-events-0.1.0
kube-auditing                   kubesphere-logging-system       1               2021-06-01 17:09:16.097696047 +0800 CST deployed        kube-auditing-0.1.0
logsidecar-injector             kubesphere-logging-system       3               2021-06-01 16:03:43.06877325 +0800 CST  deployed        logsidecar-injector-0
bglab@master:~$
bglab@master:~$ helm -n kubesphere-logging-system uninstall ks-events
release "ks-events" uninstalled
bglab@master:~$
bglab@master:~$ kubectl -n kubesphere-logging-system delete crd exporters.events.kubesphere.io
warning: deleting cluster-scoped resources, not scoped to the provided namespace
customresourcedefinition.apiextensions.k8s.io "exporters.events.kubesphere.io" deleted

2、删除cc中的status中的event信息

bglab@master:~$ kubectl edit cc -n kubesphere-system ks-installer
clusterconfiguration.installer.kubesphere.io/ks-installer edited

3、重启ks-installer

bglab@master:~$ kubectl rollout restart deploy ks-installer -n kubesphere-system
deployment.apps/ks-installer restarted

csz711

![
](https://)

wanjunlei @Cauchy 比较了下新老CRD，发现只有metadata.annotations下的version的差别，spec并无区别，所以上面的报错是由于version引起的？
ERROR controller-runtime.controller Reconciler error {"controller": "webhook", "request": "/kube-auditing-webhook", "error": "Operation cannot be fulfilled on deployments.apps \"kube-auditing-webhook-deploy\": the object has been modified; please apply your changes to the latest version and try again"}

wanjunlei

这个报错不影响的，报错之后会有重试
event explorer没有的原因是ks-events-exporter没有创建，event operator是根据这个资源创建explorer的。