之前日志收集都很正常,从11点开始日志查询发现一直没有日志进来,但是终端看是有日志的
这是fluentbit-operator容器日志
2021-05-27T12:26:20.836Z INFO controller-runtime.metrics metrics server is starting to listen {“addr”: “:8080”}
2021-05-27T12:26:20.836Z INFO setup starting manager
2021-05-27T12:26:20.836Z INFO controller-runtime.manager starting metrics server {“path”: “/metrics”}
2021-05-27T12:26:22.337Z INFO controller-runtime.controller Starting EventSource {“controller”: “fluentbitconfig”, “source”: “kind source: /, Kind=”}
2021-05-27T12:26:22.337Z INFO controller-runtime.controller Starting EventSource {“controller”: “fluentbit”, “source”: “kind source: /, Kind=”}
2021-05-27T12:26:22.437Z INFO controller-runtime.controller Starting EventSource {“controller”: “fluentbitconfig”, “source”: “kind source: /, Kind=”}
2021-05-27T12:26:22.438Z INFO controller-runtime.controller Starting EventSource {“controller”: “fluentbit”, “source”: “kind source: /, Kind=”}
2021-05-27T12:26:22.438Z INFO controller-runtime.controller Starting EventSource {“controller”: “fluentbit”, “source”: “kind source: /, Kind=”}
2021-05-27T12:26:22.438Z INFO controller-runtime.controller Starting Controller {“controller”: “fluentbit”}
2021-05-27T12:26:22.438Z INFO controller-runtime.controller Starting EventSource {“controller”: “fluentbitconfig”, “source”: “kind source: /, Kind=”}
2021-05-27T12:26:22.438Z INFO controller-runtime.controller Starting workers {“controller”: “fluentbit”, “worker count”: 1}
2021-05-27T12:26:22.538Z INFO controller-runtime.controller Starting EventSource {“controller”: “fluentbitconfig”, “source”: “kind source: /, Kind=”}
2021-05-27T12:26:22.639Z INFO controller-runtime.controller Starting EventSource {“controller”: “fluentbitconfig”, “source”: “kind source: /, Kind=”}
2021-05-27T12:26:22.736Z DEBUG controller-runtime.controller Successfully Reconciled {“controller”: “fluentbit”, “request”: “kubesphere-logging-system/fluent-bit”}
2021-05-27T12:26:22.739Z INFO controller-runtime.controller Starting EventSource {“controller”: “fluentbitconfig”, “source”: “kind source: /, Kind=”}
2021-05-27T12:26:22.839Z INFO controller-runtime.controller Starting Controller {“controller”: “fluentbitconfig”}
2021-05-27T12:26:22.839Z INFO controller-runtime.controller Starting workers {“controller”: “fluentbitconfig”, “worker count”: 1}
2021-05-27T12:26:23.146Z DEBUG controller-runtime.controller Successfully Reconciled {“controller”: “fluentbitconfig”, “request”: “kubesphere-logging-system/fluent-bit-config”}
2021-05-27T12:26:23.150Z DEBUG controller-runtime.controller Successfully Reconciled {“controller”: “fluentbitconfig”, “request”: “kubesphere-logging-system/tail-auditing”}
2021-05-27T12:26:23.154Z ERROR controller-runtime.controller Reconciler error {“controller”: “fluentbitconfig”, “request”: “kubesphere-logging-system/tail-events”, “error”: "Operation cannot be fulfilled on secrets \“fluent-bit-config\”: the object has been modified; please apply your changes to the latest version and try again"}
github.com/go-logr/zapr.(*zapLogger).Error
/go/pkg/mod/github.com/go-logr/zapr@v0.1.0/zapr.go:128
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.5.2/pkg/internal/controller/controller.go:258
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.5.2/pkg/internal/controller/controller.go:232
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker
/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.5.2/pkg/internal/controller/controller.go:211
k8s.io/apimachinery/pkg/util/wait.JitterUntil.func1
/go/pkg/mod/k8s.io/apimachinery@v0.17.2/pkg/util/wait/wait.go:152
k8s.io/apimachinery/pkg/util/wait.JitterUntil
/go/pkg/mod/k8s.io/apimachinery@v0.17.2/pkg/util/wait/wait.go:153
k8s.io/apimachinery/pkg/util/wait.Until
/go/pkg/mod/k8s.io/apimachinery@v0.17.2/pkg/util/wait/wait.go:88
2021-05-27T12:26:24.160Z DEBUG controller-runtime.controller Successfully Reconciled {“controller”: “fluentbitconfig”, “request”: “kubesphere-logging-system/docker”}
2021-05-27T12:26:24.163Z ERROR controller-runtime.controller Reconciler error {“controller”: “fluentbitconfig”, “request”: “kubesphere-logging-system/kubelet”, “error”: "Operation cannot be fulfilled on secrets \“fluent-bit-config\”: the object has been modified; please apply your changes to the latest version and try again"}
github.com/go-logr/zapr.(*zapLogger).Error
/go/pkg/mod/github.com/go-logr/zapr@v0.1.0/zapr.go:128
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.5.2/pkg/internal/controller/controller.go:258
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.5.2/pkg/internal/controller/controller.go:232
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker
/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.5.2/pkg/internal/controller/controller.go:211
k8s.io/apimachinery/pkg/util/wait.JitterUntil.func1
/go/pkg/mod/k8s.io/apimachinery@v0.17.2/pkg/util/wait/wait.go:152
k8s.io/apimachinery/pkg/util/wait.JitterUntil
/go/pkg/mod/k8s.io/apimachinery@v0.17.2/pkg/util/wait/wait.go:153
k8s.io/apimachinery/pkg/util/wait.Until
/go/pkg/mod/k8s.io/apimachinery@v0.17.2/pkg/util/wait/wait.go:88
2021-05-27T12:26:25.200Z DEBUG controller-runtime.controller Successfully Reconciled {“controller”: “fluentbitconfig”, “request”: “kubesphere-logging-system/tail”}
2021-05-27T12:26:25.239Z DEBUG controller-runtime.controller Successfully Reconciled {“controller”: “fluentbitconfig”, “request”: “kubesphere-logging-system/filter-events”}
2021-05-27T12:26:25.244Z DEBUG controller-runtime.controller Successfully Reconciled {“controller”: “fluentbitconfig”, “request”: “kubesphere-logging-system/kubernetes”}
2021-05-27T12:26:25.339Z DEBUG controller-runtime.controller Successfully Reconciled {“controller”: “fluentbitconfig”, “request”: “kubesphere-logging-system/systemd”}
2021-05-27T12:26:25.343Z DEBUG controller-runtime.controller Successfully Reconciled {“controller”: “fluentbitconfig”, “request”: “kubesphere-logging-system/filter-auditing”}
2021-05-27T12:26:25.347Z DEBUG controller-runtime.controller Successfully Reconciled {“controller”: “fluentbitconfig”, “request”: “kubesphere-logging-system/es-auditing”}
2021-05-27T12:26:25.351Z ERROR controller-runtime.controller Reconciler error {“controller”: “fluentbitconfig”, “request”: “kubesphere-logging-system/es-events”, “error”: "Operation cannot be fulfilled on secrets \“fluent-bit-config\”: the object has been modified; please apply your changes to the latest version and try again"}
github.com/go-logr/zapr.(*zapLogger).Error
/go/pkg/mod/github.com/go-logr/zapr@v0.1.0/zapr.go:128
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.5.2/pkg/internal/controller/controller.go:258
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.5.2/pkg/internal/controller/controller.go:232
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker
/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.5.2/pkg/internal/controller/controller.go:211
k8s.io/apimachinery/pkg/util/wait.JitterUntil.func1
/go/pkg/mod/k8s.io/apimachinery@v0.17.2/pkg/util/wait/wait.go:152
k8s.io/apimachinery/pkg/util/wait.JitterUntil
/go/pkg/mod/k8s.io/apimachinery@v0.17.2/pkg/util/wait/wait.go:153
k8s.io/apimachinery/pkg/util/wait.Until
/go/pkg/mod/k8s.io/apimachinery@v0.17.2/pkg/util/wait/wait.go:88
2021-05-27T12:26:26.365Z DEBUG controller-runtime.controller Successfully Reconciled {“controller”: “fluentbitconfig”, “request”: “kubesphere-logging-system/es”}
2021-05-27T12:26:26.371Z DEBUG controller-runtime.controller Successfully Reconciled {“controller”: “fluentbitconfig”, “request”: “kubesphere-logging-system/fluent-bit-config”}
2021-05-27T12:26:26.376Z DEBUG controller-runtime.controller Successfully Reconciled {“controller”: “fluentbitconfig”, “request”: “kubesphere-logging-system/tail-events”}
2021-05-27T12:26:26.380Z DEBUG controller-runtime.controller Successfully Reconciled {“controller”: “fluentbitconfig”, “request”: “kubesphere-logging-system/kubelet”}
2021-05-27T12:26:26.385Z DEBUG controller-runtime.controller Successfully Reconciled {“controller”: “fluentbitconfig”, “request”: “kubesphere-logging-system/es-events”}
2021-05-27T12:26:26.441Z DEBUG controller-runtime.controller Successfully Reconciled {“controller”: “fluentbitconfig”, “request”: “kubesphere-logging-system/fluent-bit-config”}
2021-05-27T12:26:26.446Z DEBUG controller-runtime.controller Successfully Reconciled {“controller”: “fluentbitconfig”, “request”: “kubesphere-logging-system/fluent-bit-config”}
2021-05-27T12:26:26.451Z DEBUG controller-runtime.controller Successfully Reconciled {“controller”: “fluentbitconfig”, “request”: “kubesphere-logging-system/fluent-bit-config”}
2021-05-27T12:26:26.456Z DEBUG controller-runtime.controller Successfully Reconciled {“controller”: “fluentbitconfig”, “request”: “kubesphere-logging-system/fluent-bit-config”}
2021-05-27T12:26:26.460Z DEBUG controller-runtime.controller Successfully Reconciled {“controller”: “fluentbitconfig”, “request”: “kubesphere-logging-system/fluent-bit-config”}
2021-05-27T12:26:26.464Z DEBUG controller-runtime.controller Successfully Reconciled {“controller”: “fluentbitconfig”, “request”: “kubesphere-logging-system/fluent-bit-config”}
2021-05-27T12:26:26.483Z DEBUG controller-runtime.controller Successfully Reconciled {“controller”: “fluentbitconfig”, “request”: “kubesphere-logging-system/fluent-bit-config”}
2021-05-27T12:26:26.492Z DEBUG controller-runtime.controller Successfully Reconciled {“controller”: “fluentbitconfig”, “request”: “kubesphere-logging-system/fluent-bit-config”}
2021-05-27T12:26:26.533Z DEBUG controller-runtime.controller Successfully Reconciled {“controller”: “fluentbitconfig”, “request”: “kubesphere-logging-system/fluent-bit-config”}
2021-05-27T12:26:26.539Z DEBUG controller-runtime.controller Successfully Reconciled {“controller”: “fluentbitconfig”, “request”: “kubesphere-logging-system/fluent-bit-config”}
2021-05-27T12:26:26.543Z DEBUG controller-runtime.controller Successfully Reconciled {“controller”: “fluentbitconfig”, “request”: “kubesphere-logging-system/fluent-bit-config”}
2021-05-27T12:26:26.548Z DEBUG controller-runtime.controller Successfully Reconciled {“controller”: “fluentbitconfig”, “request”: “kubesphere-logging-system/fluent-bit-config”}
2021-05-27T12:26:26.553Z DEBUG controller-runtime.controller Successfully Reconciled {“controller”: “fluentbitconfig”, “request”: “kubesphere-logging-system/fluent-bit-config”}
2021-05-27T12:26:26.558Z DEBUG controller-runtime.controller Successfully Reconciled {“controller”: “fluentbitconfig”, “request”: “kubesphere-logging-system/fluent-bit-config”}
2021-05-27T12:26:26.633Z DEBUG controller-runtime.controller Successfully Reconciled {“controller”: “fluentbitconfig”, “request”: “kubesphere-logging-system/fluent-bit-config”}
2021-05-27T12:26:26.639Z DEBUG controller-runtime.controller Successfully Reconciled {“controller”: “fluentbitconfig”, “request”: “kubesphere-logging-system/fluent-bit-config”}
2021-05-27T12:26:26.644Z DEBUG controller-runtime.controller Successfully Reconciled {“controller”: “fluentbitconfig”, “request”: “kubesphere-logging-system/fluent-bit-config”}
2021-05-27T12:26:26.649Z DEBUG controller-runtime.controller Successfully Reconciled {“controller”: “fluentbitconfig”, “request”: “kubesphere-logging-system/fluent-bit-config”}
2021-05-27T12:26:26.654Z DEBUG controller-runtime.controller Successfully Reconciled {“controller”: “fluentbitconfig”, “request”: “kubesphere-logging-system/fluent-bit-config”}
2021-05-27T12:26:26.735Z DEBUG controller-runtime.controller Successfully Reconciled {“controller”: “fluentbitconfig”, “request”: “kubesphere-logging-system/fluent-bit-config”}
我删除fluentbit-operator容器也是不行