• 监控日志

  • 如何修改fluent-bit的配置，使之不采集指定deloyment的日志。

由于我们100多个的pod产生大量日志，产生大量带宽，导致nfs压力很大，而且日志不会常看，所以想把该deloyment的日志采集屏蔽掉。
论坛上有编辑kubectl edit fluentbit -n kubesphere-logging-system fluent-bit的文章，但不知道怎么改才规范，不影响其他服务。

    errorcode7

    exclude pod logs 有两种办法：

    1. Kubernetes filter 插件开启 K8S-Logging.Exclude，并给需要屏蔽日志的 Pod 打上 fluentbit.io/exclude[_stream][-container] 注解：https://docs.fluentbit.io/manual/pipeline/filters/kubernetes#kubernetes-annotations
    2. 在 Tail input 插件里设置 Exclude_Path，排除 Pod 日志目录：https://docs.fluentbit.io/manual/pipeline/inputs/tail

    以上方法需要通过 fluent bit operator CRD 来操作配置文件，fluent bit operator 的项目地址 https://github.com/kubesphere/fluentbit-operator

    方便的话可以整理一个 guide 文档，给 fluentbit-operator 项目提一个 pr，可以在 https://github.com/kubesphere/fluentbit-operator/tree/master/docs 目录下创建一个user-guide 目录

        参考这篇：https://fluentbit.io/documentation/0.13/input/tail.html，添加Exclude_Path
        kubectl edit fluentbit -n kubesphere-logging-system fluent-bit

          input:
  - name: fluentbit-input
    parameters:
    - name: Name
      value: tail
    - name: Path
      value: /var/log/containers/*.log
    - name: Exclude_Path
      value: /var/log/containers/jackfruit810*.log
    - name: Parser
      value: docker

        这里的jackfruit810就是我的deloyment

        hongming 这是 fluent bit operator v0.1.0 的文档，outdated。现在 fluent bit operator 的版本是 v0.2.0

          huanggze

          https://github.com/kubesphere/fluentbit-operator/blob/master/docs/plugins/filter/kubernetes.md

          kubeURL	API Server end-point	string
kubeCAFile	CA certificate file	string
kubeCAPath	Absolute path to scan for certificate files	string
kubeTokenFile	Token file	string
kubeTagPrefix	When the source records comes from Tail input plugin, this option allows to specify what's the prefix used in Tail configuration.	string
mergeLog	When enabled, it checks if the log field content is a JSON string map, if so, it append the map fields as part of the log structure.	*bool
mergeLogKey	When Merge_Log is enabled, the filter tries to assume the log field from the incoming message is a JSON string message and make a structured representation of it at the same level of the log field in the map. Now if Merge_Log_Key is set (a string name), all the new structured fields taken from the original log content are inserted under the new key.	string
mergeLogTrim	When Merge_Log is enabled, trim (remove possible \n or \r) field values.	*bool
mergeParser	Optional parser name to specify how to parse the data contained in the log key. Recommended use is for developers or testing only.	string
keepLog	When Keep_Log is disabled, the log field is removed from the incoming message once it has been successfully merged (Merge_Log must be enabled as well).	*bool
tlsDebug	Debug level between 0 (nothing) and 4 (every detail).	*int32
tlsVerify	When enabled, turns on certificate validation when connecting to the Kubernetes API server.	*bool
useJournal	When enabled, the filter reads logs coming in Journald format.	*bool
regexParser	Set an alternative Parser to process record Tag and extract pod_name, namespace_name, container_name and docker_id. The parser must be registered in a parsers file (refer to parser filter-kube-test as an example).	string
k8sLoggingParser	Allow Kubernetes Pods to suggest a pre-defined Parser (read more about it in Kubernetes Annotations section)	*bool
k8sLoggingExclude	Allow Kubernetes Pods to exclude their logs from the log processor (read more about it in Kubernetes Annotations section).        *bool

          https://github.com/kubesphere/fluentbit-operator/blob/master/manifests/logging-stack/filter-kubernetes.yaml

            - kubernetes:
      kubeURL: https://kubernetes.default.svc:443
      kubeCAFile: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
      kubeTokenFile: /var/run/secrets/kubernetes.io/serviceaccount/token
      labels: false
      annotations: false

          “Kubernetes filter 插件开启 K8S-Logging.Exclude”,我找到以上两个文档，是不是修改filter-kubernetes.yaml，添加k8sLoggingExclude: true后,直接apply就可以了？

              errorcode7

              其实也还好。kubernetes filter 不会一条一条日志的解析，而是按 pod log 目录来决定的。pod目录名里包含 pod 信息，然后fluent bit 请求 kube-apiserver 解析pod 的metadada。只要 pod 没有经常漂移，不需要多次请求 kube-apiserver。

              所以区别只是 filter 可能更灵活一点。

              input 适合集群管理员预先设置 exlude 方案。filter 适合集群普通用户随 workload 动态配置

                errorcode7

                感兴趣可以给 fluent bit operator 项目提 user guide pr，这个项目的 contributor 太少了。。

                  2 年 后

                  想请教一下，如果是涉及几十个、上百个集群都要改这个配置，该如何做比较好呢？如果edit进去改有点太笨了，如果把原来的configmap文件备份，重新以新配置create一个倒是可以，但这样做总觉得不是很规范，不知道是否有更好的配置管理的方法。