v3.0.0环境不显示日志

wanjunlei

这个secret就是fluent bit的配置文件，你把上面的数据用base64解码看一下

qgr

wanjunlei 解码后是这样的
[Service]
Parsers_File parsers.conf
[Input]
Name tail
Path /var/log/containers/.log
Exclude_Path /var/log/containers/kubesphere-logging-system_events-exporter.log,/var/log/containers/kube-auditing-webhookkubesphere-logging-system_kube-auditing-webhook.log
Refresh_Interval 10
Skip_Long_Lines true
DB /fluent-bit/tail/pos.db
DB.Sync Normal
Mem_Buf_Limit 5MB
Parser docker
Tag kube.
[Filter]
Name kubernetes
Match kube.*
Kube_URL https://kubernetes.default.svc:443
Kube_CA_File /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
Kube_Token_File /var/run/secrets/kubernetes.io/serviceaccount/token
Labels false
Annotations false
[Filter]
Name nest
Match kube.*
Operation lift
Nested_under kubernetes
Add_prefix kubernetes_
[Filter]
Name modify
Match kube.*
Remove stream
Remove kubernetes_pod_id
Remove kubernetes_host
Remove kubernetes_container_hash
[Filter]
Name nest
Match kube.*
Operation nest
Wildcard kubernetes*
Nest_under kubernetes
Remove_prefix kubernetes
[Output]
Name es
Match kube.*
Host elasticsearch-logging-data.kubesphere-logging-system.svc
Port 9200
Logstash_Format true
Logstash_Prefix ks-logstash-log
Time_Key @timestamp

wanjunlei

kubectl get fluentbits.logging.kubesphere.io -n kubesphere-logging-system -oyaml
看下fluentbit的镜像版本

qgr

wanjunlei
啥也没有。
这个镜像是我自己编的，基于你们v1.4.6分支编的，因为我是在国产龙芯平台上安装kubesphere

wanjunlei

qgr 你这个镜像编译的有问题，你试试kubespheredev/fluent-bit:v1.6.9,这个支持arm64

qgr

wanjunlei 好的，那我先用这个分支编下，多谢啦，有问题再请教您

qgr

wanjunlei https://kubesphere.com.cn/forum/d/3500-v300/2 另外这个问题帮忙看下呗，这个问题很奇怪，我用你们master分支编的ks-installer镜像安装就可以看的监控数据，用v3.0.0分支编的ks-installer镜像安装就没有监控数据。我检查了clusterrole之类的，也没有权限丢失，所以不知道是不是还有啥配置的问题。

qgr

wanjunlei kubespheredev/fluent-bit:v1.6.9这个是基于哪个分支编的呢，我看你们的fluent-bit仓库没有1.6.9分支，我这个平台是mips的，arm的用不了，只能自己编

wanjunlei

你用kk装吧，https://github.com/kubesphere/kubekey/releases/tag/v1.1.0-alpha.1 这个版本是支持arm64的

qgr

wanjunlei 我的平台不是arm的，是其他架构，难受。。。

wanjunlei

qgr 那你不能用1.6.9，你自己编译1.4.6吧，用这个Dockerfile

`FROM golang:1.13.6-alpine3.11 as buildergo
RUN mkdir -p /fluent-bit
COPY main.go go.mod /fluent-bit/
WORKDIR /fluent-bit
RUN CGO_ENABLED=0 go build -o fluent-bit main.go

FROM debian:buster-slim as builderc

ENV FLB_MAJOR 1
ENV FLB_MINOR 4
ENV FLB_PATCH 6
ENV FLB_VERSION 1.4.6

ENV DEBIAN_FRONTEND noninteractive

RUN apt-get update && \
apt-get install -y –no-install-recommends \
build-essential \
curl \
ca-certificates \
cmake \
make \
tar \
libssl-dev \
libsasl2-dev \
pkg-config \
libsystemd-dev \
zlib1g-dev \
libpq-dev \
postgresql-server-dev-all \
flex \
bison

RUN mkdir -p /fluent-bit/bin /fluent-bit/etc /fluent-bit/log /tmp/src/
COPY . /tmp/src/
RUN rm -rf /tmp/src/build/*

WORKDIR /tmp/src/build/
RUN cmake -DFLB_DEBUG=Off \
-DFLB_TRACE=Off \
-DFLB_JEMALLOC=On \
-DFLB_TLS=On \
-DFLB_SHARED_LIB=Off \
-DFLB_EXAMPLES=Off \
-DFLB_HTTP_SERVER=On \
-DFLB_IN_SYSTEMD=On \
-DFLB_OUT_KAFKA=On \
-DFLB_OUT_PGSQL=On ../

RUN make -j $(getconf _NPROCESSORS_ONLN)
RUN install bin/fluent-bit /fluent-bit/bin/fluent-bit-core

COPY conf/fluent-bit.conf \
conf/parsers.conf \
conf/parsers_ambassador.conf \
conf/parsers_java.conf \
conf/parsers_extra.conf \
conf/parsers_openstack.conf \
conf/parsers_cinder.conf \
conf/plugins.conf \
/fluent-bit/etc/

COPY –from=buildergo /fluent-bit/fluent-bit /fluent-bit/bin/fluent-bit

EXPOSE 2020

CMD [“/fluent-bit/bin/fluent-bit”, “-c”, “/fluent-bit/etc/fluent-bit.conf”]

qgr

wanjunlei
这个分支吧，我看v1.4.6分支没有main.go文件

wanjunlei

是的

qgr

wanjunlei 大佬您好，换成你这个dockerfile编译的fluent-bit还是不行，同样一直打印这个日志

能帮忙看下吗？

wanjunlei

qgr 你要用operator部署，不是直接起个deployment，把你起的fluentbit删了，https://github.com/kubesphere/fluentbit-operator/blob/master/manifests/logging-stack/fluentbit-fluentBit.yaml
把这个apply一下，记得把镜像地址换成你自己的

qgr

wanjunlei 我看用这个crd部署也是创建一个fluent-bit的daemon-set，之前也是创建的ds。

现在换成用crd部署还是一样的错误，会不会和部署方式没关系，是哪里的配置有问题，麻烦大佬看下不，多谢啦。

wanjunlei

把fluent bit的yaml文件贴一下

qgr

wanjunlei
FluentBit:
`apiVersion: logging.kubesphere.io/v1alpha2
kind: FluentBit
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{“apiVersion”:“logging.kubesphere.io/v1alpha2”,“kind”:“FluentBit”,“metadata”:{“annotations”:{},“labels”:{“app.kubernetes.io/name”:“fluent-bit”},“name”:“fluent-bit”,“namespace”:“kubesphere-logging-system”},“spec”:{“fluentBitConfigName”:“fluent-bit-config”,“image”:“kubesphere/fluent-bit:v1.4.6-qgr”,“positionDB”:{“emptyDir”:{}},“tolerations”:[{“operator”:“Exists”}]}}
creationTimestamp: “2021-01-20T12:58:27Z”
finalizers:

fluentbit.logging.kubesphere.io
generation: 2
labels:
app.kubernetes.io/name: fluent-bit
managedFields:
apiVersion: logging.kubesphere.io/v1alpha2
fieldsType: FieldsV1
fieldsV1:
f:metadata:
f:annotations:
.: {}
f:kubectl.kubernetes.io/last-applied-configuration: {}
f:labels:
.: {}
f:app.kubernetes.io/name: {}
f:spec:
.: {}
f:fluentBitConfigName: {}
f:image: {}
f:positionDB:
.: {}
f:emptyDir: {}
f:tolerations: {}
manager: kubectl
operation: Update
time: “2021-01-20T12:58:27Z”
apiVersion: logging.kubesphere.io/v1alpha2
fieldsType: FieldsV1
fieldsV1:
f:metadata:
f:finalizers:
.: {}
v:“fluentbit.logging.kubesphere.io”: {}
f:status: {}
manager: manager
operation: Update
time: “2021-01-20T12:58:27Z”
name: fluent-bit
namespace: kubesphere-logging-system
resourceVersion: “1565816”
selfLink: /apis/logging.kubesphere.io/v1alpha2/namespaces/kubesphere-logging-system/fluentbits/fluent-bit
uid: a7204642-346b-447e-8965-fc82145fa930
spec:
fluentBitConfigName: fluent-bit-config
image: kubesphere/fluent-bit:v1.4.6-qgr
positionDB:
emptyDir: {}
tolerations:
operator: Exists
status: {}`

fluent-bit DaemonSet
`apiVersion: v1
items:

apiVersion: apps/v1
kind: DaemonSet
metadata:
annotations:
deprecated.daemonset.template.generation: “1”
creationTimestamp: “2021-01-20T12:58:28Z”
generation: 1
labels:
app.kubernetes.io/name: fluent-bit
managedFields:
- apiVersion: apps/v1
  fieldsType: FieldsV1
  fieldsV1:
  f:metadata:
  f:annotations:
  .: {}
  f:deprecated.daemonset.template.generation: {}
  f:labels:
  .: {}
  f:app.kubernetes.io/name: {}
  f:ownerReferences:
  .: {}
  k:{“uid”:“a7204642-346b-447e-8965-fc82145fa930”}:
  .: {}
  f:apiVersion: {}
  f:blockOwnerDeletion: {}
  f:controller: {}
  f:kind: {}
  f:name: {}
  f:uid: {}
  f:spec:
  f:revisionHistoryLimit: {}
  f:selector:
  f:matchLabels:
  .: {}
  f:app.kubernetes.io/name: {}
  f:template:
  f:metadata:
  f:labels:
  .: {}
  f:app.kubernetes.io/name: {}
  f:name: {}
  f:namespace: {}
  f:spec:
  f:containers:
  k:{“name”:“fluent-bit”}:
  .: {}
  f:image: {}
  f:imagePullPolicy: {}
  f:name: {}
  f:ports:
  .: {}
  k:{“containerPort”:2020,“protocol”:“TCP”}:
  .: {}
  f:containerPort: {}
  f:name: {}
  f:protocol: {}
  f:resources: {}
  f:terminationMessagePath: {}
  f:terminationMessagePolicy: {}
  f:volumeMounts:
  .: {}
  k:{“mountPath”:“/fluent-bit/config”}:
  .: {}
  f:mountPath: {}
  f:name: {}
  f:readOnly: {}
  k:{“mountPath”:“/fluent-bit/tail”}:
  .: {}
  f:mountPath: {}
  f:name: {}
  k:{“mountPath”:“/var/lib/docker/containers”}:
  .: {}
  f:mountPath: {}
  f:name: {}
  f:readOnly: {}
  k:{“mountPath”:“/var/log/”}:
  .: {}
  f:mountPath: {}
  f:name: {}
  f:readOnly: {}
  f:dnsPolicy: {}
  f:restartPolicy: {}
  f:schedulerName: {}
  f:securityContext: {}
  f:serviceAccount: {}
  f:serviceAccountName: {}
  f:terminationGracePeriodSeconds: {}
  f:tolerations: {}
  f:volumes:
  .: {}
  k:{“name”:“config”}:
  .: {}
  f:name: {}
  f:secret:
  .: {}
  f:defaultMode: {}
  f:secretName: {}
  k:{“name”:“positions”}:
  .: {}
  f:emptyDir: {}
  f:name: {}
  k:{“name”:“varlibcontainers”}:
  .: {}
  f:hostPath:
  .: {}
  f:path: {}
  f:type: {}
  f:name: {}
  k:{“name”:“varlogs”}:
  .: {}
  f:hostPath:
  .: {}
  f:path: {}
  f:type: {}
  f:name: {}
  f:updateStrategy:
  f:rollingUpdate:
  .: {}
  f:maxUnavailable: {}
  f:type: {}
  manager: manager
  operation: Update
  time: “2021-01-20T12:58:28Z”
- apiVersion: apps/v1
  fieldsType: FieldsV1
  fieldsV1:
  f:status:
  f:currentNumberScheduled: {}
  f:desiredNumberScheduled: {}
  f:numberAvailable: {}
  f:numberReady: {}
  f:observedGeneration: {}
  f:updatedNumberScheduled: {}
  manager: kube-controller-manager
  operation: Update
  time: “2021-01-20T12:58:33Z”
  name: fluent-bit
  namespace: kubesphere-logging-system
  ownerReferences:
- apiVersion: logging.kubesphere.io/v1alpha2
  blockOwnerDeletion: true
  controller: true
  kind: FluentBit
  name: fluent-bit
  uid: a7204642-346b-447e-8965-fc82145fa930
  resourceVersion: “1565858”
  selfLink: /apis/apps/v1/namespaces/kubesphere-logging-system/daemonsets/fluent-bit
  uid: 4e818009-9ebf-4464-9dc6-f4be2a52a700
  spec:
  revisionHistoryLimit: 10
  selector:
  matchLabels:
  app.kubernetes.io/name: fluent-bit
  template:
  metadata:
  creationTimestamp: null
  labels:
  app.kubernetes.io/name: fluent-bit
  name: fluent-bit
  namespace: kubesphere-logging-system
  spec:
  containers:
  - image: kubesphere/fluent-bit:v1.4.6-qgr
    imagePullPolicy: IfNotPresent
    name: fluent-bit
    ports:
    - containerPort: 2020
      name: metrics
      protocol: TCP
      resources: {}
      terminationMessagePath: /dev/termination-log
      terminationMessagePolicy: File
      volumeMounts:
    - mountPath: /var/lib/docker/containers
      name: varlibcontainers
      readOnly: true
    - mountPath: /fluent-bit/config
      name: config
      readOnly: true
    - mountPath: /var/log/
      name: varlogs
      readOnly: true
    - mountPath: /fluent-bit/tail
      name: positions
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      serviceAccount: fluent-bit
      serviceAccountName: fluent-bit
      terminationGracePeriodSeconds: 30
      tolerations:
  - operator: Exists
    volumes:
  - hostPath:
    path: /var/lib/docker/containers
    type: ""
    name: varlibcontainers
  - name: config
    secret:
    defaultMode: 420
    secretName: fluent-bit-config
  - hostPath:
    path: /var/log
    type: ""
    name: varlogs
  - emptyDir: {}
    name: positions
    updateStrategy:
    rollingUpdate:
    maxUnavailable: 1
    type: RollingUpdate
    status:
    currentNumberScheduled: 1
    desiredNumberScheduled: 1
    numberAvailable: 1
    numberMisscheduled: 0
    numberReady: 1
    observedGeneration: 1
    updatedNumberScheduled: 1
    kind: List
    metadata:
    resourceVersion: ""
    selfLink: ""`

wanjunlei

试试看能不能exec到fluent bit的pod里面，看看/fluent-bit/etc/fluent-bit.conf文件的内容，正确的应该是
@INCLUDE /fluent-bit/config/fluent-bit.conf，如果不是，说明镜像编译的有问题。
看看/fluent-bit/config/fluent-bit.conf文件的内容，这个应该和fluent-bit-config的内容是一样的。
把fluentbit-operator的replica改成0，然后编辑fluent-bit-config中的fluent-bit.conf的内容，你可以复制出来用base64解码，改完后再编码再复制进去，加上这个

然后重启fluent bit的daemonset，把日志贴出来

qgr

wanjunlei 现在fluent-bit的日志是这样的是不是就代表正常了

qgr

wanjunlei 现在fluent-bit和elasticsearch都能正常使用了
这是elasticsearch这不存的日志文件

为啥界面上还看不到日志，按照项目查询日志出现下图的错误

请问这是啥原因呢