notification-manager 报Secret \"global-email-config-secret\" not found

tanjunjie

请问下面这个错误，应该怎么配置？在kubesphere-monitoring-system命名空间里面配置了不起作用。
root@node1:~/tanjunjie/kubesphere/notification/notification-manager# kubectl logs notification-manager-deployment-7bd887ffb4-tl29f
level=info ts=2021-07-26T18:35:10.392245554+08:00 caller=main.go:95 msg=“Starting notification manager…” addr=:19093 timeout=5s
level=error ts=2021-07-26T18:35:14.919292792+08:00 caller=config.go:545 msg=“update receiver error” name=global-email-receiver err="conversion webhook for notification.kubesphere.io/v2beta1, Kind=Receiver failed: Post \“https://notification-manager-webhook.kubesphere-monitoring-system.svc:443/convert?timeout=30s\”: dial tcp 10.233.56.198:443: connect: connection refused"
level=error ts=2021-07-26T18:35:14.922031568+08:00 caller=config.go:560 msg=“update config error” name=default-email-config err="conversion webhook for notification.kubesphere.io/v2beta1, Kind=Config failed: Post \“https://notification-manager-webhook.kubesphere-monitoring-system.svc:443/convert?timeout=30s\”: dial tcp 10.233.56.198:443: connect: connection refused"
level=info ts=2021-07-26T18:35:15.005114079+08:00 caller=config.go:236 msg=“Setting up informers successfully”
level=error ts=2021-07-27T04:35:35.377901517+08:00 caller=config.go:545 msg=“update receiver error” name=global-email-receiver err="conversion webhook for notification.kubesphere.io/v2beta1, Kind=Receiver failed: Post \“https://notification-manager-webhook.kubesphere-monitoring-system.svc:443/convert?timeout=30s\”: dial tcp 10.233.56.198:443: connect: connection refused"
level=error ts=2021-07-27T04:35:35.38251109+08:00 caller=config.go:560 msg=“update config error” name=default-email-config err="conversion webhook for notification.kubesphere.io/v2beta1, Kind=Config failed: Post \“https://notification-manager-webhook.kubesphere-monitoring-system.svc:443/convert?timeout=30s\”: dial tcp 10.233.56.198:443: connect: connection refused"
level=info ts=2021-07-27T04:51:54.619104415+08:00 caller=config.go:583 msg=“resource change” op=add name=global-email-receiver
level=warn ts=2021-07-27T04:54:05.044444448+08:00 caller=email.go:104 msg=“EmailNotifier: ignore receiver because of empty config”
level=warn ts=2021-07-27T04:55:40.304655064+08:00 caller=email.go:104 msg=“EmailNotifier: ignore receiver because of empty config”
level=info ts=2021-07-27T05:00:20.452103416+08:00 caller=config.go:583 msg=“resource change” op=add name=default-email-config
level=error ts=2021-07-27T05:00:47.24368152+08:00 caller=email.go:168 msg=“EmailNotifier: get email config error” error="Secret \“global-email-config-secret\” not found"
level=error ts=2021-07-27T05:00:47.243823675+08:00 caller=handler.go:139 msg=“Worker: notification sent error”
level=error ts=2021-07-27T05:01:40.307108149+08:00 caller=email.go:168 msg=“EmailNotifier: get email config error” error="Secret \“global-email-config-secret\” not found"
level=error ts=2021-07-27T05:01:40.307289686+08:00 caller=handler.go:139 msg=“Worker: notification sent error”
level=error ts=2021-07-27T05:03:44.803326613+08:00 caller=email.go:168 msg=“EmailNotifier: get email config error” error="Secret \“global-email-config-secret\” not found"
level=error ts=2021-07-27T05:03:44.803548497+08:00 caller=handler.go:139 msg=“Worker: notification sent error”
level=error ts=2021-07-27T05:04:05.033299605+08:00 caller=email.go:168 msg=“EmailNotifier: get email config error” error="Secret \“global-email-config-secret\” not found"
level=error ts=2021-07-27T05:04:05.033353663+08:00 caller=handler.go:139 msg=“Worker: notification sent error”
level=error ts=2021-07-27T05:05:28.96175062+08:00 caller=email.go:168 msg=“EmailNotifier: get email config error” error="Secret \“global-email-config-secret\” not found"
level=error ts=2021-07-27T05:05:28.961862234+08:00 caller=handler.go:139 msg=“Worker: notification sent error”
level=error ts=2021-07-27T05:07:46.523273544+08:00 caller=email.go:168 msg=“EmailNotifier: get email config error” error="Secret \“global-email-config-secret\” not found"
level=error ts=2021-07-27T05:07:46.523386624+08:00 caller=handler.go:139 msg=“Worker: notification sent error”
level=error ts=2021-07-27T05:08:16.541498493+08:00 caller=email.go:168 msg=“EmailNotifier: get email config error” error="Secret \“global-email-config-secret\” not found"
level=error ts=2021-07-27T05:08:16.541588844+08:00 caller=handler.go:139 msg=“Worker: notification sent error”

yuswift

这个ns你有删过吗正常境况下是会自动初始化的请描述一下过程

tanjunjie

yuswift
3.1版本，这个问题应该是文档写的很不好，导致部署不太顺畅，强烈建议把网站上的文档写的清晰明了一些，能一次成功。
现在我按照github上的重新来过，还在测试，告警是一大堆，但是没有收到邮件。不知道是不是发送间隔时间未到，还是什么情况，反正现在日志里面没有报上面的secret未找到的错误了。
请问如何快速触发报警发送邮件进行测试呢？

yuswift

你用的ks是什么版本的

yuswift

tanjunjie 这个是3.0文档 3.1的文档正在合并中不久会更新

tanjunjie

yuswift 我已经按照github上的重新操作了一遍，notification-manager-deployment也重启了，报secret找不到，可是secret明明是存在的啊。请问怎么排查呢？

yuswift

tanjunjie 你的这些secret都在哪个ns下呢看起来是default

tanjunjie

yuswift root@node1:~/tanjunjie/kubesphere/notification# kubectl -n kubesphere-monitoring-system get secret | grep default
default-email-secret Opaque 1 16m
default-token-pddww kubernetes.io/service-account-token 3 45d
default-wechat-api-secret Opaque 1 14m
root@node1:~/tanjunjie/kubesphere/notification#

wanjunlei

你可以指定secret所在的ns

apiVersion: notification.kubesphere.io/v2beta1
kind: Config
metadata:
  name: default-config
  labels:
    app: notification-manager
    type: default
spec:
  email:
    authPassword:
      key: password
      name: defalut-email-secret
      namespace: default

tanjunjie

wanjunlei 谢谢，指定命名空间后没有报找不到了。

wanjunlei

如果secret没有明确指定ns，会在defaultSecretNamespace指定的ns中查找，这个值可以通过 kubectl edit nm notification-manager 设置，参见https://github.com/kubesphere/notification-manager/blob/master/pkg/apis/v2beta1/notificationmanager_types.go#L63

如果这个值也没有设置，那么会去notification manager Operator所在的ns下查找secret

tanjunjie

暂时结贴吧。这文档质量真是伤不起啊。又遇到secret base64编码问题，搜遍论坛才解决。