etcd安装失败：tls: failed to find any PEM data in certificate input

hongming · 2020年9月10日

.bashrc 里不能有无法执行的命令，会导致安装失败。注意nfs server 的配置

Recommended nfs server configuration: *(rw,insecure,sync,no_subtree_check,no_root_squash)

rysinal · 2020年9月10日

感谢指导，正如hongming 所说，原因是因为第一次安装后，又安装了k8s命令补全工具。然后使用kubekey卸载后，kubectl被删除了，但补全工具还在环境变量里，导致第二次安装报错。注释掉补全工具的环境配置再次安装即正常

rysinal · 2020年9月11日

hongming 我们今天确认了下nas的属性：

确定是no_root_squash模式的，请教下还有其他可能导致安装失败吗？

hongming · 2020年9月11日

rysinal 从现象判断大概率就是存储的问题，可以把mysql/ldap的crash log再贴一下，看看有没有什么解决方式

rysinal · 2020年9月11日

hongming

[root@master1 ~]# kubectl -n kubesphere-system get po
NAME                                    READY   STATUS             RESTARTS   AGE
etcd-65796969c7-hjxnd                   1/1     Running            0          18h
ks-apiserver-98484f67f-54q2j            1/1     Running            0          18h
ks-apiserver-98484f67f-bhlnt            1/1     Running            0          18h
ks-apiserver-98484f67f-t6b4m            1/1     Running            0          18h
ks-console-786b9846d4-2hz26             1/1     Running            0          18h
ks-console-786b9846d4-brpb4             1/1     Running            0          18h
ks-console-786b9846d4-tqvc5             1/1     Running            0          18h
ks-controller-manager-646595cb4-bss4w   1/1     Running            0          18h
ks-controller-manager-646595cb4-dqn47   1/1     Running            0          18h
ks-controller-manager-646595cb4-mpm6t   1/1     Running            0          18h
ks-installer-7cb866bd-9fpzc             1/1     Running            0          18h
minio-7bfdb5968b-pwbh4                  1/1     Running            0          18h
mysql-7f64d9f584-4j82n                  0/1     CrashLoopBackOff   224        18h
openldap-0                              1/1     Running            1          18h
openldap-1                              0/1     CrashLoopBackOff   267        18h
redis-ha-haproxy-5c6559d588-97kvx       1/1     Running            1          18h
redis-ha-haproxy-5c6559d588-bf5wt       1/1     Running            0          18h
redis-ha-haproxy-5c6559d588-jzxbq       1/1     Running            0          18h
redis-ha-server-0                       2/2     Running            0          18h
redis-ha-server-1                       2/2     Running            0          18h
redis-ha-server-2                       2/2     Running            0          18h

#mysql的日志：

Initializing database
2020-09-11T03:47:27.502006Z 0 [Warning] [MY-011070] [Server] 'Disabling symbolic links using --skip-symbolic-links (or equivalent) is the default. Consider not using thisoption as it' is deprecated and will be removed in a future release.
2020-09-11T03:47:27.502110Z 0 [Warning] [MY-011068] [Server] The syntax 'expire-logs-days' is deprecated and will be removed in a future release. Please use binlog_expire_logs_seconds instead.
2020-09-11T03:47:27.502235Z 0 [System] [MY-013169] [Server] /usr/sbin/mysqld (mysqld 8.0.11) initializing of server in progress as process 36
2020-09-11T03:47:27.507361Z 0 [ERROR] [MY-010457] [Server] --initialize specified but the data directory has files in it. Aborting.
2020-09-11T03:47:27.507444Z 0 [ERROR] [MY-010119] [Server] Aborting
2020-09-11T03:47:27.507985Z 0 [System] [MY-010910] [Server] /usr/sbin/mysqld: Shutdown complete (mysqld 8.0.11)  MySQL Community Server - GPL.

#openldap-0的日志

5f5af49e <= mdb_equality_candidates: (uniqueMember) not indexed
5f5af49e <= mdb_equality_candidates: (memberUid) not indexed
5f5af49e conn=1082 op=676 SEARCH RESULT tag=101 err=0 nentries=0 text=
5f5af49e conn=1082 op=677 SRCH base="ou=Users,dc=kubesphere,dc=io" scope=2 deref=3 filter="(&(objectClass=inetOrgPerson)(|(uid=admin)(mail=admin)))"
5f5af49e conn=1082 op=677 SEARCH RESULT tag=101 err=0 nentries=1 text=
5f5af49e conn=1082 op=678 SRCH base="ou=Groups,dc=kubesphere,dc=io" scope=2 deref=3 filter="(|(member=uid=admin,ou=users,dc=kubesphere,dc=io)(uniqueMember=uid=admin,ou=users,dc=kubesphere,dc=io)(memberUid=admin))"
5f5af49e conn=1082 op=678 SRCH attr=cn
5f5af49e <= mdb_equality_candidates: (member) not indexed
5f5af49e <= mdb_equality_candidates: (uniqueMember) not indexed
5f5af49e <= mdb_equality_candidates: (memberUid) not indexed
5f5af49e conn=1082 op=678 SEARCH RESULT tag=101 err=0 nentries=0 text=
5f5af4aa conn=10015 fd=13 ACCEPT from IP=172.22.215.71:25500 (IP=0.0.0.0:389)
5f5af4aa conn=10015 fd=13 closed (connection lost)
5f5af4ac conn=10016 fd=13 ACCEPT from IP=172.22.215.71:25568 (IP=0.0.0.0:389)
5f5af4ac conn=10016 fd=13 closed (connection lost)
5f5af4b2 slap_client_connect: URI=ldap://openldap-1.openldap DN="cn=admin,dc=kubesphere,dc=io" ldap_sasl_bind_s failed (-1)
5f5af4b2 slap_client_connect: URI=ldap://openldap-1.openldap DN="cn=admin,cn=config" ldap_sasl_bind_s failed (-1)
5f5af4b2 do_syncrepl: rid=102 rc -1 retrying
5f5af4b2 do_syncrepl: rid=002 rc -1 retrying
5f5af4b9 conn=10017 fd=13 ACCEPT from IP=172.22.215.71:25886 (IP=0.0.0.0:389)
5f5af4b9 conn=10017 fd=13 closed (connection lost)
5f5af4bb conn=10018 fd=13 ACCEPT from IP=172.22.215.71:25968 (IP=0.0.0.0:389)
5f5af4bb conn=10018 fd=13 closed (connection lost)
5f5af4c8 conn=10019 fd=13 ACCEPT from IP=172.22.215.71:26294 (IP=0.0.0.0:389)
5f5af4c8 conn=10019 fd=13 closed (connection lost)

#openldap-1的日志

*** An error occurred. Aborting.
*** Init system aborted.
*** Not all processes have exited in time. Forcing them to exit.

rysinal · 2020年9月11日

hongming 更新：
1、mysql的删除了pvc里的ibdata1文件后恢复正常
2、openldap重启后出现新的错误：

5f5b4726 @(#) $OpenLDAP: slapd 2.4.50+dfsg-1~bpo10+1 (May 4 2020 05:25:06) $

Debian OpenLDAP Maintainers

5f5b4726 slapd starting

5f5b4726 <= mdb_equality_candidates: (entryUUID) not indexed

5f5b4728 syncrepl_message_to_entry: rid=001 mods check (objectClass: value #1 invalid per syntax)

5f5b4728 do_syncrepl: rid=001 rc 21 retrying

5f5b4728 <= mdb_equality_candidates: (entryUUID) not indexed

5f5b4728 <= mdb_equality_candidates: (entryUUID) not indexed

5f5b4728 <= mdb_equality_candidates: (entryUUID) not indexed

5f5b4728 <= mdb_equality_candidates: (entryUUID) not indexed

hongming · 2020年9月11日

rysinal 停掉ldap, 把 ldap pvc 里的文件也都清空一下

rysinal · 2020年9月11日

hongming openldap自己正常了。。。不过我看mysql又出现重启了，我再删除文件试下

rysinal · 2020年9月18日

hongming 请教下，我想设置nfs的选项，一直报格式错误,这个格式有示例吗

  addons:
  - name: nfs-client
    namespace: kube-system
    sources:
      chart:
        name: nfs-client-provisioner
        repo: https://charts.kubesphere.io/main
        values:
        - storageClass.defaultClass=true
        - nfs.server=xxxxxx
        - nfs.path=/xxxx
        - nfs.mountOptions="nfsvers=3,timeo=600,nolock"

- nfs.mountOptions="nfsvers=3,timeo=600,nolock主要是这行nolock设置
安装报错：

Failed to deploy addons: failed parsing --set data: key "nolock\"" has no value

Cauchy · 2020年9月18日

rysinal

看了下nfs-client-provisioner这个chart，nfs.mountOptions这个参数是个列表，这个可以用values.yaml配置文件的方式配置。

在自己机器上搞个custom-nfs-client-values.yaml, 修改成自己的配置后，把这个文件路径填到values那里。

nfs:
  server: xxx
  path: xxx
  mountOptions:
  - nfsvers=3
  - timeo=600
  - nolock
storageClass:
  defaultClass: true

  addons:
  - name: nfs-client
    namespace: kube-system
    sources:
      chart:
        name: nfs-client-provisioner
        repo: https://charts.kubesphere.io/main
        values: /xxx/custom-nfs-client-values.yaml

rysinal · 2020年9月18日

Cauchy 感谢，已经成功，第二个mysql、openldap启动重启问题确实是nolock这个参数导致的

rysinal · 2021年8月2日

Cauchy 大佬，我现在使用外置文件的方式不行了，是不让这样写了吗：

Error: Failed to download cluster config: Unable to convert file to yaml: yaml: unmarshal errors:
  line 52: cannot unmarshal !!str `/root/k...` into []string

etcd安装失败：tls: failed to find any PEM data in certificate input

hongmingK零SK壹S

rysinalK零S

rysinalK零S

hongmingK零SK壹S

rysinalK零S

rysinalK零S

hongmingK零SK壹S

rysinalK零S

rysinalK零S

CauchyK零SK壹S

rysinalK零S

rysinalK零S