环境
三台 Centos 7
k8s 1.15.4 master+worker
kubesphere 3.0
问题描述
三个ks-apiserver 有一个启动失败。
相关日志
apiserver 日志
W0301 14:18:54.444097 1 client_config.go:543] Neither --kubeconfig nor --master was specified. Using the inClusterConfig. This might not work.
Error: failed to connect to redis service, please check redis status, error: dial tcp 192.168.4.149:6379: i/o timeout
2021/03/01 14:18:59 failed to connect to redis service, please check redis status, error: dial tcp 192.168.4.149:6379: i/o timeout
redis 日志
[NOTICE] 049/054639 (1) : New worker #1 (6) forked
[WARNING] 049/054639 (6) : Server check_if_redis_is_master_0/R0 is DOWN, reason: Layer4 connection problem, info: "Connection refused at step 1 of tcp-check (connect)", check duration: 3ms. 2 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
[WARNING] 049/054639 (6) : Server check_if_redis_is_master_0/R1 is DOWN, reason: Layer4 connection problem, info: "Connection refused at step 1 of tcp-check (connect)", check duration: 3ms. 1 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
[WARNING] 049/054639 (6) : Server check_if_redis_is_master_0/R2 is DOWN, reason: Layer4 connection problem, info: "Connection refused at step 1 of tcp-check (connect)", check duration: 3ms. 0 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
[ALERT] 049/054639 (6) : backend 'check_if_redis_is_master_0' has no server available!
[WARNING] 049/054640 (6) : Server check_if_redis_is_master_1/R0 is DOWN, reason: Layer4 connection problem, info: "Connection refused at step 1 of tcp-check (connect)", check duration: 1001ms. 2 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
[WARNING] 049/054640 (6) : Server check_if_redis_is_master_1/R1 is DOWN, reason: Layer4 connection problem, info: "Connection refused at step 1 of tcp-check (connect)", check duration: 1001ms. 1 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
[WARNING] 049/054640 (6) : Server check_if_redis_is_master_1/R2 is DOWN, reason: Layer4 connection problem, info: "Connection refused at step 1 of tcp-check (connect)", check duration: 1000ms. 0 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
[ALERT] 049/054640 (6) : backend 'check_if_redis_is_master_1' has no server available!
[WARNING] 049/054640 (6) : Server check_if_redis_is_master_2/R0 is DOWN, reason: Layer4 timeout, info: " at step 1 of tcp-check (connect)", check duration: 1000ms. 2 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
[WARNING] 049/054641 (6) : Server check_if_redis_is_master_2/R1 is DOWN, reason: Layer4 timeout, info: " at step 1 of tcp-check (connect)", check duration: 1000ms. 1 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
[WARNING] 049/054641 (6) : Server check_if_redis_is_master_2/R2 is DOWN, reason: Layer4 timeout, info: " at step 1 of tcp-check (connect)", check duration: 1001ms. 0 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
[ALERT] 049/054641 (6) : backend 'check_if_redis_is_master_2' has no server available!
[WARNING] 049/054641 (6) : Server bk_redis_master/R0 is DOWN, reason: Layer4 timeout, info: " at step 1 of tcp-check (connect)", check duration: 1001ms. 2 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
[WARNING] 049/054641 (6) : Server bk_redis_master/R1 is DOWN, reason: Layer4 timeout, info: " at step 1 of tcp-check (connect)", check duration: 1001ms. 1 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
[WARNING] 049/054641 (6) : Server bk_redis_master/R2 is DOWN, reason: Layer4 timeout, info: " at step 1 of tcp-check (connect)", check duration: 1001ms. 0 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
[ALERT] 049/054641 (6) : backend 'bk_redis_master' has no server available!
[WARNING] 049/054652 (6) : Server bk_redis_master/R0 is UP, reason: Layer7 check passed, code: 0, info: "(tcp-check)", check duration: 0ms. 1 active and 0 backup servers online. 0 sessions requeued, 0 total in queue.
[WARNING] 049/054652 (6) : Server check_if_redis_is_master_0/R0 is UP, reason: Layer7 check passed, code: 0, info: "(tcp-check)", check duration: 2ms. 1 active and 0 backup servers online. 0 sessions requeued, 0 total in queue.
[WARNING] 049/054709 (6) : Server check_if_redis_is_master_0/R1 is UP, reason: Layer7 check passed, code: 0, info: "(tcp-check)", check duration: 1ms. 2 active and 0 backup servers online. 0 sessions requeued, 0 total in queue.
[WARNING] 049/054726 (6) : Server check_if_redis_is_master_0/R2 is UP, reason: Layer7 check passed, code: 0, info: "(tcp-check)", check duration: 0ms. 3 active and 0 backup servers online. 0 sessions requeued, 0 total in queue.
[WARNING] 055/084847 (6) : Server check_if_redis_is_master_0/R1 is DOWN, reason: Layer4 timeout, info: " at step 1 of tcp-check (connect)", check duration: 1001ms. 2 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
[WARNING] 055/085002 (6) : Server bk_redis_master/R0 is DOWN, reason: Layer4 timeout, info: " at step 1 of tcp-check (connect)", check duration: 1001ms. 0 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
[ALERT] 055/085002 (6) : backend 'bk_redis_master' has no server available!
[WARNING] 055/085006 (6) : Server check_if_redis_is_master_0/R0 is DOWN, reason: Layer4 timeout, info: " at step 1 of tcp-check (connect)", check duration: 1000ms. 1 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
[WARNING] 055/085008 (6) : Server check_if_redis_is_master_0/R2 is DOWN, reason: Layer4 timeout, info: " at step 1 of tcp-check (connect)", check duration: 1000ms. 0 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
[ALERT] 055/085008 (6) : backend 'check_if_redis_is_master_0' has no server available!
[WARNING] 055/091127 (6) : Server check_if_redis_is_master_2/R1 is UP, reason: Layer7 check passed, code: 0, info: "(tcp-check)", check duration: 1ms. 1 active and 0 backup servers online. 0 sessions requeued, 0 total in queue.
[WARNING] 055/091220 (6) : Server bk_redis_master/R0 is UP, reason: Layer7 check passed, code: 0, info: "(tcp-check)", check duration: 1ms. 1 active and 0 backup servers online. 0 sessions requeued, 0 total in queue.
[WARNING] 055/091221 (6) : Server check_if_redis_is_master_0/R0 is UP, reason: Layer7 check passed, code: 0, info: "(tcp-check)", check duration: 8ms. 1 active and 0 backup servers online. 0 sessions requeued, 0 total in queue.
[WARNING] 055/091228 (6) : Server check_if_redis_is_master_2/R0 is UP, reason: Layer7 check passed, code: 0, info: "(tcp-check)", check duration: 0ms. 2 active and 0 backup servers online. 0 sessions requeued, 0 total in queue.
[WARNING] 055/091232 (6) : Server check_if_redis_is_master_0/R0 is DOWN, reason: Layer7 timeout, info: " at step 5 of tcp-check (expect string '192.168.169.49')", check duration: 1000ms. 0 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
[ALERT] 055/091232 (6) : backend 'check_if_redis_is_master_0' has no server available!
[WARNING] 055/091238 (6) : Server bk_redis_master/R0 is DOWN, reason: Layer7 timeout, info: " at step 5 of tcp-check (expect string 'role:master')", check duration: 1001ms. 0 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
[ALERT] 055/091238 (6) : backend 'bk_redis_master' has no server available!
