• klj890K零S

    2025/03/10 07:47:30 [error] 34#34: *45028 connect() failed (110: Operation timed out) while connecting to upstream, client: 172.1.30.21, server: 172.1.30.21.nip.io, request: "GET / HTTP/2.0", upstream: "http://10.233.127.31:8000/", host: "172.1.30.21.nip.io:31199"
    2025/03/10 07:47:40 [error] 38#38: *45480 connect() failed (110: Operation timed out) while connecting to upstream, client: 172.1.30.21, server: 172.1.30.21.nip.io, request: "GET / HTTP/1.1", upstream: "http://10.233.127.31:8000/", host: "172.1.30.21.nip.io:31199"
    2025/03/10 07:48:03 [error] 39#39: *46592 connect() failed (110: Operation timed out) while connecting to upstream, client: 172.1.30.21, server: 172.1.30.21.nip.io, request: "GET / HTTP/2.0", upstream: "http://10.233.127.31:8000/", host: "172.1.30.21.nip.io:31199"
    2025/03/10 07:49:41 [error] 34#34: *45028 connect() failed (110: Operation timed out) while connecting to upstream, client: 172.1.30.21, server: 172.1.30.21.nip.io, request: "GET / HTTP/2.0", upstream: "http://10.233.127.31:8000/", host: "172.1.30.21.nip.io:31199"
    2025/03/10 07:49:51 [error] 38#38: *45480 connect() failed (110: Operation timed out) while connecting to upstream, client: 172.1.30.21, server: 172.1.30.21.nip.io, request: "GET / HTTP/1.1", upstream: "http://10.233.127.31:8000/", host: "172.1.30.21.nip.io:31199"
    2025/03/10 07:50:14 [error] 39#39: *46592 connect() failed (110: Operation timed out) while connecting to upstream, client: 172.1.30.21, server: 172.1.30.21.nip.io, request: "GET / HTTP/2.0", upstream: "http://10.233.127.31:8000/", host: "172.1.30.21.nip.io:31199"

    这里说了,你的upstream连接 timeout了

    • cici 回复了此帖

      klj890
      我已經在ingress的annotation加入這些配置仍然是一樣的錯誤

          nginx.ingress.kubernetes.io/proxy-connect-timeout: "300"
          nginx.ingress.kubernetes.io/proxy-read-timeout: "300"
          nginx.ingress.kubernetes.io/proxy-send-timeout: "300"
      • klj890K零S

        • klj890K零S

          为什么是8000端口?默认不是80端口吗?

          • cici 回复了此帖

            klj890 我其實也不太知道這個端口是怎麼跑出來的,不過我ingress內配置確實是用80

            但我現在是在內網環境,外部確實有被防火牆擋

              me:~/kubesphere$ curl -v http://10.233.127.31:8000
              *   Trying 10.233.127.31:8000...
              * Connected to 10.233.127.31 (10.233.127.31) port 8000 (#0)
              > GET / HTTP/1.1
              > Host: 10.233.127.31:8000
              > User-Agent: curl/7.81.0
              > Accept: */*
              > 
              * Mark bundle as not supporting multiuse
              < HTTP/1.1 302 Found
              < Vary: Accept-Encoding
              < Location: /login
              < Content-Type: text/html; charset=utf-8
              < Content-Length: 43
              < Date: Tue, 11 Mar 2025 03:26:16 GMT
              < Connection: keep-alive
              < Keep-Alive: timeout=5
              < 
              * Connection #0 to host 10.233.127.31 left intact
                • klj890K零S

                  cici
                  进入ingress pod里面,看下nginx的配置。找到代理ks-console的域名,找个pod,在里面curl/nslookup这个域名看看后端IP是不是ks-console的ip

                  • cici 回复了此帖
                    • klj890K零S

                      cici 这是在哪里curl的?

                      klj890 抱歉前面沒修改到回復 我後來檢查到這個10.233.127.31是ks-console pod IP

                        • klj890K零S

                          cici 但是端口看着不对啊。。。

                          • cici 回复了此帖

                            klj890 Ingress-controller pod 裡面curl 10.233.127.31:8000

                            h00283@coverity-ms:~/kubesphere$ kubectl exec -it ingress-nginx-controller-79746f9d79-2j5b8 -n ingress-nginx -- curl -v http://10.233.127.31:8000
                            *   Trying 10.233.127.31:8000...
                            * connect to 10.233.127.31 port 8000 from 10.233.127.73 port 58146 failed: Operation timed out
                            * Failed to connect to 10.233.127.31 port 8000 after 130821 ms: Could not connect to server
                            * closing connection #0
                            curl: (28) Failed to connect to 10.233.127.31 port 8000 after 130821 ms: Could not connect to server
                            command terminated with exit code 28
                            
                            
                            #改用80試看看
                            h00283@coverity-ms:~/kubesphere$ kubectl exec -it ingress-nginx-controller-79746f9d79-2j5b8 -n ingress-nginx -- curl -v http://10.233.127.31:80
                            *   Trying 10.233.127.31:80...
                            * connect to 10.233.127.31 port 80 from 10.233.127.73 port 50282 failed: Operation timed out
                            * Failed to connect to 10.233.127.31 port 80 after 130403 ms: Could not connect to server
                            * closing connection #0
                            curl: (28) Failed to connect to 10.233.127.31 port 80 after 130403 ms: Could not connect to server
                            command terminated with exit code 28

                            我用另一台一樣配置安裝的機器試了一下,確定是ks-console pod IP:8000 port ,curl 出來應該會是

                            me2:~/kubesphere$ kubectl exec -it ingress-nginx-controller-d6f759d4d-8fflw -n ingress-nginx -- curl -v http://10.233.120.30:8000
                            *   Trying 10.233.120.30:8000...
                            * Connected to 10.233.120.30 (10.233.120.30) port 8000
                            * using HTTP/1.x
                            > GET / HTTP/1.1
                            > Host: 10.233.120.30:8000
                            > User-Agent: curl/8.11.1
                            > Accept: */*
                            >
                            * Request completely sent off
                            < HTTP/1.1 302 Found
                            < Vary: Accept-Encoding
                            < Location: /login
                            < Content-Type: text/html; charset=utf-8
                            < Content-Length: 43
                            < Date: Tue, 11 Mar 2025 06:08:11 GMT
                            < Connection: keep-alive
                            < Keep-Alive: timeout=5
                            <
                            * Connection #0 to host 10.233.120.30 left intact
                            Redirecting to <a href="/login">/login</a>.

                            不過就不知道為什麼這台不行

                            • klj890K零S

                              是我错了。pod监听的就是8000端口,所以路由是对的

                              那你还是检查ks-console pod本身的健康吧

                              • klj890K零S

                                如果pod本身没有问题,检查你的网络。

                                • klj890K零S


                                  可以问下AI

                                    klj890 看起來確實是兩個pod互聯都time out,有可能因為我外部防火牆影響嗎

                                      • klj890K零S

                                        cici
                                        怀疑它,证明它

                                        • cici 回复了此帖

                                          klj890 我進ks-console跟ingress的pod分別做了都time out

                                          me:~/kubesphere$ kubectl exec -it ks-console-5675bf657b-kzplx -n kubesphere-system -- nc -zv 10.233.127.77 80
                                          nc: 10.233.127.77 (10.233.127.77:80): Operation timed out
                                          command terminated with exit code 1
                                          me:~/kubesphere$ kubectl exec -it ingress-nginx-controller-d5c666948-8fg6h -n ingress-nginx -- nc -zv 10.233.127.31 8000
                                          nc: 10.233.127.31 (10.233.127.31:8000): Operation timed out
                                          command terminated with exit code 1