api 限流 Throttling request

xxm13504577723

Throttling request took 1.159444401s, request: GET:https://xxxxxxxxx:6443/apis/storage.k8s.io/v1beta1?timeout=32s
每次使用kubectl客户端查询资源时总是会等待几秒，然后提示上面这个问题，查询知道是api限流，请问这个问题产生的原因，如何解决这类问题

yuswift

xxm13504577723 你是在ks-console界面操作的吗?先看看kube-apiserver日志

xxm13504577723

yuswift 我是用客户端kubectl查命令的时候，就会报这个错误，我是三个master节点，kube-apiserver有一个节点也都是这类错误

hongming

xxm13504577723 看看集群监控，看看kube-apiserver 的负载高不高

xxm13504577723

hongming 负载还没有你的高呢
18.100 times/s API 每秒请求数
6.03 ms API 请求延迟
161 调度器调度次数
6 调度失败的容器组

xmapst

有解决办法吗

yuswift

xmapst 你现在后台kube-apiserver日志贴一下看看呢

alexclownfish

yuswift 有结论吗？我刚刚也遇到了，以下是apiserver的日志
I0607 01:40:56.905513 1 request.go:621] Throttling request took 1.044737692s, request: GET:https://172.22.254.57:6443/apis/app.k8s.io/v1beta1?timeout=32s I0607 01:41:28.007127 1 request.go:621] Throttling request took 1.044919562s, request: GET:https://172.22.254.57:6443/apis/servicemesh.kubesphere.io/v1alpha2?timeout=32s I0607 01:41:59.108655 1 request.go:621] Throttling request took 1.044621164s, request: GET:https://172.22.254.57:6443/apis/app.k8s.io/v1beta1?timeout=32s I0607 01:42:30.210214 1 request.go:621] Throttling request took 1.045909735s, request: GET:https://172.22.254.57:6443/apis/servicemesh.kubesphere.io/v1alpha2?timeout=32s I0607 01:43:01.311781 1 request.go:621] Throttling request took 1.044440237s, request: GET:https://172.22.254.57:6443/apis/app.k8s.io/v1beta1?timeout=32s I0607 01:43:32.413453 1 request.go:621] Throttling request took 1.044467063s, request: GET:https://172.22.254.57:6443/apis/app.k8s.io/v1beta1?timeout=32s I0607 01:44:03.514948 1 request.go:621] Throttling request took 1.046405775s, request: GET:https://172.22.254.57:6443/apis/scheduling.k8s.io/v1beta1?timeout=32s I0607 01:44:34.616449 1 request.go:621] Throttling request took 1.044620289s, request: GET:https://172.22.254.57:6443/apis/app.k8s.io/v1beta1?timeout=32s I0607 01:45:05.718023 1 request.go:621] Throttling request took 1.044623853s, request: GET:https://172.22.254.57:6443/apis/app.k8s.io/v1beta1?timeout=32s

yuswift

alexclownfish 这个是集群内部有一些异常的调用过快的api导致的建议排查一下集群内部的调用者可以一个个停掉试试

ruiyaoOps

这个问题俺也遇到了，然后自己盘了下给解决了，出现这个问题的环境都不是直接用 root 用户，所以就会出现权限不足的问题
解决方法很简单： sudo chmod 777 -R ~/.kube/
当然还会由其他原因导致该问题，欢迎补充。: D

alexd

ruiyaoOps 报错是api限流，和权限没关系吧

24sama

alexd
如果只是kubectl报限流的错是有可能的。因为权限问题，kubectl 无法读取~/.kube/目录下的cache目录就会出现这个问题。