kubesphere的web页面中存储类型一直刷不出

cntsp

[环境ubuntu系统18.04，kubeadm安装的一个master两个node,版本是v1.18.5，已经创建好了存储类，借助nfs，然后根据这个文档在k8s最小化安装kubesphere，只不过我对cluster-configuration.yaml 进行了修改，启用了一些devops和日志相关的组件同时也指明了存储类：

persistence: storageClass: "managed-nfs-storage" # If there is not a default StorageClass in your cluster, you need to specify an existing StorageClass here.

在部署kubesphere我都已经部署好了NFS服务器，创建好了存储类:
root@k8smaster1:~# kubectl get sc NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE managed-nfs-storage fuseim.pri/ifs Delete Immediate false 54d
现在就是不知道为啥存储类型这个页面刷不出来我的存储类

stoneshi-yunify

cntsp 可能是遇到 https://kubesphere.com.cn/forum/d/3304/4 同样的问题了，请参考那个帖子的方法解决

yuswift

看一下后台ks-apiserver日志呢

cntsp

yuswift 这个是ks-apiserver的日志
`
2021/02/08 12:53:17 http: panic serving 10.244.9.253:59586: runtime error: invalid memory address or nil pointer dereference
goroutine 55702 [running]:
net/http.(conn).serve.func1(0xc002a00780)
/opt/hostedtoolcache/go/1.13.15/x64/src/net/http/server.go:1795 +0×139
panic(0×2d36da0, 0×5450320)
/opt/hostedtoolcache/go/1.13.15/x64/src/runtime/panic.go:679 +0×1b2
kubesphere.io/kubesphere/pkg/models/resources/v1alpha2/storageclass.(storageClassesSearcher).countPersistentVolumeClaims(0xc000912e00, 0xc0012b4d80, 0×13, 0×54c6d68)
/home/runner/work/kubesphere/kubesphere/pkg/models/resources/v1alpha2/storageclass/storageclasses.go:110 +0×12c
kubesphere.io/kubesphere/pkg/models/resources/v1alpha2/storageclass.(storageClassesSearcher).Search(0xc000912e00, 0×0, 0×0, 0xc00313ab70, 0xc002d20402, 0xa, 0×1, 0×2e5db20, 0xc003140180, 0×0, …)
/home/runner/work/kubesphere/kubesphere/pkg/models/resources/v1alpha2/storageclass/storageclasses.go:92 +0×2f0
kubesphere.io/kubesphere/pkg/models/resources/v1alpha2/resource.(ResourceGetter).ListResources(0xc00096a100, 0×0, 0×0, 0xc002d203ec, 0xe, 0xc00313ab70, 0xc002d20402, 0xa, 0×299a801, 0xa, …)
/home/runner/work/kubesphere/kubesphere/pkg/models/resources/v1alpha2/resource/resources.go:123 +0×23e
kubesphere.io/kubesphere/pkg/kapis/resources/v1alpha3.(Handler).fallback(0xc000912e20, 0xc002d203ec, 0xe, 0×0, 0×0, 0xc0030c3700, 0×0, 0×3856240, 0xc0004f0a50)
/home/runner/work/kubesphere/kubesphere/pkg/kapis/resources/v1alpha3/handler.go:155 +0xb0b
kubesphere.io/kubesphere/pkg/kapis/resources/v1alpha3.(Handler).handleListResources(0xc000912e20, 0xc003139f50, 0xc000de7420)
/home/runner/work/kubesphere/kubesphere/pkg/kapis/resources/v1alpha3/handler.go:97 +0×2cb
github.com/emicklei/go-restful.(Container).dispatch.func5(0xc003139f50, 0xc000de7420)
/home/runner/go/pkg/mod/github.com/emicklei/go-restful@v2.9.5+incompatible/container.go:283 +0×3e
github.com/emicklei/go-restful.(FilterChain).ProcessFilter(0xc003140000, 0xc003139f50, 0xc000de7420)
/home/runner/go/pkg/mod/github.com/emicklei/go-restful@v2.9.5+incompatible/filter.go:21 +0×8b
kubesphere.io/kubesphere/pkg/apiserver.logRequestAndResponse(0xc003139f50, 0xc000de7420, 0xc003140000)
/home/runner/work/kubesphere/kubesphere/pkg/apiserver/apiserver.go:531 +0×82
github.com/emicklei/go-restful.(FilterChain).ProcessFilter(0xc003140000, 0xc003139f50, 0xc000de7420)
/home/runner/go/pkg/mod/github.com/emicklei/go-restful@v2.9.5+incompatible/filter.go:19 +0×65
github.com/emicklei/go-restful.(Container).dispatch(0xc000984630, 0×38c1500, 0xc003139ef0, 0xc002ef2100)
/home/runner/go/pkg/mod/github.com/emicklei/go-restful@v2.9.5+incompatible/container.go:285 +0×866
net/http.HandlerFunc.ServeHTTP(0xc0004251c0, 0×38c1500, 0xc003139ef0, 0xc002ef2100)
/opt/hostedtoolcache/go/1.13.15/x64/src/net/http/server.go:2036 +0×44
net/http.(ServeMux).ServeHTTP(0xc00095f340, 0×38c1500, 0xc003139ef0, 0xc002ef2100)
/opt/hostedtoolcache/go/1.13.15/x64/src/net/http/server.go:2416 +0×1bd
github.com/emicklei/go-restful.(Container).ServeHTTP(0xc000984630, 0×38c1500, 0xc003139ef0, 0xc002ef2100)
/home/runner/go/pkg/mod/github.com/emicklei/go-restful@v2.9.5+incompatible/container.go:303 +0×4d
kubesphere.io/kubesphere/pkg/apiserver/filters.WithKubeAPIServer.func1(0×38c1500, 0xc003139ef0, 0xc002ef2100)
/home/runner/work/kubesphere/kubesphere/pkg/apiserver/filters/kubeapiserver.go:71 +0×37f
net/http.HandlerFunc.ServeHTTP(0xc000951b00, 0×38c1500, 0xc003139ef0, 0xc002ef2100)
/opt/hostedtoolcache/go/1.13.15/x64/src/net/http/server.go:2036 +0×44
kubesphere.io/kubesphere/pkg/apiserver/filters.WithAuditing.func1(0×38c1b80, 0xc00276c460, 0xc002ef2100)
/home/runner/work/kubesphere/kubesphere/pkg/apiserver/filters/auditing.go:54 +0×1b8
net/http.HandlerFunc.ServeHTTP(0xc0009259e0, 0×38c1b80, 0xc00276c460, 0xc002ef2100)
/opt/hostedtoolcache/go/1.13.15/x64/src/net/http/server.go:2036 +0×44
kubesphere.io/kubesphere/pkg/apiserver/filters.WithAuthorization.func1(0×38c1b80, 0xc00276c460, 0xc002ef2100)
/home/runner/work/kubesphere/kubesphere/pkg/apiserver/filters/authorization.go:50 +0×37c
net/http.HandlerFunc.ServeHTTP(0xc000951b80, 0×38c1b80, 0xc00276c460, 0xc002ef2100)
/opt/hostedtoolcache/go/1.13.15/x64/src/net/http/server.go:2036 +0×44
kubesphere.io/kubesphere/pkg/apiserver/filters.WithAuthentication.func1(0×38c1b80, 0xc00276c460, 0xc00361b500)
/home/runner/work/kubesphere/kubesphere/pkg/apiserver/filters/authentication.go:95 +0xb6b
net/http.HandlerFunc.ServeHTTP(0xc000142eb0, 0×38c1b80, 0xc00276c460, 0xc00361b500)
/opt/hostedtoolcache/go/1.13.15/x64/src/net/http/server.go:2036 +0×44
kubesphere.io/kubesphere/pkg/apiserver/filters.WithRequestInfo.func1(0×38c1b80, 0xc00276c460, 0xc00361b400)
/home/runner/work/kubesphere/kubesphere/pkg/apiserver/filters/requestinfo.go:67 +0×3c5
net/http.HandlerFunc.ServeHTTP(0xc000819950, 0×38c1b80, 0xc00276c460, 0xc00361b400)
/opt/hostedtoolcache/go/1.13.15/x64/src/net/http/server.go:2036 +0×44
net/http.serverHandler.ServeHTTP(0xc0009b4000, 0×38c1b80, 0xc00276c460, 0xc00361b400)
/opt/hostedtoolcache/go/1.13.15/x64/src/net/http/server.go:2831 +0xa4
net/http.(conn).serve(0xc002a00780, 0×38d4780, 0xc002d2cc80)
/opt/hostedtoolcache/go/1.13.15/x64/src/net/http/server.go:1919 +0×875
created by net/http.(Server).Serve
/opt/hostedtoolcache/go/1.13.15/x64/src/net/http/server.go:2957 +0×384

cntsp

stoneshi-yunify
查看一下所有的pvc

stoneshi-yunify

cntsp 看起来都有storageclassname。。。。再看看是否所有pvc都有annotations. 如果pvc没有任何annotations，也会触发这个bug。

kubectl get sc -o name | xargs -n 1 -I % sh -c '{ echo %;kubectl get % -o jsonpath='{.metadata.annotations}'; }'

cntsp

stoneshi-yunify
root@k8smaster1:~/k8s/db/mysql# kubectl get sc -o name | xargs -n 1 -I % sh -c '{ echo %;kubectl get % -o jsonpath='{.metadata.annotations}'; }' storageclass.storage.k8s.io/managed-nfs-storage map[kubectl.kubernetes.io/last-applied-configuration:{"apiVersion":"storage.k8s.io/v1","kind":"StorageClass","metadata":{"annotations":{},"name":"managed-nfs-storage"},"parameters":{"archiveOnDelete":"false"},"provisioner":"fuseim.pri/ifs"} storageclass.kubesphere.io/support-snapshot:false]

cntsp

stoneshi-yunify
这个是annotations 是不是需要自己单独添加呢？但是查看pvc的yaml文件都是由注解的

cntsp

stoneshi-yunify
部署定义这个存储类的yaml文件中需要定义注解anntations，我目前往这个文件中添加了annotations。重新执行了 kubectl apply -f class.yml

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: managed-nfs-storage
  annotations:
    volume.beta.kubernetes.io/storage-class: "nfs-stroage-class"
provisioner: fuseim.pri/ifs # or choose another name, must match deployment's env PROVISIONER_NAME'
parameters:
  archiveOnDelete: "false"

执行后，现在还是不行的

stoneshi-yunify

cntsp 你打开浏览器的开发者模式，切换到网络，刷新下页面，看看具体哪个request返回了internal server error，request和response分别是什么

stoneshi-yunify

cntsp sorry, 搞错了。是查看pvc是否有annotations。没有annotation的pvc会触发这个bug。你看下

cntsp

stoneshi-yunify

cntsp

stoneshi-yunify

cntsp 查看是否所有的pvc都有annotations

stoneshi-yunify

cntsp 截response

cntsp

stoneshi-yunify
多谢，解决问题了！

cntsp

stoneshi-yunify

cntsp okay，目前能hit到这个bug的只有两种情况：1. 至少有一个pvc的storageclassname为空 2.至少有一个pvc的annotations为空。目前看来1应该排除了，那可能就是2. 你看下。如果也不是2。那可能就是前端bug了。。。

cntsp

stoneshi-yunify
搞定了，问题就出在你说的1

cntsp

stoneshi-yunify
有几个的pvc的 spec 下缺少： storageClassName: “managed-nfs-storage” 这个字段

cntsp

stoneshi-yunify

cntsp okay，解决了就好。那为啥kubectl get pvc -A 的时候显示了storageclass了呢，奇怪啊