安装bookinfo示例总是没有注入sidecar

for-mat

service:
      name: istio-sidecar-injector
      namespace: istio-system
      path: /inject
  failurePolicy: Fail
  name: sidecar-injector.istio.io
  namespaceSelector:
    matchExpressions:
    - key: kubesphere.io/workspace
      operator: Exists
    - key: istio-injection
      operator: NotIn
      values:
      - disabled
  rules:
  - apiGroups:
    - ""
    apiVersions:
    - v1
    operations:
    - CREATE
    resources:
    - pods
  sideEffects: Unknown

for-mat

Jeff kubectl get mutatingwebhookconfigurations.admissionregistration.k8s.io istio-sidecar-injector -o yaml
看到了一个disabled

Jeff

for-mat 贴下全部内容

for-mat

[root@master100 ~]# kubectl -n istio-system get cm istio-sidecar-injector -o yaml                                        
apiVersion: v1
data:
  config: "policy: disabled\ntemplate: |-\n  rewriteAppHTTPProbe: false\n  initContainers:\n
    \ [[ if ne (annotation .ObjectMeta `sidecar.istio.io/interceptionMode` .ProxyConfig.InterceptionMode)
    \"NONE\" ]]\n  - name: istio-init\n    image: \"istio/proxy_init:1.1.1\"\n    args:\n
    \   - \"-p\"\n    - [[ .MeshConfig.ProxyListenPort ]]\n    - \"-u\"\n    - 1337\n
    \   - \"-m\"\n    - [[ annotation .ObjectMeta `sidecar.istio.io/interceptionMode`
    .ProxyConfig.InterceptionMode ]]\n    - \"-i\"\n    - \"[[ annotation .ObjectMeta
    `traffic.sidecar.istio.io/includeOutboundIPRanges`  \"*\"  ]]\"\n    - \"-x\"\n
    \   - \"[[ annotation .ObjectMeta `traffic.sidecar.istio.io/excludeOutboundIPRanges`
    \ \"\"  ]]\"\n    - \"-b\"\n    - \"[[ annotation .ObjectMeta `traffic.sidecar.istio.io/includeInboundPorts`
    (includeInboundPorts .Spec.Containers) ]]\"\n    - \"-d\"\n    - \"[[ excludeInboundPort
    (annotation .ObjectMeta `status.sidecar.istio.io/port`  15020 ) (annotation .ObjectMeta
    `traffic.sidecar.istio.io/excludeInboundPorts`  \"\" ) ]]\"\n    [[ if (isset
    .ObjectMeta.Annotations `traffic.sidecar.istio.io/kubevirtInterfaces`) -]]\n    -
    \"-k\"\n    - \"[[ index .ObjectMeta.Annotations `traffic.sidecar.istio.io/kubevirtInterfaces`
    ]]\"\n    [[ end -]]\n    imagePullPolicy: IfNotPresent\n    resources:\n      requests:\n
    \       cpu: 10m\n        memory: 10Mi\n      limits:\n        cpu: 100m\n        memory:
    50Mi\n    securityContext:\n      capabilities:\n        add:\n        - NET_ADMIN\n
    \   restartPolicy: Always\n  [[ end -]]\n  containers:\n  - name: istio-proxy\n
    \   image: [[ annotation .ObjectMeta `sidecar.istio.io/proxyImage`  \"istio/proxyv2:1.1.1\"
    \ ]]\n    ports:\n    - containerPort: 15090\n      protocol: TCP\n      name:
    http-envoy-prom\n    args:\n    - proxy\n    - sidecar\n    - --domain\n    -
    $(POD_NAMESPACE).svc.cluster.local\n    - --configPath\n    - [[ .ProxyConfig.ConfigPath
    ]]\n    - --binaryPath\n    - [[ .ProxyConfig.BinaryPath ]]\n    - --serviceCluster\n
    \   [[ if ne \"\" (index .ObjectMeta.Labels \"app\") -]]\n    - [[ index .ObjectMeta.Labels
    \"app\" ]].$(POD_NAMESPACE)\n    [[ else -]]\n    - [[ valueOrDefault .DeploymentMeta.Name
    \"istio-proxy\" ]].[[ valueOrDefault .DeploymentMeta.Namespace \"default\" ]]\n
    \   [[ end -]]\n    - --drainDuration\n    - [[ formatDuration .ProxyConfig.DrainDuration
    ]]\n    - --parentShutdownDuration\n    - [[ formatDuration .ProxyConfig.ParentShutdownDuration
    ]]\n    - --discoveryAddress\n    - [[ annotation .ObjectMeta `sidecar.istio.io/discoveryAddress`
    .ProxyConfig.DiscoveryAddress ]]\n    - --zipkinAddress\n    - [[ .ProxyConfig.GetTracing.GetZipkin.GetAddress
    ]]\n    - --connectTimeout\n    - [[ formatDuration .ProxyConfig.ConnectTimeout
    ]]\n    - --proxyAdminPort\n    - [[ .ProxyConfig.ProxyAdminPort ]]\n    [[ if
    gt .ProxyConfig.Concurrency 0 -]]\n    - --concurrency\n    - [[ .ProxyConfig.Concurrency
    ]]\n    [[ end -]]\n    - --controlPlaneAuthPolicy\n    - [[ annotation .ObjectMeta
    `sidecar.istio.io/controlPlaneAuthPolicy` .ProxyConfig.ControlPlaneAuthPolicy
    ]]\n  [[- if (ne (annotation .ObjectMeta `status.sidecar.istio.io/port`  15020
    ) \"0\") ]]\n    - --statusPort\n    - [[ annotation .ObjectMeta `status.sidecar.istio.io/port`
    \ 15020  ]]\n    - --applicationPorts\n    - \"[[ annotation .ObjectMeta `readiness.status.sidecar.istio.io/applicationPorts`
    (applicationPorts .Spec.Containers) ]]\"\n  [[- end ]]\n    env:\n    - name:
    POD_NAME\n      valueFrom:\n        fieldRef:\n          fieldPath: metadata.name\n
    \   - name: POD_NAMESPACE\n      valueFrom:\n        fieldRef:\n          fieldPath:
    metadata.namespace\n    - name: INSTANCE_IP\n      valueFrom:\n        fieldRef:\n
    \         fieldPath: status.podIP\n    - name: ISTIO_META_POD_NAME\n      valueFrom:\n
    \       fieldRef:\n          fieldPath: metadata.name\n    - name: ISTIO_META_CONFIG_NAMESPACE\n
    \     valueFrom:\n        fieldRef:\n          fieldPath: metadata.namespace\n
    \   - name: ISTIO_META_INTERCEPTION_MODE\n      value: [[ or (index .ObjectMeta.Annotations
    \"sidecar.istio.io/interceptionMode\") .ProxyConfig.InterceptionMode.String ]]\n
    \   [[ if .ObjectMeta.Annotations ]]\n    - name: ISTIO_METAJSON_ANNOTATIONS\n
    \     value: |\n             [[ toJSON .ObjectMeta.Annotations ]]\n    [[ end
    ]]\n    [[ if .ObjectMeta.Labels ]]\n    - name: ISTIO_METAJSON_LABELS\n      value:
    |\n             [[ toJSON .ObjectMeta.Labels ]]\n    [[ end ]]\n    [[- if (isset
    .ObjectMeta.Annotations `sidecar.istio.io/bootstrapOverride`) ]]\n    - name:
    ISTIO_BOOTSTRAP_OVERRIDE\n      value: \"/etc/istio/custom-bootstrap/custom_bootstrap.json\"\n
    \   [[- end ]]\n    imagePullPolicy: IfNotPresent\n    [[ if (ne (annotation .ObjectMeta
    `status.sidecar.istio.io/port`  15020 ) \"0\") ]]\n    readinessProbe:\n      httpGet:\n
    \       path: /healthz/ready\n        port: [[ annotation .ObjectMeta `status.sidecar.istio.io/port`
    \ 15020  ]]\n      initialDelaySeconds: [[ annotation .ObjectMeta `readiness.status.sidecar.istio.io/initialDelaySeconds`
    \ 1  ]]\n      periodSeconds: [[ annotation .ObjectMeta `readiness.status.sidecar.istio.io/periodSeconds`
    \ 2  ]]\n      failureThreshold: [[ annotation .ObjectMeta `readiness.status.sidecar.istio.io/failureThreshold`
    \ 30  ]]\n    [[ end -]]securityContext:\n      readOnlyRootFilesystem: true\n
    \     [[ if eq (annotation .ObjectMeta `sidecar.istio.io/interceptionMode` .ProxyConfig.InterceptionMode)
    \"TPROXY\" -]]\n      capabilities:\n        add:\n        - NET_ADMIN\n      runAsGroup:
    1337\n      [[ else -]]\n      \n      runAsUser: 1337\n      [[- end ]]\n    resources:\n
    \     [[ if or (isset .ObjectMeta.Annotations `sidecar.istio.io/proxyCPU`) (isset
    .ObjectMeta.Annotations `sidecar.istio.io/proxyMemory`) -]]\n      requests:\n
    \       [[ if (isset .ObjectMeta.Annotations `sidecar.istio.io/proxyCPU`) -]]\n
    \       cpu: \"[[ index .ObjectMeta.Annotations `sidecar.istio.io/proxyCPU` ]]\"\n
    \       [[ end ]]\n        [[ if (isset .ObjectMeta.Annotations `sidecar.istio.io/proxyMemory`)
    -]]\n        memory: \"[[ index .ObjectMeta.Annotations `sidecar.istio.io/proxyMemory`
    ]]\"\n        [[ end ]]\n    [[ else -]]\n      limits:\n        cpu: 2000m\n
    \       memory: 128Mi\n      requests:\n        cpu: 100m\n        memory: 128Mi\n
    \     \n    [[ end -]]\n    volumeMounts:\n    [[- if (isset .ObjectMeta.Annotations
    `sidecar.istio.io/bootstrapOverride`) ]]\n    - mountPath: /etc/istio/custom-bootstrap\n
    \     name: custom-bootstrap-volume\n    [[- end ]]\n    - mountPath: /etc/istio/proxy\n
    \     name: istio-envoy\n    - mountPath: /etc/certs/\n      name: istio-certs\n
    \     readOnly: true\n      [[- if isset .ObjectMeta.Annotations `sidecar.istio.io/userVolumeMount`
    ]]\n      [[ range $index, $value := fromJSON (index .ObjectMeta.Annotations `sidecar.istio.io/userVolumeMount`)
    ]]\n    - name: \"[[ $index ]]\"\n      [[ toYaml $value | indent 4 ]]\n      [[
    end ]]\n      [[- end ]]\n  volumes:\n  [[- if (isset .ObjectMeta.Annotations
    `sidecar.istio.io/bootstrapOverride`) ]]\n  - name: custom-bootstrap-volume\n
    \   configMap:\n      name: [[ annotation .ObjectMeta `sidecar.istio.io/bootstrapOverride`
    `` ]]\n  [[- end ]]\n  - emptyDir:\n      medium: Memory\n    name: istio-envoy\n
    \ - name: istio-certs\n    secret:\n      optional: true\n      [[ if eq .Spec.ServiceAccountName
    \"\" -]]\n      secretName: istio.default\n      [[ else -]]\n      secretName:
    [[ printf \"istio.%s\" .Spec.ServiceAccountName ]]\n      [[ end -]]\n    [[-
    if isset .ObjectMeta.Annotations `sidecar.istio.io/userVolume` ]]\n    [[ range
    $index, $value := fromJSON (index .ObjectMeta.Annotations `sidecar.istio.io/userVolume`)
    ]]\n  - name: \"[[ $index ]]\"\n    [[ toYaml $value | indent 2 ]]\n    [[ end
    ]]\n    [[ end ]]"
kind: ConfigMap
metadata:
  creationTimestamp: "2019-10-16T10:42:41Z"
  labels:
    app: istio
    chart: istio-1.1.0
    heritage: Tiller
    istio: sidecar-injector
    release: istio
  name: istio-sidecar-injector
  namespace: istio-system
  resourceVersion: "9551076"
  selfLink: /api/v1/namespaces/istio-system/configmaps/istio-sidecar-injector
  uid: ae06ba5e-f001-11e9-9015-52560ade2365

for-mat

[root@master100 ~]# kubectl get mutatingwebhookconfigurations.admissionregistration.k8s.io istio-sidecar-injector -o yamlapiVersion: admissionregistration.k8s.io/v1beta1
kind: MutatingWebhookConfiguration
metadata:
  creationTimestamp: "2019-10-16T10:42:42Z"
  generation: 2
  labels:
    app: sidecarInjectorWebhook
    chart: sidecarInjectorWebhook
    heritage: Tiller
    release: istio
  name: istio-sidecar-injector
  resourceVersion: "9551721"
  selfLink: /apis/admissionregistration.k8s.io/v1beta1/mutatingwebhookconfigurations/istio-sidecar-injector
  uid: ae3fa913-f001-11e9-9015-52560ade2365
webhooks:
- clientConfig:
    caBundle: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUMzakNDQWNhZ0F3SUJBZ0lSQU1LTUpLL051MGJON0Ezb1ZYQlFLeE13RFFZSktvWklodmNOQVFFTEJRQXcKR0RFV01CUUdBMVVFQ2hNTlkyeDFjM1JsY2k1c2IyTmhiREFlRncweE9URXdNVFl4TURReU5ETmFGdzB5TURFdwpNVFV4TURReU5ETmFNQmd4RmpBVUJnTlZCQW9URFdOc2RYTjBaWEl1Ykc5allXd3dnZ0VpTUEwR0NTcUdTSWIzCkRRRUJBUVVBQTRJQkR3QXdnZ0VLQW9JQkFRRFpCeUgzSmNKMVh1WlFtQVVyZG41OXNlSlNzSkNGZzJEdWdyMHYKNUlVaFpldzVSaHJaWVcvalllTnFOWXhIdzVBektZYUZLa0xadWhrcmR5dStMR0hkM0lVQjZvVWhORVVjZ2xxbwozQ1ZhTkpnU1ljQU9DblM5ZUZLcDBIYmI1WUUyS0xCUSsrWmQ5YVhOQjF0MElsUGMxZmJYMjFsWUZBRExTUnR4Cnc3ZG5VV3R2RFFqdWhRVnZjS1lkNUNsbEFxbEpEWWlnUEJ0TktFUG0zcW4wcU5GdVNUTzZJMXlZelNrZ3VRZ1cKMm1UTERHZzVIU1J4RGR4aVh6dXdBY241SWlSVGI2VnBtNENZK2Mwcmk4bURQa2pXdWNGYVdGNWhQMncxWkQ5Vwp6eHk3OHVGV0xWUW9DeXl4TjlKcCtsM1BjVi9JTHlsVURqdlJEcnZiU1RiQnZ3SGxBZ01CQUFHakl6QWhNQTRHCkExVWREd0VCL3dRRUF3SUNCREFQQmdOVkhSTUJBZjhFQlRBREFRSC9NQTBHQ1NxR1NJYjNEUUVCQ3dVQUE0SUIKQVFCYURxZ01yOXk5eWVzcUFacHJZN1lSU0RJTXNhRUowM0s4QnhPM2poMUw0d3l1MlJRVXM0T0hVZ0FaRXZUNgpaL2lEUSt2NW5EVnc5c0RZY1UwOHNCSHp3SXowODkrTmJoUUVsN2RqdnJuU3lEZXhOeE0yM3l5Z2ZXOXpxU2duCjUyeUZFcWVYNmgvZjRINDgvVUF4Rmg4YkpvNUJhS3llOWZSSVN1NXd2SExYUjcrREgzMS80dzJGZTZaTE40N04KT3hLN1BkTGFKaSsvNDJncVdlaTZaRitqRFFlSUxydThnRlUvb0hseC9ROFBSakNkaHgrRXlncU9ZQWZNWXNrZgptZUVRNS9aRHBJazFmWVZmQkt1ZTlyWWRFSTR1QkQvemNGNjdJcW9semNSdUV1R1dKSmRtUXYveFRLWFZwSzl6CmxWQ3gyS0xrWnk4MUpVRzViTTQ1aERqdAotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==
    service:
      name: istio-sidecar-injector
      namespace: istio-system
      path: /inject
  failurePolicy: Fail
  name: sidecar-injector.istio.io
  namespaceSelector:
    matchExpressions:
    - key: kubesphere.io/workspace
      operator: Exists
    - key: istio-injection
      operator: NotIn
      values:
      - disabled
  rules:
  - apiGroups:
    - ""
    apiVersions:
    - v1
    operations:
    - CREATE
    resources:
    - pods
  sideEffects: Unknown

Jeff

策略配置看着是对的，再贴下下面这个命令的执行结果吧

kubectl -n [namespace] get deployment productpage-v1 -o yaml

for-mat

Jeff

[root@master100 ~]# kubectl -n test-namespace get deployment productpage-v1 -o yaml  
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  annotations:
    creator: admin
    deployment.kubernetes.io/revision: "1"
    kubesphere.io/isElasticReplicas: "false"
    servicemesh.kubesphere.io/enabled: "true"
  creationTimestamp: "2019-10-17T03:29:35Z"
  generation: 1
  labels:
    app: productpage
    app.kubernetes.io/name: bookinfo
    app.kubernetes.io/version: v1
    version: v1
  name: productpage-v1
  namespace: test-namespace
  ownerReferences:
  - apiVersion: app.k8s.io/v1beta1
    blockOwnerDeletion: true
    controller: false
    kind: Application
    name: bookinfo
    uid: 57b01a89-f08e-11e9-93ab-52560ade2364
  resourceVersion: "9687755"
  selfLink: /apis/extensions/v1beta1/namespaces/test-namespace/deployments/productpage-v1
  uid: 57b037eb-f08e-11e9-93ab-52560ade2364
spec:
  progressDeadlineSeconds: 600
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app: productpage
      app.kubernetes.io/name: bookinfo
      app.kubernetes.io/version: v1
      version: v1
  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
    type: RollingUpdate
  template:
    metadata:
      annotations:
        sidecar.istio.io/inject: "true"
      creationTimestamp: null
      labels:
        app: productpage
        app.kubernetes.io/name: bookinfo
        app.kubernetes.io/version: v1
        version: v1
    spec:
      containers:
      - image: kubesphere/examples-bookinfo-productpage-v1:1.13.0
        imagePullPolicy: IfNotPresent
        name: productpage
        ports:
        - containerPort: 9080
          name: http-web
          protocol: TCP
        resources:
          limits:
            cpu: "1"
            memory: 1000Mi
          requests:
            cpu: 10m
            memory: 10Mi
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      serviceAccount: default
      serviceAccountName: default
      terminationGracePeriodSeconds: 30
status:
  availableReplicas: 1
  conditions:
  - lastTransitionTime: "2019-10-17T03:29:56Z"
    lastUpdateTime: "2019-10-17T03:29:56Z"
    message: Deployment has minimum availability.
    reason: MinimumReplicasAvailable
    status: "True"
    type: Available
  - lastTransitionTime: "2019-10-17T03:29:35Z"
    lastUpdateTime: "2019-10-17T03:29:56Z"
    message: ReplicaSet "productpage-v1-579dfbcddd" has successfully progressed.
    reason: NewReplicaSetAvailable
    status: "True"
    type: Progressing
  observedGeneration: 1
  readyReplicas: 1
  replicas: 1
  updatedReplicas: 1

Jeff

这个配置看着也是对的，你的电脑能够远程么，我看下，可以把登录方式发到 kubesphere@yunify.com

for-mat

没法远程。。

我到这一步，都是正常的，这里也显示了istio-proxy，创建后是就绪的，但是查看pod状态，就没有这个istio-proxy了

for-mat

不行我再手动部署istio官方的实例试试

Jeff

for-mat 执行下这个命令看下，ns是否打上label了

kubectl get ns test-namespace -o yaml

for-mat

Jeff label是我看istio文档后手动打的

[root@master100 ~]# kubectl get ns test-namespace -o yaml
apiVersion: v1
kind: Namespace
metadata:
  annotations:
    creator: admin
    openpitrix_runtime: runtime-BVzjOO3LRJQA
  creationTimestamp: "2019-10-16T11:22:14Z"
  finalizers:
  - finalizers.kubesphere.io/namespaces
  labels:
    istio-injection: enabled
    kubesphere.io/workspace: test-workspace
  name: test-namespace
  ownerReferences:
  - apiVersion: tenant.kubesphere.io/v1alpha1
    blockOwnerDeletion: true
    controller: true
    kind: Workspace
    name: test-workspace
    uid: 233958f4-f007-11e9-93ab-52560ade2364
  resourceVersion: "9674211"
  selfLink: /api/v1/namespaces/test-namespace
  uid: 3486cf19-f007-11e9-9044-52560ade2365
spec:
  finalizers:
  - kubernetes
status:
  phase: Active

Jeff

for-mat 把这个label去掉 istio-injection: enabled，重新部署应用试下

for-mat

Jeff 去掉试了还是不行，另外我照着istio官方文档操作了下
也没有注入sidecar，我检查了apiserver启动项
–admission-control=NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,ResourceQuota,NodeRestriction \
也开启了自动sidecar注入，但就是不生效。
我的k8s是1.13.10
奇了怪了，我再手动试试。。。

for-mat

Jeff 大佬，istio-sidecar-injector的configmap没有values
[root@master100 bin]# ./istioctl kube-inject -f ../samples/sleep/sleep.yaml | kubectl apply -f -
Error: missing configuration map key “values” in “istio-sidecar-injector”
error: no objects passed to apply

Jeff

for-mat 你的环境是默认安装的么，还是在已有的k8s上安装的？最好能有个环境我看下

for-mat

我滴妈呀，终于解决了
1、我是已有k8s集群上搭的kubesphere
kube-apiserver的启动参数中，要有–admission-control=MutatingAdmissionWebhook，开启自动注入
2、MutatingWebhookConfiguration配置有问题
kubectl -n istio-system edit MutatingWebhookConfiguration istio-sidecar-injector
下面两个参数改成In和enabled
operator: In
values:
- enabled
3、给用到的namespace打标签
kubectl label namespace test-namespace istio-injection=enabled
4、感谢大佬帮忙

Jeff

for-mat 👍️

for-mat

不知道是不是默认都会这样，我装了两次是都不行。也可能是因为我自己搭的集群

« 上一页