Istio API Security in Kubernetes with JWT

Earlier we installed Istio and deployed vecho into K8s.

This should be up and running, if we call the endpoint.

curl -i vadal.local/echo

We now want to secure access to this service like we did with Kong.

For security in Istio, we need to ensure the following is set for the service or else you will get a 503 error.

kubectl edit svc/vecho -n vadal

Make sure it has name: http set

  ports:
  - name: http
    port: 80
    protocol: TCP
    targetPort: 8080

Add authorization policy

apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  name: vecho-get
  namespace: vadal
spec:
  selector:
    matchLabels:
      app: vecho
  action: ALLOW
  rules:
  - to:
    - operation:
        methods: ["GET"]

kubectl apply -f <above file>

curl -i vadal.local/echo -X POST
HTTP/1.1 403 Forbidden
content-length: 19
content-type: text/plain
date: Sun, 12 Jul 2020 23:05:03 GMT
server: istio-envoy
x-envoy-upstream-service-time: 3
RBAC: access denied

curl -i vadal.local/echo -X GET
HTTP/1.1 200 OK
content-type: application/json
date: Sun, 12 Jul 2020 23:05:38 GMT
x-envoy-upstream-service-time: 8
server: istio-envoy
transfer-encoding: chunked
{"timestamp":"2020-07-12T23:05:38.318","headers":{"host":"vadal.local","user-agent":"curl/7.64.1","accept":"/","x-b3-sampled":"1","x-forwarded-proto":"http","x-request-id":"6c0ded60-6b20-9eea-b599-1392d1053e56","x-envoy-original-path":"/echo","content-length":"0","x-envoy-internal":"true","x-forwarded-client-cert":"By=spiffe://cluster.local/ns/vadal/sa/default;Hash=aae1e29c7a34b9f1f614d868f212f936019bd2caa7560221224ef70578705c31;Subject="";URI=spiffe://cluster.local/ns/istio-system/sa/istio-ingressgateway-service-account","x-b3-traceid":"c11e0dcd7b8727b19adf3e7a5dd5b589","x-b3-spanid":"294e70a066ee29a7","x-b3-parentspanid":"9adf3e7a5dd5b589"}}

All good.

Key Header Access

apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  name: vecho-get
  namespace: vadal
spec:
  selector:
    matchLabels:
      app: vecho
  action: ALLOW
  rules:
  - to:
    - operation:
        methods: ["GET"]
    when:
      - key: request.headers[x-vadal-key]
        values: ["secret"]

Access will be denied unless you provide the correct header and value.

curl -i http://vadal.local/echo -H "x-vadal-key:secret"

JWT Access

In Istio the JWT settings are defined with JSON Web Key Sets (JWKS).

First create a JWT test token (RS256) which we will use to secure our API.

https://jwt.io/#debugger-io?token=eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJ2YWRhbCIsImlhdCI6MTUxNjIzOTAyMn0.fZjUjMZA4Zk06HMwUhWWyN_9390eyPq1NHc8PkCD_9ZX2N6hqoPyoKeoUJlz2-914F-zSYa31YF1Z0tLPkf8oUCoC-8NaUmgcpB178g957b0ugg39mZGRi2DDiJSPPaQ1S_0xfHjQQwl46S0UoNnDDzcFuEngZ_Rnpviyoi-LZ4CcwamYP19N-WmXOZvESnPJ9MhSzudBWArLAarFQ74tg6Lmf3GWRuTKzdECd29IBq-CdQvdP6YxsMvceoxZXfy76jrRLbFkjfEn6fT_SWPL-FN3zX-emaiYGyfFGB9nmOjgCph0F20oy5Gf5ffEVxM4uzMerzheawqtg3HDResYQ&publicKey=-----BEGIN PUBLIC KEY----- MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAnzyis1ZjfNB0bBgKFMSv vkTtwlvBsaJq7S5wA%2BkzeVOVpVWwkWdVha4s38XM%2Fpa%2Fyr47av7%2Bz3VTmvDRyAHc aT92whREFpLv9cj5lTeJSibyr%2FMrm%2FYtjCZVWgaOYIhwrXwKLqPr%2F11inWsAkfIy tvHWTxZYEcXLgAXFuUuaS3uF9gEiNQwzGTU1v0FqkqTBr4B8nW3HCN47XUu0t8Y0 e%2Blf4s4OxQawWD79J9%2F5d3Ry0vbV3Am1FtGJiJvOwRsIfVChDpYStTcHTCMqtvWb V6L11BWkpzGXSW4Hv43qa%2BGSYOD2QU68Mb59oSk2OB%2BBtOLpJofmbGEGgvmwyCI9 MwIDAQAB -----END PUBLIC KEY-----

Now we need the corresponding JWK, copy the public key to this site:

https://russelldavies.github.io/jwk-creator/

Copy the converted JWK to the block below within keys: []:

apiVersion: security.istio.io/v1beta1
kind: "RequestAuthentication"
metadata:
  name: vecho-jwt
  namespace: vadal
spec:
  selector:
    matchLabels:
      app: vecho
  jwtRules:
    - outputPayloadToHeader: vadaltoken
      issuer: "vadal"
      jwks: |
        {
          keys: [
            {
              "kty": "RSA",
              "n": "nzyis1ZjfNB0bBgKFMSvvkTtwlvBsaJq7S5wA-kzeVOVpVWwkWdVha4s38XM_pa_yr47av7-z3VTmvDRyAHcaT92whREFpLv9cj5lTeJSibyr_Mrm_YtjCZVWgaOYIhwrXwKLqPr_11inWsAkfIytvHWTxZYEcXLgAXFuUuaS3uF9gEiNQwzGTU1v0FqkqTBr4B8nW3HCN47XUu0t8Y0e-lf4s4OxQawWD79J9_5d3Ry0vbV3Am1FtGJiJvOwRsIfVChDpYStTcHTCMqtvWbV6L11BWkpzGXSW4Hv43qa-GSYOD2QU68Mb59oSk2OB-BtOLpJofmbGEGgvmwyCI9Mw",
              "e": "AQAB",
              "alg": "RS256",
              "use": "sig"
            }
          ]
        }

---
apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  name: vecho-get
  namespace: vadal
spec:
  selector:
    matchLabels:
      app: vecho
  action: ALLOW
  rules:
  - to:
    - operation:
        methods: ["GET"]
    when:
      - key: request.auth.claims[iss]
        values: ["vadal"]

kubectl apply -f <above file>

We have added RequestAuthentication and added to AuthorizationPolicy the when clause which expects an iss equal to vadal.

curl -i vadal.local/echo
HTTP/1.1 403 Forbidden
content-length: 19
content-type: text/plain
date: Sun, 12 Jul 2020 23:08:17 GMT
server: istio-envoy
x-envoy-upstream-service-time: 6

RBAC: access denied

Now use the token above:

TOKEN=eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJ2YWRhbCIsImlhdCI6MTUxNjIzOTAyMn0.fZjUjMZA4Zk06HMwUhWWyN_9390eyPq1NHc8PkCD_9ZX2N6hqoPyoKeoUJlz2-914F-zSYa31YF1Z0tLPkf8oUCoC-8NaUmgcpB178g957b0ugg39mZGRi2DDiJSPPaQ1S_0xfHjQQwl46S0UoNnDDzcFuEngZ_Rnpviyoi-LZ4CcwamYP19N-WmXOZvESnPJ9MhSzudBWArLAarFQ74tg6Lmf3GWRuTKzdECd29IBq-CdQvdP6YxsMvceoxZXfy76jrRLbFkjfEn6fT_SWPL-FN3zX-emaiYGyfFGB9nmOjgCph0F20oy5Gf5ffEVxM4uzMerzheawqtg3HDResYQ

curl -i http://vadal.local/echo -H "Authorization: Bearer $TOKEN"

HTTP/1.1 200 OK
content-type: application/json
date: Mon, 13 Jul 2020 00:12:59 GMT
x-envoy-upstream-service-time: 8
server: istio-envoy
transfer-encoding: chunked

{"timestamp":"2020-07-13T00:32:42.107","headers":{"host":"vadal.local","user-agent":"curl/7.64.1","accept":"/","authorization":"Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJ2YWRhbCIsImlhdCI6MTUxNjIzOTAyMn0.fZjUjMZA4Zk06HMwUhWWyN_9390eyPq1NHc8PkCD_9ZX2N6hqoPyoKeoUJlz2-914F-zSYa31YF1Z0tLPkf8oUCoC-8NaUmgcpB178g957b0ugg39mZGRi2DDiJSPPaQ1S_0xfHjQQwl46S0UoNnDDzcFuEngZ_Rnpviyoi-LZ4CcwamYP19N-WmXOZvESnPJ9MhSzudBWArLAarFQ74tg6Lmf3GWRuTKzdECd29IBq-CdQvdP6YxsMvceoxZXfy76jrRLbFkjfEn6fT_SWPL-FN3zX-emaiYGyfFGB9nmOjgCph0F20oy5Gf5ffEVxM4uzMerzheawqtg3HDResYQ","x-b3-sampled":"1","x-forwarded-proto":"http","x-request-id":"54b2bab0-8b7b-90d5-ad4d-d5c61ddb5c17","x-envoy-original-path":"/echo","content-length":"0","x-envoy-internal":"true","x-forwarded-client-cert":"By=spiffe://cluster.local/ns/vadal/sa/default;Hash=aae1e29c7a34b9f1f614d868f212f936019bd2caa7560221224ef70578705c31;Subject="";URI=spiffe://cluster.local/ns/istio-system/sa/istio-ingressgateway-service-account","vadaltoken":"eyJpc3MiOiJ2YWRhbCIsImlhdCI6MTUxNjIzOTAyMn0","x-b3-traceid":"df7bf539adf2fe23035466d917e4b60c","x-b3-spanid":"b937e924a214b861","x-b3-parentspanid":"035466d917e4b60c"}}

Access is now controlled by JWT RS256.

Similarly we can also use a HS256 JWT.

Generate JWK from here

https://mkjwk.org/

Add a new RequestAuthentication with a metadata name of vadal-jwt-oct and the following keys copied from the site above.

{
    "keys": [
        {
            "kty": "oct",
            "use": "sig",
            "kid": "vadal",
            "k": "_kR1V8tTZ6z4-u2sQ3B4BfAx96773EHZQMi5g6baUQXsxHfSLU5EucNDycQqW3QkA2V0oXOLn4zoj6nSehJz-pidWVjP3-QGOZcj4lU5B2bU0nzjUNwgvTQM0h2ooyhlG8zOgaSPbmXYRh9eXlLVK2Zn8GFBk-YRciTejrkMpP1bLw70LB5rHtMwSY-hUJ-xEx1v7GeibNCglVp5BnAqXZM47fEhFxxcz1vw7sNnBM-s_sV58Hg4tY0nSKO7-iMCuCs2XAPfcxVY8JaVwZjrN9hLITsqJsvQlDj48uB6W1f4T94Xydc5cCjKsXkSk169ii5gNJLZFaWPCNupk1FFSA",
            "alg": "HS256"
        }
    ]
}

To obtain the corresponding HS256 JWT we need to grab the K value.

The k value -> _KR1V... needs to go into the secret field in the link below (and secret is based 64,  selected)

https://jwt.io/#debugger-io?token=eyJhbGciOiJIUzI1NiIsImtpZCI6InZhZGFsIiwidHlwIjoiSldUIn0.eyJpc3MiOiJ2YWRhbCIsImlhdCI6MTUxNjIzOTAyMn0.wKGe7fJG4Vp0ZDXtf0RaHyXTUL0tnWGA10ptGlB1Cp4

Set a new variable with the resulting token.

TOKENH=eyJhbGciOiJIUzI1NiIsImtpZCI6InZhZGFsIiwidHlwIjoiSldUIn0.eyJpc3MiOiJ2YWRhbCIsImlhdCI6MTUxNjIzOTAyMn0.wKGe7fJG4Vp0ZDXtf0RaHyXTUL0tnWGA10ptGlB1Cp4

curl -i http://vadal.local/echo -H "Authorization: Bearer $TOKENH"

HTTP/1.1 200 OK
content-type: application/json
date: Mon, 13 Jul 2020 01:07:04 GMT
x-envoy-upstream-service-time: 6
server: istio-envoy
transfer-encoding: chunked

{"timestamp":"2020-07-13T01:07:04.348","headers":{"host":"vadal.local","user-agent":"curl/7.64.1","accept":"/","x-b3-sampled":"1","x-forwarded-proto":"http","x-request-id":"097c3aa6-5284-993b-89be-167556119b13","x-envoy-original-path":"/echo","content-length":"0","x-envoy-internal":"true","x-forwarded-client-cert":"By=spiffe://cluster.local/ns/vadal/sa/default;Hash=aae1e29c7a34b9f1f614d868f212f936019bd2caa7560221224ef70578705c31;Subject="";URI=spiffe://cluster.local/ns/istio-system/sa/istio-ingressgateway-service-account","vadaltoken":"eyJpc3MiOiJ2YWRhbCIsImlhdCI6MTUxNjIzOTAyMn0","x-b3-traceid":"c658d2e664c825a7435899abdefffd7c","x-b3-spanid":"fee9e51531bd63a6","x-b3-parentspanid":"435899abdefffd7c"}}

JWT HS256 access.

As seen above, the one authorization policy can have multiple separate request authentication definitions.

Conclusion

We have added a API Header key, and RS256 as well as HS256 JWT to control access to our service.

This is a bit more involved than Kong due to its use of JWKS, but works in the same way. We saw how Authorisation policies could be applied to workloads directly.

Further Details

https://istio.io/latest/docs/concepts/security/

https://tools.ietf.org/html/rfc7517#section-4.3

https://mkjwk.org/

https://russelldavies.github.io/jwk-creator/

https://www.keycloak.org/