Istio API Security with Keycloak in Kubernetes

Having added JWT directly into Istio API service security, we now instead use Keycloak to act as our OIDC/JWT provider.

Install Keycloak

helm repo add codecentric https://codecentric.github.io/helm-charts
helm install codecentric/keycloak --name keycloak

NAME:   keycloak
NAMESPACE: default
STATUS: DEPLOYED

RESOURCES:
==> v1/ConfigMap
NAME              DATA  AGE
keycloak-sh       1     0s
keycloak-startup  1     0s
keycloak-test     1     0s

==> v1/Pod(related)
NAME        READY  STATUS             RESTARTS  AGE
keycloak-0  0/1    ContainerCreating  0         0s

==> v1/Secret
NAME           TYPE    DATA  AGE
keycloak-db    Opaque  2     0s
keycloak-http  Opaque  1     0s

==> v1/Service
NAME               TYPE       CLUSTER-IP     EXTERNAL-IP  PORT(S)          AGE
keycloak-headless  ClusterIP  None                  80/TCP,8443/TCP  0s
keycloak-http      ClusterIP  10.96.125.143         80/TCP,8443/TCP  0s

==> v1/StatefulSet
NAME      READY  AGE
keycloak  0/1    0s

NOTES:
Keycloak can be accessed:
Within your cluster, at the following DNS name at port 80:
keycloak-http.default.svc.cluster.local

Add Realm and User to Keycloak

To access the Keycloak GUI carry out the following steps.

Edit the keycloak-http service and change ClusterIP to NodePort and add nodePort: say 30006 (assuming it doesn't clash with anything you have already).

kubectl edit svc/keycloak-http

spec:
  ports:
    - name: http
      protocol: TCP
      port: 80
      targetPort: http
      nodePort: 30006
    - name: https
      protocol: TCP
      port: 8443
      targetPort: https
      nodePort: 30090
  selector:
    app.kubernetes.io/instance: keycloak
    app.kubernetes.io/name: keycloak
  clusterIP: 10.98.46.34
  type: NodePort

Login to http://localhost:30006

Username: keycloak
Initial password:

kubectl get secret --namespace default keycloak-http -o jsonpath="{.data.password}" | base64 --decode; echo

Set up a realm (vadal), a client (vadal), a user (vadal) with password (vadal) and a role (vadaluser) assigned to vadal user (see https://labs.consol.de/development/2020/05/07/istio-and-keycloak.html ).

You should end up with something like this.

Configure Istio authentication and authorisation.

cat <<EOF | kubectl apply -f -
apiVersion: security.istio.io/v1beta1
kind: "RequestAuthentication"
metadata:
  name: vecho-jwt
  namespace: vadal
spec:
  selector:
    matchLabels:
      app: vecho
  jwtRules:
    - outputPayloadToHeader: vadaltoken
      issuer: "http://localhost:30006/auth/realms/vadal"
      jwksUri: "http://keycloak-http.default.svc.cluster.local/auth/realms/vadal/protocol/openid-connect/certs"

---

apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  name: vecho-get
  namespace: vadal
spec:
  selector:
    matchLabels:
      app: vecho
  action: ALLOW
  rules:
  - to:
    - operation:
        methods: ["GET"]
    when:
      - key: request.auth.claims[iss]
        values: ["http://localhost:30006/auth/realms/vadal"]
EOF

Note the issuer and jwksUri.

Get a token from keycloak.

curl \
  -sk \
  --data "username=vadal&password=vadal&grant_type=password&client_id=vadal" \
  http://localhost:30006/auth/realms/vadal/protocol/openid-connect/token | jq ".access_token"

(if you do not have jq installed, then remove | jq ".access_token" from the query above, and pickout just the access_token part)

{"access_token":"eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICJuZEkydGFPcHViZjltQ2RSNEhLRkJ4dXYxNGl0Q1dxTkxsWmR0X19HaWlNIn0.eyJleHAiOjE1OTQ2ODIzMjcsImlhdCI6MTU5NDY4MjAyNywianRpIjoiOTMwZGM5YmEtNDllZi00OWM5LTliOGEtOGJmNWUwY2M1YjcwIiwiaXNzIjoiaHR0cDovL2xvY2FsaG9zdDozMDAwNi9hdXRoL3JlYWxtcy92YWRhbCIsImF1ZCI6....}

NOTE: Need to use it before the expiry time.

TOKEN=eyJhbGciOiJSUzI1....

curl -i http://vadal.local/echo -H "Authorization: Bearer $TOKEN"
HTTP/1.1 200 OK
content-type: application/json
date: Mon, 13 Jul 2020 23:36:53 GMT
x-envoy-upstream-service-time: 334
server: istio-envoy
transfer-encoding: chunked

{"timestamp":"2020-07-13T23:36:53.675","headers":{"host":"vadal.local","user-agent":"curl/7.64.1","accept":"/","x-b3-sampled":"1","x-forwarded-proto":"http","x-request-id":"e250b31a-903e-9ca4-ae1a-edde00f97718","x-envoy-original-path":"/echo","content-length":"0","x-envoy-internal":"true","x-forwarded-client-cert":"By=spiffe://cluster.local/ns/vadal/sa/default;Hash=cd101b753b794f67a67f25bbcdcb391a7505d52f24e0cd340c9365d05c241a80;Subject="";URI=spiffe://cluster.local/ns/istio-system/sa/istio-ingressgateway-service-account","vadaltoken":"eyJleHAiOjE1OTQ2ODM2ODcsImlhdCI6MTU5NDY4MzM4NywianRpIjoiYzhjZTRiNGUtMzVjMC00NzQ1LTk3M2YtNTQzMjZlMzM0Njc3IiwiaXNzIjoiaHR0cDovL2xvY2FsaG9zdDozMDAwNi9hdXRoL3JlYWxtcy92YWRhbCIsImF1ZCI6ImFjY291bnQiLCJzdWIiOiI5MDNhMDUwMS03Mjc5LTQxOTMtYjkxYS03NGI5MjJmMDMyMjQiLCJ0eXAiOiJCZWFyZXIiLCJhenAiOiJ2YWRhbCIsInNlc3Npb25fc3RhdGUiOiIxM2YwOWM3Yy1iYWY3LTQ0NTMtYTVmNC1jMmEwMWExNjlhYWMiLCJhY3IiOiIxIiwiYWxsb3dlZC1vcmlnaW5zIjpbImh0dHBzOi8vd3d3LmtleWNsb2FrLm9yZyJdLCJyZWFsbV9hY2Nlc3MiOnsicm9sZXMiOlsidmFkYWx1c2VyIiwib2ZmbGluZV9hY2Nlc3MiLCJ1bWFfYXV0aG9yaXphdGlvbiJdfSwicmVzb3VyY2VfYWNjZXNzIjp7ImFjY291bnQiOnsicm9sZXMiOlsibWFuYWdlLWFjY291bnQiLCJtYW5hZ2UtYWNjb3VudC1saW5rcyIsInZpZXctcHJvZmlsZSJdfX0sInNjb3BlIjoicHJvZmlsZSBlbWFpbCIsImVtYWlsX3ZlcmlmaWVkIjpmYWxzZSwibmFtZSI6InZhZGFsIGNsb3VkIiwicHJlZmVycmVkX3VzZXJuYW1lIjoidmFkYWwiLCJnaXZlbl9uYW1lIjoidmFkYWwiLCJmYW1pbHlfbmFtZSI6ImNsb3VkIn0","x-b3-traceid":"db81914bb9612948cb7d38748d119f8e","x-b3-spanid":"daebf61620cb8212","x-b3-parentspanid":"cb7d38748d119f8e"}}

If it has expired you'll get the following and you'll need a new token.

HTTP/1.1 401 Unauthorized
content-length: 14
content-type: text/plain
date: Mon, 13 Jul 2020 23:55:15 GMT
server: istio-envoy
x-envoy-upstream-service-time: 1

Jwt is expired

Nice. With KeyCloak you can manage your token delivery and other parameters.

Conclusion

We installed keycloak, added our client, user credentials and realm to it. It then was used to provide us with a JWT which we were able to secure our API access. We had Istio authorisation defined to connect to keycloak to verify access.

Note: This keycloak installation used H2 db for it's persistence so the details will be lost once shutdown.