This post has been updated for Istio version 1.11.4
A service mesh is an architectural pattern that provides common network services as a feature of the infrastructure. This typically includes features such as service discovery and policy enforcement to control how services within the mesh can communicate with each other.
Istio is a service mesh implementation that works by running an instance of Envoy alongside each instance of each of your services to intercept and proxy service traffic. Additionally, fleets of standalone Envoys are deployed to handle traffic entering and leaving the mesh; Istio’s main purpose then is to configure and expose the functionality of Envoy.
In addition to the core features, Istio also supports powerful extension points, as well as the ability to apply custom configuration to the Envoy sidecars. Here we will describe how Istio can be configured to manage the OpenID Connect (OIDC) authentication flow for applications running within the mesh to allow both authentication and authorisation decisions to be offloaded to Istio. There are a number of ways to achieve this with Istio however here we look at two solutions and how their integration points have been affected by changes to Istio’s architecture.
OIDC is an identity layer built upon the OAuth 2.0 protocol which allows the identity of a user to be verified based on authentication to an identity provider. There are different types of authentication flow which dictate how authentication is handled by the identity provider, but the most common is the Authorization Code Flow, which we will use here.
Typically, when a user first visits an HTTP service that implements the OIDC Authorization Code Flow, they are redirected to an identity provider (for example Google, Azure or dex) where they would login to obtain an Authorization Code. The user is then redirected again back to the original service, passing the Authorization Code as an HTTP query parameter. The service (or OIDC client) can then exchange this Authorization Code (using its Client ID and Client Secret) for a JWT called an ID Token. This JWT is signed by the identity provider (which the service should verify) with fields (or claims) that contain information about the user who logged in (for example their email address) and can be used to make authentication and authorization decisions. Note that since the user does not have access to the service’s Client Secret, they are unable to obtain such a JWT locally. Instead, in order to prevent the user from having to login on every request, the service can implement login sessions; one way in which this can be achieved is by setting an HTTP cookie and then injecting the obtained JWT into subsequent requests based on the presence of this cookie. Session management is discussed further for our setup below.
OIDC is a common way of delegating the responsibility of managing user credentials to a third-party identity provider and a powerful feature of Istio is that it can be leveraged to manage this flow without your backend service needing to be aware OIDC is even being used.
For more information on OIDC and associated terminology, Okta has a great primer.
Istio 1.4 and earlier included a component called Mixer which formed part of the Istio control plane. When policy checks were enabled, before Envoy made an upstream connection it would make a logical request to Mixer in order to determine whether the connection was allowed and what action to take. Mixer therefore provided an extension point for Istio, allowing integrations with external components that could make these policy decisions on its behalf.
The App Identity and Access Adapter is an example of an external component that interfaces with Mixer in exactly this way; by analyzing attributes sent by Envoy to Mixer when making policy decisions it can work out whether a user has already authenticated to a configured identity provider and if not trigger the OIDC flow. The resulting JWT can then be compared against policy configuration to either allow or deny access to the upstream service.
Istio 1.5 and Above
Since Istio 1.5, Mixer has been deprecated in favour of implementing these extensions within Envoy itself. In particular this reduces the latency of policy decisions that would otherwise require a network call to Mixer.
Here we describe in detail an alternative way to configure Istio to manage the OIDC authentication flow and authorization decisions but without Mixer. For the purpose of this description we will assume we are running on a 1.19 GKE cluster with Istio 1.11.4 installed, but this setup should be compatible with recent versions of both Kubernetes and Istio:
We will deploy Nginx to the cluster to act as the test application we want to configure authenticaiton and authorisation for. You will need to configure a domain (I will use
nginx.lukeaddison.co.uk) to expose Nginx on. It should be pointing at the following IP address (this may take a few moments to provision):
WARNING: Once OIDC authentication is enforced on the Istio ingress gateway, cert-manager will no longer be able to renew the certificate using the HTTP-01 challenge solver configured below. One solution would be to use a DNS-01 challenge solver instead
After a few moments Nginx should become available to the internet which you can verify by curling your domain:
We can now configure OIDC authentication. The first thing we need to do is to configure our OIDC identity provider; we will be using Google here but any compatible provider should work.
For Google specifically, we need to create a Google OAuth application. The Kubeflow documentation walks through the steps to achieve this for IAP. The functional differences for us is that we want to specify
Authorized domains to be whatever domain Nginx is exposed on (so
nginx.lukeaddison.co.uk for me) and in
Authorized redirect URIs specify the
/oauth2/callback path of the Nginx domain (so
https://nginx.lukeaddison.co.uk/oauth2/callback for me).
For other providers the configuration (especially the requirement to whitelist redirect URIs) should be similar. Whichever provider you use the setup process should return a Client ID and Client Secret which you should remember for a later step.
We can now enforce that access to the Nginx service be authenticated using our OIDC provider. Using the discovery URL (supported by most providers) we can retrieve the information required to configure Istio.
The RequestAuthentication resource says that if a request to the ingress gateway contains a bearer token in the Authorization header then it must be a valid JWT signed by the specified OIDC provider. Istio will concatenate the
sub fields of the JWT with a
/ separator which will form the principal of the request. The AuthorizationPolicy says to contact oauth2-proxy for authorization decisions.
Since we have not deployed oauth2-proxy yet, visiting your domain again should now show:
RBAC: access denied, so the final thing we need to do is to deploy oauth2-proxy to manage the OIDC flow, retrieve a JWT and inject it into each request.
Envoy supports pluggable functionality known as filters which can be chained together and affect how requests are handled. One of Istio’s main roles is to configure these filters across a fleet of Envoys so that they form a mesh, supporting high-level APIs such as VirtualService and DestinationRule for operators to declare their desired mesh behaviour. The extension provider configuration we specified when we first installed Istio, together with the AuthorizationPolicy configuration above, configures the external authorization HTTP filter which is responsible for calling out to oauth2-proxy:
Before Istio 1.9, the same external authorization configuration could be supplied by applying an EnvoyFilter
Note that the above configuration tells oauth2-proxy to store session state as a browser cookie. This has the advantage of being stateless but can lead to large HTTP headers if the JWT returned from the OIDC flow is large; an alternative is to use Redis which adds a further operational burden but the size of the cookie is small and constant.
Visiting Nginx again you should be redirected to your OIDC provider. After signing in successfully oauth2-proxy should set an encrypted cookie which on subsequent requests will be decrypted to a JWT and attached as the Authorization header which Istio can validate.
If you want to authenticate multiple services you may want to configure the
--whitelist-domain flags on the oauth2-proxy Deployment to include multiple subdomains. For example, I could set both of those flags to
.lukeaddison.co.uk and then the cookie would be sent for any subdomain of
lukeaddison.co.uk meaning I would only need to sign in once. To see the full set of configuration options go here.
Another powerful use case is to combine the OIDC authentication configuration with Istio’s ability to proxy to external services. For example, if I have a service running outside of Kubernetes but that does not have its own identity-aware authentication mechanism, Istio could be used as a reverse proxy to configure access to that service in a similar way to if it was running within the mesh.
With the above configuration in place, we can now make further authorization decisions based on the attached JWT and corresponding claims. For example, to ensure that the user signed into our configured OAuth application (instead of another Google one) we can restrict access based on the audience claim. This is straightforward since the audience claim must contain our Client ID:
Since oauth2-proxy is making a decision about whether to allow a request depending on the existence of a JWT stored in an encrypted cookie, it shouldn’t be possible for a user to gain access using a JWT from a different source. Nevertheless, representing the expected value natively in Istio provides defence in depth.
We can also configure the ingress gateway Envoys to forward the Authorization header to our upstream service to allow for service specific policy (the following assumes that the
Restricting access to an email address which differs from yours should give
RBAC: access denied as before. Unfortunately, currently only string and string list claims are extracted from the JWT, so in particular the boolean
email_verified claim (signifying whether the provider took steps to ensure the email address was controlled by the end user) cannot be matched on. However, as described in the Claim Stability and Uniqueness section of the OIDC specification, the
sub claims used together are the only claims that can provide a stable identifier for a user, so if that is the goal then they should be used instead of the
The Dex example-app is a useful tool for retrieving a JWT locally to see the claims your identity provider returns for a particular set of scopes. Note the scopes requested by oauth2-proxy by default. By configuring oauth2-proxy to request different scopes, you can adjust the claims that are present on the returned JWT and thus the attributes that can be matched on for authorization decisions. Returning group membership for example allows access to particular services to be granted and revoked by simply moving users within your provider, without any changes to the Istio configuration.
As we have demonstrated, a really powerful aspect of this is that our backend service can be completely unaware that OIDC is being used and does not need to support it itself. However, if the service has support for parsing the JWT, then it can also be used to authorize granular access to different features of the service.
Going with the theme of WASM extensibility in Istio, managing the OIDC flow may be a good candidate as a WASM extension so that Envoy no longer needs to call out to an external authorization implementation.
Get in Touch
TLS Protect for Kubernetes integrates directly with Istio for managing the security layer of your mesh at scale. If you found this post useful we’d encourage you to check it out, contact us directly to speak to an expert on service mesh security or download our guide to Zero Trust cloud-native infrastructure.