July 03, 2020
Below are a number of different istio network topologies that can be constructed and a depiction of how the Istio networking CRDs are used together to enable connectivity within an Istio service mesh.
Given a Kubernetes Namespace
has been configured to allow for Istio sidecar injection,
then services deployed into this namespace with be accompanied by an Envoy Sidecar Proxy.
In this way the service will be augmented into the Service Mesh.
kubectl get pods
NAME READY STATUS RESTARTS AGE
api-auth-5f74b8c466-4fcl6 2/2 Running 0 1h
api-information-bdc664c59-dtscd 2/2 Running 0 1h
The 2/2
signifies that the given Pod
has two containers, one being the microservice application and the other being the sidecar proxy.
The Gateway
resource can be consider the mechanism to define a front-end listener, in terms of port and hostname.
Additionally how to map terminated TLS certificates to a given host listener.
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: api-gateway
spec:
selector:
istio: ingressgateway # use istio default gateyway
servers:
- port:
number: 80
name: http
protocol: HTTP
hosts:
- "api.training.local"
- port:
number: 443
name: https
protocol: HTTPS
hosts:
- "api.training.local"
tls:
mode: SIMPLE
serverCertificate: /etc/certs/https-api/tls.crt
privateKey: /etc/certs/https-api/tls.key
A VirtualService
CRD defines the instructions for how requests will be routed for a given hostname. It allows a list of matches to be specified, with corresponding destinations.
The match rules will be evaluated from top to bottom in terms of precedence. This resource has fined grained control on where it will reside, depending on the ingress and egress topology
within the service mesh.
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: product-gateway
spec:
hosts:
- "api.training.local"
gateways:
- api-gateway
http:
- match:
- uri:
prefix: "/api/product"
route:
- destination:
host: api-product
A DestinationRule
is responsible for providing session information for when a connection is being established to a given outbound endpoint, either
either within the mesh or externally.
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: api-product
spec:
host: api-product
trafficPolicy:
tls:
mode: ISTIO_MUTUAL
subsets:
- name: v1
labels:
version: v1
- name: v2
labels:
version: v2
A more granular, or service specific VirtualService
resource can be applied to the mesh. This provides a mechanism to decouple traffic steering from infrastructure scaling. What this means is,
we can now making a routing/steering decision from the sidecar instead of having to route the traffic to a main frontend-proxy or internal load balancing node.
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: api-product
spec:
gateways:
- mesh
hosts:
- api-product
http:
- route:
- destination:
host: api-product
subset: v1
The DestinationRule
configuration will reside on both the ingressgateway and the sidecar.
There will also be a DestinationRule
used to establish a mTLS connection for the communication between the ingressgateway and the sidecar, as shown below
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: default
namespace: istio-system
spec:
host: '*.local'
trafficPolicy:
tls:
mode: ISTIO_MUTUAL
An Istio service mesh will operate as a whitelist filter for all traffic egressing from the mesh.
Consequently we need to add ServiceEntry
resources to effectively allow traffic out of the service mesh. There is a mechanism to break
out of this default behavior, where we can enable certain subnets to bypass the Envoy sidecar when making extern/upstream requests.
It is enabled by add the following annotation to your Deployment
resource as follows:
apiVersion: apps/v1
kind: Deployment
metadata:
name: microservice-without-istio-egress
spec:
replicas: 1
selector:
matchLabels:
app: microservice-without-istio-egress
template:
metadata:
...
annotations:
...
traffic.sidecar.istio.io/excludeOutboundIPRanges: 0.0.0.0/0
...
...
The ServiceEntry
CRD resource is used to create an outbound object inside of the service mesh, so that services within the mesh are able to access
the defined upstream endpoint.
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: external-svc-https
spec:
hosts:
- fernago.core.gcp.anz
location: MESH_EXTERNAL
ports:
- number: 443
name: https
protocol: HTTPS
resolution: DNS
In order to define more granular session information for a given endpoint or destination we utilise the DestinationRule
CRD.
This will inform an Envoy proxy how it should establish a connection with a service either internal or external to the service mesh
In the example below, we are specifying that we should use a mTLS based connection and which certificates to use when establishing the connection.
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: api-stock-mtls
spec:
host: stock.training.local
trafficPolicy:
tls:
mode: MUTUAL
clientCertificate: /etc/certs/fernago/tls.crt
privateKey: /etc/certs/fernago/fernago.key
caCertificates: /etc/certs/fernago/ca.pem
Istio defines two types of endpoints, either MESH_EXTERNAL or MESH_INTERNAL. Internal services are, for the most part, entirely
auto discovered via the built-in Kubernetes service discovery mechanisms. However for service endpoints outside the Istio service mesh, we need
to more explicitly define these as ServiceEntry
resources so that the service mesh can intelligently route when required. An "external" endpoint
can be a microservice which lives outside of the service mesh, but within the same Kubernetes cluster. In this scenario we can define the following
DestinationRule
to disable mTLS when communicating to the service.
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: fernago
namespace: ext-ns
spec:
host: fernago.ext-ns.svc.cluster.local
trafficPolicy:
tls:
mode: DISABLE
A more complex service mesh topology can be deployed which facilitates enabling more robust security boundaries.
By having a egressgateway service act as a "gateway" out of the mesh, it is possible to enforce traffic flows via NetworkPolicy
to only allow egress from the egressgateway service. This prevents a malicious actor bypassing the default routing behavior that
services within the mesh will use.
Creates an outbound listener on the sidecar to handle outbound requests on a given Port
(if it doesn't already exist). This then allows for a VirtualService
policy to handle the request and
route it to the given destination. This ServiceEntry
will effectuate both Sidecars and Gateways, it is a global policy (spans across namespaces and mesh components).
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: stock-to-gateway
spec:
hosts:
- stock.training.local
location: MESH_EXTERNAL
ports:
- number: 443
name: https
protocol: HTTPS
resolution: DNS
The VirtualService
will be targeted to apply only to the Sidecars via the gateways:
selector. This VirtualService
policy aims to route all traffic on
a specific port (80) and route it to the egressgateway. Note that we specify a subset, which allows us to use more targeting or specific policy for the target destination.
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: stock-to-egressgateway
spec:
hosts:
- stock.training.local
gateways:
- mesh
http:
- match:
- gateways:
- mesh
port: 80
route:
- destination:
host: egressgateway.istio-system.svc.cluster.local
subset: stock-to-gateway
port:
number: 443
weight: 100
The DestinationRule
which is intended to be used by the Sidecar defines the required information to enable a mTLS connection to be established between. The key being
that is sets the sni: stock.training.local
and mode: ISTIO_MUTUAL
, for all traffic routed by the VirtualService
shown previously
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: stock-to-gateway
spec:
host: istio-egressgateway.istio-system.svc.cluster.local
subsets:
- name: stock-to-gateway
trafficPolicy:
loadBalancer:
simple: ROUND_ROBIN
portLevelSettings:
- port:
number: 443
tls:
mode: ISTIO_MUTUAL
sni: stock.training.local
Similar to what we've seen so far for an ingressgateway, we have the egressgateway. This is working in a similar fashion but in reverse in terms of the direction of traffic.
The Gateway
resource will enable the egressgateway to handle requests for the given hostname, as well as specifies what certificates to terminate on the given port. In this case
it terminates the internal istio certificates which are distributed by Citadel.
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: stock
spec:
selector:
istio: egressgateway
servers:
- hosts:
- stock.training.local
port:
name: https-stock
number: 443
protocol: HTTPS
tls:
caCertificates: /etc/certs/root-cert.pem
mode: MUTUAL
privateKey: /etc/certs/key.pem
serverCertificate: /etc/certs/cert-chain.pem
Once traffic has been routed from Envoy Sidecars within the mesh, the next stage of routing takes place on the egressgateway.
The VirtualService
shown below, routes manages all traffic for the stock.training.local domain, and will utilise the external DestinationRule
subset.
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: stock-to-egressgateway
spec:
hosts:
- stock.training.local
gateways:
- istio-egressgateway
http:
- match:
- gateways:
- istio-egressgateway
port: 443
route:
- destination:
host: stock.training.local
subset: external
port:
number: 443
weight: 100
For the final stage of the request handling that occurs within the Service Mesh, we have the below DestinationRule
. This policy effectively instructs the eggressgateway
to originate a mTLS connection to the target destination, with the specified certificates for the mutual authentication handshake and validation.
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: stock-to-external
spec:
host: stock.training.local
subsets:
- name: external
trafficPolicy:
loadBalancer:
simple: ROUND_ROBIN
portLevelSettings:
- loadBalancer:
simple: ROUND_ROBIN
port:
number: 443
tls:
mode: MUTUAL
clientCertificate: /etc/certs/stock/tls.crt
privateKey: /etc/certs/stock/tls.key
caCertificates: /etc/certs/stock/ca.pem
The ServiceEntry
policy will be deployed to effectively all mesh components, enabling outbound resolution by the mesh components.
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: stockt-to-gateway
spec:
hosts:
- stock.training.local
location: MESH_EXTERNAL
ports:
- number: 443
name: https
protocol: HTTPS
resolution: DNS
Written by Ben Ebsworth, thoughts are their own and any material is representative of an attempt to self-educate and explore ideas and technology You should follow him on Twitter