How to apply Istio DestinationRule across the mesh?
Published on

How to apply Istio DestinationRule across the mesh?

Author
Written by Peter Jausovec
    Contents
  1. How about merging multiple DestinationRules?
Discuss on Discord
An interesting question came up today in Istio Slack where someone asked if and how to apply DestinationRules globally to all workloads inside the cluster. The short answer is yes, you can do that, and this article will explain how to do it.
The DestinationRule resource allows us to configure policies that get applied to traffic once routing has occurred. Under the hood, the Istio translates this config to the Envoy config's clusters section.
Typically you use DestinationRule to configure different subsets (e.g., v1, v2 or prod, dev, and so on) and traffic policy settings, such as the load balancing policy, connection pool sizes, and outlier detection.
If you're using only subsets, you'll probably have one DestinationRule per host, as those typically don't make sense to apply globally across multiple workloads.
On the other hand, settings such as outlier detection or load balancer policies might make sense to apply globally.
For example:
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: ?
  namespace: ?
spec:
  host: ?
  trafficPolicy:
    loadBalancer:
      simple: LEAST_REQUEST
    outlierDetection:
      consecutive5xxErrors: 10
      interval: 2m
      baseEjectionTime: 10m
It would be best if you looked first at the Istio's global mesh options. The global mesh options are the place for any settings and configurations Istio applies globally across the whole mesh. However, in the case of the outlier detection, load balancer policy, or other configuration that can be set in the DestinationRule, we can't define those through the global options.
Looking at the above YAML, we know we need to answer two questions -- where to create the DestinationRule (name and namespace), and what do we set the host field to?
Istio has a concept of a root namespace (rootNamespace field in the global mesh options). So let's say we're running a workload in ns1; when processing the configuration, Istio looks in ns1 first, and if there's no configuration there, it will look in the root namespace. If not explicitly set, the root namespace defaults to istio-system. The answer to the first question is that we must deploy the resource to the istio-system namespace.
But how can we target multiple hosts? The host field in the DestinationRule expects the name of a service that exists in the service registry - for example, product-page or product-page.svc.cluster.local. Note that using a fully qualified name is the preferred way to address the service, so you can avoid misinterpretation and know precisely which service the rule applies to.
The "feature" that will allow us to apply the rule globally is the support for wildcards (*) in the service name. Therefore, to apply a destination rule globally, we could use the notation *.cluster.local or just leave the host field off all together.

Note

If you're using a different trust domain (configurable through trustDomain field in the global mesh settings), replace the cluster.local with your trust domain.
This notation, coupled with the resource being in the root namespace, will apply the configuration across all services in the mesh:
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: global-dr
  namespace: istio-system
spec:
  # Note we don't specify the `host` field at all.
  trafficPolicy:
    loadBalancer:
      simple: LEAST_REQUEST
    outlierDetection:
      consecutive5xxErrors: 15
      interval: 2m
      baseEjectionTime: 10m
We can try this out quickly by deploying workloads into two separate namespaces and then applying this global DestinationRule.
Then, we can use the istioctl pc cluster command to check the settings are applied globally to all Envoy clusters in the mesh:
istioctl pc cluster deploy/httpbin.httpbin -o yaml | grep -B3 consecutive5xx

 name: outbound|8000||httpbin.httpbin.svc.cluster.local
  outlierDetection:
    baseEjectionTime: 900s
    consecutive5xx: 15
--
  name: outbound|80||istio-ingressgateway.istio-system.svc.cluster.local
  outlierDetection:
    baseEjectionTime: 900s
    consecutive5xx: 15
--
  name: outbound|443||istio-ingressgateway.istio-system.svc.cluster.local
  outlierDetection:
    baseEjectionTime: 900s
    consecutive5xx: 15
...
Another option we have for targeting workloads is by using the workloadSelector and specifying the labels. Note that having a mesh or namespace destination rule together with the workloadSelector isn't going to work. You'll have to specify the exact host name.

Note

I've tried setting the host to *.cluster.local, exporting the destination rule to the current namespace only (default and exportTo set to .), however, the outlier detection (for example) was still applied to workloads outside of the default namespace and the workload selector labels were also ignored. Make sure you check the configuration is applied correctly or per your expectations.
Here's how you can apply a destination rule for v1 versions of the workloads:
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: sleep-dr
  namespace: default
spec:
  host: "sleep.default.svc.cluster.local"
  workloadSelector:
    matchLabels:
      version: v1
  trafficPolicy:
    loadBalancer:
      simple: LEAST_REQUEST
    outlierDetection:
      consecutive5xxErrors: 15
      interval: 2m
      baseEjectionTime: 10m

How about merging multiple DestinationRules?

Another interesting scenario is having a global DestinationRule and having workload or host specific DestinationRules. In this case we'd expect Istio to merge the two DestinationRules.
However, that only happens if you set the PILOT_ENABLE_DESTINATION_RULE_INHERITANCE environment variable to true when installing Istio. By default, the destination rule inheritance is turned off.

Note

You can pass the environment variables in when installing Istio: istioctl install --set values.pilot.env.PILOT_ENABLE_DESTINATION_RULE_INHERITANCE=true
So let's apply a global destination rule (without a host set) to the istio-system namespace:
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: global-dr
  namespace: istio-system
spec:
  # Note we don't specify the `host` field at all.
  trafficPolicy:
    loadBalancer:
      simple: LEAST_REQUEST
    outlierDetection:
      consecutive5xxErrors: 15
      interval: 2m
      baseEjectionTime: 10m
Just like before, these settings will be applied to all services in the mesh. If we want to customize the settings per namespace, we can deploy another DestinationRule and only modify the specific values we want to change.
For example, let's say we want to increase the consecutive5xxErrors to 25, for the sleep host:
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: sleep-dr
  namespace: default
spec:
  host: "sleep.default.svc.cluster.local"
  trafficPolicy:
    outlierDetection:
      consecutive5xxErrors: 25
If we run the istioctl pc cluster command, we'll see that the sleep workload inherited the settings from the global DestinationRule that got merged with the local DestinationRule where we changed the consecutive5xxErrors value:
...
         name: httpbin
          namespace: default
  name: outbound|8000||httpbin.default.svc.cluster.local
  outlierDetection:
    baseEjectionTime: 600s
    consecutive5xx: 15
    enforcingConsecutive5xx: 100
    enforcingSuccessRate: 0
    interval: 120s
  transportSocketMatches:
  - match:
--
...
          name: sleep
          namespace: default
  name: outbound|80||sleep.default.svc.cluster.local
  outlierDetection:
    baseEjectionTime: 600s
    consecutive5xx: 25
    enforcingConsecutive5xx: 100
    enforcingSuccessRate: 0
    interval: 120s
  transportSocketMatches:
  - match:
...
Join the discussion
SHARE THIS ARTICLE
Peter Jausovec

Peter Jausovec

Peter Jausovec is a platform advocate at Solo.io. He has more than 15 years of experience in the field of software development and tech, in various roles such as QA (test), software engineering and leading tech teams. He's been working in the cloud-native space, focusing on Kubernetes and service meshes, and delivering talks and workshops around the world. He authored and co-authored a couple of books, latest being Cloud Native: Using Containers, Functions, and Data to Build Next-Generation Applications.

Related posts

Running Hugo on free Ampere VM (Oracle Cloud Infrastructure)
General

Running Hugo on free Ampere VM (Oracle Cloud Infrastructure)

In this article, I'll take you on a journey of setting up a free-for-life virtual machine instance running on OCI. We'll be creating the account, a virtual machine instance, creating a Github repository with Hugo, setting up Nginx on the VM, and obtaining a free SSL certificate.

Peter Jausovec

Peter Jausovec

All About the Ingress Resource
Kubernetes

All About the Ingress Resource

You deployed your applications to Kubernetes and now you need a way for your users to access them. You can do that with a Kubernetes Ingress resource and an Ingress controller. This article explains all about the Ingress resource and shows you how to expose multiple applications through a domain, subdomain and on a different URL path.

Peter Jausovec

Peter Jausovec

Kubernetes and Istio service mesh workshop materials
Service Mesh

Kubernetes and Istio service mesh workshop materials

In the past year, I have done multiple workshops on Kubernetes, Istio and cloud-native development. As part of my workshops I usually start with theory and explain the concepts using slides, show some demos, but then it's on you, the participant to try out the technology yourself.

Peter Jausovec

Peter Jausovec

How to use kwatch to detect crashes in Kubernetes clusters
Kubernetes

How to use kwatch to detect crashes in Kubernetes clusters

kwatch is an open-source tool that detects crashes in Kubernetes Pods in real-time and sends the notifications to supported backends. In this post, we'll show you how to deploy and configure kwatch to send Kubernetes Pod crash notifications to PagerDuty.

Amgad Ramses

Amgad Ramses

Minikube Basics and How to Get Started with Kubernetes
Kubernetes

Minikube Basics and How to Get Started with Kubernetes

This post and accompanying video guides you through the Minikube installation process. It explains and introduces a couple of essential Minikube commands you can use to work with your Kubernetes cluster, and shows you how to access your applications inside the cluster when using Minikube.

Peter Jausovec

Peter Jausovec

How to configure rate limiter in Istio
Service Mesh

How to configure rate limiter in Istio

In this article you'll learn how rate limiting works in Envoy and how to configure a local rate limiter on the worloads inside the mesh, as well as at the edge of the mesh for the ingress and ingress gateways.

Peter Jausovec

Peter Jausovec

;