- Published on
How to apply Istio DestinationRule across the mesh?
- Author
- Written by Peter Jausovec
- Name Peter Jausovec
- @pjausovec
An interesting question came up today in Istio Slack where someone asked if and how to apply DestinationRules globally to all workloads inside the cluster. The short answer is yes, you can do that, and this article will explain how to do it.
The DestinationRule resource allows us to configure policies that get applied to traffic once routing has occurred. Under the hood, the Istio translates this config to the Envoy config's clusters section.
Typically you use DestinationRule to configure different subsets (e.g., v1, v2 or prod, dev, and so on) and traffic policy settings, such as the load balancing policy, connection pool sizes, and outlier detection.
If you're using only subsets, you'll probably have one DestinationRule per host, as those typically don't make sense to apply globally across multiple workloads.
On the other hand, settings such as outlier detection or load balancer policies might make sense to apply globally.
For example:
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: ?
namespace: ?
spec:
host: ?
trafficPolicy:
loadBalancer:
simple: LEAST_REQUEST
outlierDetection:
consecutive5xxErrors: 10
interval: 2m
baseEjectionTime: 10m
It would be best if you looked first at the Istio's global mesh options. The global mesh options are the place for any settings and configurations Istio applies globally across the whole mesh. However, in the case of the outlier detection, load balancer policy, or other configuration that can be set in the DestinationRule, we can't define those through the global options.
Looking at the above YAML, we know we need to answer two questions -- where to create the DestinationRule (name and namespace), and what do we set the host field to?
Istio has a concept of a root namespace (rootNamespace
field in the global mesh options). So let's say we're running a workload in ns1
; when processing the configuration, Istio looks in ns1
first, and if there's no configuration there, it will look in the root namespace. If not explicitly set, the root namespace defaults to istio-system
. The answer to the first question is that we must deploy the resource to the istio-system
namespace.
But how can we target multiple hosts? The host
field in the DestinationRule expects the name of a service that exists in the service registry - for example, product-page
or product-page.svc.cluster.local
. Note that using a fully qualified name is the preferred way to address the service, so you can avoid misinterpretation and know precisely which service the rule applies to.
The "feature" that will allow us to apply the rule globally is the support for wildcards (*
) in the service name. Therefore, to apply a destination rule globally, we could use the notation *.cluster.local
or just leave the host
field off all together.
Note
If you're using a different trust domain (configurable through
trustDomain
field in the global mesh settings), replace thecluster.local
with your trust domain.
This notation, coupled with the resource being in the root namespace, will apply the configuration across all services in the mesh:
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: global-dr
namespace: istio-system
spec:
# Note we don't specify the `host` field at all.
trafficPolicy:
loadBalancer:
simple: LEAST_REQUEST
outlierDetection:
consecutive5xxErrors: 15
interval: 2m
baseEjectionTime: 10m
We can try this out quickly by deploying workloads into two separate namespaces and then applying this global DestinationRule.
Then, we can use the istioctl pc cluster
command to check the settings are applied globally to all Envoy clusters in the mesh:
istioctl pc cluster deploy/httpbin.httpbin -o yaml | grep -B3 consecutive5xx
name: outbound|8000||httpbin.httpbin.svc.cluster.local
outlierDetection:
baseEjectionTime: 900s
consecutive5xx: 15
--
name: outbound|80||istio-ingressgateway.istio-system.svc.cluster.local
outlierDetection:
baseEjectionTime: 900s
consecutive5xx: 15
--
name: outbound|443||istio-ingressgateway.istio-system.svc.cluster.local
outlierDetection:
baseEjectionTime: 900s
consecutive5xx: 15
...
Another option we have for targeting workloads is by using the workloadSelector
and specifying the labels. Note that having a mesh or namespace destination rule together with the workloadSelector
isn't going to work. You'll have to specify the exact host name.
Note
I've tried setting the host to
*.cluster.local
, exporting the destination rule to the current namespace only (default
andexportTo
set to.
), however, the outlier detection (for example) was still applied to workloads outside of thedefault
namespace and the workload selector labels were also ignored. Make sure you check the configuration is applied correctly or per your expectations.
Here's how you can apply a destination rule for v1
versions of the workloads:
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: sleep-dr
namespace: default
spec:
host: "sleep.default.svc.cluster.local"
workloadSelector:
matchLabels:
version: v1
trafficPolicy:
loadBalancer:
simple: LEAST_REQUEST
outlierDetection:
consecutive5xxErrors: 15
interval: 2m
baseEjectionTime: 10m
How about merging multiple DestinationRules?
Another interesting scenario is having a global DestinationRule and having workload or host specific DestinationRules. In this case we'd expect Istio to merge the two DestinationRules.
However, that only happens if you set the PILOT_ENABLE_DESTINATION_RULE_INHERITANCE
environment variable to true
when installing Istio. By default, the destination rule inheritance is turned off.
Note
You can pass the environment variables in when installing Istio:
istioctl install --set values.pilot.env.PILOT_ENABLE_DESTINATION_RULE_INHERITANCE=true
So let's apply a global destination rule (without a host set) to the istio-system
namespace:
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: global-dr
namespace: istio-system
spec:
# Note we don't specify the `host` field at all.
trafficPolicy:
loadBalancer:
simple: LEAST_REQUEST
outlierDetection:
consecutive5xxErrors: 15
interval: 2m
baseEjectionTime: 10m
Just like before, these settings will be applied to all services in the mesh. If we want to customize the settings per namespace, we can deploy another DestinationRule and only modify the specific values we want to change.
For example, let's say we want to increase the consecutive5xxErrors
to 25
, for the sleep
host:
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: sleep-dr
namespace: default
spec:
host: "sleep.default.svc.cluster.local"
trafficPolicy:
outlierDetection:
consecutive5xxErrors: 25
If we run the istioctl pc cluster
command, we'll see that the sleep
workload inherited the settings from the global DestinationRule that got merged with the local DestinationRule where we changed the consecutive5xxErrors
value:
...
name: httpbin
namespace: default
name: outbound|8000||httpbin.default.svc.cluster.local
outlierDetection:
baseEjectionTime: 600s
consecutive5xx: 15
enforcingConsecutive5xx: 100
enforcingSuccessRate: 0
interval: 120s
transportSocketMatches:
- match:
--
...
name: sleep
namespace: default
name: outbound|80||sleep.default.svc.cluster.local
outlierDetection:
baseEjectionTime: 600s
consecutive5xx: 25
enforcingConsecutive5xx: 100
enforcingSuccessRate: 0
interval: 120s
transportSocketMatches:
- match:
...