Why my canary deployment does not work with istio?

2/18/2021

I am trying to learn the basics about istio so I have gone through the official documentation here in order to create a 80/20 canary deployment, I have also follow this guide from digitalocean which explains it very easy for a simple deployment https://www.digitalocean.com/community/tutorials/how-to-do-canary-deployments-with-istio-and-kubernetes.

I have created a simple app with 2 different messages on the homepage, and then created the virtualService, Gateway and the destination rules. I (as mentioned in the guide) get the external ip with kubectl -n istio-system get svc and the try to navigate to that address but I get a 503 error. It seems very simple but I have to be missing something. These are my 3 files (as far as I understand there are no more necessary files):

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  namespace: istio
  name: flask-gateway
spec:
  selector:
    istio: ingressgateway
  servers:
  - hosts:
    - "*"
    port:
      name: http
      number: 80
      protocol: HTTP
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: flask-app
  namespace: istio
spec:
  hosts:
  - "*"
  gateways:
  - flask-gateway
  http:
  - route:
    - destination:
        host: flask-app
        subset: v1
      weight: 80
    - destination:
        host: flask-app
        subset: v2
      weight: 20
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: flask-app
  namespace: istio
spec:
  host: flask-app
  subsets:
  - name: v1
    labels:
      version: v1
  - name: v2
    labels:
      version: v2

Here are the yaml with deployment and services for v1 and v2:

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    version: v1
  name: flask-deployment-v1
  namespace: istio
spec:
  replicas: 1
  selector:
    matchLabels:
      app: flask-app
  template:
    metadata:
      labels:
        version: v1
        app: flask-app
    spec:
      containers:
      - name: flask-app
        image: latalavera/flask-app:1.3
        ports:
          - containerPort: 5000
        resources:
          requests:
            cpu: 100m
            memory: 128Mi
          limits:
            cpu: 250m
            memory: 256Mi

---
apiVersion: v1
kind: Service
metadata:
  name: flask-service
  namespace: istio
spec:
  selector:
    app: flask-app
  ports:
  - port: 5000
    protocol: TCP
    targetPort: 5000
  type: ClusterIP
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    version: v2
  name: flask-deployment-v2
  namespace: istio
spec:
  replicas: 1
  selector:
    matchLabels:
      app: flask-app
  template:
    metadata:
      labels:
        app: flask-app
        version: v2
    spec:
      containers:
      - name: flask-app
        image: latalavera/flask-app:2.0
        ports:
          - containerPort: 5000
        resources:
          requests:
            cpu: 100m
            memory: 128Mi
          limits:
            cpu: 250m
            memory: 256Mi

---
apiVersion: v1
kind: Service
metadata:
  name: flask-service2
  namespace: istio
spec:
  selector:
    app: flask-app
  ports:
  - port: 5000
    protocol: TCP
    targetPort: 5000
  type: ClusterIP

I have added the labels version: v1and version: v2to my deployments, and I have also used the kubectl label ns istio istio-injection=enabled command, but they are not working anyways

-- Jesus Fernandez
docker
istio
kubernetes

1 Answer

2/19/2021

You named the service flask-service and set the host in your VirtualService to flask-app.

The host field is not a selector but the FQDN of the service you want to route the traffic to. So it should be called flask-service or better flask-service.istio.svc.cluster.local:

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: flask-app
  namespace: istio
spec:
  hosts:
  - "*"
  gateways:
  - flask-gateway
  http:
  - route:
    - destination:
        host: flask-service.istio.svc.cluster.local
        subset: v1
      weight: 80
    - destination:
        host: flask-service.istio.svc.cluster.local
        subset: v2
      weight: 20

Alternatively you could just call the service flask-app like the Deployment. But using the full FQDN <service-name>.<namespace-name>.svc.cluster.local is recommended in any case. From docs:

Note for Kubernetes users: When short names are used (e.g. “reviews” instead of “reviews.default.svc.cluster.local”), Istio will interpret the short name based on the namespace of the rule, not the service. A rule in the “default” namespace containing a host “reviews” will be interpreted as “reviews.default.svc.cluster.local”, irrespective of the actual namespace associated with the reviews service. To avoid potential misconfigurations, it is recommended to always use fully qualified domain names over short names.

https://istio.io/latest/docs/reference/config/networking/virtual-service/#VirtualService -> hosts

You btw don't need 2 services, just one. Your service has a selector for app: flask-app so it can route traffic to v1 and v2. How the traffic is routed is defined by the VirtualService and DestionationRule. I would recommand to remove service flask-service2. If you need to route the traffic inside the mesh, add mesh as gateways in the VirtualService or create a new one for mesh internal traffic, to reach both versions. More on that topic:

https://istio.io/latest/docs/reference/config/networking/virtual-service/#VirtualService -> gateways

-- Chris
Source: StackOverflow