Route TCP traffic trough Istio egressgateway

8/13/2021

I have a deployment running frontier-squid proxy for caching, and I need to route all outbound TCP traffic through an istio-egressgateway in order to exit the mesh always from the same host/ip.

I tried to follow the egress-mongo example but unsuccessfully.

For reference the inbound part is working fine and traffic is also exiting the mesh, from the host where the pod is running (unwanted behavior).

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: cvmfs
  namespace: lcg
  annotations:
    co.elastic.logs/enabled: "true"
    co.elastic.logs/module: "squid"
  labels:
    app: cvmfs
spec:
  replicas: 1
  selector:
    matchLabels:
      app: cvmfs
  template:
    metadata:
      labels:
        app: cvmfs
    spec:
      containers:
      - name: cvmfs
        image: opensciencegrid/frontier-squid:stable
        env:
        ## SQUID_IPRANGE: Don't add external ip here. Proxied traffic dosen't have external ip. See below AuthorizationPolicy ingress-policy-cvmfs
        - name: "SQUID_IPRANGE"
          value: "10.0.0.0/8 172.16.0.0/12 192.168.0.0/16 fc00::/7 fe80::/10 127.0.0.0/24"
        - name: "SQUID_CACHE_DISK"
          value: "50000"
        - name: "SQUID_CACHE_MEM"
          value: "16 GB"
        ports:
        - containerPort: 3128
        resources:
          requests:
            cpu: 4
            memory: 20G
          limits:
            cpu: 8
            memory: 32G
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: cvmfs
  name: cvmfs
spec:
  ports:
    - port: 3128
  selector:
    app: cvmfs
---
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: lcg-cvmfs-gw
  namespace: lcg
spec:
  selector:
     app: clustername-vlan320-gateway-cvmfs1
     istio: ingressgateway
  servers:
  - port:
      number: 3128
      name: tcp-squid
      protocol: TCP
    hosts:
    - cvmfs1.sub.domain.com
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: lcg-cvmfs-vs
  namespace: lcg
spec:
  hosts:
  - cvmfs1.sub.domain.com
  gateways:
  - lcg-cvmfs-gw
  tcp:
  - match:
    - port: 3128
    route:
    - destination:
        host: cvmfs
        port:
          number: 3128
---
apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  name: ingress-policy-cvmfs
  namespace: istio-system
spec:
  selector:
    matchLabels:
      app: clustername-vlan320-gateway-cvmfs1
      istio: ingressgateway
  action: ALLOW
  rules:
  - from:
    - source:
       remoteIpBlocks:
        - aa.aa.aa.aa/19
        - bb.bb.bb.bb/25

Here my attempt to setup all steps to exit the mesh from the istio-egressgateway:

---
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
  name: external-stratum-1
  namespace: lcg
spec:
  hosts:
    - external.tcp.svc # Not used in TCP mode
  addresses:
    - xx.xx.xx.xx/32
    - xx.xx.xx.xx/32
  ports:
  - number: 8000
    name: tcp-stratum1
    protocol: TCP
  location: MESH_EXTERNAL
  resolution: STATIC
  endpoints:
  - address: xx.xx.xx.xx/32
  - address: xx.xx.xx.xx/32
---
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: istio-egressgateway
  namespace: lcg
spec:
  selector:
    istio: egressgateway
  servers:
  - port:
      number: 55555
      name: tcp-exit1
      protocol: TCP
    hosts:
    - external.tcp.svc
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: egressgateway-for-cvmfs-stratum1
  namespace: lcg
spec:
  host: istio-egressgateway.istio-system.svc.clustername.domain.com
  subsets:
  - name: external-stratum-1
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: external-stratum-1
spec:
  host: external.tcp.svc
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: direct-stratum1-through-egress-gateway
  namespace: lcg
spec:
  hosts:
  - external.tcp.svc
  gateways:
  - mesh
  - istio-egressgateway
  tcp:
  - match:
    - gateways:
      - mesh
      destinationSubnets:
      - xx.xx.xx.xx/32
      - xx.xx.xx.xx/32
      port: 8000
    route:
    - destination:
        host: istio-egressgateway.istio-system.svc.clustername.domain.com
        subset: external-stratum-1
        port:
          number: 55555
  - match:
    - gateways:
      - istio-egressgateway
      port: 55555
    route:
    - destination:
        host: external.tcp.svc
        port:
          number: 8000
      weight: 100

I'm running vanilla kubernetes version 1.21 and instio version 1.10

Can someone help me setting up this egressgateway or maybe suggest a better istio example?

-- Dino
istio
kubernetes

0 Answers