I have a deployment running frontier-squid proxy for caching, and I need to route all outbound TCP traffic through an istio-egressgateway
in order to exit the mesh always from the same host/ip.
I tried to follow the egress-mongo example but unsuccessfully.
For reference the inbound part is working fine and traffic is also exiting the mesh, from the host where the pod is running (unwanted behavior).
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: cvmfs
namespace: lcg
annotations:
co.elastic.logs/enabled: "true"
co.elastic.logs/module: "squid"
labels:
app: cvmfs
spec:
replicas: 1
selector:
matchLabels:
app: cvmfs
template:
metadata:
labels:
app: cvmfs
spec:
containers:
- name: cvmfs
image: opensciencegrid/frontier-squid:stable
env:
## SQUID_IPRANGE: Don't add external ip here. Proxied traffic dosen't have external ip. See below AuthorizationPolicy ingress-policy-cvmfs
- name: "SQUID_IPRANGE"
value: "10.0.0.0/8 172.16.0.0/12 192.168.0.0/16 fc00::/7 fe80::/10 127.0.0.0/24"
- name: "SQUID_CACHE_DISK"
value: "50000"
- name: "SQUID_CACHE_MEM"
value: "16 GB"
ports:
- containerPort: 3128
resources:
requests:
cpu: 4
memory: 20G
limits:
cpu: 8
memory: 32G
---
apiVersion: v1
kind: Service
metadata:
labels:
app: cvmfs
name: cvmfs
spec:
ports:
- port: 3128
selector:
app: cvmfs
---
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: lcg-cvmfs-gw
namespace: lcg
spec:
selector:
app: clustername-vlan320-gateway-cvmfs1
istio: ingressgateway
servers:
- port:
number: 3128
name: tcp-squid
protocol: TCP
hosts:
- cvmfs1.sub.domain.com
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: lcg-cvmfs-vs
namespace: lcg
spec:
hosts:
- cvmfs1.sub.domain.com
gateways:
- lcg-cvmfs-gw
tcp:
- match:
- port: 3128
route:
- destination:
host: cvmfs
port:
number: 3128
---
apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
name: ingress-policy-cvmfs
namespace: istio-system
spec:
selector:
matchLabels:
app: clustername-vlan320-gateway-cvmfs1
istio: ingressgateway
action: ALLOW
rules:
- from:
- source:
remoteIpBlocks:
- aa.aa.aa.aa/19
- bb.bb.bb.bb/25
Here my attempt to setup all steps to exit the mesh from the istio-egressgateway
:
---
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: external-stratum-1
namespace: lcg
spec:
hosts:
- external.tcp.svc # Not used in TCP mode
addresses:
- xx.xx.xx.xx/32
- xx.xx.xx.xx/32
ports:
- number: 8000
name: tcp-stratum1
protocol: TCP
location: MESH_EXTERNAL
resolution: STATIC
endpoints:
- address: xx.xx.xx.xx/32
- address: xx.xx.xx.xx/32
---
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: istio-egressgateway
namespace: lcg
spec:
selector:
istio: egressgateway
servers:
- port:
number: 55555
name: tcp-exit1
protocol: TCP
hosts:
- external.tcp.svc
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: egressgateway-for-cvmfs-stratum1
namespace: lcg
spec:
host: istio-egressgateway.istio-system.svc.clustername.domain.com
subsets:
- name: external-stratum-1
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: external-stratum-1
spec:
host: external.tcp.svc
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: direct-stratum1-through-egress-gateway
namespace: lcg
spec:
hosts:
- external.tcp.svc
gateways:
- mesh
- istio-egressgateway
tcp:
- match:
- gateways:
- mesh
destinationSubnets:
- xx.xx.xx.xx/32
- xx.xx.xx.xx/32
port: 8000
route:
- destination:
host: istio-egressgateway.istio-system.svc.clustername.domain.com
subset: external-stratum-1
port:
number: 55555
- match:
- gateways:
- istio-egressgateway
port: 55555
route:
- destination:
host: external.tcp.svc
port:
number: 8000
weight: 100
I'm running vanilla kubernetes version 1.21 and instio version 1.10
Can someone help me setting up this egressgateway or maybe suggest a better istio example?