I can route HTTP traffic (e.g. Elasticsearch and various dashboards) through Istio Gateway, but I can't get raw TCP traffic through. I have two examples below (postgres and redpanda). I have no trouble accessing the underlying services (mypostgres.default.svc.cluster.local
and three-node-cluster-0.three-node-cluster.redpanda-system.svc.cluster.local
) internally with postgres and kafka clients.
My Gateway:
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: gateway
namespace: istio-system
spec:
selector:
istio: ingressgateway
servers:
- port:
number: 80
name: http
protocol: HTTP
hosts:
- 'mydomain.cloud'
tls:
httpsRedirect: true
- port:
number: 443
name: https
protocol: HTTPS
hosts:
- 'mydomain.cloud'
tls:
mode: SIMPLE
credentialName: letsencrypt-staging-tls
- port:
number: 9092
name: redpanda
protocol: TCP
hosts:
- 'mydomain.cloud'
- port:
number: 5432
name: postgres
protocol: TCP
hosts:
- 'mydomain.cloud'
Postgres spec:
apiVersion: kubegres.reactive-tech.io/v1
kind: Kubegres
metadata:
name: mypostgres
namespace: postgres
spec:
replicas: 3
image: postgres:13.2
database:
size: 50Gi
storageClassName: postgres
env:
- name: POSTGRES_PASSWORD
valueFrom:
secretKeyRef:
name: postgressecret
key: superUserPassword
- name: POSTGRES_REPLICATION_PASSWORD
valueFrom:
secretKeyRef:
name: postgressecret
key: replicationUserPassword
Virtual service:
spec:
hosts:
- "*"
gateways:
- istio-system/gateway
tcp:
- match:
- port: 5432
route:
- destination:
host: mypostgres.default.svc.cluster.local
port:
number: 5432
Destination rule
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: postgres-destination-rule
namespace: default
spec:
host: mypostgres.default.svc.cluster.local
trafficPolicy:
tls:
mode: DISABLE
Redpanda
apiVersion: redpanda.vectorized.io/v1alpha1
kind: Cluster
metadata:
name: three-node-cluster
spec:
image: "vectorized/redpanda"
version: "latest"
replicas: 2
resources:
requests:
cpu: 1
memory: 2Gi
limits:
cpu: 1
memory: 2Gi
configuration:
rpcServer:
port: 33145
kafkaApi:
- port: 9092
pandaproxyApi:
- port: 8082
adminApi:
- port: 9644
developerMode: true
storage:
storageClassName: redpanda
Virtual service
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
name: redpanda-vts
namespace: redpanda-system
spec:
hosts:
- "*"
gateways:
- istio-system/gateway
tcp:
- match:
- port: 9092
route:
- destination:
host: three-node-cluster-0.three-node-cluster.redpanda-system.svc.cluster.local
port:
number: 9092
Destination rule:
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: redpanda-destination-rule
namespace: redpanda-system
spec:
host: three-node-cluster-0.three-node-cluster.redpanda-system.svc.cluster.local
trafficPolicy:
tls:
mode: DISABLE
Any ideas? I've tried playing around with the host names, using asterisks instead of domain names, but no effect. Getting TLS will be another day's fight, but now I'd just like to get some traffic through.
For example, the following works for RedPanda from inside the cluster with the standard kafka-python client:
from kafka.admin import KafkaAdminClient, NewTopic
nodes = {'bootstrap.servers':'three-node-cluster-0.three-node-cluster.redpanda-system.svc.cluster.local, three-node-cluster-1.three-node-cluster.redpanda-system.svc.cluster.local'}
admin_client = KafkaAdminClient(
bootstrap_servers=nodes['bootstrap.servers'],
client_id='test'
)
topic_list = []
topic_list.append(NewTopic(name="test-topic", num_partitions=1, replication_factor=1))
admin_client.create_topics(new_topics=topic_list, validate_only=False)
Similarly, I would like to be able to do the following from outside K8s through Istio Gateway:
from kafka.admin import KafkaAdminClient, NewTopic
nodes = {'bootstrap.servers':'mydomain.cloud/kafka'}
admin_client = KafkaAdminClient(
bootstrap_servers=nodes['bootstrap.servers'],
client_id='test'
)
topic_list = []
topic_list.append(NewTopic(name="test-topic", num_partitions=1, replication_factor=1))
admin_client.create_topics(new_topics=topic_list, validate_only=False)
Based on the documentation about Istio Protocol Selection
Istio supports proxying any TCP traffic. This includes HTTP, HTTPS, gRPC, as well as raw TCP protocols. In order to provide additional capabilities, such as routing and rich metrics, the protocol must be determined. This can be done automatically or explicitly specified.
And the answer to your problem should be in this fragment:
Protocols can be specified manually in the Service definition.
This can be configured in two ways:
- By the name of the port:
name: <protocol>[-<suffix>]
.- In Kubernetes 1.18+, by the
appProtocol
field:appProtocol: <protocol>
.Note that behavior at the Gateway differs in some cases as the gateway can terminate TLS and the protocol can be negotiated.
Look at the example yaml:
Below is an example of a Service that defines a
https
port byappProtocol
and anhttp
port by name:
kind: Service metadata: name: myservice spec: ports: - number: 3306 name: database appProtocol: https <-change here 'https' to 'tcp' - number: 80 name: http-web
In your situation try to replace appProtocol: https
and put appProtocol: tcp
in your Service yaml
Bare in mind that Server First protocols, such as MySQL, are incompatible with automatic protocol selection. See Server first protocols for more information.