K
Q

Bitnami Postgresql with Kubernetes Istio

10/19/2020

I started to build a Postgresql Ha cluster on Kubernetes with sesrvice mesh istio. Before I enabled istion injection in namespace I tried it without istio. It worked well. After I enabled the istio injection the second pod in the statefulset cannot connect to the first pod in the cluster via headless service .

postgreslq-test-postgresql-ha-pgpool-779d7f588-qmbmf   1/1     Running   9          69m
postgreslq-test-postgresql-ha-postgresql-0             3/3     Running   0          10m
postgreslq-test-postgresql-ha-postgresql-1             2/3     Running   6          9m50s

I deleted the networkpolicy what was created under the helm install.

kubectl logs postgreslq-test-postgresql-ha-postgresql-1 postgresql

postgresql-repmgr 13:00:54.29 DEBUG ==> Host 'postgreslq-test-postgresql-ha-postgresql-0.postgreslq-test-postgresql-ha-postgresql-headless.gitlab-test.svc.cluster.local:5432' is not accessible
psql: error: could not connect to server: server closed the connection unexpectedly
	This probably means the server terminated abnormally
	before or while processing the request.

To test it myself I created a pod with postgresql-client

root@testpod:/# psql -h postgreslq-test-postgresql-ha-postgresql-0.postgreslq-test-postgresql-ha-postgresql-headless.gitlab-test.svc.cluster.local -U posgres
psql: server closed the connection unexpectedly
	This probably means the server terminated abnormally
	before or while processing the request.

Istio-proxy log:

"response_flags": "UF,URX",
    "start_time": "2020-10-19T14:21:44.631Z",
    "method": "-",
    "request_id": "-",
    "upstream_host": "10.93.202.188:5432",
    "x_forwarded_for": "-",
    "requested_server_name": "-",
    "bytes_received": "0",
    "istio_policy_status": "-",
    "bytes_sent": "0",
    "upstream_cluster": "outbound|5432||postgresql-ha-postgresql-headless.gitlab-prod.svc.cluster.local",
    "downstream_remote_address": "10.93.203.197:42222",
    "authority": "-",
    "path": "-",
    "protocol": "-",
    "upstream_service_time": "-",
    "upstream_local_address": "-",
    "duration": "3",
    "upstream_transport_failure_reason": "-",
    "route_name": "-",
    "downstream_local_address": "10.93.202.188:5432",
    "user_agent": "-",
    "response_code": "0"

Kubernetes Version: k8s.gcr.io/hyperkube:v1.18.4

Istio Version: 1.6.3

Helm chart: bitnami/postgresql-ha CHART VERSION:5.0.0 APP VERSION:11.9.1

I think some kind of Istio configuration cause the problem because it work without Istio.

Any idea what could be wrong ?

-- CsharpJoe
bitnami
istio
kubernetes
postgresql

Similar Questions

Kubernetes PHP NGINX Pods Intercommunication
Migrate from Deployment to StatefulSet without losing persistent volume access
K8S Resource Requests and Limits = Are not used request percentages temporarily given to other pods or do they lead to idle status?
Cannot reach resource through ingress on GKE
Automatic deployment of server plugin updates

1 Answer

2/3/2021

I find a problem and the solution. The problem caused by the port name.

apiVersion: v1
kind: Service
metadata:
  labels:
    app.kubernetes.io/instance: pg-istio
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: postgresql-ha
    helm.sh/chart: postgresql-ha-6.5.0
  namespace: postgre-test-istio
spec:
  clusterIP: None
  ports:
  - name: postgresql
    port: 5432
    protocol: TCP
    targetPort: 5432
  selector:
    app.kubernetes.io/component: postgresql
    app.kubernetes.io/instance: pg-istio
    app.kubernetes.io/name: postgresql-ha
  sessionAffinity: None
  type: ClusterIP

Change to :

apiVersion: v1
kind: Service
metadata:
  labels:
    app.kubernetes.io/instance: pg-istio
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: postgresql-ha
    helm.sh/chart: postgresql-ha-6.5.0
  namespace: postgre-test-istio
spec:
  clusterIP: None
  ports:
  - name: tcp-postgresql
    port: 5432
    protocol: TCP
    targetPort: 5432
  selector:
    app.kubernetes.io/component: postgresql
    app.kubernetes.io/instance: pg-istio
    app.kubernetes.io/name: postgresql-ha
  sessionAffinity: None
  type: ClusterIP
-- CsharpJoe
Source: StackOverflow