I'm using a Kubernetes/Istio setup and my list of pods and services are as below:
NAME READY STATUS RESTARTS AGE
hr--debug-deployment-86575cffb6-wl6rx 2/2 Running 0 33m
hr--hr-deployment-596946948d-jrd7g 2/2 Running 0 33m
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
hr--debug-service ClusterIP 10.104.160.61 <none> 80/TCP 33m
hr--hr-service ClusterIP 10.102.117.177 <none> 80/TCP 33mI'm attempting to curl into hr--hr-service from hr--debug-deployment-86575cffb6-wl6rx
pasan@ubuntu:~/product-vick$ kubectl exec -it hr--debug-deployment-86575cffb6-wl6rx /bin/bash
Defaulting container name to debug.
Use 'kubectl describe pod/hr--debug-deployment-86575cffb6-wl6rx -n default' to see all of the containers in this pod.
root@hr--debug-deployment-86575cffb6-wl6rx:/# curl hr--hr-service -v
* Rebuilt URL to: hr--hr-service/
* Trying 10.102.117.177...
* Connected to hr--hr-service (10.102.117.177) port 80 (#0)
> GET / HTTP/1.1
> Host: hr--hr-service
> User-Agent: curl/7.47.0
> Accept: */*
>
< HTTP/1.1 403 Forbidden
< date: Thu, 03 Jan 2019 04:06:17 GMT
< server: envoy
< content-length: 0
<
* Connection #0 to host hr--hr-service left intact
Can you please explain why I'm getting a 403 forbidden by envoy and how I can troubleshoot it?
If you have the envoy sidecar injected it really depends on what type of authentication policy you have between your services. Are you using a MeshPolicy or a Policy?
You can also try disabling authentication between your services to debug. Something like this (if your policy is defined like this):
apiVersion: "authentication.istio.io/v1alpha1"
kind: "Policy"
metadata:
name: "hr--hr-service"
spec:
targets:
- name: hr--hr-service
peers:
- mTLS:
mode: PERMISSIVE