I have an AWS EKS cluster up and running. I have a bastion host set up with admin rights in a public subnet and worker nodes are in private subnets.
The k8s services are deployed on the workers in private subnets and also our main API pods are there and are exposed to the internet using the nginx ingress controller with which we have setup an internet-facing elastic load balancer and nginx pods tied to it. We have path-based routing due to the ingress rules. The ELB has a DNS record we've setup.
This all works fine but recently I've setup the Elastic Stack for logging and monitoring following this tutorial: https://nick.sarbicki.com/blog/visualising-services-kubernetes-elk-kubernetes/ an have my pods up and running but can't reach kibana. When I put the path I specified in the ingress file for kibana, I get back default backend - 404
but at the same time I can see the request in the kibana pod's log.
I'm playing around with different nginx settings but can't seem to get it to work. Has anyone been in this situation? Any help would be greatly appreciated.