Kubernetes service - service IP address is not accessible

9/3/2019

I am working with aks service. I have a Tensorflow serving image at azure container registry. Now when I deploy my service, the public service endpoint is not accessible neither is it pingable.

My image is exposed at port 8501 , so I am using it as a target port in my yaml.

Here is the yaml file I am using for this deployment.

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: my-model-gpu
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: my-model-gpu
    spec:
      containers:
      - name: my-model-gpu
        image: dsdemocr.azurecr.io/work-place-safety-gpu
        ports:
        - containerPort: 8501
        resources:
         limits:
           nvidia.com/gpu: 1
      imagePullSecrets:
        - name: registrykey

---
apiVersion: v1
kind: Service
metadata:
  name: my-model-gpu
spec:
  type: LoadBalancer
  ports:
  - port: 8501
    protocol: "TCP"
    targetPort: 8501
  selector:
    app: my-model-gpu

below are my svc description : kubectl describe svc my-model-gpu

Name:                     my-model-gpu
Namespace:                default
Labels:                   <none>
Annotations:              kubectl.kubernetes.io/last-applied-configuration:
                            {"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"name":"my-model-gpu","namespace":"default"},"spec":{"ports":[{"port":850...
Selector:                 app=my-model-gpu
Type:                     LoadBalancer
IP:                       10.0.244.106
LoadBalancer Ingress:     52.183.17.101
Port:                     <unset>  8501/TCP
TargetPort:               8501/TCP
NodePort:                 <unset>  31546/TCP
Endpoints:                10.244.0.22:8501
Session Affinity:         None
External Traffic Policy:  Cluster
Events:
  Type    Reason                Age   From                Message
  ----    ------                ----  ----                -------
  Normal  EnsuringLoadBalancer  10m   service-controller  Ensuring load balancer
  Normal  EnsuredLoadBalancer   9m8s  service-controller  Ensured load balancer

Looks like I am making some mistake with port mapping. Any help is much appreciated.

-- ashwini prakash
azure-aks
kubectl
kubernetes
tensorflow-serving

4 Answers

9/4/2019

From the information you provided, there is no problem with the service having the load balancer type. As I think, the possible reasons are all about your application and I would list below:

  1. the port which you expose is not right, so you need to make sure what is the right port to expose.
  2. I see you want to use GPU in AKS, so you need to choose the right VM size for GPU in the creation time. This may also cause the problem that your application not in the running state.
  3. there are other problems happen to your application and it cause your application not run well. So you also need to check your application state.
  4. To the imagePullSecret, maybe you do not assign enough permission for the service principal to pull the image. This reason is less likely, but I also list here.

Hope this helps you.

-- Charles Xu
Source: StackOverflow

9/10/2019

I think you need to access application on 52.183.17.101:8501

Because you didn't defined the routing the traffic to 80 port of loadbalancer. By default it will create loadbalancer listening on 8501.

-- Sachin Arote
Source: StackOverflow

9/9/2019

The container i was trying to access had no port open on 8501 , once i fixed it it worked well.

-- ashwini prakash
Source: StackOverflow

9/5/2019

Please follow the advice provided by community:

Researching this topic I would suggest to take a look for those Azure specific information:

If the static IP address defined in the loadBalancerIP property of the Kubernetes service manifest does not exist, or has not been created in the node resource group and no additional delegations configured, the load balancer service creation fails.

There is very similar case on github.

If using Advanced networking it creates the vNet in the same resource group as the AKS service by default.

Note:

Currently only Basic IP SKUis supported. Work is in progress to support the Standard IP resource SKU. For more information, see IP address types and allocation methods in Azure.

Additional resources:

Azure-LoadBalnacer related:

Hope this help.

-- Hanx
Source: StackOverflow