K
Q

Question

How to configure correctly to receive the prediction result in Seldon Core python client?

5/20/2019

I'm checking out with Seldon Core on Minikube and have successfully deployed a model on a cluster. I tested with below code:

seldon-core-api-tester ../seldon-core/examples/models/keras_mnist/contract.json     `minikube ip` `kubectl get svc -l app=seldon-apiserver-container-app -o jsonpath='{.items[0].spec.ports[0].nodePort}'`     --oauth-key oauth-key --oauth-secret oauth-secret -p

and got the right prediction result looking like this.

RECEIVED RESPONSE:
meta {
  puid: "gn83vb2ag419k547eqkhfduos2"
  requestPath {
    key: "mnist"
    value: "mnist:0.1"
  }
}
data {
  names: "t:0"
  names: "t:1"
  names: "t:2"
  names: "t:3"
  names: "t:4"
  names: "t:5"
  names: "t:6"
  names: "t:7"
  names: "t:8"
  names: "t:9"
  ndarray {
    values {
      list_value {
        values {
          number_value: 0.00026227490161545575
        }
        values {
          number_value: 0.0007252057548612356
        }
        values {
          number_value: 0.028986405581235886
        }
        values {
          number_value: 0.8030332922935486
        }
        values {
          number_value: 7.914198795333505e-05
        }
        values {
          number_value: 0.14541368186473846
        }
        values {
          number_value: 0.002676495350897312
        }
        values {
          number_value: 0.015001941472291946
        }
        values {
          number_value: 0.0034872409887611866
        }
        values {
          number_value: 0.00033424459979869425
        }
      }
    }
  }
}

However, when I was trying to use the python client,

from seldon_core.seldon_client import SeldonClient
sc = SeldonClient(deployment_name="mnist",namespace="seldon", seldon_rest_endpoint= '127.0.0.1:30790')
r = sc.predict(transport="rest")

I got this error.

HTTPConnection object at 0xb2bb5a780>: Failed to establish a new connection: [Errno 61] Connection refused'))

Could someone help me find out what's wrong?

$kubectl get svc
mnist-deployment-mnist   ClusterIP      10.99.10.81      <none>        8000/TCP,5001/TCP               2d22h
kubernetes                           ClusterIP      10.96.0.1        <none>        443/TCP                         4d22h
seldon-core-redis-master             ClusterIP      10.107.217.176   <none>        6379/TCP                        2d22h
seldon-core-seldon-apiserver         NodePort       10.106.34.6      <none>        8080:30790/TCP,5000:31866/TCP   2d22h
seldon-mnist-0-1-4249605       ClusterIP      10.101.205.227   <none>        9000/TCP                        2d22h

-- user3368526

kubeflow

kubernetes

minikube

seldon

1 Answer

8/1/2019

When you run the seldon-core-api-tester script, you provide minikube ip as an argument (along with the ambassador port). You'll need this address for the endpoint when you initialize the client instead of 127.0.0.1. So first run in your shell

minikube ip

and take a note of the ip, then find the ambassador port

kubectl get svc ambassador -o jsonpath='{.spec.ports[0].nodePort}'

then your client and call will look sth like this

from seldon_core.seldon_client import SeldonClient
import numpy as np

# this is the ip from `minikube ip` and port from `kubectl get svc ambassador -o jsonpath='{.spec.ports[0].nodePort}'`
minikube_ambassador_endpoint = "192.168.99.108:32667"

deployment_name = "mnist"
namespace = "default"

sc = SeldonClient(
    gateway="ambassador",
    gateway_endpoint=minikube_ambassador_endpoint,
    transport="rest",
    deployment_name=deployment_name,
    namespace=namespace
)

response = sc.predict(
    data=np.ones((5,)), 
    deployment_name=deployment_name, 
    payload_type="ndarray"
)
print(response)

-- PSL

Source: StackOverflow

KQ

How to configure correctly to receive the prediction result in Seldon Core python client?

Similar Questions

1 Answer

K
Q