How to run kubernetes custom scheduler?

8/2/2017

I would like to run the custom scheduler given here: http://blog.kubernetes.io/2017/03/advanced-scheduling-in-kubernetes.html

which is a bash script. I need to use it in a google cloud cluster. Does anyone have the experience of implementing a scheduler like that? What are the steps to have that running?

-- samanta
gcloud
kubernetes
scheduler

1 Answer

8/27/2017

Assuming you've configured your kubectl for Google Cloud cluster, i.e. you can get nodes with:

$ kubectl get nodes

In terminal A, just run the following:

$ kubectl proxy

this will let your scheduler script to interact with api-server over localhost:8001

In terminal B, run the following to create deployment file for pod with custom scheduler (note "schedulerName: my-scheduler"):

$ cat > pod-to-schedule.yaml <<EOF
apiVersion: v1
kind: Pod
metadata:
  name: nginx
  labels:
    app: nginx
spec:
  schedulerName: my-scheduler
  containers:
  - name: nginx
    image: nginx:1.10
EOF

deploy the pod:

$ kubectl create -f pod-to-schedule.yaml

check the pod status:

$ kubectl get pods nginx
NAME      READY     STATUS    RESTARTS   AGE
nginx     0/1       Pending   0          12s

Pod status is "Pending" as there is no scheduler named "my-scheduler".

Now, save the scheduler script in the blog you mentioned:

$ cat << 'EOF' > my-scheduler.sh
#!/bin/bash
SERVER='localhost:8001'
while true;
do
    for PODNAME in $(kubectl --server $SERVER get pods -o json | jq '.items[] | select(.spec.schedulerName == "my-scheduler") | select(.spec.nodeName == null) | .metadata.name' | tr -d '"');
    do
        NODES=($(kubectl --server $SERVER get nodes -o json | jq '.items[].metadata.name' | tr -d '"'))
        NUMNODES=${#NODES[@]}
        CHOSEN=${NODES[$[ $RANDOM % $NUMNODES ]]}
        curl --header "Content-Type:application/json" --request POST --data '{"apiVersion":"v1", "kind": "Binding", "metadata": {"name": "'$PODNAME'"}, "target": {"apiVersion": "v1", "kind"
: "Node", "name": "'$CHOSEN'"}}' http://$SERVER/api/v1/namespaces/default/pods/$PODNAME/binding/
        echo "Assigned $PODNAME to $CHOSEN"
    done
    sleep 1
done
EOF

Run your scheduler:

$ bash my-scheduler.sh
{
  "kind": "Status",
  "apiVersion": "v1",
  "metadata": {},
  "status": "Success",
  "code": 201
}Assigned nginx to gke-cluster-1-default-pool-ac152967-nd30

Observe the output stating a node assignment (scheduling) is done for your pod. Verify that the pod is no longer pending:

$ kubectl get pods nginx
NAME      READY     STATUS    RESTARTS   AGE
nginx     1/1       Running   0          10m
-- turkenh
Source: StackOverflow