Python kubernetes pod watch state

5/2/2020

I have not been successfully in getting the actual state of pod when it changes states. From my code below, it is stuck in the waiting and in my terminal ContainerCreating is stuck in an endless loop.

    for event in watch.stream(func=v1.list_namespaced_pod, namespace=ns, timeout_seconds=120):
      response = event['object'].status
      pod_data = {
          "name": data.container_statuses[0].name,
          "status": data.container_statuses[0].state,
          "phase": data.phase,
      }
      if f'{pod_name}' in data[0]['name']:
        running = data[0]['status'].running
        wait = data[0]['status'].waiting
        while wait != None:
          print(f"waiting for {pod_name} state to Running...")
          print(wait.reason)
          time.sleep(10)
        if running.started_at != None:
          print("State has changed to Running. Pod has been created and started... ' ")
          watch.stop()

What is the best approach to get the current state of my pod? I have a copy action that is triggered when the state changes from Pending to Running.

ContainerCreating
ContainerCreating
ContainerCreating
ContainerCreating
ContainerCreating
ContainerCreating
ContainerCreating
ContainerCreating
ContainerCreating

I have also tried using event['object'].status.phase. But it's still stuck in the loop.

 'container_statuses': [{'container_id': 'container-xxx',
                         'image': 'image-xxx',
                         'image_id': 'image-xxxx',
                         'last_state': {'running': None,
                                        'terminated': None,
                                        'waiting': None},
                         'name': 'hub',
                         'ready': True,
                         'restart_count': 0,
                         'state': {'running': {'started_at': datetime.datetime(2020, 4, 11, 8, 15, 53, tzinfo=tzutc())},
                                   'terminated': None,
                                   'waiting': None}}],
 'host_ip': 'xx.xx.xx.xxx',
 'init_container_statuses': None,
 'message': None,
 'nominated_node_name': None,
 'phase': 'Running',
 'pod_ip': 'x.xx.xx.xxx',
 'qos_class': 'Burstable',
 'reason': None,
 'start_time': datetime.datetime(2020, 4, 11, 23, 48, 1, tzinfo=tzutc())}

I might be doing something wrong in the python call or using the wrong kubernetes method. I am out of ideas at this point!

-- ghenzi83
azure-kubernetes
kubernetes-pod
python

1 Answer

5/2/2020

I think you can use watch.stop():

watch = kubernetes.watch.Watch()
core_v1 = k8s.CoreV1Api()
for event in watch.stream(func=core_v1.list_namespaced_pod,
                          namespace=namespace,
                          label_selector=label,
                          timeout_seconds=60):
    if event["object"].status.phase == "Running":
        watch.stop()
        end_time = time.time()
        logger.info("%s started in %0.2f sec", full_name, end_time-start_time)
        return
    # event.type: ADDED, MODIFIED, DELETED
    if event["type"] == "DELETED":
        # Pod was deleted while we were waiting for it to start.
        logger.debug("%s deleted before it started", full_name)
        watch.stop()
        return
-- irvifa
Source: StackOverflow