K
Q

Question

Proper way to perform different requests in parallel

2/21/2019

I know there are multiple different solutions to do what I am looking for, but I am looking for a/the proper way to perform some requests in parallel. I am new to Go but it feels cumbersome what I am doing at the moment.

Use case:

I need to query 4 different REST endpoints (kubernetes client requests) in parallel. Once I got all these 4 results I need to do some processing.

My problem:

I know I need to use go routines for that, but what's the best way to collect the results. What I am doing at the moment (see code sample below) is probably very cumbersome, but I am not sure what else I could do to improve the code.

Code:

This code is the easiest to understand but I don't want to perform the requests in sequence:

// Get node resource usage metrics
nodeMetricsList, err := c.kubernetesClient.NodeMetricses()
if err != nil {
    log.Warn("Failed to get node usage list from Kubernetes", err)
    return err
}

// Get pod resource usage metrics
podMetricsList, err := c.kubernetesClient.PodMetricses()
if err != nil {
    log.Warn("Failed to get pod usage list from Kubernetes", err)
    return err
}

This is how I would run the requests in parallel. This makes the code so much less readable and it feels very cumbersome to implement too:

var nodeMetricsList *v1beta1.NodeMetricsList
var nodeMetricsListError error
var podMetricsList *v1beta1.PodMetricsList
var podMetricsListError error

go func() {
    nodeMetricsList, nodeMetricsListError = c.kubernetesClient.NodeMetricses()
}()
if nodeMetricsListError != nil {
    log.Warn("Failed to get podList from Kubernetes", err)
    return err
}

// Get pod resource usage metrics
go func() {
    podMetricsList, podMetricsListError = c.kubernetesClient.PodMetricses()
}()
if podMetricsListError != nil {
    log.Warn("Failed to get pod usage list from Kubernetes", err)
    return err
}

What's the proper way to perform the requests in my given example in parallel?

-- kentor

go

kubernetes

1 Answer

2/21/2019

Your code has 2 race conditions, and likely will never correctly report an error.

You need to wait for the goroutines to complete before you can read the values they operate on, which is easily done with a sync.WaitGroup like so:

var nodeMetricsList *v1beta1.NodeMetricsList
var podMetricsList *v1beta1.PodMetricsList
var nodeMetricsListError, podMetricsListError error
var wg sync.WaitGroup

// Get node resource usage metrics
wg.Add(1)
go func() {
    defer wg.Done()
    nodeMetricsList, nodeMetricsListError = c.kubernetesClient.NodeMetricses()
}()

// Get pod resource usage metrics
wg.Add(1)
go func() {
    defer wg.Done()
    podMetricsList, podMetricsListError = c.kubernetesClient.PodMetricses()
}()

wg.Wait()
if nodeMetricsListError != nil {
    log.Warn("Failed to get podList from Kubernetes", err)
    return err
}
if podMetricsListError != nil {
    log.Warn("Failed to get pod usage list from Kubernetes", err)
    return err
}

fmt.Println("Hello, playground")

-- JimB

Source: StackOverflow

KQ

Proper way to perform different requests in parallel

Similar Questions

1 Answer

K
Q