Accessing firestore data within GKE from another region

4/26/2020

I'm hoping someone could answer or share some lights,

I'm currently trying to access firestore data from GKE, however since firestore only has several regions so it's currently at a different region than our cluster.

  • GKE: asia-east1
  • Firestore: aisa-east2

Here's the sample code that I'm using:

import (
    "cloud.google.com/go/firestore"
    "context"
    "golang.org/x/oauth2/google"
    "google.golang.org/api/option"
)

func main() {
    ctx := context.Background()
    cred, err := google.FindDefaultCredentials(ctx, "https://www.googleapis.com/auth/datastore")
    if err != nil {
        log.Error(err, "failed to get cred")
    }

    client, err = firestore.NewClient(ctx, PROJECT_ID, option.WithCredentials(cred))
    if err != nil {
        log.Error(err, "Failed to connect firestore: %s")
        return nil, err
    }
    // I can retrieve data for docs with local environment, but in GKE it's always empty
    docs := client.Collection(COLLECT_NAME).Doc(id).Get(ctx)
    ...
}    

I can retrieve data for docs with local environment, but in GKE it's always empty. I suspect it's due the region, since I can see the error message if there's any permission issue. Is there a way to force setting the region for client? I couldn't find anyway to set region, or is there any workarounds?

Note kubernetes permissions is already enabled for Cloud Datastore as well as the node pool. I also tried setting GOOGLE_APPLICATION_CREDENTIALS and mounting it for the container but still no luck.

Any hints/helps are greatly appreciated.

Thanks!!

-- Bill Chung
google-cloud-datastore
google-cloud-firestore
google-cloud-platform
google-kubernetes-engine

1 Answer

4/26/2020

Solved...It was multiple mistakes and data issue from our application, now it's working.

For anyone that might suspect the region issue of firestore (since once it's set cannot be changed, and it has only one region, etc), the firestore data can still be accessed from other regions.

The steps I took to debug:

  1. Create a new GKE cluster with VPC native, since someone mentioned about VPC, but no luck.
  2. Create a vm in same region as GKE, found the script can get data! correctly
  3. Update the library I use [but this might not matter]
  4. Checked the data we use, found one data point accidentally contain null that caused our script to return null.
  5. Fixed the data and the program started to work (yay!)

So the lesson learned here is double check the data, since firestore can have null values so don't expect it to always return values.

-- Bill Chung
Source: StackOverflow