Deploy kubeflow on GCP CloudShell using cli: /home/user/.kube/config: no such file or directory

6/10/2019

I need to deploy Kubeflowfor ML pipeline and TFX on GCP. Unfortunately I cannot use the following UI for the installation, because I need to setup manually, the Zone, Network and Subnet. enter image description here

There is a nice documentation page: https://www.kubeflow.org/docs/gke/deploy/deploy-cli/

I tried with both OAuth credentials and basic authentication. I also tried to install kfctl_v0.5.1_linux.tar.gz and kfctl_v0.5.0_linux.tar.gz. I always got the following:

WARN[0036] could not open /home/user/.kube/config Error stat /home/user/.kube/config: no such file or directory  filename=“apps/group.go:188”
WARN[0036] could not load config Error: open /home/user/.kube/config: no such file or directory  filename=“apps/group.go:208"

I have kubectl installed:

kubectl version
Client Version: version.Info{Major:"1", Minor:"12+", GitVersion:"v1.12.8-dispatcher", GitCommit:"1215389331387f57594b42c5dd024a2fe27334f8", GitTreeState:"clean", BuildDate:"2019-05-1
3T18:09:56Z", GoVersion:"go1.10.8", Compiler:"gc", Platform:"linux/amd64"}
The connection to the server localhost:8080 was refused - did you specify the right host or port?

Clearly ~/.kube/config doesn't exist. Which application is suppose to create it ?

ls ~/.kube/config
ls: cannot access '/home/user/.kube/config': No such file or directory

I did the following:

ls home/user/folder/kubeflow
kfctl  kfctl_v0.5.1_linux.tar.gz


export KUBEFLOW_USERNAME=xxx
export KUBEFLOW_PASSWORD=xxx
export PATH=$PATH:/home/user/folder/kubeflow
export ZONE=europe-west1-b
export PROJECT=project
export KFAPP=kubeflow-test

from home/user/folder/kubeflow:

kfctl init ${KFAPP} --platform gcp --project ${PROJECT} --use_basic_auth -V

INFO[0014] Not skipping GCP project init, running gcpInitProject.  filename="gcp/gcp.go:1619"
WARN[0017] batch API enabling is running: [deploymentmanager.googleapis.com servicemanagement.googleapis.com container.googleapis.com cloudresourcemanager.googleapis.com endpoints.googleapis.com file.googleapis.com ml.googleapis.com iam.googleapis.com sqladmin.googleapis.com] (op = operations/acf.954cc3b6-f1f4-46a2-832d-596ccb5a3d5a)  filename="gcp/gcp.go:1594"
WARN[0018] batch API enabling is running: [deploymentmanager.googleapis.com servicemanagement.googleapis.com container.googleapis.com cloudresourcemanager.googleapis.com endpoints.googleapis.com file.googleapis.com ml.googleapis.com iam.googleapis.com sqladmin.googleapis.com] (op = operations/acf.954cc3b6-f1f4-46a2-832d-596ccb5a3d5a)  filename="gcp/gcp.go:1594"
WARN[0019] batch API enabling is running: [deploymentmanager.googleapis.com servicemanagement.googleapis.com container.googleapis.com cloudresourcemanager.googleapis.com endpoints.googleapis.com file.googleapis.com ml.googleapis.com iam.googleapis.com sqladmin.googleapis.com] (op = operations/acf.954cc3b6-f1f4-46a2-832d-596ccb5a3d5a)  filename="gcp/gcp.go:1594"
WARN[0021] batch API enabling is running: [deploymentmanager.googleapis.com servicemanagement.googleapis.com container.googleapis.com cloudresourcemanager.googleapis.com endpoints.googleapis.com file.googleapis.com ml.googleapis.com iam.googleapis.com sqladmin.googleapis.com] (op = operations/acf.954cc3b6-f1f4-46a2-832d-596ccb5a3d5a)  filename="gcp/gcp.go:1594"
WARN[0024] batch API enabling is running: [deploymentmanager.googleapis.com servicemanagement.googleapis.com container.googleapis.com cloudresourcemanager.googleapis.com endpoints.googleapis.com file.googleapis.com ml.googleapis.com iam.googleapis.com sqladmin.googleapis.com] (op = operations/acf.954cc3b6-f1f4-46a2-832d-596ccb5a3d5a)  filename="gcp/gcp.go:1594"
WARN[0027] batch API enabling is running: [deploymentmanager.googleapis.com servicemanagement.googleapis.com container.googleapis.com cloudresourcemanager.googleapis.com endpoints.googleapis.com file.googleapis.com ml.googleapis.com iam.googleapis.com sqladmin.googleapis.com] (op = operations/acf.954cc3b6-f1f4-46a2-832d-596ccb5a3d5a)  filename="gcp/gcp.go:1594"
WARN[0030] batch API enabling is running: [deploymentmanager.googleapis.com servicemanagement.googleapis.com container.googleapis.com cloudresourcemanager.googleapis.com endpoints.googleapis.com file.googleapis.com ml.googleapis.com iam.googleapis.com sqladmin.googleapis.com] (op = operations/acf.954cc3b6-f1f4-46a2-832d-596ccb5a3d5a)  filename="gcp/gcp.go:1594"
INFO[0037] batch API enabling is completed: [deploymentmanager.googleapis.com servicemanagement.googleapis.com container.googleapis.com cloudresourcemanager.googleapis.com endpoints.googleapis.com file.googleapis.com ml.googleapis.com iam.googleapis.com sqladmin.googleapis.com]  filename="gcp/gcp.go:1590"
INFO[0037] reading from /home/user/folder/kubeflow/kubeflow-test/app.yaml  filename="coordinator/coordinator.go:341"
WARN[0037] could not open /home/user/.kube/config Error stat /home/user/.kube/config: no such file or directory  filename="apps/group.go:188"
WARN[0037] could not load config Error: open /home/user/.kube/config: no such file or directory  filename="apps/group.go:208"

Some file were created:

ls kubeflow-test/
app.yaml

Some check:

kubectl config view
apiVersion: v1
clusters: []
contexts: []
current-context: ""
kind: Config
preferences: {}
users: []

I also try to update kubectl

gcloud components update kubectl
You have specified individual components to update.  If you are trying
 to install new components, use:
  $ gcloud components install kubectl
Do you want to run install instead (y/N)?  y
All components are up to date.

Any idea why I don't have ~/.kube/config doesn't exist ? I try to create it manually but then I have other issue later. What should I do to create the mising configuration. Is there another recommend way than using CloudShell to deploy Kubelow with cli on GCP ?

-- Dr. Fabien Tarrade
google-cloud-platform
google-cloud-shell
google-kubernetes-engine
kubeflow

1 Answer

6/11/2019

GKE has a great description of how you can configure access to a cluster via kubectl (which uses the ~/.kube/config file as the default location to store credentials). There are two ways to populate the file:

  1. Create your cluster through the command line using gcloud container clusters create CLUSTER_NAME
  2. Fetching credentials for an existing cluster using gcloud container clusters get-credentials CLUSTER_NAME

From the Kubeflow docs, it looks like the init step doesn't create a cluster; the apply step should create a cluster. You don't describe what issue you have later if you create an empty file.

Also, you pointed to the deploy cli docs, but the customizing Kubeflow on GKE page sounds more like what you are trying to accomplish so you might see if that page answers any of your questions.

-- Robert Bailey
Source: StackOverflow