K
Q

Google Cloud PubSub on GKE : FileNotFoundException while attempting to retrieve the credentials

11/22/2019

I am developing a Spring application that uses the Spring Google PubSub library (spring-cloud-gcp-starter-pubsub 1.1.4) which will be deployed in my GKE cluster.

As mentionned in this page, I created a service account with the suitable rights on PubSub functionalities.

I created the secret with this command:

kubectl create secret generic pub-sub-key --from-file ~/Documents/kubernetes/gke/service\ accounts/pub-sub/D-Vine-d36104130624.json

Here is the yaml file used to deploy my application :

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: d-vine-machine-dev
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: d-vine-machine-dev
        version: dev
    spec:
      volumes:
        - name: pub-sub-service-account
          secret:
            secretName: pub-sub-key
      containers:
        - name: d-vine-machine
          image: gcr.io/........:0.0.47
          imagePullPolicy: Always
          env:
            - name: SPRING_PROFILES_ACTIVE
              value: dev
            - name: GOOGLE_APPLICATION_CREDENTIALS
              value: /var/secrets/google/D-Vine-d36104130624.json
          volumeMounts:
            - name: pub-sub-service-account
              mountPath: /var/secrets/google
          ports:
            - containerPort: 8080

But my container ends up not finding the file:

Caused by: java.io.FileNotFoundException: Could not open ServletContext resource [/var/secrets/google/D-Vine-d36104130624.json] 
at org.springframework.web.context.support.ServletContextResource.getInputStream(ServletContextResource.java:158) ~[spring-web-5.1.3.RELEASE.jar:5.1.3.RELEASE] 
at org.springframework.cloud.gcp.core.DefaultCredentialsProvider.<init>(DefaultCredentialsProvider.java:94) ~[spring-cloud-gcp-core-1.1.0.RC2.jar:1.1.0.RC2] 
at org.springframework.cloud.gcp.autoconfigure.core.GcpContextAutoConfiguration.googleCredentials(GcpContextAutoConfiguration.java:57) ~[spring-cloud-gcp-autoconfigure-1.1.0.RC2.jar:1.1.0.RC2] 
at org.springframework.cloud.gcp.autoconfigure.core.GcpContextAutoConfiguration$EnhancerBySpringCGLIB$16f5032c.CGLIB$googleCredentials$0(<generated>) ~[spring-cloud-gcp-autoconfigure-1.1.0.RC2.jar:1.1.0.RC2] 
at org.springframework.cloud.gcp.autoconfigure.core.GcpContextAutoConfiguration$EnhancerBySpringCGLIB$16f5032c$FastClassBySpringCGLIB$ce730653.invoke(<generated>) ~[spring-cloud-gcp-autoconfigure-1.1.0.RC2.jar:1.1.0.RC2] 
at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:244) ~[spring-core-5.1.3.RELEASE.jar:5.1.3.RELEASE] 
at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:363) ~[spring-context-5.1.4.RELEASE.jar:5.1.4.RELEASE] 
at org.springframework.cloud.gcp.autoconfigure.core.GcpContextAutoConfiguration$EnhancerBySpringCGLIB$16f5032c.googleCredentials(<generated>) ~[spring-cloud-gcp-autoconfigure-1.1.0.RC2.jar:1.1.0.RC2] 
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_232] 
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_232] 
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_232] at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_232] 
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:154) ~[spring-beans-5.1.3.RELEASE.jar:5.1.3.RELEASE]

Out of curiosity, I've modified my Go application deployment just to check if the file was correctly retrieved. And it was... So it seems to be related to the Spring application and not the deployment.

My image is built using the jib maven plugin if that makes any difference.

Has anyone encountered this issue (or sees any mistake) ?

-- Jean-Baptiste Piraux
google-cloud-pubsub
google-kubernetes-engine
java
kubernetes
spring

Similar Questions

How to redirect HTTP to HTTPS with Nginx Ingress Controller, AWS NLB and TLS certificate managed by AWS Certificate Manager?
Docker for windows kubernetes helm - add public repo and get mailhog
Should I Disable TCP Timestamp for Kubernetes?
Using istio to secure kubernetes traffic (clusterIP)
Error trying run react Docker image on k8s

1 Answer

11/22/2019

Your program is looking for /var/run/secret/google/D-Vine-d36104130624.json

But you are mounting secrets at: /var/secrets/google

Since this stack trace is in spring code related to fetching based on a property, I suspect that there might be something incorrect in your spring configuration. Is it possible your spring configuration lists the /var/run location as spring.cloud.gcp.credentials.location? (If not, spring should fall back to the GOOGLE_APPLICAITON_CREDENTIALS location -- but that wouldn't result in this stack trace).

Assuming this is the case, you should update the spring configuration to point at the /var/secrets location (or, as the docs recommend, omit it to fall back to the environment variable).

Alternatively, I suspect moving the mount point (and changing the environment variable) to the /var/run location will resolve this.

-- robsiemb
Source: StackOverflow