i've found two similar posts here but one hasn't been answered and the other was about android. I have a spring boot project and I want to access GCP Storage files within my application.
Locally everything works fine I can access my bucket and read as well as store files in my storage. But when i upload it to gcp kubernetes I get the following exception:
"java.nio.file.FileSystemNotFoundException: Provider "gs" not installed at java.nio.file.Paths.get(Paths.java:147) ~na:1.8.0_212 at xx.xx.StorageService.saveFile(StorageService.java:64) ~classes!/:0.3.20-SNAPSHOT
My line of code where it appears is like follows:
public void saveFile(MultipartFile multipartFile, String path) {
String completePath = filesBasePath + path;
Path filePath = Paths.get(URI.create(completePath)); // <- exception appears here
Files.createDirectories(filePath);
multipartFile.transferTo(filePath);
}
}
The completePath could result in something like "gs://my-storage/path/to/file/image.jpg"
I have the following dependencies:
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-gcp-starter-storage</artifactId>
<version>1.2.6.RELEASE</version>
</dependency>
<dependency>
<groupId>com.google.cloud</groupId>
<artifactId>google-cloud-nio</artifactId>
<version>0.122.5</version>
</dependency>
Does anyone have a clue where to look at? The only real difference except for the infrastructure is that i don't explicitly don't use authentication on kubernetes as it is not required according to the documentation
When using Google Cloud libraries from a Google Cloud Platform environment such as Compute Engine, Kubernetes Engine, or App Engine, no additional authentication steps are necessary.
The gs:// syntax is not universal; it's supported by certain Google tools and services (gsutil, for example) but it's not a known by many URL libraries.
You could change to use the HTTP syntax for one of the GCS APIs, e.g.,
https://my-storage.storage.googlapis.com/path/to/file/image.jpg
Note that the above HTTP URL will require authentication, so probably instead you would want to use a library like google-cloud-java that supports making authenticated GCS requests.
It looks like the conventional Spring boot packaging here isn't packaging the dependency in the needed way. Usually you'll see something like:
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<version>2.4.5</version>
<executions>
<execution>
<goals>
<goal>repackage</goal>
</goals>
</execution>
</executions>
</plugin>
However, for the "gs" provider to be accessible it needs to be in the 'lib/' folder. You can package it manually by copying the dependencies and then creating the JAR (this is based on the springboot-helloworld sample:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<executions>
<execution>
<id>copy-dependencies</id>
<phase>prepare-package</phase>
<goals>
<goal>copy-dependencies</goal>
</goals>
<configuration>
<outputDirectory>
${project.build.directory}/lib
</outputDirectory>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<configuration>
<archive>
<manifest>
<addClasspath>true</addClasspath>
<classpathPrefix>lib/</classpathPrefix>
<mainClass>
com.example.appengine.springboot.SpringbootApplication
</mainClass>
</manifest>
</archive>
</configuration>
</plugin>
Originally posted on GitHub.
Following Averi Kitsch's answer and using the same springboot-helloworld example, I was able to get it working locally after updating pom.xml. However, much like it did for you, it only worked when I ran it locally and would fail when I deployed it on Google Cloud. The issue was that the Dockerfile I was using was ignoring all of the jar files in the /target/lib directory, and I needed to copy that directory to the image. Note that I used Google Cloud Run, but I believe this should work for most deployments using Dockerfile.
Here's what I ended up with:
Dockerfile
FROM maven:3.8-jdk-11 as builder
WORKDIR /app
COPY pom.xml .
COPY src ./src
RUN mvn package -DskipTests
FROM adoptopenjdk/openjdk11:alpine-jre
COPY --from=builder /app/target/springboot-helloworld-*.jar /springboot-helloworld.jar
# IMPORTANT! - Copy the library jars to the production image!
COPY --from=builder /app/target/lib /lib
CMD ["java", "-Djava.security.egd=file:/dev/./urandom", "-jar", "/springboot-helloworld.jar"]
pom.xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.example.appengine</groupId>
<artifactId>springboot-helloworld</artifactId>
<version>0.0.1-SNAPSHOT</version>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.5.4</version>
<relativePath/>
</parent>
<properties>
<maven.compiler.target>11</maven.compiler.target>
<maven.compiler.source>11</maven.compiler.source>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>com.google.cloud</groupId>
<artifactId>google-cloud-nio</artifactId>
<version>0.123.8</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<executions>
<execution>
<id>copy-dependencies</id>
<phase>prepare-package</phase>
<goals>
<goal>copy-dependencies</goal>
</goals>
<configuration>
<outputDirectory>
${project.build.directory}/lib
</outputDirectory>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<configuration>
<archive>
<manifest>
<addClasspath>true</addClasspath>
<classpathPrefix>lib/</classpathPrefix>
<mainClass>
com.example.appengine.springboot.SpringbootApplication
</mainClass>
</manifest>
</archive>
</configuration>
</plugin>
</plugins>
</build>
</project>