I have library hosted on Google Cloud storage. I am using Jenkins to deploy my application into GKE.
I am using CloudStorageMaven library (V1.0) to push project to GCS.
I am able to deploy my library project but, when I am trying to run test for microservice on Jenkins server, maven is not able to download my library.
I have tried making GCS bucket public but it did not make any difference.
Jenkinsfile
#!/usr/bin/env groovy
podTemplate(
label: 'slavepod',
containers: [
containerTemplate (
name: 'maven',
image: 'maven:3.5.2-alpine',
command: 'cat',
ttyEnabled: true,
resourceRequestCpu: '256m',
resourceLimitCpu: '512m',
resourceRequestMemory: '512Mi',
resourceLimitMemory: '1024Mi'
),
containerTemplate (
name: 'docker',
image: 'gcr.io/cloud-builders/docker',
command: 'cat',
ttyEnabled: true,
resourceRequestCpu: '512m',
resourceLimitCpu: '1024m',
resourceRequestMemory: '512Mi',
resourceLimitMemory: '1024Mi'
),
containerTemplate (
name: 'kubectl',
image: 'gcr.io/cloud-builders/kubectl',
command: 'cat',
ttyEnabled: true,
resourceRequestCpu: '512m',
resourceLimitCpu: '1024m',
resourceRequestMemory: '512Mi',
resourceLimitMemory: '1024Mi'
),
containerTemplate (
name: 'gcloud',
image: 'gcr.io/cloud-builders/gcloud',
command: 'cat',
ttyEnabled: true,
resourceRequestCpu: '512m',
resourceLimitCpu: '1024m',
resourceRequestMemory: '512Mi',
resourceLimitMemory: '1024Mi'
)
],
volumes: [
hostPathVolume(mountPath: '/var/run/docker.sock', hostPath: '/var/run/docker.sock'),
persistentVolumeClaim(claimName: 'maven-repo', mountPath: '/home/jenkins/.m2')
]
)
{
node('slavepod') {
stage('Clone repository') {
checkout scm
}
stage('Unit Test') {
container('maven') {
echo 'Running Maven tests'
sh("mvn -Dmaven.repo.local=/home/jenkins/.m2/ test")
}
}
} // end of pipeline
I have extension and repository configured in pom.xml
<build>
....
<extensions>
<extension>
<groupId>com.gkatzioura.maven.cloud</groupId>
<artifactId>google-storage-wagon</artifactId>
<version>1.0</version>
</extension>
</extensions>
</build>
<repositories>
<repository>
<id>ca.performance.common</id>
<url>gs://performance-repository</url>
</repository>
</repositories>Error I am getting in Jenkins pipeline console is as below:
Downloading from ca.performance.common: gs://url/to/repository/jar-1.0.0.pom
[ERROR] Could not establish connection with google cloud
com.google.cloud.storage.StorageException: Insufficient Permission
at com.google.cloud.storage.spi.v1.HttpStorageRpc.translate (HttpStorageRpc.java:219)
at com.google.cloud.storage.spi.v1.HttpStorageRpc.list (HttpStorageRpc.java:314)
at com.google.cloud.storage.StorageImpl$6.call (StorageImpl.java:272)
at com.google.cloud.storage.StorageImpl$6.call (StorageImpl.java:269)
at com.google.api.gax.retrying.DirectRetryingExecutor.submit (DirectRetryingExecutor.java:89)
at com.google.cloud.RetryHelper.run (RetryHelper.java:74)
at com.google.cloud.RetryHelper.runWithRetries (RetryHelper.java:51)
at com.google.cloud.storage.StorageImpl.listBuckets (StorageImpl.java:268)
at com.google.cloud.storage.StorageImpl.list (StorageImpl.java:257)
at com.gkatzioura.maven.cloud.gcs.GoogleStorageRepository.connect (GoogleStorageRepository.java:55)
at com.gkatzioura.maven.cloud.gcs.GoogleStorageWagon.connect (GoogleStorageWagon.java:135)
at org.eclipse.aether.transport.wagon.WagonTransporter.connectWagon (WagonTransporter.java:342)
at org.eclipse.aether.transport.wagon.WagonTransporter.pollWagon (WagonTransporter.java:382)
at org.eclipse.aether.transport.wagon.WagonTransporter.execute (WagonTransporter.java:431)
at org.eclipse.aether.transport.wagon.WagonTransporter.get (WagonTransporter.java:412)
at org.eclipse.aether.connector.basic.BasicRepositoryConnector$GetTaskRunner.runTask (BasicRepositoryConnector.java:453)
at org.eclipse.aether.connector.basic.BasicRepositoryConnector$TaskRunner.run (BasicRepositoryConnector.java:360)
at org.eclipse.aether.util.concurrency.RunnableErrorForwarder$1.run (RunnableErrorForwarder.java:75)
at org.eclipse.aether.connector.basic.BasicRepositoryConnector$DirectExecutor.execute (BasicRepositoryConnector.java:583)
at org.eclipse.aether.connector.basic.BasicRepositoryConnector.get (BasicRepositoryConnector.java:259)
at org.eclipse.aether.internal.impl.DefaultArtifactResolver.performDownloads (DefaultArtifactResolver.java:498)
at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve (DefaultArtifactResolver.java:399)
at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifacts (DefaultArtifactResolver.java:224)
at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifact (DefaultArtifactResolver.java:201)
at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.loadPom (DefaultArtifactDescriptorReader.java:261)
at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.readArtifactDescriptor (DefaultArtifactDescriptorReader.java:192)
at org.eclipse.aether.internal.impl.DefaultDependencyCollector.resolveCachedArtifactDescriptor (DefaultDependencyCollector.java:539)
at org.eclipse.aether.internal.impl.DefaultDependencyCollector.getArtifactDescriptorResult (DefaultDependencyCollector.java:522)
at org.eclipse.aether.internal.impl.DefaultDependencyCollector.processDependency (DefaultDependencyCollector.java:411)
at org.eclipse.aether.internal.impl.DefaultDependencyCollector.processDependency (DefaultDependencyCollector.java:365)
at org.eclipse.aether.internal.impl.DefaultDependencyCollector.process (DefaultDependencyCollector.java:353)
at org.eclipse.aether.internal.impl.DefaultDependencyCollector.collectDependencies (DefaultDependencyCollector.java:256)
at org.eclipse.aether.internal.impl.DefaultRepositorySystem.collectDependencies (DefaultRepositorySystem.java:282)
at org.apache.maven.project.DefaultProjectDependenciesResolver.resolve (DefaultProjectDependenciesResolver.java:169)
at org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.getDependencies (LifecycleDependencyResolver.java:223)
at org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.resolveProjectDependencies (LifecycleDependencyResolver.java:145)
at org.apache.maven.lifecycle.internal.MojoExecutor.ensureDependenciesAreResolved (MojoExecutor.java:246)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:200)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:154)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:146)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:51)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:309)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:194)
at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:107)
at org.apache.maven.cli.MavenCli.execute (MavenCli.java:955)
at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:290)
at org.apache.maven.cli.MavenCli.main (MavenCli.java:194)
at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke (Method.java:498)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:289)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:229)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:415)
at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:356)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 403 Forbidden
{
"code" : 403,
"errors" : [ {
"domain" : "global",
"message" : "Insufficient Permission",
"reason" : "insufficientPermissions"
} ],
"message" : "Insufficient Permission"
}
Ensure that you follow the instructions in the full guide from the CloudStorageMaven link you posted?
As the official docs states, you need to grant sufficient permissions to specific users, or allow public (allUsers) readability at bucket-level to make all objects in that given bucket accessible to specified users or allUsers.
The error code from your attempt points to access permission issues:
{
"code" : 403,
"errors" : [ {
"domain" : "global",
"message" : "Insufficient Permission",
"reason" : "insufficientPermissions"
} ],
"message" : "Insufficient Permission"
}
If you haven't already tried it, you could grant the allUsers access at the bucket-level by running:
$ gsutil iam ch allUsers:objectViewer gs://[BUCKET_NAME]However, keeping with best practices, I would recommend granting granular access at the bucket-level for only the user accounts or service accounts you wish to have read/write access to the objects within a given bucket. You can read more about how to do that here.
If the problem persists then the "Insufficient Permissions" issue is with the GKE cluster. Check what Storage permissions your cluster has like so:
From the Cloud Console --> Click on Kubernets Engine
Click on the 'Cluster Name' ---> Find 'Storage' under Permissions section.
If you only have 'Read Only' then you'll need to recreate your cluster like so: