We are trying to deploy our java project on a tomcat server in an OpenShift environment. Weird thing is that it works on localhost, it also works when opening the terminal in the OpenShift client pod and using curl. But when the java application running in the client pod tries it returns a 403 error.

Does anyone have any idea what we are doing wrong / are missing? Here is some information regarding the applications and the environment:

Server part of POM dependencies:

<dependency>
    <groupId>org.glassfish.jersey.core</groupId>
    <artifactId>jersey-client</artifactId>
    <version>2.24</version>
</dependency>
<dependency>
    <groupId>org.glassfish.jersey.core</groupId>
    <artifactId>jersey-server</artifactId>
    <version>2.24</version>
</dependency>
<dependency>
    <groupId>org.glassfish.jersey.containers</groupId>
    <artifactId>jersey-container-servlet</artifactId>
    <version>2.24</version>
</dependency>

Server part of web.xml:

<servlet>
    <servlet-name>Our REST Endpoint</servlet-name>
    <servlet-class>org.glassfish.jersey.servlet.ServletContainer</servlet-class>
    <init-param>
        <param-name>javax.ws.rs.Application</param-name>
        <param-value>com.mycompany.ourapplication.business.rest.ServiceApplication</param-value>
    </init-param>
    <load-on-startup>1</load-on-startup>
</servlet>
<servlet-mapping>
    <servlet-name>Our REST Endpoint</servlet-name>
    <url-pattern>/rest/*</url-pattern>
</servlet-mapping>

Server contents of the com.mycompany.ourapplication.business.rest package:

AuthenticationFilter.java

@Provider
public class AuthenticationFilter implements ContainerRequestFilter
{
    @Override
    public void filter(final ContainerRequestContext requestContext)
    {
        /* When using curl these are printed, when accessed via the client application
         * these printlns are not printed, which can only mean that it doesn't even reach this code
         */
        System.out.println("WENT INTO AUTHENTICATION FILTER");
        final Response ACCESS_DENIED = Response.status(Response.Status.UNAUTHORIZED)
            .entity("You cannot access this resource")
            .build();

        // These were also just to see if we were missing some headers, but sadly didn't fix it
        requestContext.getHeaders().add("Access-Control-Allow-Origin", "*");
        requestContext.getHeaders().add("Access-Control-Allow-Headers", "origin, content-type, accept, authorization");
        requestContext.getHeaders().add("Access-Control-Allow-Credentials", "true");
        requestContext.getHeaders().add("Access-Control-Allow-Methods", "GET, POST, PUT, DELETE, OPTIONS, HEAD");
        requestContext.getHeaders().add("Access-Control-Max-Age", "1209600");

        /* Here is some commented out authentication code (not shown)
         * to see if it was part of the problem...but doesn't seem to be
         */

        System.out.println("AUTHENTICATION FILTER SUCCESSFULL");
    }
}

GETServices.java (snippet)

@Path("/")
public class GETServices
{
    private static final Gson   GSON        = new Gson();
    private static final String UTF8JSON    = MediaType.APPLICATION_JSON + ";charset=utf-8";

    @GET
    @Produces(UTF8JSON)
    @Consumes(UTF8JSON)
    @Path("something/category")
    public Response getCategoryList()
    {
        // This is also not reached from the client
        System.out.println("something/CATEGORY ENDPOINT REACHED!");
        final List<String> allCategories = new CategoryDAO()
            .findAll()
            .stream()
            .map(Category::getCategory)
            .collect(Collectors.toList());
        System.out.println("BUILDING RESPONSE HEADER 200...");
        return Response
            .status(200)
            .entity(GSON.toJson(allCategories))
            .build();
    }
}

ServiceApplication.java

public class ServiceApplication extends ResourceConfig
{
    public ServiceApplication()
    {
        packages(getClass().getPackage().getName());
        register(AuthenticationFilter.class);
    }
}

This is what the client looks like:

POM dependencies:

<dependency>
    <groupId>org.glassfish.jersey.core</groupId>
    <artifactId>jersey-client</artifactId>
    <version>2.29</version>
</dependency>
<dependency>
    <groupId>org.glassfish.jersey.containers</groupId>
    <artifactId>jersey-container-servlet</artifactId>
    <version>2.29</version>
</dependency>
<dependency>
    <groupId>org.glassfish.jersey.inject</groupId>
    <artifactId>jersey-hk2</artifactId>
    <version>2.29</version>
</dependency>

With this class the servers services are called (the target() method is used with the appended path to be accessed) OurService.java

public final class OurService
{
    private static final String serviceUrl = ApplicationProperties.serviceUrl();
    private static final Client client;
    private static final Logger logger = Logger.getLogger(OurService.class.getName());

    /* On localhost it always works, we thought maybe it is because of ssl on the OpenShift platform?
     * So we implemented this dummy trustmanager to trust everything, but this didn't change anything
     * We then deactivated ssl on the OpenShift platform...still no changes
     */

    static
    {
        final TrustManager[] certs = new TrustManager[]{new X509TrustManager()
        {
            @Override
            public void checkClientTrusted(final X509Certificate[] chain, final String authType)
                throws CertificateException
            {
            }

            @Override
            public void checkServerTrusted(final X509Certificate[] chain, final String authType)
                throws CertificateException
            {
            }

            @Override
            public X509Certificate[] getAcceptedIssuers()
            {
                return null;
            }
        }};

        SSLContext ctx = null;
        try
        {
            ctx = SSLContext.getInstance("SSL");
            ctx.init(null, certs, new SecureRandom());
        }
        catch(NoSuchAlgorithmException | KeyManagementException e)
        {
            e.printStackTrace();
        }
        client = ClientBuilder
            .newBuilder()
            .register(new LoggingFeature(logger, Level.INFO, null, null))
            .build();
    }

    public static WebTarget target(final String path)
    {
        return client.target(serviceUrl + path);
    }

    private OurService()
    {
    }
}

One of the service classes: CategoryService.java

public final class CategoryService
{
    public static List<String> getCategories()
    {
        final WebTarget target = GrappService.target("something/category");

        // REST contains a plain new Gson() object
        // REST.UTF8JSON is...MediaType.APPLICATION_JSON + ";charset=utf-8"
        return REST.gson.fromJson(
            target.request(REST.UTF8JSON).get(String.class),
            new TypeToken<List<String>>(){}.getType()
        );
    }

    private CategoryService() {}
}

OpenShift configuration and dump

Now I do not know a lot here so I might be missing some crucial information Here is the service network for the client application:

Type: ClusterIP
route: our-dashboard-app
service port: 8080/TCP (8080-tcp)
target port: 8080

This is what happens if I call the service from the pod terminal with curl:
sh-4.2$ curl -v "http://our-cool.external.url.given.by.openshift.local:8080/rest/something/category
* About to connect() to our-cool.external.url.given.by.openshift.local port 8080 (#0)
*   Trying (ip of it)...
* Connected to our-cool.external.url.given.by.openshift.local (ip of it) port 8080 (#0)
> GET /rest/something/category HTTP/1.1
> User-Agent: curl/7.29.0
> Host: our-cool.external.url.given.by.openshift.local:8080
> Accept: */*
>
< HTTP/1.1 200 OK
< Server: Apache-Coyote/1.1
< Content-Type: application/json;charset=utf-8
< Content-Length: 96
< Date: Tue, 25 Feb 2020 08:39:56 GMT
<
* Connection #0 to host our-cool.external.url.given.by.openshift.local left intact
["It works","and gives us","the categories","like requested"]

Stack trace of client exception when it tries to do the rest call

...........
Caused by:javax.ws.rs.ForbiddenException:HTTP 403 Forbidden
   at org.glassfish.jersey.client.JerseyInvocation.convertTo(JerseyInvocation.java:1059)
   at org.glassfish.jersey.client.JerseyInvocation.translate(JerseyInvocation.java:859)
   at org.glassfish.jersey.client.JerseyInvocation.lambda$invoke$1(JerseyInvocation.java:743)
   at org.glassfish.jersey.internal.Errors.process(Errors.java:292)
   at org.glassfish.jersey.internal.Errors.process(Errors.java:274)
   at org.glassfish.jersey.internal.Errors.process(Errors.java:205)
   at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:390)
   at org.glassfish.jersey.client.JerseyInvocation.invoke(JerseyInvocation.java:741)
   at org.glassfish.jersey.client.JerseyInvocation$Builder.method(JerseyInvocation.java:404)
   at org.glassfish.jersey.client.JerseyInvocation$Builder.get(JerseyInvocation.java:300)
   at path.to.our.software.services.CategoryService.getCategories(CategoryService.java:17)
   at path.to.our.software.services.Category...the view
   ... 59 more

This is what the log of the client gives us:

INFO: 3 * Sending client request on thread http-apr-8080-exec-10
3 > GET http://our-cool.external.url.given.by.openshift.local:8080/rest/something/category
3 > Accept: application/json;charset=utf-8

Feb 25, 2020 8:50:03 AM org.glassfish.jersey.logging.LoggingInterceptor log
INFO: 3 * Client response received on thread http-apr-8080-exec-10
3 < 403
3 < Cache-Control: no-cache
3 < Connection: Keep-Alive
3 < Content-Length: 2723
3 < Content-Type: text/html; charset=utf-8
3 < Pragma: no-cache
3 < Proxy-Connection: Keep-Alive
<HTML>Some html content telling us that it's a 403 access denied exception</HTML>

Network service of the server application:

Type: ClusterIP
Route: our-app-route
Serivce Port: 8080/TCP (8080-tcp)
Target Port: 8080
Hostname: http://our-cool.external.url.given.by.openshift.local

Any help is appreciated!