How to scale horizontally a spring-boot oauth2 server with JDBC implementation

10/23/2019

I have a spring boot oauth2 server that uses a JDBC implementation. It is configured as an authorization server with @EnableAuthorizationServer.

I'd like to scale that application horyzontally but it doesn't seem to work properly.

I can connect only if I have one instance (pods) of the server.

I use autorisation_code_client grant from another client service to get the token. So first the client service redirect the user to the oauth2 server form, then once the user is authenticated he is supposed to be redirect to the client-service with a code attached to the url, finally the client use that code to request the oauth2 server again and obtain the token.

Here the user is not redirected at all if I have several instance of the oauth2-server. With one instance it works well.

When I check the log of the two instances in real time, I can see that the authentication works on one of them. I don't have any specific error the user is just not redirected.

Is there a way to configure the oauth2-server to be stateless or other way to fix that issue ?

Here is my configuration, the AuthorizationServerConfigurerAdapter implementation.

@Configuration
public class AuthorizationServerConfiguration extends AuthorizationServerConfigurerAdapter {


    @Bean
    @ConfigurationProperties(prefix = "spring.datasource")
    public DataSource oauthDataSource() {
        return DataSourceBuilder.create().build();
    }

    @Autowired
    @Qualifier("authenticationManagerBean")
    private AuthenticationManager authenticationManager;

    @Bean
    public JdbcClientDetailsService clientDetailsSrv() {
        return new JdbcClientDetailsService(oauthDataSource());
    }

    @Bean
    public TokenStore tokenStore() {
        return new JdbcTokenStore(oauthDataSource());
    }

    @Bean
    public ApprovalStore approvalStore() {
        return new JdbcApprovalStore(oauthDataSource());
    }

    @Bean
    public AuthorizationCodeServices authorizationCodeServices() {
        return new JdbcAuthorizationCodeServices(oauthDataSource());
    }

    @Bean
    public TokenEnhancer tokenEnhancer() {

        return new CustomTokenEnhancer();
    }

    @Bean
    @Primary
    public AuthorizationServerTokenServices tokenServices() {


        DefaultTokenServices tokenServices = new DefaultTokenServices();

        tokenServices.setTokenStore(tokenStore());

        tokenServices.setTokenEnhancer(tokenEnhancer());

        return tokenServices;
    }

    @Override
    public void configure(ClientDetailsServiceConfigurer clients) throws Exception {

        clients.withClientDetails(clientDetailsSrv());
    }

    @Override
    public void configure(AuthorizationServerSecurityConfigurer oauthServer)  {

        oauthServer
                .tokenKeyAccess("permitAll()")
                .checkTokenAccess("isAuthenticated()")
                .allowFormAuthenticationForClients();

    }

    @Override
    public void configure(AuthorizationServerEndpointsConfigurer endpoints)  {
        endpoints
                .authenticationManager(authenticationManager)
                .approvalStore(approvalStore())
                //.approvalStoreDisabled()
                .authorizationCodeServices(authorizationCodeServices())
                .tokenStore(tokenStore())
                .tokenEnhancer(tokenEnhancer());
    }

}

The main class

@SpringBootApplication
@EnableResourceServer
@EnableAuthorizationServer
@EnableConfigurationProperties
@EnableFeignClients("com.oauth2.proxies")
public class AuthorizationServerApplication {


    public static void main(String[] args) {

        SpringApplication.run(AuthorizationServerApplication.class, args);

    }

}

The Web Security Configuration

@Configuration
@Order(1)
public class WebSecurityConfiguration extends WebSecurityConfigurerAdapter {


    @Bean
    @Override
    public UserDetailsService userDetailsServiceBean() throws Exception {
        return new JdbcUserDetails();
    }

    @Override
    @Bean
    public AuthenticationManager authenticationManagerBean() throws Exception {
        return super.authenticationManagerBean();
    }

    @Override
    protected void configure(HttpSecurity http) throws Exception { // @formatter:off

        http.requestMatchers()
                .antMatchers("/",
                        "/login",
                        "/login.do",
                        "/registration",
                        "/registration/confirm/**",
                        "/registration/resendToken",
                        "/password/forgot",
                        "/password/change",
                        "/password/change/**",
                        "/oauth/authorize**")
                .and()
                .authorizeRequests()//autorise les requetes
                .antMatchers(
                        "/",
                        "/login",
                        "/login.do",
                        "/registration",
                        "/registration/confirm/**",
                        "/registration/resendToken",
                        "/password/forgot",
                        "/password/change",
                        "/password/change/**")
                .permitAll()
                .and()
                .requiresChannel()
                .anyRequest()
                .requiresSecure()
                .and()
                .authorizeRequests()
                .anyRequest()
                .authenticated()
                .and()
                .formLogin()
                .loginPage("/login")
                .loginProcessingUrl("/login.do")
                .usernameParameter("username")
                .passwordParameter("password")
                .and()
                .userDetailsService(userDetailsServiceBean());


    } // @formatter:on


    @Override
    protected void configure(AuthenticationManagerBuilder auth) throws Exception {
        auth.userDetailsService(userDetailsServiceBean()).passwordEncoder(passwordEncoder());
    }


}

Client side the WebSecurityConfigurerAdapter

@EnableOAuth2Sso
@Configuration
public class UiSecurityConfig extends WebSecurityConfigurerAdapter {

    @Override
    public void configure(HttpSecurity http) throws Exception {

        http.antMatcher("/**")
                .authorizeRequests()
                .antMatchers(
                        "/",
                        "/index.html",
                        "/login**",
                        "/logout**",
                        //resources
                        "/assets/**",
                        "/static/**",
                        "/*.ico",
                        "/*.js",
                        "/*.json").permitAll()
                .anyRequest()
                .authenticated()
                .and()
                .csrf().csrfTokenRepository(csrfTokenRepository())
                .and()
                .addFilterAfter(csrfHeaderFilter(), SessionManagementFilter.class);
    }

}

the oauth2 configuration properties

oauth2-server is the service name (load balancer) on kubernetes and also the server path that is why it appears twice.

security:
    oauth2:
        client:
            clientId: **********
            clientSecret: *******
            accessTokenUri: https://oauth2-server/oauth2-server/oauth/token
            userAuthorizationUri: https://oauth2.mydomain.com/oauth2-server/oauth/authorize
        resource:
            userInfoUri: https://oauth2-server/oauth2-server/me

Here an important detail, the value of userAuthorizationUri is the address to access the oauth2-server from the outside of the k8s cluster. The client-service send back that address into the response with a 302 http code if the user is not connected and tries to access to the /login path of the client-service. then the user is redirected to the /login path of the oauth2-server.
https://oauth2.mydomain.com target an Nginx Ingress controller that handle the redirection to the load balancer service.

-- Kaizokun
horizontal-scrolling
kubernetes
oauth2-server
spring-boot

2 Answers

10/23/2019

By default an in-memory TokenStore is used.

The default InMemoryTokenStore is perfectly fine for a single server

If you want multiple pods, you probably should go for JdbcTokenStore

The JdbcTokenStore is the JDBC version of the same thing, which stores token data in a relational database. Use the JDBC version if you can share a database between servers, either scaled up instances of the same server if there is only one, or the Authorization and Resources Servers if there are multiple components. To use the JdbcTokenStore you need "spring-jdbc" on the classpath.

Source Spring Security: OAuth 2 Developers Guide

-- Jonas
Source: StackOverflow

10/24/2019

Here is a solution to the problem. It's not a Spring issue at all but a bad configuration of the Nginx Ingress controller.

The authentication process is done in several stages :

1 - the user clic on a login button that target the /login path of the client-server

2 - the client-server, if the user is not authenticated yet, send a response to the browser with a 302 http code to redirect the user to the oauth2-server, the value of the redirection is composed with the value of the security.oauth2.client.userAuthorizationUri property and the redirection url that will be used by the user browser to allow the client-server to get the Token once the user is authenticated. That url look like this :

h*tps://oauth2.mydomain.com/oauth2-server/oauth/authorize?client_id=autorisation_code_client&redirect_uri=h*tps://www.mydomain.com/login&response_type=code&state=bSWtGx

3 - the user is redirected to the previous url

4 - the oauth2-server send a 302 http code to the browser with the login url of the oauth2-server, h*tps://oauth2.mydomain.com/oauth2-server/login

5 - the user submit his credentials and the token is created if they are correct.

6 - the user is redirected to the same address as in step two, and the oauth-server add informations to the redirect_uri value

7 - the user is redirected to the client-server. The redirection part of the response look like this :

location: h*tps://www.mydomain.com/login?code=gnpZ0r&state=bSWtGx

8 - the client-server contact the oauth2-server and obtain the token from the code and the state that authenticates it. It doesn't matter if the instance of the oauth2 server is different than the one used by the user to authenticate himself. Here the client-server use the value of security.oauth2.client.accessTokenUri to get the token, this is the internal load balancing service address that targets the oauth2 server pods, so it doesn't pass through any Ingress controller.

So at the steps 3 to 6 the user must communicate with the same instance of the oauth2-server throught the Ingress controller in front of the load balancer service.

Its is possible by configuring the Nginx Ingress controller with a few annotations :

"annotations": {
  ...
  "nginx.ingress.kubernetes.io/affinity": "cookie",
  "nginx.ingress.kubernetes.io/session-cookie-expires": "172800",
  "nginx.ingress.kubernetes.io/session-cookie-max-age": "172800",
  "nginx.ingress.kubernetes.io/session-cookie-name": "route"
}

That way we ensure that the user will be redirected to the same pods/instance of the oauth2-server during the authentication process as long he's identified with the same cookie.

The affinity session mecanism is a great way to scale the authentication server and also the client-server. Once the user is authenticated he will always use the same instance of the client and keep his session informations.

Thanks to Christian Altamirano Ayala for his help.

-- Kaizokun
Source: StackOverflow