I'm developing a microservices application with Spring Boot and Postgres. Using Docker, the REST API in one container and Postgres in another container, everything works fine. But when I try to run this with Kubernetes , it always gives an API error.
The Dockerfile
where my Spring Boot-based API is defined looks as follows:
FROM alpine
RUN apk add openjdk8
MAINTAINER rjdesenvolvimento.com
COPY target/apipessoas-0.0.1.jar /opt/gaia/apipessoas.jar
ENTRYPOINT ["/usr/bin/java"]
CMD ["-jar", "/opt/gaia/apipessoas.jar"]
EXPOSE 8080
The Dockerfile
for Postgres:
FROM postgres:10-alpine
MAINTAINER rjdesenvolvimento.com
ENV POSTGRES_PASSWORD=postgres
ENV POSTGRES_PORT=5432
CMD ["postgres"]
ADD postgres.sql /docker-entrypoint-initdb.d/
EXPOSE 5432
My docker-compose
file:
version: "3.0"
services:
postgres:
build:
dockerfile: postgres.dockerfile
context: .
image: rjdesenvolvimento/postgres-apipessoas
container_name: postgres
ports:
- "5432:5432"
volumes:
- /home/rodrigo/Projetos/Volumes:/var/lib/postgresql/data
networks:
- gaia_network
apipessoas:
build:
dockerfile: api-pessoa.dockerfile
context: .
image: rjdesenvolvimento/api-pessoas
container_name: api_pessoas
ports:
- "8080:8080"
depends_on:
- postgres
networks:
- gaia_network
networks:
gaia_network:
driver: bridge
Now, my Kubernetes Postgres mainfest:
apiVersion: v1
kind: Service
metadata:
name: postgres
labels:
app: postgres
spec:
ports:
- port: 5432
name: postgres-api-pessoas
clusterIP: None
selector:
app: postgres
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: postgres-api-pessoas
spec:
serviceName: postgres
selector:
matchLabels:
app: postgres
template:
metadata:
labels:
app: postgres
spec:
containers:
- name: container-postgres-api-pessoas
image: rjdesenvolvimento/postgres-apipessoas
imagePullPolicy: Never
env:
- name: POSTGRES0_USER
value: postgres
- name: POSTGRES_PASSWORD
value: postgres
- name: POSTGRES_DB
value: zeus
ports:
- containerPort: 5432
name: zeus
volumeMounts:
- name: volume-postgres-api-pessoas
mountPath: /var/lib/postgresql/data
volumeClaimTemplates:
- metadata:
name: volume-postgres-api-pessoas
spec:
accessModes: [ "ReadWriteOnce" ]
resources:
requests:
storage: 1Gi
And the API manifest:
kind: Service
apiVersion: v1
metadata:
name: servico-api-pessoas
spec:
type: LoadBalancer
selector:
app: api-pessoas-pod
ports:
- port: 80
targetPort: 8080
apiVersion: apps/v1
kind: Deployment
metadata:
name: deployment-api-pessoas
labels:
app: api-pessoas-pod
spec:
selector:
matchLabels:
app: api-pessoas-pod
template:
metadata:
labels:
app: api-pessoas-pod
spec:
containers:
- name: contaienr-api-pessoas
image: rjdesenvolvimento/api-pessoas
imagePullPolicy: Never
ports:
- containerPort: 8080
When I deploy above YAML manifest to Kubernetes I get this error:
deployment-api-pessoas-8cfd5c6c5-dpln4 0/1 CrashLoopBackOff 6 10m
postgres-api-pessoas-0 1/1 Running 0 28m
What im doing wrong?
ADD:
describe pod
Name: deployment-api-pessoas-8cfd5c6c5-nhtff
Namespace: default
Node: minikube/10.0.2.15
Start Time: Sun, 16 Sep 2018 11:19:02 -0400
Labels: app=api-pessoas-pod
pod-template-hash=479817271
Annotations: <none>
Status: Running
IP: 172.17.0.5
Controlled By: ReplicaSet/deployment-api-pessoas-8cfd5c6c5
Containers:
contaienr-api-pessoas:
Container ID: docker://a6fb9a254895bb31effdcadd66675cfb5197f72d526f805d20cbbde90c0677cc
Image: rjdesenvolvimento/api-pessoas
Image ID: docker://sha256:f326d42c0afd7b4f3d3e7a06e8a2625f7a841a7451a08c2b326e90e13459b244
Port: 8080/TCP
Host Port: 0/TCP
State: Terminated
Reason: Error
Exit Code: 1
Started: Sun, 16 Sep 2018 11:19:48 -0400
Finished: Sun, 16 Sep 2018 11:20:02 -0400
Last State: Terminated
Reason: Error
Exit Code: 1
Started: Sun, 16 Sep 2018 11:19:11 -0400
Finished: Sun, 16 Sep 2018 11:19:42 -0400
Ready: False
Restart Count: 1
Environment: <none>
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-x9s8k (ro)
Conditions:
Type Status
Initialized True
Ready False
PodScheduled True
Volumes:
default-token-x9s8k:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-x9s8k
Optional: false
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 1m default-scheduler Successfully assigned deployment-api-pessoas-8cfd5c6c5-nhtff to minikube
Normal SuccessfulMountVolume 1m kubelet, minikube MountVolume.SetUp succeeded for volume "default-token-x9s8k"
Normal Pulled 26s (x2 over 1m) kubelet, minikube Container image "rjdesenvolvimento/api-pessoas" already present on machine
Normal Created 23s (x2 over 1m) kubelet, minikube Created container
Normal Started 21s (x2 over 58s) kubelet, minikube Started container
Warning BackOff 6s kubelet, minikube Back-off restarting failed container
the log is too big, but I think I found the problem:
attempt: org.springframework.beans.factory.UnsatisfiedDependencyException: Error
creating bean with name 'pessoaJuridicaFuncionarioResource' defined in URL
[jar:file:/opt/gaia/apipessoas.jar!/BOOT-
INF/classes!/com/rjdesenvolvimento/apipessoas/resource/funcionario/PessoaJ
uridicaFuncionarioResource.class]: Unsatisfied dependency expressed
through constructor parameter 0; nested exception is
org.springframework.beans.factory.UnsatisfiedDependencyException: Error
creating bean with name 'pessoaJuridicaFuncionarioService' defined in URL
[jar:file:/opt/gaia/apipessoas.jar!/BOOT-
INF/classes!/com/rjdesenvolvimento/apipessoas/service/funcionario/PessoaJu
ridicaFuncionarioService.class]: Unsatisfied dependency expressed through
constructor parameter 0; nested exception is
org.springframework.beans.factory.BeanCreationException: Error creating
bean with name 'pessoaJuridicaFuncionarioRepository': Cannot create inner
bean '(inner bean)#37ddb69a' of type
[org.springframework.orm.jpa.SharedEntityManagerCreator] while setting
bean property 'entityManager'; nested exception is
org.springframework.beans.factory.BeanCreationException: Error creating
bean with name '(inner bean)#37ddb69a': Cannot resolve reference to bean
'entityManagerFactory' while setting constructor argument; nested
exception is org.springframework.beans.factory.BeanCreationException:
Error creating bean with name 'entityManagerFactory': Post-processing of
FactoryBean's singleton object failed; nested exception is
org.springframework.jdbc.datasource.init.ScriptStatementFailedException:
Failed to execute SQL script statement #1 of URL
[jar:file:/opt/gaia/apipessoas.jar!/BOOT-INF/classes!/data.sql]: INSERT
INTO zeus.endereco.pais (nome) VALUES ('Brasil'); nested exception is
org.postgresql.util.PSQLException: ERROR: relation "endereco.pais" does
not exist
Position: 13
HIBERNATE can not create any table.
Hibernate:
create table endereco.endereco_pessoa_fisica_funcionario (
id bigserial not null,
bairro varchar(255) not null,
cep varchar(255) not null,
complemento varchar(255) not null,
logradouro varchar(255) not null,
numero varchar(255) not null,
fk_cidade int8,
fk_pessoafisicafuncionario int8,
primary key (id)
)
2018-09-16 15:27:03.480 WARN 1 --- [ main] o.h.t.s.i.ExceptionHandlerLoggedImpl : GenerationTarget encountered exception accepting command : Error executing DDL via JDBC Statement
org.hibernate.tool.schema.spi.CommandAcceptanceException: Error executing DDL via JDBC Statement
at org.hibernate.tool.schema.internal.exec.GenerationTargetToDatabase.accept(GenerationTargetToDatabase.java:67) ~[hibernate-core-5.2.17.Final.jar!/:5.2.17.Final]
at org.hibernate.tool.schema.internal.SchemaCreatorImpl.applySqlString(SchemaCreatorImpl.java:440) [hibernate-core-5.2.17.Final.jar!/:5.2.17.Final]
at org.hibernate.tool.schema.internal.SchemaCreatorImpl.applySqlStrings(SchemaCreatorImpl.java:424) [hibernate-core-5.2.17.Final.jar!/:5.2.17.Final]
at org.hibernate.tool.schema.internal.SchemaCreatorImpl.createFromMetadata(SchemaCreatorImpl.java:315) [hibernate-core-5.2.17.Final.jar!/:5.2.17.Final]
at org.hibernate.tool.schema.internal.SchemaCreatorImpl.performCreation(SchemaCreatorImpl.java:166) [hibernate-core-5.2.17.Final.jar!/:5.2.17.Final]
at org.hibernate.tool.schema.internal.SchemaCreatorImpl.doCreation(SchemaCreatorImpl.java:135) [hibernate-core-5.2.17.Final.jar!/:5.2.17.Final]
at org.hibernate.tool.schema.internal.SchemaCreatorImpl.doCreation(SchemaCreatorImpl.java:121) [hibernate-core-5.2.17.Final.jar!/:5.2.17.Final]
at org.hibernate.tool.schema.spi.SchemaManagementToolCoordinator.performDatabaseAction(SchemaManagementToolCoordinator.java:155) [hibernate-core-5.2.17.Final.jar!/:5.2.17.Final]
at org.hibernate.tool.schema.spi.SchemaManagementToolCoordinator.process(SchemaManagementToolCoordinator.java:72) [hibernate-core-5.2.17.Final.jar!/:5.2.17.Final]
at org.hibernate.internal.SessionFactoryImpl.<init>(SessionFactoryImpl.java:312) [hibernate-core-5.2.17.Final.jar!/:5.2.17.Final]
at org.hibernate.boot.internal.SessionFactoryBuilderImpl.build(SessionFactoryBuilderImpl.java:462) [hibernate-core-5.2.17.Final.jar!/:5.2.17.Final]
at org.hibernate.jpa.boot.internal.EntityManagerFactoryBuilderImpl.build(EntityManagerFactoryBuilderImpl.java:892) [hibernate-core-5.2.17.Final.jar!/:5.2.17.Final]
at org.springframework.orm.jpa.vendor.SpringHibernateJpaPersistenceProvider.createContainerEntityManagerFactory(SpringHibernateJpaPersistenceProvider.java:57) [spring-orm-5.0.8.RELEASE.jar!/:5.0.8.RELEASE]
at org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean.createNativeEntityManagerFactory(LocalContainerEntityManagerFactoryBean.java:365) [spring-orm-5.0.8.RELEASE.jar!/:5.0.8.RELEASE]
at org.springframework.orm.jpa.AbstractEntityManagerFactoryBean.buildNativeEntityManagerFactory(AbstractEntityManagerFactoryBean.java:390) [spring-orm-5.0.8.RELEASE.jar!/:5.0.8.RELEASE]
at org.springframework.orm.jpa.AbstractEntityManagerFactoryBean.afterPropertiesSet(AbstractEntityManagerFactoryBean.java:377) [spring-orm-5.0.8.RELEASE.jar!/:5.0.8.RELEASE]
at org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean.afterPropertiesSet(LocalContainerEntityManagerFactoryBean.java:341) [spring-orm-5.0.8.RELEASE.jar!/:5.0.8.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1758) [spring-beans-5.0.8.RELEASE.jar!/:5.0.8.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1695) [spring-beans-5.0.8.RELEASE.jar!/:5.0.8.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:573) [spring-beans-5.0.8.RELEASE.jar!/:5.0.8.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:495) [spring-beans-5.0.8.RELEASE.jar!/:5.0.8.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:317) [spring-beans-5.0.8.RELEASE.jar!/:5.0.8.RELEASE]
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222) ~[spring-beans-5.0.8.RELEASE.jar!/:5.0.8.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:315) [spring-beans-5.0.8.RELEASE.jar!/:5.0.8.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199) [spring-beans-5.0.8.RELEASE.jar!/:5.0.8.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:1089) ~[spring-context-5.0.8.RELEASE.jar!/:5.0.8.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:859) ~[spring-context-5.0.8.RELEASE.jar!/:5.0.8.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:550) ~[spring-context-5.0.8.RELEASE.jar!/:5.0.8.RELEASE]
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:140) ~[spring-boot-2.0.4.RELEASE.jar!/:2.0.4.RELEASE]
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:762) ~[spring-boot-2.0.4.RELEASE.jar!/:2.0.4.RELEASE]
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:398) ~[spring-boot-2.0.4.RELEASE.jar!/:2.0.4.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:330) ~[spring-boot-2.0.4.RELEASE.jar!/:2.0.4.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1258) ~[spring-boot-2.0.4.RELEASE.jar!/:2.0.4.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1246) ~[spring-boot-2.0.4.RELEASE.jar!/:2.0.4.RELEASE]
at com.rjdesenvolvimento.apipessoas.ApipessoasApplication.main(ApipessoasApplication.java:10) ~[classes!/:0.0.1]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_171]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_171]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_171]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_171]
at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48) ~[apipessoas.jar:0.0.1]
at org.springframework.boot.loader.Launcher.launch(Launcher.java:87) ~[apipessoas.jar:0.0.1]
at org.springframework.boot.loader.Launcher.launch(Launcher.java:50) ~[apipessoas.jar:0.0.1]
at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:51) ~[apipessoas.jar:0.0.1]
Caused by: org.postgresql.util.PSQLException: ERROR: schema "endereco" does not exist
Position: 19
at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2440) ~[postgresql-42.2.4.jar!/:42.2.4]
at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2183) ~[postgresql-42.2.4.jar!/:42.2.4]
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:308) ~[postgresql-42.2.4.jar!/:42.2.4]
at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:441) ~[postgresql-42.2.4.jar!/:42.2.4]
at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:365) ~[postgresql-42.2.4.jar!/:42.2.4]
at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:307) ~[postgresql-42.2.4.jar!/:42.2.4]
at org.postgresql.jdbc.PgStatement.executeCachedSql(PgStatement.java:293) ~[postgresql-42.2.4.jar!/:42.2.4]
at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:270) ~[postgresql-42.2.4.jar!/:42.2.4]
at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:266) ~[postgresql-42.2.4.jar!/:42.2.4]
at com.zaxxer.hikari.pool.ProxyStatement.execute(ProxyStatement.java:95) ~[HikariCP-2.7.9.jar!/:na]
at com.zaxxer.hikari.pool.HikariProxyStatement.execute(HikariProxyStatement.java) ~[HikariCP-2.7.9.jar!/:na]
at org.hibernate.tool.schema.internal.exec.GenerationTargetToDatabase.accept(GenerationTargetToDatabase.java:54) ~[hibernate-core-5.2.17.Final.jar!/:5.2.17.Final]
... 42 common frames omitted
but if i run with docker-compose without kubertes, everthing works ok
ADD:
in my application.properties:
1) if i run in my pc withou docker: spring.datasource.url= jdbc:postgresql://locahost:port/db_name
2) if i run in docker spring.datasource.url= jdbc:postgresql://docker_db_container_name:port/db_name
Now I do not know how to make the api rest "see" the database with kubernetes
ADD:
First I wanted to apologize for my absence. I had personal problems. Secondly, I would like to thank you all for your help so far.
I modified my Kubernetes files:
postgres
kind: ConfigMap
apiVersion: v1
metadata:
name: postgres-config
labels:
app: postgres
camada: banco-de-dados
data:
POSTGRES_DB: zeus
POSTGRES_PASSWORD: lk85Kab5aCn904Ad
---
kind: PersistentVolume
apiVersion: v1
metadata:
name: postgres-volume
labels:
app: postgres
tipo: local
spec:
capacity:
storage: 5Gi
accessModes:
- ReadWriteOnce
hostPath:
path: /home/rodrigo/Projetos/VolumeKube
---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: postgres-volume-claim
labels:
app: postgres
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 5Gi
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: postgres
spec:
replicas: 1
selector:
matchLabels:
app: postgres
template:
metadata:
labels:
app: postgres
spec:
containers:
- name: postgres
image: rjdesenvolvimento/postgres-apipessoas
imagePullPolicy: Never
ports:
- containerPort: 5432
envFrom:
- configMapRef:
name: postgres-config
volumeMounts:
- mountPath: /var/lib/postgresql/data
name: postgres-database
volumes:
- name: postgres-database
persistentVolumeClaim:
claimName: postgres-volume-claim
---
kind: Service
apiVersion: v1
metadata:
name: postgres
labels:
app: postgres
spec:
type: NodePort
selector:
app: postgres
ports:
- port: 5432
and API
apiVersion: apps/v1
kind: Deployment
metadata:
name: api-pessoas
labels:
app: api-pessoas
camada: backend
spec:
replicas: 1
selector:
matchLabels:
app: api-pessoas
template:
metadata:
labels:
app: api-pessoas
camada: backend
spec:
containers:
- name: api-pessoas
image: rjdesenvolvimento/api-pessoas
imagePullPolicy: Never
ports:
- containerPort: 8080
---
kind: Service
apiVersion: v1
metadata:
name: api-pessoas
labels:
app: api-pessoas
camada: backend
spec:
type: LoadBalancer
selector:
app: api-pessoas
camada: backend
ports:
- port: 8080
The error continued. So I read quietly and realized that kubernetes created "zeus DB" but did not create the SCHEMAS.
when I create my Docker image, from DockerFile, I add the following file:
CREATE DATABASE zeus;
\connect zeus;
CREATE SCHEMA funcionario;
CREATE SCHEMA endereco;
CREATE SCHEMA tabela_auxiliar;
CREATE SCHEMA cliente;
But this information is not passed on to Kubernetes. So I have to manually enter and add them. And then everything works.
How do I make the information automatically added?
As far as I understand your issue, your service k8s pod cannot reach the postgres. As you may already know, in k8s, pods are ephemeral and you need services abstraction if you need to invoke a service by its url.
I would ssh into the App pod and try to ping the hostname with which you are trying to connect to the postgres (in your app's config)
kubectl exec -it <podname> sh
I don't know how docker compose sets up its internal network, but I can see you are pointing to the same network for both postgres and your service within your docker-compose definition.
As for k8s, you need a way to locate your services as they can be deployed on any node. Either through an internal k8s DNS application or define a Postgres service with a static address (the same way you defined your application service from port 80 to port 8080).
Solution for API not connected to Postgres: Create a Postgres service resource. Also get a read at k8s services for more help.
Solution for Postgres not having the expected schemas: Make sure that your database gets initialized with your user and data by adding the relevant SQL scripts under the /docker-entrypoint-initdb.d/ folder in your Docker image. See a nice answer on how it is done here.