K
Q

terraform kubernetes provider - tls secret not created properly

12/17/2019
  1. i created a self-signed tls certificate and private key via terraform. The files are called server.key and server.crt

  2. i create a kubernetes tls secret with this certificate and private key using this command:
    kubectl create secret tls dpaas-secret -n dpaas-prod --key server.key --cert server.crt

  3. this works fine, nginx ingress ssl termination works, and the following kubectl command: kubectl get secret test-secret -o yaml -n dpaas-prod
    returns correct output with tls.crt pem data and tls.key pem data (see correct output in step 6 below)

  4. Since we use terraform, i tried creating the same secret via terraform kubernetes provider with same server.key and servert.crt files. However this time, the command :
    kubectl get secret test-secret -o yaml -n dpaas-prod
    returned weird output for crt pem and key pem (see output in step 5 below) and the ssl termination on my nginx ingress does not work.
    This is how i create the kubernetes secret via terraform:

      resource "kubernetes_secret" "this" {
          metadata {
            name = "dpaas-secret"
            namespace = "dpaas-prod"
          }

          data = {
            "tls.crt" = "${path.module}/certs/server.crt"
            "tls.key" = "${path.module}/certs/server.key"
          }

          type = "kubernetes.io/tls"
       }
  1. bad output following step 4. notice the short values of tls.crt and tls.key (secret created via terraform) :
    *apiVersion: v1
    data:
      tls.crt: bXktdGVzdHMvY2VydHMvc2VydmVyLmNydA==
      tls.key: bXktdGVzdHMvY2VydHMvc2VydmVyLmtleQ==
   `enter code here` kind: Secret
    metadata:
      creationTimestamp: "2019-12-17T16:18:22Z"
      name: dpaas-secret
      namespace: dpaas-prod
      resourceVersion: "9879"
      selfLink: /api/v1/namespaces/dpaas-prod/secrets/dpaas-secret
      uid: d84db7f0-20e8-11ea-aa92-1269ad9fd693
    type: kubernetes.io/tls*
  1. correct output following step 2 (secret created via kubectl command)
*apiVersion: v1
data:
  tls.crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUR4RENDQXF5Z0F3SUJBZ0lRZGM0cmVoOHRQNFZPN3NkTWpzc1lTVEFOQmdrcWhraUc5dzBCQVFzRkFEQkUKTVJBd0RnWURWUVFLRXdkbGVHRnRjR3hsTVJRd0VnWURWUVFMRXd0bGVHRnRjR3hsS
UdSd2N6RWFNQmdHQTFVRQpBd3dSS2k1a2NITXVaWGhoYlhCc1pTNWpiMjB3SGhjTk1Ua3hNakUzTVRZeE5UVXpXaGNOTWpreE1qRTNNRE14Ck5UVXpXakJaTVFzd0NRWURWUVFHRXdKVlV6RVVNQklHQTFVRUNoTUxaWGhoYlhCc1pTQmtjSE14R0RBV0JnTlYKQkFzV
EQyVjRZVzF3YkdVdVpIQnpMbU52YlRFYU1CZ0dBMVVFQXd3UktpNWtjSE11WlhoaGJYQnNaUzVqYjIwdwpnZ0VpTUEwR0NTcUdTSWIzRFFFQkFRVUFBNElCRHdBd2dnRUtBb0lCQVFDaDU0enBwVTBod1hscm1qVXpOeVl0Ckp5WG9NSFU4WFpXTzhoVG9KZ09YUDU5N
nZFVmJQRXJlQ1VxM1BsZXB5SkRrcHNWbHo1WWc1TWp4NkVGTnlxNVQKOHVLUlVZUXNPVzNhd1VCbzM2Y3RLZEVvci8wa0JLNXJvYTYyR2ZFcHJmNVFwTlhEWnY3T1Y1YU9VVjlaN2FFTwpNNEl0ejJvNWFVYm5mdHVDZVdqKzhlNCtBS1phVTlNOTFCbFROMzFSUUFSR
3RnUzE4MFRzcVlveGV3YXBoS3FRCmUvTm5TeWF6ejUyTU5jeml6WTRpWXlRUU9EbUdEOEtWRGRJbWxJYXFoYXhiVGVTMldWZFJzdmpTa2xVZ0pGMUUKb2VWaWo1KytBd0FBczYwZkI2M1A4eFB1NEJ3cmdGTmhTV2F2ZXdJV1RMUXJPV1I2V2wvWTY1Q3lnNjlCU0xse
gpBZ01CQUFHamdad3dnWmt3RGdZRFZSMFBBUUgvQkFRREFnV2dNQjBHQTFVZEpRUVdNQlFHQ0NzR0FRVUZCd01CCkJnZ3JCZ0VGQlFjREFqQU1CZ05WSFJNQkFmOEVBakFBTUI4R0ExVWRJd1FZTUJhQUZPanRBdTJoNDN0WjhkS1YKaHUzc2xVS3VJYTlHTURrR0ExV
WRFUVF5TURDQ0VTb3VaSEJ6TG1WNFlXMXdiR1V1WTI5dGdoc3FMbVJ3Y3k1MQpjeTFsWVhOMExURXVaWGhoYlhCc1pTNWpiMjB3RFFZSktvWklodmNOQVFFTEJRQURnZ0VCQUlxRVlubHdwQnEyCmNmSnhNUUl0alF4ZTlDK2FDTnZXS1VOZjlwajlhZ3V6YXNxTW9wU
URWTFc1dnZxU21FbHJrVXNBQzJPemZ3K2UKRkxKNUFvOFg3VFcxTHBqbk01Mm1FVjRZYUcvM05hVTg5dWhOb0FHd0ZPbU5TK3ZldU12N3RKQjhsUHpiQ1k3VApKaG9TL2lZVE9jUEZUN1pmNkVycjFtd1ZkWk1jbEZuNnFtVmxwNHZGZk1pNzRFWnRCRXhNaDV3aWU3Q
Wl4Z2tTCmZaVno4QUEzTWNpalNHWFB6YStyeUpJTnpYY0gvM1FRaVdLbzY5SUQrYUlSYTJXUUtxVlhVYmk0bmlZaStDUXcKeTJuaW5TSEVCSDUvOHNSWVZVS1ZjNXBPdVBPcFp0RmdqK1l6d1VsWGxUSytLRTR0R21Ed09teGxvMUNPdGdCUAorLzFXQWdBN1p0QT0KL
S0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=
  tls.key: LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFb3dJQkFBS0NBUUVBb2VlTTZhVk5JY0Y1YTVvMU16Y21MU2NsNkRCMVBGMlZqdklVNkNZRGx6K2ZlcnhGCld6eEszZ2xLdHo1WHFjaVE1S2JGWmMrV0lPVEk4ZWhCVGNxdVUvTGlrVkdFT
ERsdDJzRkFhTituTFNuUktLLzkKSkFTdWE2R3V0aG54S2EzK1VLVFZ3MmIremxlV2psRmZXZTJoRGpPQ0xjOXFPV2xHNTM3YmdubG8vdkh1UGdDbQpXbFBUUGRRWlV6ZDlVVUFFUnJZRXRmTkU3S21LTVhzR3FZU3FrSHZ6WjBzbXM4K2RqRFhNNHMyT0ltTWtFRGc1C
mhnL0NsUTNTSnBTR3FvV3NXMDNrdGxsWFViTDQwcEpWSUNSZFJLSGxZbytmdmdNQUFMT3RId2V0ei9NVDd1QWMKSzRCVFlVbG1yM3NDRmt5MEt6bGtlbHBmMk91UXNvT3ZRVWk1Y3dJREFRQUJBb0lCQUZaNjFqdmUvY293QytrNwozM3JSMUdSOTZyT1JYcTIxMXpNW
mY2MVkwTVl6Ujc1SlhrcVRjL0lSeUlVRW1kS292U3hGSUY5M2VGdHRtU0FOCnpRUCtaUXVXU3dzUUhhZDVyWUlSZzVRQkVzeis3eWZxaVM1NkNhaVlIamhLdHhScVNkTk5tSmpkSlBHV3UyYWQKZEc4V2pOYUhFTnZqVkh3Q0RjdU5hVGJTSHhFOTAwSjhGQTg0c3d2M
lZFUGhSbExXVjJudVpLTko5aGIrY2IzVQpsZ2JrTnVxMkFsd2Y3MkRaTVRXZ21DM3N1Z004eGYwbWFCRWV3UXdETVdBZis2dWV6MEJ5V0hLdThwNHZRREJvCjBqQVYzOGx6UHppTDU3UTZYbFdnYjIxWUh2QmJMSVVKcEFRMGdrcGthaEFNVmJlbHdiSDJVR25wOXcrb
zU3MnIKTmhWMFJXRUNnWUVBeEtFU3FRWEV5dmpwWU40MGxkbWNhTmdIdFd1d3RpUThMMWRSQitEYXBUbWtZbzFlWXJnWgpzNi9HbStRalFtNlU1cG04dHFjTm1nL21kTDJzdk9uV1Y1dnk5eThmZHo3OFBjOFlobXFJRE5XRE9tZG9wUVJsCmxsVUZ6S0NwRmVIVTJRW
URYYjBsNWNvZzYyUVFUaWZIdjFjUGplWlltc2I5elF0cDd6czJZMGtDZ1lFQTBzcFoKTWRRUUxiRkZkWDlYT05FU2xvQlp3Slg5UjFQZVA0T2F4eUt2a01jQXFFQ0Npa05ZU3FvOU55MkZQNVBmQlplQgpWbzYvekhHR0dqVkFPQUhBczA5My8zZUZxSFRleWVCSzhQR
kJWMHh5em9ZZThxYUhBR1JxVnpjS240Zy9LVjhWClpjVGMwTm5aQzB5b09NZkhYUTVnQm1kWnpBVXBFOHlqZzhucGV0c0NnWUVBd0ZxU1ZxYytEUkhUdk4ranNiUmcKUG5DWG1mTHZ2RDlXWVRtYUc0cnNXaFk1cWUrQ0ZqRGpjOVRSQmsvMzdsVWZkVGVRVlY2Mi82L
3VVdVg2eGhRNwppeGtVWnB2Q3ZIVHhiY1hheUNRUFUvN0xrYWIzeC9hMUtvdWlVTHdhclQxdmE1OW1TNTF1SlkzSEJuK3RNOGZXCnNHZ0szMVluOThJVEp6T3pQa1UrdjRFQ2dZQUpRQkFCKzhocCtPbVBhbk10Yng5ZHMydzg0MWdtRlN3ZnBXcloKYWxCQ0RqbWRLS
mVSOGJxaUxDNWJpWWZiYm1YUEhRTDBCWGV0UlI0WmNGVE5JR2FRZCtCUU9iS0gzZmtZNnRyZgpEL2RLR1hUQVUycHdRNWFSRWRjSTFNV0drcmdTM0xWWHJmZnl3bHlmL2xFempMRFhDSloyTVhyalZTYWtVOHFwCk1lY3BHUUtCZ0c0ZjVobDRtL1EvaTNJdGZJbGw4W
DFRazFUSXVDK0JkL1NGM0xVVW5JUytyODA4dzVuaTNCUnEKNXgvQjFnRUhZbTNjSTROajR5ZzEvcE1CejhPMk1PeFhVbVNVZVh6dit1MG5oOFQxUE96eDJHOTNZaVlOL0cvNQpjMlBMSFMvTTlmVjhkTEVXL0hBVFM3K0hsMDFGQlVlREhrODQrVXlha2V2ZFU2djdVZ
2ErCi0tLS0tRU5EIFJTQSBQUklWQVRFIEtFWS0tLS0tCg==
kind: Secret
metadata:
  creationTimestamp: "2019-12-17T16:16:56Z"
  name: dpaas-secret
  namespace: dpaas-prod
  resourceVersion: "9727"
  selfLink: /api/v1/namespaces/dpaas-prod/secrets/dpaas-secret
  uid: a5739134-20e8-11ea-a977-0a0ee9b505f9
type: kubernetes.io/tls*

Question: why is the kubernetes secret pem data short (corrupt?) when adding secret with terraform?
we use terraform version 0.12.8 and kubernetes 1.13

for reproducing this here are the file links:

-- Assaf
kubernetes
ssl
terraform

Similar Questions

Do I need a service for exposing every app running in a pod?
OpenShift 4.1 | Horizontal POD Auto scaling (HPA) | All PODS are not updating with latest image
Persistent volume to windows not working on kubernetes
ServiceAccount cannot list resource "pods" in namespace though it has role with appropriate resources
is it ok to use the ingress-nginx nodeport to access the cluster in production?

1 Answer

12/17/2019

The problem is that you are encoding the paths of the certificate files into the secret and not the contents of the files.

You can see this is the case if you base64-decode the secret strings in your example:

$ echo -n bXktdGVzdHMvY2VydHMvc2VydmVyLmNydA== | base64 -d
my-tests/certs/server.crt
$ echo -n bXktdGVzdHMvY2VydHMvc2VydmVyLmtleQ== | base64 -d
my-tests/certs/server.key

So, instead of this:

data = {
    "tls.crt" = "${path.module}/certs/server.crt"
    "tls.key" = "${path.module}/certs/server.key"
}

do this:

data = {
    "tls.crt" = file("${path.module}/certs/server.crt")
    "tls.key" = file("${path.module}/certs/server.key")
}
-- John
Source: StackOverflow