@grizzthedj, Here is what I've found inside front50.log (I wiped out ID's of course for security reasons) You may be right.

2017-11-20 12:40:29.151  INFO 682 --- [0.0-8080-exec-1] com.amazonaws.latency                  : ServiceName=[Amazon S3], AWSErrorCode=[NoSuchKey], StatusCode=[404], ServiceEndpoint=[https://s3-us-west-2.amazonaws.com], Exception=[com.amazonaws.services.s3.model.AmazonS3Exception: The specified key does not exist. (Service: Amazon S3; Status Code: 404; Error Code: NoSuchKey; Request ID: ...; S3 Extended Request ID: ...), S3 Extended Request ID: ...], RequestType=[GetObjectRequest], AWSRequestID=[...], HttpClientPoolPendingCount=0, RetryCapacityConsumed=0, HttpClientPoolAvailableCount=1, RequestCount=1, Exception=1, HttpClientPoolLeasedCount=0, ClientExecuteTime=[39.634], HttpClientSendRequestTime=[0.072], HttpRequestTime=[39.213], RequestSigningTime=[0.067], CredentialsRequestTime=[0.001, 0.0], HttpClientReceiveResponseTime=[39.059],

I had a similar issue on kubernetes/aws, when I opened up the chrome dev console I was getting lots of 404 errors trying to connect to localhost:8084, I had to reconfigure the deck and gate baseurls. This is what I did using halyard:

hal config security ui edit --override-base-url http://<deck-loadbalancer-dns-entry>:9000
hal config security api edit --override-base-url http://<gate-loadbalancer-dns-entry>:8084

i did hal deploy apply and when it came back I noticed the developer console was throwing cors errors so I had to do the following.

echo "host: 0.0.0.0" | tee \ ~/.hal/default/service-settings/gate.yml \ ~/.hal/default/service-settings/deck.yml

You may note the lack of TLS and cors config, this is a test system so make better choices in production :)

@grizzthedj

thanks again for recommendations. It doesn't seem, however, solved the issue. I wonder if it has something to do with my Docker Registry or Kubernetes. Here is what I have in my .hal/config:

dockerRegistry:
  enabled: true
  accounts:
  - name: <hidden-name>
    requiredGroupMembership: []
    address: https://docker-registry.<hidden-name>.net/
    cacheIntervalSeconds: 30
    repositories:
    - hellopod
    - demoapp
  primaryAccount: <hidden-name>

kubernetes:
  enabled: true
  accounts:
  - name: <username>
    requiredGroupMembership: []
    dockerRegistries:
    - accountName: <hidden-name>
      namespaces: []
    context: sre-os1-dev
    namespaces:
    - spinnaker
    omitNamespaces: []
    kubeconfigFile: /home/<username>/.kube/config

I suspect you may be using redis as the persistent storage type(I ran into the same issue).

If this is the case, persistent storage using redis doesn't seem to be working properly out-of-the-box, and it is not supported. I would try using an S3 target, if available.

More info here on support for redis

To configure S3 using Halyard, use the following commands:

echo <SECRET_ACCESS_KEY> | hal config storage s3 edit --access-key-id <ACCESS_KEY_ID> --endpoint <S3_ENDPOINT> --bucket <BUCKET_NAME> --root-folder spinnaker --secret-access-key
hal config storage edit --type s3
hal deploy apply