ACR to AKS deployment ERROR: The container operating system does not match the host operating system

5/30/2019

I have deployed a windows container which runs successfully in my local system using docker. Moved the image to Azure container registry and deployed the image from ACR to AKS.

I have created a windows node using AKS windows preview

C:\Users\HTECH>kubectl get nodes -o wide
NAME                                STATUS   ROLES   AGE    VERSION   INTERNAL-IP   EXTERNAL-IP   OS-IMAGE                    KERNEL-VERSION      CONTAINER-RUNTIME
aks-nodepool1-36563144-vmss000000   Ready    agent   2d3h   v1.14.0   10.240.0.4    <none>        Ubuntu 16.04.6 LTS          4.15.0-1042-azure   docker://3.0.4
akssample000000                     Ready    agent   2d2h   v1.14.0   10.240.0.35   <none>        Windows Server Datacenter   10.0.17763.379      docker://18.9.2

Docker File:

FROM microsoft/iis:latest
SHELL ["powershell"]

RUN Install-WindowsFeature NET-Framework-45-ASPNET ; \
    Install-WindowsFeature Web-Asp-Net45

COPY . ewims
RUN Remove-WebSite -Name 'Default Web Site'
RUN New-Website -Name 'sample' -Port 80 \
    -PhysicalPath 'c:\sample' -ApplicationPool '.NET v4.5'
EXPOSE 80

Manifest YAML

apiVersion: apps/v1
kind: Deployment
metadata:
  name: samplecloudpoc-v1
spec:
  replicas: 1
  strategy:
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 1
  minReadySeconds: 5   
  selector:
    matchLabels:
      app: samplecloudpoc-v1
  template:
    metadata:
      labels:
        app: samplecloudpoc-v1
    spec:
      containers:
      - name: samplecloudpoc-v1
        image: samplecloud.azurecr.io/sample:v1
        ports:
        - containerPort: 80
        resources:
          requests:
            cpu: 100m
          limits:
            cpu: 100m
        env:
        - name: dev
          value: "samplecloudpoc-v1"
      imagePullSecrets:
        - name: sampleauth
      nodeSelector:
        beta.kubernetes.io/os: windows
---
apiVersion: v1
kind: Service
metadata:
  name: samplecloudpoc-v1
spec:
  loadBalancerIP: 13.90.205.141
  type: LoadBalancer
  ports:
  - port: 80
  selector:
    app: samplecloudpoc-v1

While checking the deployement status using below command, I'm getting the following error.

D:\Cloud>kubectl describe po samplecloudpoc-v1-5d567d48d9-7gtx8
Name:               samplecloudpoc-v1-5d567d48d9-7gtx8
Namespace:          default
Priority:           0
PriorityClassName:  <none>
Node:               akssample000000/10.240.0.35
Start Time:         Thu, 30 May 2019 13:05:13 +0530
Labels:             app=samplecloudpoc-v1
                    pod-template-hash=5d567d48d9
Annotations:        <none>
Status:             Running
IP:                 10.240.0.44
Controlled By:      ReplicaSet/samplecloudpoc-v1-5d567d48d9
Containers:
  sample:
    Container ID:  docker://0cf6c92b15738c2786caca5b989aa2773c9375352cb4f1d95472ff63cc7b5112
    Image:         samplecloud.azurecr.io/sample:v1
    Image ID:      docker-pullable://samplecloud.azurecr.io/sample@sha256:55ac14dc512abc0f8deebb8b87ee47d51fdfbfd997ce6cee0ab521bd69d42b08
    Port:          80/TCP
    Host Port:     0/TCP
    Args:
      -it
    State:       Waiting
      Reason:    CrashLoopBackOff
    Last State:  Terminated
      Reason:    ContainerCannotRun
      Message:   CreateComputeSystem 0cf6c92b15738c2786caca5b989aa2773c9375352cb4f1d95472ff63cc7b5112: The container operating system does not match the host operating system.
(extra info: {"SystemType":"Container","Name":"0cf6c92b15738c2786caca5b989aa2773c9375352cb4f1d95472ff63cc7b5112","Owner":"docker","VolumePath":"\\\\?\\Volume{f5ff1135-4e83-4baa-961d-f4533dcb6985}","IgnoreFlushesDuringBoot":true,"LayerFolderPath":"C:\\ProgramData\\docker\\windowsfilter\\0cf6c92b15738c2786caca5b989aa2773c9375352cb4f1d95472ff63cc7b5112","Layers":[{"ID":"b3b88c23-310f-5e95-86bc-117e9f6a6184","Path":"C:\\ProgramData\\docker\\windowsfilter\\0cc15446c028e2fe68601b10a1921a809dedb2a981162c4ed90344d2fde58f0e"},{"ID":"fb1ae57e-89dc-502c-996b-75804b972adc","Path":"C:\\ProgramData\\docker\\windowsfilter\\7217ca2e8bbd2c431c9db44050946ec4f8040c42fdd79f7ceae321e48dc5ca0d"},{"ID":"bb5e3864-b1af-51c8-a8e4-63d88749f082","Path":"C:\\ProgramData\\docker\\windowsfilter\\16f07ffe70a600c95bea2e8297c04cbb6af877562c2cc2ac1693267b152d3793"},{"ID":"2fae8c16-582f-5ab1-acfe-0a88980adec3","Path":"C:\\ProgramData\\docker\\windowsfilter\\a325070d766dd4af490b433d75eac6e1d71297961d89011e5798949eae2e7e4a"},{"ID":"dffd6df2-a500-5985-9c9c-1bc03c9efce3","Path":"C:\\ProgramData\\docker\\windowsfilter\\1221f773d66647fd1dc7aad44693f28843c8385612edb520231c1cb754eb2f97"},{"ID":"7e349a26-81b9-554e-aa13-a6e4286de93e","Path":"C:\\ProgramData\\docker\\windowsfilter\\67d6d22eae7f829e590fde792c6b8129aff3d9f9242850fe72e8d167e284a6b7"},{"ID":"8730db1a-385d-5e9a-a4ec-c45525b5fcb3","Path":"C:\\ProgramData\\docker\\windowsfilter\\2a53ed97b10bd4f67e62e8511e8922496651f3d343dd1889425ba1bedca134fa"},{"ID":"d1e23520-6c0b-5909-8e52-bb6961f80876","Path":"C:\\ProgramData\\docker\\windowsfilter\\d3a27083556be1bb7e36997f0eee2b544f6a16eab94797715bc21db99bf42e88"},{"ID":"18d8ab30-09e9-54e3-a991-f48cca651c8d","Path":"C:\\ProgramData\\docker\\windowsfilter\\9b4143f537ff70f6b1e05b2a5e38e3b05dd2a4b2f624822e32bb2b7cd17b7cca"},{"ID":"2acb6fa3-f27c-50cf-9033-eedb06d5bf32","Path":"C:\\ProgramData\\docker\\windowsfilter\\f71b6708cc4045bf9633f971dd4d6eddb1c5ffeda52d38e648c740e0e277b2df"},{"ID":"0dc40cf1-482a-5fed-af35-c5d1902b95ae","Path":"C:\\ProgramData\\docker\\windowsfilter\\100f3380579a77f2fb2c0f997201e34a0dd2c42e4b0d9a39fb850706aa16e474"}],"ProcessorWeight":5000,"HostName":"samplecloudpoc-v1-5d567d48d9-7gtx8","MappedDirectories":[{"HostPath":"c:\\var\\lib\\kubelet\\pods\\75f48f65-82ad-11e9-8e99-a24a72224ed5\\volumes\\kubernetes.io~secret\\default-token-67g2m","ContainerPath":"c:\\var\\run\\secrets\\kubernetes.io\\serviceaccount","ReadOnly":true,"BandwidthMaximum":0,"IOPSMaximum":0,"CreateInUtilityVM":false}],"HvPartition":false,"NetworkSharedContainerName":"d11237887aec604bfbb9b3cd56fca586975e5a92e04dab4d4ba19b1fcc56ed99"})
      Exit Code:    128
      Started:      Thu, 30 May 2019 13:25:00 +0530
      Finished:     Thu, 30 May 2019 13:25:00 +0530
    Ready:          False
    Restart Count:  5
    Limits:
      cpu:  100m
    Requests:
      cpu:  100m
    Environment:
      dev:  samplecloudpoc-v1
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-67g2m (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             False
  ContainersReady   False
  PodScheduled      True
Volumes:
  default-token-67g2m:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-67g2m
    Optional:    false
QoS Class:       Burstable
Node-Selectors:  beta.kubernetes.io/os=windows
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type     Reason     Age                    From                      Message
  ----     ------     ----                   ----                      -------
  Normal   Scheduled  21m                    default-scheduler         Successfully assigned default/samplecloudpoc-v1-5d567d48d9-7gtx8 to akssample000000
  Normal   Pulling    21m                    kubelet, akssample000000  Pulling image "samplecloud.azurecr.io/sample:v1"
  Normal   Pulled     4m31s                  kubelet, akssample000000  Successfully pulled image "samplecloud.azurecr.io/sample:v1"
  Normal   Created    3m (x5 over 4m31s)     kubelet, akssample000000  Created container sample
  Normal   Pulled     3m (x4 over 4m28s)     kubelet, akssample000000  Container image "samplecloud.azurecr.io/sample:v1" already present on machine
  Warning  Failed     2m59s (x5 over 4m30s)  kubelet, akssample000000  Error: failed to start container "sample": Error response from daemon: CreateComputeSystem sample: The container operating system does not match the host operating system.
(extra info: {"SystemType":"Container","Name":"sample","Owner":"docker","VolumePath":"\\\\?\\Volume{f5ff1135-4e83-4baa-961d-f4533dcb6985}","IgnoreFlushesDuringBoot":true,"LayerFolderPath":"C:\\ProgramData\\docker\\windowsfilter\\sample","Layers":[{"ID":"b3b88c23-310f-5e95-86bc-117e9f6a6184","Path":"C:\\ProgramData\\docker\\windowsfilter\\0cc15446c028e2fe68601b10a1921a809dedb2a981162c4ed90344d2fde58f0e"},{"ID":"fb1ae57e-89dc-502c-996b-75804b972adc","Path":"C:\\ProgramData\\docker\\windowsfilter\\7217ca2e8bbd2c431c9db44050946ec4f8040c42fdd79f7ceae321e48dc5ca0d"},{"ID":"bb5e3864-b1af-51c8-a8e4-63d88749f082","Path":"C:\\ProgramData\\docker\\windowsfilter\\16f07ffe70a600c95bea2e8297c04cbb6af877562c2cc2ac1693267b152d3793"},{"ID":"2fae8c16-582f-5ab1-acfe-0a88980adec3","Path":"C:\\ProgramData\\docker\\windowsfilter\\a325070d766dd4af490b433d75eac6e1d71297961d89011e5798949eae2e7e4a"},{"ID":"dffd6df2-a500-5985-9c9c-1bc03c9efce3","Path":"C:\\ProgramData\\docker\\windowsfilter\\1221f773d66647fd1dc7aad44693f28843c8385612edb520231c1cb754eb2f97"},{"ID":"7e349a26-81b9-554e-aa13-a6e4286de93e","Path":"C:\\ProgramData\\docker\\windowsfilter\\67d6d22eae7f829e590fde792c6b8129aff3d9f9242850fe72e8d167e284a6b7"},{"ID":"8730db1a-385d-5e9a-a4ec-c45525b5fcb3","Path":"C:\\ProgramData\\docker\\windowsfilter\\2a53ed97b10bd4f67e62e8511e8922496651f3d343dd1889425ba1bedca134fa"},{"ID":"d1e23520-6c0b-5909-8e52-bb6961f80876","Path":"C:\\ProgramData\\docker\\windowsfilter\\d3a27083556be1bb7e36997f0eee2b544f6a16eab94797715bc21db99bf42e88"},{"ID":"18d8ab30-09e9-54e3-a991-f48cca651c8d","Path":"C:\\ProgramData\\docker\\windowsfilter\\9b4143f537ff70f6b1e05b2a5e38e3b05dd2a4b2f624822e32bb2b7cd17b7cca"},{"ID":"2acb6fa3-f27c-50cf-9033-eedb06d5bf32","Path":"C:\\ProgramData\\docker\\windowsfilter\\f71b6708cc4045bf9633f971dd4d6eddb1c5ffeda52d38e648c740e0e277b2df"},{"ID":"0dc40cf1-482a-5fed-af35-c5d1902b95ae","Path":"C:\\ProgramData\\docker\\windowsfilter\\100f3380579a77f2fb2c0f997201e34a0dd2c42e4b0d9a39fb850706aa16e474"}],"ProcessorWeight":5000,"HostName":"samplecloudpoc-v1-5d567d48d9-7gtx8","MappedDirectories":[{"HostPath":"c:\\var\\lib\\kubelet\\pods\\75f48f65-82ad-11e9-8e99-a24a72224ed5\\volumes\\kubernetes.io~secret\\default-token-67g2m","ContainerPath":"c:\\var\\run\\secrets\\kubernetes.io\\serviceaccount","ReadOnly":true,"BandwidthMaximum":0,"IOPSMaximum":0,"CreateInUtilityVM":false}],"HvPartition":false,"NetworkSharedContainerName":"d11237887aec604bfbb9b3cd56fca586975e5a92e04dab4d4ba19b1fcc56ed99"})
  Warning  BackOff  119s (x10 over 3m59s)  kubelet, akssample000000  Back-off restarting failed container

I have tried to fix this by pulling the latest version IIS image from docker also throws the same error.

-- Hub
azure
azure-aks
azure-container-registry
azure-kubernetes
kubernetes

3 Answers

5/30/2019

It seems you just need to change the apiVersion into apps/v1, then the pod will run well. I test the apiVersion as yours and also get the error "Back-off restarting failed container".

-- Charles Xu
Source: StackOverflow

10/28/2019

I had the problem, the issue was that my node in Kubernetes was running in Windows 2019 and my images was built with Windows 2016.

To check the Windows version for your image:

docker inspect <image>:<tag>

Then you can get nods with kubectl to view the Windows version of your node:

kubectl get nodes -o wide

To resolve it, I re-build my image from a Windows 2019 machine and then it worked from AKS.

-- glm
Source: StackOverflow

5/30/2019

I suppose the issue is with the fact AKS doesnt yet support windows nodes, so you dont really have windows nodes. You can create AKS with windows nodes, but its in preview at this point in time.

https://github.com/Azure/AKS/blob/master/previews.md#windows

If you have windows nodes, you'd need to make sure that the base image you are using matches the windows host os version

https://docs.microsoft.com/en-us/virtualization/windowscontainers/deploy-containers/version-compatibility

-- 4c74356b41
Source: StackOverflow