I'm trying to add customresourcedeployment in aks. Earlier we used V1beta1 version which I think, deprecated. Now I am trying with v1 version which is throwing schema validation error. The V1beta1 version is as follows.
{
'apiVersion': 'apiextensions.k8s.io/v1beta1',
'kind': 'CustomResourceDefinition',
'metadata': {
'name': 'azureassignedidentities.aadpodidentity.k8s.io'
},
'spec': {
'group': 'aadpodidentity.k8s.io',
'version': 'v1',
'names': {
'kind': 'AzureAssignedIdentity',
'plural': 'azureassignedidentities'
},
'scope': 'Namespaced'
}
}
Now i'm trying with the following template.
let collection = {
'apiVersion': 'apiextensions.k8s.io/v1',
'kind': 'CustomResourceDefinition',
'metadata': {
"annotations": {
"api-approved.kubernetes.io": "unapproved",
"controller-gen.kubebuilder.io/version": "v0.5.0"
},
'name': 'azureassignedidentities.aadpodidentity.k8s.io'
},
'spec': {
'group': 'aadpodidentity.k8s.io',
'names': {
'kind': 'AzureAssignedIdentity',
'listKind': 'AzureAssignedIdentityList',
'plural': 'azureassignedidentities',
'singular': 'azureassignedidentity'
},
'scope': 'Namespaced',
'versions': {
'name': 'v1',
'schema': {
'openAPIV3Schema': {
'description': 'AzureAssignedIdentity contains the identity <-> pod mapping which is matched.',
'properties': {
'apiVersion': {
'description': 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources',
'type': 'string'
},
'kind': {
'description': 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds',
'type': 'string'
},
'metadata': {
'type': 'object'
},
'spec': {
'description': 'AzureAssignedIdentitySpec contains the relationship between an AzureIdentity and an AzureIdentityBinding.',
'properties': {
'azureBindingRef': {
'description': 'AzureBindingRef is an embedded resource referencing the AzureIdentityBinding used by the AzureAssignedIdentity, which requires x-kubernetes-embedded-resource fields to be true',
'properties': {
'apiVersion': {
'description': 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources',
'type': 'string'
},
'kind': {
'description': 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds',
'type': 'string'
},
'metadata': {
'type': 'object'
},
'spec': {
'description': 'AzureIdentityBindingSpec matches the pod with the Identity. Used to indicate the potential matches to look for between the pod/deployment and the identities present.',
'properties': {
'azureIdentity': {
'type': 'string'
},
'metadata': {
'type': 'object'
},
'selector': {
'type': 'string'
},
'weight': {
'description': 'Weight is used to figure out which of the matching identities would be selected.',
'type': 'integer'
}
},
'type': 'object'
},
'status': {
'description': 'AzureIdentityBindingStatus contains the status of an AzureIdentityBinding.',
'properties': {
'availableReplicas': {
'format': 'int32',
'type': 'integer'
},
'metadata': {
'type': 'object'
}
},
'type': 'object'
}
},
'type': 'object',
'x-kubernetes-embedded-resource': 'true'
},
'azureIdentityRef': {
'description': 'AzureIdentityRef is an embedded resource referencing the AzureIdentity used by the AzureAssignedIdentity, which requires x-kubernetes-embedded-resource fields to be true',
'properties': {
'apiVersion': {
'description': 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources',
'type': 'string'
},
'kind': {
'description': 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds',
'type': 'string'
},
'metadata': {
'type': 'object'
},
'spec': {
'description': 'AzureIdentitySpec describes the credential specifications of an identity on Azure.',
'properties': {
'adEndpoint': {
'type': 'string'
},
'adResourceID': {
'description': 'For service principal. Option param for specifying the AD details.',
'type': 'string'
},
'auxiliaryTenantIDs': {
'description': 'Service principal auxiliary tenant ids',
'items': {
'type': 'string'
},
'nullable': 'true',
'type': 'array'
},
'clientID': {
'description': 'Both User Assigned MSI and SP can use this field.',
'type': 'string'
},
'clientPassword': {
'description': 'Used for service principal',
'properties': {
'name': {
'description': 'Name is unique within a namespace to reference a secret resource.',
"type": "string"
},
'namespace': {
'description': 'Namespace defines the space within which the secret name must be unique.',
'type': 'string'
}
},
'type': 'object'
},
'metadata': {
'type': 'object'
},
'replicas': {
'format': 'int32',
'nullable': 'true',
'type': 'integer'
},
'resourceID': {
'description': 'User assigned MSI resource id.',
'type': 'string'
},
'tenantID': {
'description': 'Service principal primary tenant id.',
'type': 'string'
},
'type': {
'description': 'UserAssignedMSI or Service Principal',
'type': 'integer'
}
},
'type': 'object'
},
'status': {
'description': 'AzureIdentityStatus contains the replica status of the resource.',
'properties': {
'availableReplicas': {
'format': 'int32',
'type': 'integer'
},
'metadata': {
'type': 'object'
}
},
'type': 'object'
}
},
'type': 'object',
'x-kubernetes-embedded-resource': 'true'
},
'metadata': {
'type': 'object'
},
'nodename': {
'type': 'string'
},
'pod': {
'type': 'string'
},
'podNamespace': {
'type': 'string'
},
'replicas': {
'format': 'int32',
'nullable': 'true',
'type': 'integer'
}
},
'type': 'object'
},
'status': {
'description': 'AzureAssignedIdentityStatus contains the replica status of the resource.',
'properties': {
'availableReplicas': {
'format': 'int32',
'type': 'integer'
},
'metadata': {
'type': 'object'
},
'status': {
'type': 'string'
}
},
'type': 'object'
}
},
'type': 'object'
}
},
'served': 'true',
'storage': 'true',
}
},
'status': {
'acceptedNames': {
'kind': "",
'plural': ""
},
'conditions': [],
'storedVersions': []
}
}
I am trying to deploy using following code.
const client = this.kubeConfig.makeApiClient(k8s.ApiextensionsV1Api);
return new Promise<string>((resolve, reject) => {
client.createCustomResourceDefinition(collection).then(
(response) => {
resolve(response);
},
(err) => {
reject(err.response && err.response.body ? err.response.body : err);
},
);
});The error is as follows.
{ "message": "CustomResourceDefinition.apiextensions.k8s.io \"azureassignedidentities.aadpodidentity.k8s.io\" is invalid: [spec.versions0.schema.openAPIV3Schema: Required value: schemas are required, spec.versions1.schema.openAPIV3Schema: Required value: schemas are required, spec.versions2.schema.openAPIV3Schema: Required value: schemas are required, spec.versions3.schema.openAPIV3Schema: Required value: schemas are required, spec.versions0.name: Invalid value: \"\": a DNS-1035 label must consist of lower case alphanumeric characters or '-', start with an alphabetic character, and end with an alphanumeric character (e.g. 'my-name', or 'abc-123', regex used for validation is 'a-z(-a-z0-9a-z0-9)?'), spec.versions1.name: Invalid value: \"\": a DNS-1035 label must consist of lower case alphanumeric characters or '-', start with an alphabetic character, and end with an alphanumeric character (e.g. 'my-name', or 'abc-123', regex used for validation is 'a-z?'), spec.versions2.name: Invalid value: \"\": a DNS-1035 label must consist of lower case alphanumeric characters or '-', start with an alphabetic character, and end with an alphanumeric character (e.g. 'my-name', or 'abc-123', regex used for validation is 'a-z?'), spec.versions3.name: Invalid value: \"\": a DNS-1035 label must consist of lower case alphanumeric characters or '-', start with an alphabetic character, and end with an alphanumeric character (e.g. 'my-name', or 'abc-123', regex used for validation is 'a-z?'), spec.versions: Invalid value: []apiextensions.CustomResourceDefinitionVersion{apiextensions.CustomResourceDefinitionVersion{Name:\"\", Served:false, Storage:false, Deprecated:false, DeprecationWarning:(string)(nil), Schema:(apiextensions.CustomResourceValidation)(nil), Subresources:(apiextensions.CustomResourceSubresources)(nil), AdditionalPrinterColumns:[]apiextensions.CustomResourceColumnDefinition(nil)}, apiextensions.CustomResourceDefinitionVersion{Name:\"\", Served:false, Storage:false, Deprecated:false, DeprecationWarning:(string)(nil), Schema:(apiextensions.CustomResourceValidation)(nil), Subresources:(apiextensions.CustomResourceSubresources)(nil), AdditionalPrinterColumns:[]apiextensions.CustomResourceColumnDefinition(nil)}, apiextensions.CustomResourceDefinitionVersion{Name:\"\", Served:false, Storage:false, Deprecated:false, DeprecationWarning:(string)(nil), Schema:(apiextensions.CustomResourceValidation)(nil), Subresources:(apiextensions.CustomResourceSubresources)(nil), AdditionalPrinterColumns:[]apiextensions.CustomResourceColumnDefinition(nil)}, apiextensions.CustomResourceDefinitionVersion{Name:\"\", Served:false, Storage:false, Deprecated:false, DeprecationWarning:(string)(nil), Schema:(apiextensions.CustomResourceValidation)(nil), Subresources:(apiextensions.CustomResourceSubresources)(nil), AdditionalPrinterColumns:[]apiextensions.CustomResourceColumnDefinition(nil)}}: must contain unique version names, spec.versions: Invalid value: []apiextensions.CustomResourceDefinitionVersion{apiextensions.CustomResourceDefinitionVersion{Name:\"\", Served:false, Storage:false, Deprecated:false, DeprecationWarning:(string)(nil), Schema:(apiextensions.CustomResourceValidation)(nil), Subresources:(apiextensions.CustomResourceSubresources)(nil), AdditionalPrinterColumns:[]apiextensions.CustomResourceColumnDefinition(nil)}, apiextensions.CustomResourceDefinitionVersion{Name:\"\", Served:false, Storage:false, Deprecated:false, DeprecationWarning:(string)(nil), Schema:(apiextensions.CustomResourceValidation)(nil), Subresources:(apiextensions.CustomResourceSubresources)(nil), AdditionalPrinterColumns:[]apiextensions.CustomResourceColumnDefinition(nil)}, apiextensions.CustomResourceDefinitionVersion{Name:\"\", Served:false, Storage:false, Deprecated:false, DeprecationWarning:(string)(nil), Schema:(apiextensions.CustomResourceValidation)(nil), Subresources:(apiextensions.CustomResourceSubresources)(nil), AdditionalPrinterColumns:[]apiextensions.CustomResourceColumnDefinition(nil)}, apiextensions.CustomResourceDefinitionVersion{Name:\"\", Served:false, Storage:false, Deprecated:false, DeprecationWarning:(string)(nil), Schema:(apiextensions.CustomResourceValidation)(nil), Subresources:(apiextensions.CustomResourceSubresources)(nil), AdditionalPrinterColumns:[]apiextensions.CustomResourceColumnDefinition(nil)}}: must have exactly one version marked as storage version, status.storedVersions: Invalid value: []string(nil): must have at least one stored version]" }*
PLEASE HELP ME TO SOLVE THIS
Please check if possible cause is that versions (at least 1)not being stored in stored versions status.
See the error which says
status.storedVersions: Invalid value: []string(nil): must have at least one stored version]" }
When deprecating versions, select a storage upgrade procedure. Remove the old version from the CustomResourceDefinition status.storedVersions field and make sure new version (at least one) is present.
Manually upgrade the existing objects to a new stored version : Write an upgrade procedure to list all existing objects and write them with the same content. This forces the backend to write objects in the current storage version, which is v1
1.First created the resource crd1.yaml with previous version.
versions:
- name: v1beta1
served: true
storage: trueapply it using kubectl.
kubectl apply -f crd1.yaml2.Update resource with definition of later version say v1. Set v1 as the storage in the CustomResourceDefinition file crd12.yaml .
versions:
- name: v1
served: true
storage: trueand apply it using kubectl.
kubectl apply -f crd12.yamlkubectl get CustomResourceDefinition $resouce_name -o yaml
storedVersions: [v1beta1 ,v1](*Note: It's totally an option for us to keep the v1beta1 version in the CRD. If specified with served: false, any get/create/update operation on a v1alpha1 resource is prevented.* )
Verify that the storage is set to true for the new version in the spec.versions list in the CustomResourceDefinition.
You can save the CustomResourceDefinition in a YAML file, then use kubectl apply to create it.
example:
spec:
group: example.com
# list of versions supported by this CustomResourceDefinition
versions:
- name: v1
served: true
storage: true # One and only one version must be marked as the storage version.
schema:
openAPIV3Schema:
type: object
properties:
host:
type: string
port:
type: string
........
- name: v1beta1
served: false # Each version can be enabled/disabled by Served flag.
storage: false
# A schema is required
schema:
openAPIV3Schema:
type: object
properties:
host:
type: string
port:
type: string
......
conversion:
strategy: None
# None conversion assumes the same schema for all versions and only sets the apiVersion
field of custom resources to the proper valueThen remove older version.Fields under status are modified via the status subresource,
e.g.:
curl -d '[{ "op": "replace", "path":"/status/storedVersions", "value": ["v1"] }]' \
-H "Content-Type: application/json-patch+json" \
-X PATCH \ http://localhost:8080 /apis/apiextensions.k8s.io/v1beta1/customresourcedefinitions/$resource_name/statusVerify that the old version is no longer listed in the CustomResourceDefinition status.storedVersions
Please refer these: 1. Remove version from status.storedVersions (kubernetesquestions.com) or how to remove version from status.storedVersions (github.com) 2. custom-resource-definition-versioning example. 3. Versions in CustomResourceDefinitions - Kubernetes