Component parsing does not support allOf: -if

[danielshiplett]

I have a schema that has a component that is defined like:

            AFooSpec:
              type: "object"
              properties:
                displayName:
                  type: "string"
                  description: "Name to display on user screens ([a-zA-Z0-9 _-])."
                  pattern: "^[a-zA-Z0-9][a-zA-Z0-9_\\- ]+[a-zA-Z0-9]$"
                enabled:
                  type: "boolean"
                  title: "Automatically Provision"
                  description: "Auto Provision the Shared Cluster"
                  default: true
              x-kubernetes-preserve-unknown-fields: true
              allOf:
                - if:
                    properties:
                      enabled:
                        const: true
                  then:
                    required:
                      - displayName
                  else:
                    required: [ ]

When parsed, I end up with:

components:
    schemas:
        AFooSpec:
            allOf:
                - {}
            properties:
                displayName:
                    description: Name to display on user screens ([a-zA-Z0-9 _-]).
                    pattern: ^[a-zA-Z0-9][a-zA-Z0-9_\- ]+[a-zA-Z0-9]$
                    type: string
                enabled:
                    default: true
                    description: Auto Provision the Shared Cluster
                    title: Automatically Provision
                    type: boolean
            type: object
            x-kubernetes-preserve-unknown-fields: true
            description: 'generated by: ../services/afoo/afoo-xrd.yaml'

[danielshiplett]

Note. The real problem here isn't the missing required in the output structure. The problem is that I have more complex nested objects that add structure based on allOf: -if patterns. When it ends up as the empty allOf, I loose all of my structure for the nested objects.

This is from parsing complex OpenAPI coming out of K8S CRDs.

[daveshanley]

working on it.

[daveshanley]

Fixed in v0.36

[danielshiplett]

I don't think the fix addresses my concern. Let me talk to my team some and see if I can get you a full copy of the YAML that is causing me the issue and then explain exactly what it is that I'm doing/trying. Maybe you can use something like my use case in the test code. Give me a bit and I will get back to you.

[danielshiplett]

OK. I think I have a minimal YAML that covers my use cases.

We are using Crossplane heavily. We make extensive use of their Composite Resource (XR) feature. Each XR starts with a Composite Resource Definition (XRD). It is a YAML file that is sort of like a normal K8S Custom Resource Definition. Inside of an XRD you have an OpenAPI schema section. Typically, this section will describe two objects. The 'spec' and the 'status'. You can see our 'spec' starting on line 26.

---
apiVersion: apiextensions.crossplane.io/v1
kind: CompositeResourceDefinition
metadata:
  name: afoo.aws.platform.cp.com
  annotations:
    grouping: platform
    description: "management cluster for ees."
spec:
  defaultCompositeDeletePolicy: Foreground
  group: aws.platform.cp.com
  names:
    kind: AFoo
    plural: afoos
  claimNames:
    kind: Foo
    plural: foos
  versions:
    - name: v1alpha1
      served: true
      referenceable: true
      schema:
        openAPIV3Schema:
          type: object
          properties:
            spec:
              type: object
              properties:
                guid:
                  type: string
                  description: A globally unique alphanumeric and dash identifier. Ideally a UUID.
                  pattern: "^[a-zA-Z0-9][a-zA-Z0-9\\-]+[a-zA-Z0-9]$"
                sharedCluster:
                  type: object
                  description: Cluster configuration options.
                  properties:
                    enabled:
                      type: "boolean"
                      title: "Automatically Provision"
                      description: "Auto Provision the Shared Cluster"
                      default: true
                    guid:
                      type: "string"
                      description: "A globally unique alphanumeric and dash identifier. Ideally a UUID."
                      pattern: "^[a-zA-Z0-9][a-zA-Z0-9\\-]+[a-zA-Z0-9]$"
                      x-guid-gen: true
                    displayName:
                      type: "string"
                      description: "Name of shared cluster to display on user screens ([a-zA-Z0-9 _-])."
                      pattern: "^[a-zA-Z0-9][a-zA-Z0-9_\\- ]+[a-zA-Z0-9]$"
                      x-advanced: true
                    platform:
                      type: "string"
                      description: "Platform to deploy the cluster on"
                      default: "aws"
                      x-advanced: true
                      enum:
                        - "aws"
                    version:
                      type: "string"
                      description: "The Kubernetes version."
                      default: "1.33"
                      x-advanced: true
                      enum:
                        - "1.31"
                        - "1.32"
                        - "1.33"
                        - "1.34"
                    region:
                      title: "Region"
                      type: "string"
                      description: "AWS region override for cluster (defaults to CP region)."
                      x-advanced: true
                    profiles:
                      type: "object"
                      description: "Cluster Profile Configurations"
                      properties:
                        cp:
                          type: "object"
                          description: "CP profile installation on cluster"
                          properties:
                            enabled:
                              type: "boolean"
                              title: "Install CP Apps"
                              description: "Install CP applications on the cluster when ready"
                              default: true
                          x-kubernetes-preserve-unknown-fields: true
                          allOf:
                            - if:
                                properties:
                                  enabled:
                                    const: true
                              then:
                                properties:
                                  registry:
                                    type: "string"
                                    description: "Registry for container images, Helm charts, and OCI artifacts"
                                    ui:placeholder: "123456789012.dkr.ecr.us-east-1.amazonaws.com"
                                    pattern: "^([A-Za-z0-9-_~]+.){1,}[a-z]+$"
                                  teleport:
                                    type: "object"
                                    description: "Teleport installation on cluster"
                                    properties:
                                      enabled:
                                        type: "boolean"
                                        title: "Install Teleport"
                                        description: "Install Teleport on the cluster"
                                        default: false
                                    x-kubernetes-preserve-unknown-fields: true
                                    allOf:
                                      - if:
                                          properties:
                                            enabled:
                                              const: true
                                        then:
                                          properties:
                                            license:
                                              type: "string"
                                              description: "Paste the contents of Teleport license.pem"
                                              ui:widget: "textarea"
                                              ui:options:
                                                rows: 10
                                          required:
                                            - license
                                        else:
                                          required: [ ]
                                required:
                                  - registry
                              else:
                                required: [ ]
                  x-kubernetes-preserve-unknown-fields: true
                  allOf:
                    - if:
                        properties:
                          enabled:
                            const: true
                      then:
                        required:
                          - platform
                          - clusterType
                      else:
                        required: [ ]
                providerContext:
                  type: "object"
                  x-advanced: true
                  description: "Configuration items required to provision accounts"
                  properties:
                    createContexts:
                      type: "boolean"
                      description: "Create a new AWS account for each CP/EE that is created"
                      default: false
                  x-kubernetes-preserve-unknown-fields: true
                  if:
                    properties:
                      createContexts:
                        const: true
                  then:
                    properties:
                      roleName:
                        type: "string"
                        description: |
                          If creating accounts for EEs, provide the role name that will be inherited:
                          - Kion: this role must be configured on an OU and designated for inheritance on new accounts
                      kionPayerId:
                        type: "integer"
                        description: "The Kion-ID for the Billing Account"
                      kionProjectId:
                        type: "integer"
                        description: "The Kion project to add created Accounts to"
                    required:
                      - "roleName"
                      - "kionPayerId"
                      - "kionProjectId"
                  else:
                    required: [ ]
                providerConfigName:
                  type: "string"
                  description: "Gives the name of the provider config to use for this CP. Required if this this is a root CP"
                  default: "aws-provider"
                  x-advanced: true
            status:
              type: object
              properties:
                ntz:
                  type: object
                  description: Network Trust Zone status information
                  properties:
                    lb:
                      type: object
                      description: NTZ internet-facing load balancer status
                      properties:
                        name:
                          type: string
                          description: Load balancer name
                        id:
                          type: string
                          description: Load balancer ARN or ID

We are in the process of converting much of our XR support code from YAML (which is normal for Crossplane) to Golang (which is supported, but less common). As part of this process, we want to have Golang types for everything. This includes the XRD spec and status objects.

What I have been working on is a way to generate all the types/constants/etc. as Golang from the YAML. We still want to treat the XRD YAML as the original schema and have everything based off of it. I have gotten a basic set of generators working. However, several of our XRDs are tripping up still. The YAML above is a stripped down version of one of our worst cases. It shows how we mix if/then/else with allOf in different ways to set up different required lists and additional properties in the objects.

The first step of my generation flow is to extract the spec and status OpenAPI components from this original YAML. From that, I generate a temporary YAML file that will be fed into a Golang type generator. I am using libopenapi to do the first step of the extraction because, along with everything else, I want to make sure the YAML is valid OpenAPI v3.1.

Here's what I'm still getting out in my temporary YAML even after updating to v0.36.1 today.

openapi: 3.1.0
info:
    title: AFoo
    version: v1alpha1
components:
    schemas:
        AFooSpec:
            properties:
                guid:
                    description: A globally unique alphanumeric and dash identifier. Ideally a UUID.
                    pattern: ^[a-zA-Z0-9][a-zA-Z0-9\-]+[a-zA-Z0-9]$
                    type: string
                providerConfigName:
                    default: aws-provider
                    description: Gives the name of the provider config to use for this CP. Required if this this is a root CP
                    type: string
                providerContext:
                    description: Configuration items required to provision accounts
                    properties:
                        createContexts:
                            default: false
                            description: Create a new AWS account for each CP/EE that is created
                            type: boolean
                    type: object
                    x-kubernetes-preserve-unknown-fields: true
                sharedCluster:
                    allOf:
                        - {}
                    description: Cluster configuration options.
                    properties:
                        displayName:
                            description: Name of shared cluster to display on user screens ([a-zA-Z0-9 _-]).
                            pattern: ^[a-zA-Z0-9][a-zA-Z0-9_\- ]+[a-zA-Z0-9]$
                            type: string
                        enabled:
                            default: true
                            description: Auto Provision the Shared Cluster
                            title: Automatically Provision
                            type: boolean
                        guid:
                            description: A globally unique alphanumeric and dash identifier. Ideally a UUID.
                            pattern: ^[a-zA-Z0-9][a-zA-Z0-9\-]+[a-zA-Z0-9]$
                            type: string
                        platform:
                            default: aws
                            description: Platform to deploy the cluster on
                            enum:
                                - aws
                            type: string
                        profiles:
                            description: Cluster Profile Configurations
                            properties:
                                cp:
                                    allOf:
                                        - {}
                                    description: CP profile installation on cluster
                                    properties:
                                        enabled:
                                            default: true
                                            description: Install CP applications on the cluster when ready
                                            title: Install CP Apps
                                            type: boolean
                                    type: object
                                    x-kubernetes-preserve-unknown-fields: true
                            type: object
                        region:
                            description: AWS region override for cluster (defaults to CP region).
                            title: Region
                            type: string
                        version:
                            default: "1.33"
                            description: The Kubernetes version.
                            enum:
                                - "1.31"
                                - "1.32"
                                - "1.33"
                                - "1.34"
                            type: string
                    type: object
                    x-kubernetes-preserve-unknown-fields: true
            type: object
            description: 'generated by: ../services/afoo/afoo-xrd.yaml'
        AFooStatus:
            properties:
                ntz:
                    description: Network Trust Zone status information
                    properties:
                        lb:
                            description: NTZ internet-facing load balancer status
                            properties:
                                id:
                                    description: Load balancer ARN or ID
                                    type: string
                                name:
                                    description: Load balancer name
                                    type: string
                            type: object
                    type: object
            type: object
            description: 'generated by: ../services/afoo/afoo-xrd.yaml'

You can see that both the AFooSpec.providerContext and the AFooSpec.sharedCluster are missing some information. The first is missing some additional properties that are part of the then block. The second is missing properties and required lists that are part of an allOf:-if block.

In both cases, I think the input is valid, according to my reading of OpenAPI v3. And, so far, our Kubernetes and Crossplane have had no problem ingesting the original XRD and interpreting it correctly.

Let me know if you need any more information to help test/debug this issue.

(oh. it looks like I cannot reopen this issue. if @daveshanley could do that, that would be great.)

[daveshanley]

OK, will re-run this again, thank you for the repro.

[danielshiplett]

Yeah. Sorry to re-open this one. I wasn't sure if you wanted a new issue or not. I blame myself as my original request wasn't very clear about the complexities that I am facing. I should have generated a more complete example XRD to begin with.