Exposing ability to Submit results when used as a library.

[naemono]

Is your feature request related to a problem? Please describe.

I know there's been recent work to expose running checks on both containers, and operators (see #809), which I've found useful when attempting to simplify our openshift/operator release flow. Unfortunately there doesn't seem to be any way to actually submit the results, as the work to move most of the flow to /internal has progressed. There also seems to be some discussion about packaging the results to be able to be submitted without internet access.

Describe the solution you'd like.

containerCheck.Run returns runtime.Results. And the ResultSubmitter returned by

openshift-preflight/internal/lib/lib.go

Line 13 in 3560a24

func ResolveSubmitter(pc PyxisClient, projectID, dockerconfig, logfile string) ResultSubmitter {

requires nearly all of the data that NewCheck can accept, given the variadic options that can be sent.

Exposing some ability to submit the results of checks seems like a logical step that could be taken to allow users the ability to integrate preflight checks fully into a custom release flow.

Describe alternatives you've considered.

There do not seem to be alternatives outside of running the preflight binary.

[komish]

Thanks for opening this! It's a great topic to discuss.

As you mentioned, the work to clarify our exported APIs is still in progress. At the time of this writing, I don't believe we intend on supporting result submission through our library (as in, exporting functionality to submit results).

This, of course, may change as we get closer to completing issues related to supporting the offline-focused bundler logic.

I'll leave this open for further discussion as we work through these.

[naemono]

@komish Thanks very much for the quick reply. Any chance you could discuss the reasoning for not including submission of results through the library? Is there some security concern that is trying to be avoided, or some other reason?

Thanks very much for your input.

[komish]

@naemono Sure thing, I'm happy to share my line of thought on this. And no security concern to note; our submission code is viewable in this project for folks to review.

The CLI gives you a way to submit your results because it's intending to be a single tool that can help partners test and certify their images and Operators. We're subject to the submission workflow potentially changing over time, and need the flexibility to change how it works internally for the CLI use case so that our CLI consumers don't feel a significant impact.

To support a public submission library, we would want to make sure it behaves identically to our CLI submission functionality today, and we'd almost certainly want to call our own library in our CLI to help with that goal. And to provide some degree of API compatibility guarantee over time, we would need to write it in such a way that is flexible enough to serve our future needs within the CLI, but also provide some degree of guardrail to prevent a subpar experience in submission for our partners when trying to use it.

There are ways to go about it, for sure! But given our current requirements and goals, we just aren't committed to adding and maintaining result submission via public library calls to the scope today. It may be considered in the future, just based on what's necessary to complete the mentioned issues.