refactor operator-framework-e2e

[everettraven]

Description

• Adds a script called setup.sh to perform all the setup logic for the coast-to-coast e2e test
  • This makes it easier to read and modify the setup logic in the future
• Removes obsolete Go code and updates the Ginkgo tests as necessary
• Removes obsolete Makefile targets and updates the Makefile as necessary

The combination of the changes here makes the test still generally do the same things of:

• Create a temp directory for building operators
• Use the operator-sdk to:
  • create two operators and build their images, one for the registry+v1 and one for the plain+v0 bundle formats
  • generate and build the bundle image for the registry+v1 bundle format
• For the plain+v0 operator, create and build the plain+v0 bundle image
• Creates a File-Based Catalog (FBC) image that contains both the operators
• Load the operator, bundle, and catalog images onto a KinD cluster
• Clean up the temp directories
• Run the Ginkgo test suite that:
  • Creates a Catalog that references a catalog image, either the one with the plain+v0 bundle or the one with the registry+v1 bundle
  • Creates an Operator that references a package, either the one with the plain+v0 bundle or the one with the registry+v1 bundle
  • Verifies that the Operator eventually has the condition Installed with a status of True and a reason of Success

Motivation

• Makes it easier to read and modify the operator-framework-e2e test. Particularly it will help with making the e2e changes needed for (vendor) k8s + catalogd bump #475 easier
• resolves operator-framework-e2e: we should verify eventual consistency between catalog unpacking and operator resolution #482
• resolves make operator-developer-e2e fails with podman #349
• resolves make operator-developer-e2e fails when container with name "local-registry" is already exists #350

Open Questions

• The previous implementation included "upgrading" an operator by changing its .spec.version to be the "new" version. This implementation does not have that and instead focuses on the coast-to-coast of build, package, deploy. Is it necessary to include the "upgrading" of an operator as part of this e2e test?

Reviewer Checklist

• API Go Documentation
• Tests: Unit Tests (and E2E Tests, if appropriate)
• Comprehensive Commit Messages
• Links to related GitHub Issue(s)

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (7156464) 83.75% compared to head (cc277fc) 83.75%.
Report is 1 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #503   +/-   ##
=======================================
  Coverage   83.75%   83.75%           
=======================================
  Files          23       23           
  Lines         868      868           
=======================================
  Hits          727      727           
  Misses         96       96           
  Partials       45       45           

Flag	Coverage Δ
e2e	65.43% <ø> (+0.34%)	⬆️
unit	78.44% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[m1kola]

Correct me if I'm wrong, but it seems that both #349 and #350 are resolved by this since we no longer use local-registry.

[everettraven]

Correct me if I'm wrong, but it seems that both #349 and #350 are resolved by this since we no longer use local-registry.

Looks like it. I looked through the issues to try and find the ones this resolved and missed these. Will add them as resolved to the PR description, thanks!

[m1kola]

Looks good overall. Just one question

[m1kola]

Re: go-apidiff - we removed a bunch of e2e helper functions which were exported. I think we can ignore this.