@grokspawn yeah, I'm not in favor of running a 15 minute job (which will only grow) on a PR basis for something that seems more like stale documentation to me. Don't get me wrong, it matters. I'm just not sure we want to hold up PRs because of it, or run it with this level of frequency. We may want to hold a release back, but PRs, probably not.

The unit tests and e2e suites should already give us pretty good signal that things are working as they should. I don't think we should use demos for that. In any case, the biggest vector for demo failures would probably be changes in API surface, which should occur fairly seldomly. The failure rate of the catalogd-demo GHA is extremely low (and in the failures I've seen, it's not even exactly clear that the RC was something wrong with the demo).

Tbh, thinking a bit more about it, I'd start by making this a non-blocking step of the release pipeline and use fewer resources.

If we feel strongly about running this more frequently than once before release, maybe a good compromise would be to only run it when there are relevant manifest/API changes?

cc @camilamacedo86