Use precalculated set objects in more places.

This is similar to #19385. Some cases were still unoptimized.

I used trace logging (#19457) to identify functions where we were
creating many set objects, and I noticed that these were unncessary.

This is a part of a set of micro-optimizations that improve self check
performance by ~5.5%.