refactor Config() and _ENV_CONFIG

Apache Iceberg version

None

Please describe the bug 🐞

Follow up to #3006, i want to see if we can refactor Config() and _ENV_CONFIG
a couple of issues ive seen

  • Config() __init__ reads both env vars and local config files implicitly, this can be a potentially footgun.
  • _ENV_CONFIG is loaded on import time... which calls Config() above and reads env vars and local config files.

#3006 is a fix for pytests. Ideally, I think we can be more explicit here

Willingness to contribute

  • I can contribute a fix for this bug independently
  • I would be willing to contribute a fix for this bug with guidance from the Iceberg community
  • I cannot contribute a fix for this bug at this time