Updating Macro to be lazy loaded by Tom-Fynes · Pull Request #5708 · SQLMesh/sqlmesh
Fixes #5692
This pull request refactors the macro evaluation logic in sqlmesh/core/macros.py to introduce lazy loading for Python environments, improve error handling for missing macro dependencies, and enhance code readability. The changes enable projects to share state databases without requiring all external dependencies to be present, and provide clearer error messages when macros fail to load. Additionally, the code has been reformatted for better readability, and error messages have been made more consistent throughout the file.
Macro Environment Loading and Error Handling:
- Introduced lazy loading for Python environment executables, deferring imports and macro definitions until they are actually needed. This allows projects to share state databases without requiring all dependencies to be present at load time. Added
_unloaded_executablesand_failed_importstracking, and refactored macro loading logic into a new_load_python_envmethod. (sqlmesh/core/macros.py) [1] [2] - Enhanced error handling in macro invocation: if a macro fails to load due to a missing dependency, a clear error message is raised, explaining the likely cause and how to proceed. (
sqlmesh/core/macros.py)
Code Readability and Consistency:
- Reformatted code throughout the file for improved readability, including line breaks in long expressions and function arguments, and consistent error message formatting. (
sqlmesh/core/macros.py) [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20]
Error Message Improvements:
- Standardized error messages for missing variables, schema issues, and invalid arguments, making them more descriptive and user-friendly. (
sqlmesh/core/macros.py) [1] [2] [3] [4] [5] [6] [7] [8]
General Cleanup:
- Removed unused import
prepare_envfrom the file. (sqlmesh/core/macros.py)
These changes make the macro evaluation system more robust, maintainable, and easier for users to debug when issues arise.