Home Original page

[AFE-53] feat(airflow-client): paginate Get* requests and add short retries by nickypy · Pull Request #2039 · astronomer/astro-cli

Conversation

@nickypy

Description

This PR fixes an issue around copying pools, variables, and connections between deployments:

  • paginate across all resources to make sure everything is copied, not just the first page of the results
  • add retries for scenarios when deployments are still spinning up (e.g. deploy previews)

🎟 Issue(s)

Related #XXX

🧪 Functional Testing

  1. Create a test deployment with >100 pools, variables and connections
  2. Use that to spin up a preview deployment
  3. Make sure that all pools, variables and connections are copied over

📸 Screenshots

Add screenshots to illustrate the validity of these changes.

📋 Checklist

  • Rebased from the main (or release if patching) branch (before testing)
  • Ran make test before taking out of draft
  • Ran make lint before taking out of draft
  • Added/updated applicable tests
  • Tested against Astro-API (if necessary).
  • Tested against Houston-API and Astronomer (if necessary).
  • Communicated to/tagged owners of respective clients potentially impacted by these changes.
  • Updated any related documentation

@coveralls-official

@nickypy nickypy marked this pull request as ready for review

March 18, 2026 19:59

nickstenning

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally this is looking good. Left a few comments and questions. Please also make sure we've run the integration tests :)

nickstenning

nickstenning

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I do think using synctest rather than providing a test seam to override the backoff interval would be more reliable, but maybe I'm missing something?

Co-authored-by: Nick Stenning <nick@whiteink.com>

@nickstenning

Looks great! Thanks for updating Go and switching to synctest 🙇‍♂️

2 participants

@nickypy @nickstenning