feat: add standalone mode for Docker-free local development by jlaneve · Pull Request #2012 · astronomer/astro-cli

@jlaneve @claude

…pment

Add a new `astro dev standalone` command that runs Airflow locally
without Docker, using `airflow standalone` and `uv` for dependency
management. This provides a dramatically faster dev loop for Airflow 3
projects.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

@pre-commit-ci

Standalone mode now works with both Airflow 2 (runtime 4.0.0+) and
Airflow 3 (runtime 3.x). The health check endpoint, image registry,
and settings version are determined dynamically based on the detected
Airflow major version.

Changes:
- Accept airflowMajor "2" or "3" (was "3" only)
- Health check: /health + webserver (AF2) vs /api/v2/monitor/health + api-server (AF3)
- Image registry: quay.io/astronomer/astro-runtime (AF2) vs astrocrpublic.azurecr.io/runtime (AF3)
- Settings version passed dynamically (was hardcoded 3)
- Kill/reset cleans up both AF2 and AF3 credential files

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

@pre-commit-ci

Default `astro dev standalone` to background mode — the CLI starts the
airflow process, writes a PID file, waits for the health check, prints
status, and returns.  A `--foreground` flag preserves the previous
stream-to-terminal behaviour.

New subcommands:
  - `astro dev standalone stop`  — SIGTERM the process group, clean up PID file
  - `astro dev standalone logs [-f]` — dump or tail the log file

Also wires `reset` to stop a running process before cleaning up files.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

jedcunningham

@jlaneve @claude

…install

- Remove airflowMajor field and AF2 code paths (standalone is AF3-only)
- Replace Docker-based constraint extraction with HTTP fetch from
  pip.astronomer.io/runtime-constraints
- Implement 2-step install: first install airflow with full constraints,
  then install user requirements with only airflow/task-sdk version locks
- Add parsePackageVersionFromConstraints helper for task-sdk version
- Remove runtimeImageName, execDockerRun, constraintsFileInImage
- Simplify healthEndpoint to always return AF3 endpoint

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…an project root

- Fix CDN base URLs: constraintsBaseURL → cdn.astronomer.io/runtime-constraints
- Add freezeBaseURL (cdn.astronomer.io/runtime-freeze) for full 191-package list
- getConstraints() now fetches and caches both constraints + freeze files;
  returns freeze path for use as pip -c arg in step 1 install
- Move "already running" check to top of Start() before any install work
- Add ensureCredentials() to seed passwords file with admin:admin on first run
- Add readCredentials() to display username/password in startup output
- Redirect AIRFLOW_HOME → .astro/standalone/ so airflow.cfg, airflow.db,
  and logs/ all live there instead of cluttering the project root
- Set AIRFLOW__CORE__SIMPLE_AUTH_MANAGER_PASSWORDS_FILE to .astro/standalone/
- Update Kill() to clean up .venv/ and .astro/standalone/ only
- Update tests: freeze file routing in fetch mock, AIRFLOW_HOME assertions,
  new TestStandaloneEnsureCredentials, TestStandaloneReadCredentials tests

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add Standalone.Build() stub (errStandaloneNotSupported) to satisfy
  the updated ContainerHandler interface which gained a Build method in main
- Resolve conflict in cmd/airflow_test.go: keep both TestAirflowStandalone
  and new TestAirflowBuild from main, preserving all subtests from both

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Rename the command from 'standalone' to 'local' for a more intuitive UX.
The internal Go types (Standalone, StandaloneHandlerInit) are unchanged.

- cmd/airflow.go: Use:"standalone" → Use:"local", all function/var names
- cmd/airflow_hooks.go: EnsureStandaloneRuntime → EnsureLocalRuntime
- cmd/airflow_test.go: update all test names and assertions
- airflow/standalone.go: add Build() stub (satisfies updated ContainerHandler)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

@jlaneve jlaneve changed the title feat: add astro dev standalone for Docker-free local development feat: add astro dev local for Docker-free local development

Feb 19, 2026
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Parse the optional -python-X.Y suffix from runtime image tags
(e.g. 3.1-12-python-3.11) instead of hardcoding Python 3.12.
Falls back to 3.12 (the default for all Runtime 3.x images)
when no suffix is present.

Cache filenames now include the Python version to avoid stale
lookups when switching between Python versions.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

@jlaneve

…lution

- Fix standaloneExecAirflowCommand using non-existent .venv/bin/bash;
  use system bash instead (venv PATH is already set in env)
- Fix .env file overriding standalone-critical vars (AIRFLOW_HOME, etc.);
  .env is now applied first, then critical settings override on top
- Add 3-tier Python version resolution: Dockerfile tag → runtime JSON → fallback 3.12
- Add DefaultPythonVersion/PythonVersions fields to RuntimeVersionMetadata
- Add GetDefaultPythonVersion() to look up Python version from updates.astronomer.io

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…s subcommand

- Remove redundant "already running" check in startBackground (Start already guards it)
- Strip matching quotes from .env values to match Docker Compose behavior
- Add --port flag and config.CFG.APIServerPort fallback for custom webserver port
- Add `astro dev local ps` subcommand to check process status
- Make readCredentials deterministic by preferring the admin user

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Remove comments that restate the code, duplicate doc comments from
elsewhere, or are stale context from incremental development. Keep
numbered steps in Start() and comments that explain non-obvious "why".

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
config.CFG.APIServerPort.GetString() returns "0" (not "") when the port
is not configured in .astro/config.yaml. This caused the health check to
hit localhost:0 and time out. Add explicit check for "0" in webserverPort().

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Settings import runs airflow CLI commands (connections list, variables
list) that require the database to be initialized. Previously settings
were applied before `airflow standalone` was started, causing the DB to
not exist yet. Move applySettings to after the health check passes in
both foreground and background modes.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

@jlaneve jlaneve marked this pull request as ready for review

February 24, 2026 19:58

@jlaneve @claude

Replace the separate `astro dev local` command tree with a unified
mode-based approach. A `dev.mode` project config ("docker" or
"standalone") controls which backend is used, with `--standalone` and
`--docker` persistent flags for per-call overrides. All existing
commands (start, stop, logs, ps, kill, restart) now work in both modes.

- Add `dev.mode` config with "docker" default and validator
- Add `--standalone`/`--docker` mutually exclusive persistent flags
- Add resolveDevMode/resolveHandlerInit for mode-based handler dispatch
- Update all pre-run hooks to skip Docker in standalone mode
- Add --foreground/-f and --port/-p flags to start and restart
- Remove `astro dev local` command tree entirely
- Update user-facing strings from `astro dev local` to `astro dev`
- Update command descriptions to be mode-agnostic
- build/upgrade-test always use Docker regardless of mode

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

@jlaneve jlaneve changed the title feat: add astro dev local for Docker-free local development feat: add standalone mode for Docker-free local development

Feb 24, 2026
Instead of scattering isStandaloneMode() checks in every hook,
assign a noOpContainerRuntime in standalone mode so containerRuntime
is never nil. Downstream hooks work unchanged — the no-op methods
just return nil. Eliminates a nil-pointer landmine for future contributors.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add a custom usage template that splits flags into Common, Docker Mode,
and Standalone Mode sections using cobra flag annotations. Applied to
the start and restart commands which have mode-specific flags. Other
commands keep the default template.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…, parse, settings)

Implement 6 ContainerHandler methods on the Standalone struct that
previously returned errStandaloneNotSupported:

- Run: executes arbitrary commands in the venv environment
- Bash: opens an interactive shell with venv env (AIRFLOW_HOME, PATH)
- ImportSettings: imports connections/variables/pools from settings file
- ExportSettings: exports to settings file or .env file
- Pytest: runs pytest on DAGs with file path resolution and args passthrough
- Parse: validates DAGs by running the default integrity test

Add ensureVenv() helper to validate the venv exists before running
commands, and resolveInEnvPath() to look up binaries in the env's PATH
(needed because exec.Command resolves using the parent process's PATH,
not cmd.Env).

Update error messages for Build, ComposeExport, and UpgradeTest to be
more descriptive about why they're unavailable in standalone mode.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Map the existing --scheduler, --triggerer, --api-server, --webserver,
and --dag-processor flags to standalone log prefixes so users can filter
log output by component. The --webserver flag maps to api-server since
AF3 standalone has no separate webserver process.

When no flags are passed, all lines are shown (including standalone
orchestrator startup messages). When any component filter is active,
only matching lines appear.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
… mode dispatch

Pre-flight check dials localhost:<port> before starting to prevent
silently connecting to the wrong Airflow instance. Also routes build
and upgrade-test commands through resolveHandlerInit() so they properly
error in standalone mode, and fixes the custom port env var to use the
AF3 config key (AIRFLOW__API__PORT).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

jeremybeard

Move standalone mode implementation and tests to standalone.go and
standalone_test.go to better reflect their contents.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Floating tags like "3.1" (without the -Z patch) are now resolved to
the latest pinned version (e.g., "3.1-12") via the runtime versions
JSON from updates.astronomer.io. Unrecognizable tags (custom images)
get a clear error explaining that a pinned runtime image is required.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Match Docker mode behavior by setting SIMPLE_AUTH_MANAGER_ALL_ADMINS=True
instead of requiring login with seeded credentials.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Match Docker image build log behavior — hide uv install output by
default and only show it when --verbose / debug logging is active.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
# Conflicts:
#	config/config.go
#	config/types.go
The LocalExecutor communicates with the api-server via the execution API.
Without this, custom ports cause Connection refused errors because the
executor defaults to localhost:8080 while the api-server is elsewhere.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ions in standalone mode

Add an experimental notice when starting standalone mode. Fix a goroutine
leak in startForeground where the signal handler blocked forever on normal
exit. Replace concrete type assertions (containerHandler.(*Standalone))
with a SetStartOpts interface method so the cmd layer no longer depends on
the Standalone concrete type.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Python's _scproxy calls SCDynamicStoreCopyProxies which is not fork-safe.
When Airflow's LocalExecutor forks, this can spin at 100% CPU indefinitely.
Setting NO_PROXY=* tells Python to skip _scproxy entirely. We only do this
when no proxy is configured (env vars, .env file, or macOS system settings)
so corporate proxy users aren't affected.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

jeremybeard

@jlaneve @claude

…s struct

Fold the 8 positional Start parameters and the separate SetStartOpts
call into a single StartOptions struct, reducing the interface surface
and eliminating the two-step setup pattern.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

jeremybeard

@jlaneve jlaneve deleted the feat/standalone-mode branch

March 3, 2026 14:38