Add script to print GitHub workflow run errors by jonsimantov · Pull Request #1761 · firebase/firebase-cpp-sdk

bot and others added 13 commits

June 27, 2025 20:59
This script allows users to specify a workflow name and branch to find
the latest run. It then identifies any failed jobs within that run and
prints the last N lines of logs for each failed step.

Key features:
- Fetches the most recent workflow run for a given workflow and branch.
- Identifies jobs within the run that have a 'failure' conclusion.
- For each failed job, attempts to identify failed steps and extracts their logs.
- Prints the last 500 lines (configurable) of the log for each failed step.
- Handles various scenarios including successful runs, running workflows,
  and missing data.
- Supports GitHub token via CLI arg, env var, or ~/.github_token.
- Auto-detects repository from git remote, or accepts via CLI args.
This commit updates the `print_workflow_run_errors.py` script:

- Workflow name and branch are now optional arguments:
    - `--workflow` (or `--workflow-name`) defaults to "integration_test.yml".
    - `--branch` defaults to the current Git branch.
- Changed default log lines printed from 500 to 100 (`--log-lines`).
- Added `--all-failed-steps` flag:
    - If false (default), only logs for the first failed step in a job are printed.
    - If true, logs for all failed steps in a job are printed.

These changes provide more flexibility and sensible defaults for common use cases.
This commit introduces a grep-like feature to the
`print_workflow_run_errors.py` script.

New features:
- Added `--grep-pattern` (`-g`) argument to specify an Extended
  Regular Expression (ERE) for searching within fetched logs.
- Added `--grep-context` (`-C`) argument to specify the number of
  lines of context to show around matches (default is 5).

Behavior:
- If a grep pattern is provided, the script will use the system `grep`
  command to filter the logs of failed steps (or the full job log
  if a specific step's log cannot be isolated).
- Output clearly indicates when grep results are shown, the pattern used,
  and the context lines.
- Handles cases where `grep` finds no matches or if the `grep` command
  itself fails (e.g., not found, bad pattern).
- If no grep pattern is provided, the script defaults to its previous
  behavior of printing the last N lines of the log.
This commit refines the grep functionality in the
`print_workflow_run_errors.py` script by updating default values:

- The default `--grep-pattern` is now `"[Ee]rror[: ]"`.
  Logs will be automatically filtered for this pattern if no other
  pattern is specified.
- The default `--grep-context` is now 10 lines.

If an empty string is explicitly passed to `--grep-pattern`
(e.g., `--grep-pattern ""`), grep functionality will be disabled,
and the script will fall back to printing the last N lines of the log.
This commit introduces a new feature to filter jobs by name using a
regular expression in the `print_workflow_run_errors.py` script.

New features:
- Added `--job-pattern` argument, which accepts a regex string.
  Only jobs whose names match this pattern will be processed for failures.
- The default value for `--job-pattern` is `'^build.*'`, meaning by
  default, the script will only look at jobs starting with 'build'.

Behavior:
- Job name filtering is applied before checking for job failures.
- If an invalid regex is provided for `--job-pattern`, the script prints
  an error and exits gracefully.
- This new filter works in conjunction with existing log processing
  options (like `--grep-pattern` and `--all-failed-steps`), which are
  applied to the jobs that pass the name pattern filter and are failed.
This commit updates the default regular expression for the
`--grep-pattern` argument in the `print_workflow_run_errors.py` script.

The default pattern is changed from `"[Ee]rror[: ]"` to
`"[Ee][Rr][Rr][Oo][Rr][: ]"` for more specific matching of "Error"
(case-insensitive) followed by a colon or space.
This commit updates the `print_workflow_run_errors.py` script to format
its standard output using Markdown for improved readability when viewed
in Markdown-aware environments.

Changes include:
- Job and step headings are now formatted as Markdown headers (H1-H4).
- Workflow run and job URLs are presented as clickable Markdown links.
- Actual log content (both grep results and last N lines) is enclosed
  in ```log ... ``` fenced code blocks.
- Horizontal rules (`---`) are used to better separate sections.
- Minor textual adjustments for clarity within the Markdown structure.

The stderr output remains plain text for informational messages.
This commit adds two main enhancements to the
`print_workflow_run_errors.py` script:

1.  **Log Download Status (stderr)**:
    - Prints messages to stderr indicating the progress of log downloads
      (e.g., "INFO: Downloading log X/Y for job 'job_name'...").
    - Prints a summary to stderr after all jobs are processed, showing how
      many logs were successfully fetched and processed (e.g.,
      "INFO: Processed logs for S/T targeted failed jobs.").
    - Includes a warning on stderr if a specific job's log fails to download.

2.  **Timestamp Stripping (stdout)**:
    - Implemented a function `strip_initial_timestamp` that uses a regex
      to remove ISO 8601-like timestamps from the beginning of log lines.
    - This stripping is applied to all log lines (from specific step
      segments or full job log fallbacks) before they are further
      processed by `grep` or printed as the 'last N lines'.
      This makes the logs cleaner and potentially easier for other tools
      or users to parse.
This commit introduces a prioritized, cascading job pattern checking
feature to the `print_workflow_run_errors.py` script.

Key changes:
- The `--job-pattern` argument now uses `action='append'`, allowing users
  to specify multiple patterns. These are checked in the order provided.
- If no `--job-pattern` is specified by the user, a default prioritized
  sequence is used: `['^build.*', '^test.*', '.*']`.
- The script iterates through the determined list of patterns:
    - For the first pattern that matches jobs AND has failures among those
      matched jobs, the script processes and displays logs for those
      failures.
    - It then stops checking subsequent patterns.
- If a pattern results in no matching jobs, or if all matching jobs
  succeeded, the script moves to the next pattern in the sequence.
- Informative messages are printed to stderr indicating which pattern is
  being checked, the outcome for that pattern, and if subsequent
  patterns are skipped.
- The main log processing loop has been refactored into a helper function
  `_process_and_display_logs_for_failed_jobs` for better organization.
- Handles invalid regular expressions within the pattern list by warning
  the user and skipping the invalid pattern.
This commit changes the default value for the `--grep-context`
argument in the `print_workflow_run_errors.py` script back to 5
(from 10).
This commit enhances `print_workflow_run_errors.py` by adding a
`--run-id` command-line option. This allows users to specify a
particular workflow run ID for processing, bypassing the default
behavior of searching for the latest run by workflow name and branch.

Key changes:
- Added `--run-id <ID>` optional argument.
- If `--run-id` is provided, the script fetches details for that specific
  run using a new helper function `get_workflow_run_details_by_id`.
  The `--workflow` and `--branch` arguments are ignored in this mode.
- If `--run-id` is not provided, the script retains its existing behavior
  of using `--workflow` and `--branch` to find the latest run.
- The new helper function includes error handling for invalid or
  non-existent run IDs.
- Standard error messages have been updated to reflect whether the script
  is processing a run by specified ID or by search criteria.
Corrects a NameError in the `main` function when calling
`_process_and_display_logs_for_failed_jobs`. The script was
attempting to use `run.get('html_url')` where it should have been
`run_details.get('html_url')` after a previous refactor.

This fix ensures the correct variable is used, allowing the script
to properly pass the workflow run's HTML URL to the log processing
function.