Home Original page

[CU-86b7p1xgv] Add --local-federated flag to bypass Explorer federation bottleneck by angelo-DNAStack · Pull Request #130 · DNAstack/dnastack-client

@angelo-DNAStack @claude 

…on bottleneck

When concurrent requests to federated questions overwhelm the Explorer service,
circuit breakers and connection timeouts cause failures. This happens because
Explorer's federation layer becomes a bottleneck when coordinating responses
from multiple publishers with queuing systems.

This change adds a --local-federated flag that queries each collection directly
in parallel, bypassing the Explorer federation layer. This allows workloads to
proceed while the long-term solution (converting Explorer's federated API to
use Data Connect-style streaming responses) is implemented.

The flag maintains full compatibility with existing behavior and output format,
making it a safe workaround for high-concurrency scenarios.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

and others added 5 commits

December 3, 2025 10:24
… correctly

The initial implementation only returned the first page of results from each
collection, which would miss data when results span multiple pages. This is
critical because Data Connect APIs return empty pages with next_page_url links
until data is ready, then continue pagination until all results are delivered.

This fix implements proper pagination handling by:
- Making the initial POST request to start the query
- Following next_page_url links with GET requests until exhausted
- Aggregating all data across all pages before returning
- Preventing infinite loops by tracking visited URLs

Without this fix, users would get incomplete results when querying collections
that return paginated responses, making the --local-federated flag unreliable
for production use.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
…eature

- Add 29 tests across LocalFederatedQuestionQueryResultLoader, ExplorerClient, and CLI
- Test Data Connect pagination handling, parallel collection queries, error scenarios
- Test CLI --local-federated flag integration and parameter parsing
- Fix ClientError constructor to use trace_context parameter
- Ensure compatibility with existing utilities and result formats

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
…lag-to-explorer-questions-CU-86b7p1xgv
…lag-to-explorer-questions-CU-86b7p1xgv
…lag-to-explorer-questions-CU-86b7p1xgv