FOUR-20929: Implement ETag Caching by eiresendez · Pull Request #7892 · ProcessMaker/processmaker

and others added 25 commits

November 26, 2024 14:09
- Introduced IfNoneMatch middleware to handle ETag-based caching:
  - Supports safe HTTP methods (GET, HEAD).
  - Returns "304 Not Modified" for matching ETags.
  - Handles multiple ETags and restores original request methods.

- Added EtagManager for flexible ETag generation:
  - Default ETag generation uses MD5 hash.
  - Supports custom hash algorithms like SHA-256 via callbacks.

- Included test cases:
  - Validate default ETag generation.
  - Test custom callback logic for ETag creation.

This commit adds foundational support for ETag-based caching in the application.
…ware

- Merged `etag.set` and `etag.if-none-match` middlewares into a single `etag.handle` middleware.
- Simplified logic to reduce redundancy and improve maintainability.
- Ensured ETag validation (`If-None-Match`) and generation are handled in the same flow.
- Preserved compatibility with HEAD requests for consistency.

This refactor improves clarity, reduces potential misconfigurations, and keeps the ETag logic centralized.
…d tests

- Created tests for the new `HandleEtag` middleware:
  - Validates ETag generation and correctness.
  - Tests responses for both matching and non-matching `If-None-Match` headers.
  - Ensures proper handling of weak ETags (`W/`).
- Removed old tests for `SetEtag` and `IfNoneMatch` middlewares as they are no longer needed.

This commit improves test clarity and ensures the new ETag middleware behaves as expected.
- Added a test to validate ETag generation for user-specific routes using `etag:user`.
  - Simulates an authenticated user and verifies the ETag includes the user ID.

These tests ensure the ETag middleware behaves correctly for both user-specific and common routes.
FOUR-20933: Implement ETag Caching for Screens Data
…on to prevent ETag inconsistencies
- Process only GET and HEAD methods to ensure middleware relevance and avoid unnecessary processing for non-cacheable HTTP methods.
- Add a check to determine if the response is cacheable, filtering out non-cacheable responses (e.g., those with 'no-store' directive or non-cacheable status codes).
- Default ETag generation now includes user-specific data (auth()->id()) to enforce personalized caching by default.
- Removed 'scope' and 'includeUser' logic for simplified and consistent caching behavior.
- Refactored `EtagManager` to support dynamic ETag generation based on configurable sources (`updated_at`).
- Introduced `generateEtagFromTables` with a `source` parameter for flexibility in determining the source of truth.

This update prepares the app for future scalability and allows switching between different ETag generation strategies.
- Applied ETag middleware to the 'startProcesses' route for improved caching and reduced payload size.
- Added default 'etag_tables' parameter set to 'processes' to optimize ETag generation for this route.
FOUR-20944: Implement ETag Caching for Task and Case Data
- Includes an overview of ETag functionality and benefits.
- Provides implementation details for the HandleEtag middleware.
- Covers ETag generation based on tables and response content.
- Adds examples for conditional responses and route configuration.
- Documents unit and manual testing approaches for ETag functionality.
- Suggests future improvements for ETag caching and metrics collection.
- Introduced a new test class `HandleEtagCacheInvalidationTest` to verify ETag behavior upon database updates.
- Implemented tests to ensure ETag changes when the underlying data is modified and that the correct ETag is returned for subsequent requests.
- Updated existing `HandleEtagTest` to include a test for returning 304 Not Modified when the ETag matches the client-provided value.
- Introduced logging for highly dynamic endpoints by tracking ETag history for each endpoint.
- Implemented a mechanism to limit the number of tracked ETags and log when all tracked ETags are unique.
- Updated documentation to reflect the new logging feature and its implications for performance optimization.
- Integrated `config/etag.php` for dynamic configuration of ETag functionality.
- Added `enabled` and `log_dynamic_endpoints` flags to control feature behavior.

This update improves flexibility and allows disabling ETag processing entirely when `enabled` is set to false.
- Added custom Trend metrics to measure and compare durations of 200 OK and 304 Not Modified responses
- Validates that 304 responses are faster using If-None-Match header
- Improved test clarity by focusing on ETag performance under load
- Implemented tracking of ETag values for specified endpoints
- Added logic to identify dynamic endpoints when ETag history shows diff values
FOUR-20954: Monitor ETag Caching Performance

estebangallego

@eiresendez