feat: add System Status page with Celery queue monitoring and purge by valentijnscholten · Pull Request #14349

feat: add System Status page with Celery queue monitoring and purge by valentijnscholten · Pull Request #14349 · DefectDojo/django-DefectDojo

- Add dedicated System Status page (/system_status) with superuser-only
  access, accessible from the navigation menu alongside System Settings
- Display Celery worker liveness, pending queue length with human-readable
  duration formatting, and active task timeout/expiry configuration
- Add Purge queue button that POSTs to the new API endpoint and reloads
  the page on success
- Fix get_celery_worker_status() to use app.control.ping() via the
  pidbox control channel, which works correctly even when the task queue
  is clogged (previously dispatched a task that would never be picked up)
- Add purge_celery_queue() utility using a direct broker connection
- Add two new superuser-only REST API endpoints:
    GET  /api/v2/celery/status/       - worker status, queue length, config
    POST /api/v2/celery/queue/purge/  - purge all pending tasks
  Both use the same permission guards as SystemSettingsViewSet
  (IsSuperUser + DjangoModelPermissions against System_Settings)
- Add DD_CELERY_TASK_TIME_LIMIT (default 12h), DD_CELERY_TASK_SOFT_TIME_LIMIT
  (default disabled), and DD_CELERY_TASK_DEFAULT_EXPIRES (default 12h)
  environment variables to settings.dist.py with explanatory comments
- Move celery status rendering from server-side Django view to client-side
  AJAX so dojo-pro can consume the same API endpoints

feat: add Refresh button next to Purge button on System Status page

remove plan

- Split status display into separate Redis Broker and Celery Worker badges
- Add loading spinners on page load and refresh for all status indicators
- Add per-task queue breakdown table behind a 'View Details' button (O(N)
  scan with warning); shows task name, count, oldest/newest queue position,
  and expiry timestamps with human-readable time-left
- Add per-row purge button to remove all queued tasks by task name
- Add global queue purge and per-task purge API endpoints
- Move Celery settings table into its own panel section

…figurable settings

- Add INFO log on start/finish and DEBUG log per batch in purge_celery_queue_by_task_name
- Cap per-task purge at 10,000 tasks with a WARNING when the cap is hit
- Make batch size and max tasks configurable via DD_CELERY_QUEUE_PURGE_BATCH_SIZE and DD_CELERY_QUEUE_PURGE_MAX_TASKS
- After per-task purge, refresh both status and details table instead of removing the row
- Refresh details table when clicking the Refresh button while details are visible
- Add note below details table referencing the two env variables

…d purge

- Replaced Kombu channel.client with redis.from_url() in both
  get_celery_queue_details() and purge_celery_queue_by_task_name()
  so both functions use the same connection mechanism
- Batched pipeline approach for per-task purge to avoid hitting
  Valkey's max query buffer limit on large queues