feat(parsers): add Qualys VMDR CSV parser by skywalke34 · Pull Request #14453 · DefectDojo/django-DefectDojo

Design document for new Qualys VMDR parser supporting QID and CVE
CSV export formats. Includes field mappings, architecture decisions,
and test strategy.

Authored by T. Walker - DefectDojo
Detailed TDD implementation plan with 13 tasks covering:
- Package structure and test files
- helpers.py, qid_parser.py, cve_parser.py, parser.py
- Comprehensive test coverage
- Documentation following enhanced format structure

Authored by T. Walker - DefectDojo
Authored by T. Walker - DefectDojo
Authored by T. Walker - DefectDojo
Authored by T. Walker - DefectDojo
TDD: Tests written before implementation.

Authored by T. Walker - DefectDojo
Shared utilities for severity mapping, date parsing, description
building, endpoint parsing, and tag handling.

Authored by T. Walker - DefectDojo
Parses QID-centric CSV exports from Qualys VMDR.

Authored by T. Walker - DefectDojo
Parses CVE-centric CSV exports with CVSS scores from NVD.

Authored by T. Walker - DefectDojo
Auto-detects QID vs CVE format and delegates to appropriate parser.

Authored by T. Walker - DefectDojo
Comprehensive tests for severity mapping, endpoints, tags, CVE fields.
Also fixed CSV test files to use standard format and updated parser
format detection for proper CVE format recognition.

Authored by T. Walker - DefectDojo
Includes field mapping tables, severity conversion, and processing notes.

Authored by T. Walker - DefectDojo
The Qualys VMDR export uses a non-standard CSV format where fields are
delimited by ,"" instead of the standard "," format. This caused the
parser to fail when processing real Qualys exports.

Changes:
- Add format detection to distinguish standard vs non-standard CSV
- Add custom parsing functions for non-standard Qualys format
- Handle multi-line records with embedded newlines
- Both parsers (QID and CVE) now use the unified parsing logic

The parser now correctly handles both test files (standard CSV) and
real Qualys exports (non-standard format).

Authored by T. Walker - DefectDojo
The previous parsing logic used simple string splitting on ,"" which
failed when field values contained escaped quotes (""""). This caused
field misalignment and empty/default values in parsed findings.

The fix:
1. Remove outer quotes from the row
2. Unescape row-level quote doubling ("" -> ")
3. Parse the result as standard CSV using Python's csv module

This correctly handles fields containing embedded quotes like:
  "Description with ""quoted text"" inside"

Authored by T. Walker - DefectDojo
The previous end-of-record detection incorrectly treated any line ending
with a single quote as a complete record. This caused multi-line records
(where Results field contains embedded newlines) to be split incorrectly.

In Qualys non-standard format, multi-field records always end with """
(the last field's closing "" plus the record's closing "). Single quote
endings within a record are just field content, not record terminators.

Authored by T. Walker - DefectDojo
Map the CVE field to unsaved_vulnerability_ids so it appears in the
Vulnerability IDs column in DefectDojo, in addition to vuln_id_from_tool.

Authored by T. Walker - DefectDojo
Add documentation that CVE is mapped to both vuln_id_from_tool and
unsaved_vulnerability_ids for proper CVE tracking in DefectDojo.

Authored by T. Walker - DefectDojo
…ction

Replace field-count-based record boundary detection in the Qualys VMDR
nonstandard CSV parser with a trailing-quote heuristic. The old approach
re-parsed accumulated rows each iteration and failed on malformed quote
patterns (e.g. #table cols=""3"") that produce incorrect field counts.

The new _is_record_end_line() helper counts trailing quotes: exactly 3
means record end, 4+ means record end only if preceded by a comma
(empty field). This is O(1) per line and correctly handles all known
Qualys export patterns. Also fixes pre-existing ruff lint issues in the
state machine parser.

Authored by T. Walker - DefectDojo
These design/plan files were used during development and should not
be included in the upstream PR.

Authored by T. Walker - DefectDojo
…R docs

Document the non-standard CSV format, multi-line record support,
metadata line detection, HTML stripping, and null marker filtering.

Authored by T. Walker - DefectDojo