FEAT: streaming support in fetchone for varcharmax data type by gargsaumya · Pull Request #219 · microsoft/mssql-python

AI review requested due to automatic review settings

September 3, 2025 12:30

@gargsaumya gargsaumya changed the title FEAT: adding streaming support in fetch for varcharmax type FEAT: streaming support in fetchone for varcharmax data type

Sep 3, 2025

sumitmsft

bewithgaurav

bewithgaurav

@gargsaumya

### Work Item / Issue Reference  
<!-- 
IMPORTANT: Please follow the PR template guidelines below.
For mssql-python maintainers: Insert your ADO Work Item ID below (e.g.
AB#37452)
For external contributors: Insert Github Issue number below (e.g. #149)
Only one reference is required - either GitHub issue OR ADO Work Item.
-->

<!-- mssql-python maintainers: ADO Work Item -->
>
[AB#38110](https://sqlclientdrivers.visualstudio.com/c6d89619-62de-46a0-8b46-70b92a84d85e/_workitems/edit/38110)

[AB#34162](https://sqlclientdrivers.visualstudio.com/c6d89619-62de-46a0-8b46-70b92a84d85e/_workitems/edit/34162)
<!-- External contributors: GitHub Issue -->
> GitHub Issue: #<ISSUE_NUMBER>

-------------------------------------------------------------------
### Summary   
<!-- Insert your summary of changes below. Minimum 10 characters
required. -->
This pull request improves NVARCHAR data handling in the SQL Server
Python bindings and adds comprehensive tests for NVARCHAR(MAX)
scenarios. The main changes include switching to streaming for large
NVARCHAR values, optimizing direct fetch for smaller values, and adding
tests for edge cases and boundaries to ensure correctness.

**NVARCHAR data handling improvements:**

* Updated the logic in `ddbc_bindings.cpp` to use streaming for large
NVARCHAR/NCHAR columns (over 4000 characters or unknown size) and direct
fetch for smaller values, optimizing performance and reliability.
* Refactored data conversion for NVARCHAR fetches, using `std::wstring`
for conversion and simplifying platform-specific handling for both
macOS/Linux and Windows.
* Improved handling of empty strings and NULLs for NVARCHAR columns,
ensuring correct Python types are returned and logging is more
descriptive.

**Testing enhancements:**

* Added new tests in `test_004_cursor.py` for NVARCHAR(MAX) covering
short strings, boundary conditions (4000 chars), streaming (4100+
chars), large values (100,000 chars), empty strings, NULLs, and
transaction rollback scenarios to verify correct behavior across all
edge cases.

**VARCHAR/CHAR fetch improvements:**

* Improved direct fetch logic for small VARCHAR/CHAR columns and fixed
string conversion to use the actual data length, preventing potential
issues with null-termination and buffer size.
[[1]](diffhunk://#diff-dde2297345718ec449a14e7dff91b7bb2342b008ecc071f562233646d71144a1R1825-R1830)
[[2]](diffhunk://#diff-dde2297345718ec449a14e7dff91b7bb2342b008ecc071f562233646d71144a1L1841-L1850)

<!-- 
### PR Title Guide

> For feature requests
FEAT: (short-description)

> For non-feature requests like test case updates, config updates ,
dependency updates etc
CHORE: (short-description) 

> For Fix requests
FIX: (short-description)

> For doc update requests 
DOC: (short-description)

> For Formatting, indentation, or styling update
STYLE: (short-description)

> For Refactor, without any feature changes
REFACTOR: (short-description)

> For release related changes, without any feature changes
RELEASE: #<RELEASE_VERSION> (short-description) 

### Contribution Guidelines

External contributors:
- Create a GitHub issue first:
https://github.com/microsoft/mssql-python/issues/new
- Link the GitHub issue in the "GitHub Issue" section above
- Follow the PR title format and provide a meaningful summary

mssql-python maintainers:
- Create an ADO Work Item following internal processes
- Link the ADO Work Item in the "ADO Work Item" section above  
- Follow the PR title format and provide a meaningful summary
-->

@gargsaumya

@gargsaumya

@gargsaumya

@gargsaumya

sumitmsft

jahnvi480