pref: Optimize xrefmap download performance by filzrev · Pull Request #9637 · dotnet/docfx

This PR is intended to resolve #9558

What's included in this PR

  • Add settings to use HTTP Compression (if available)
  • Change HttpClient's timeout to 30 minutes (Default: 100 seconds)
  • Change StreamReader's buffer size to 80KB (Default: 1024 byte)
    • 80KB is default buffer size that used by Stream::CopyTo.

What's tested

I've manually tested xrefmap download with following configuration.

  "build": {
    "xref": [
      "https://learn.microsoft.com/en-us/dotnet/.xrefmap.json"
    ]
  }

And confirmed server returns following HTTP Response headers. And download size is reduced. (316MB -> 16MB)

HTTP/1.1 200 OK
Content-Type: application/json
Content-Length: 15978441
Content-Encoding: gzip

TODO Tasks
I'll create another PR later relating #9558.

  • Add special deserialization code path for JSON file.
    (Because YamlDotNet based JsonDeserializer is 5x slower than System.Text.Json based deserialization)
  • Add support for loading xrefmap compressed by gzip (xrefmap.json.gz) from local disk.