[Config Registry] 1/5 Add gitlab step and json configuration file and yaml for doc by anna-git · Pull Request #7548 · DataDog/dd-trace-dotnet
anna-git
changed the title
[Config Inversion] Add gitlab step and json configuration file
[Config Registry] Add gitlab step and json configuration file
This was referenced
Oct 8, 2025
anna-git
marked this pull request as ready for review
anna-git
changed the title
[Config Registry] Add gitlab step and json configuration file and yaml for doc
[Config Registry] 1/5 Add gitlab step and json configuration file and yaml for doc
anna-git
deleted the
anna/config-inversion-supported-config-file
branch
anna-git added a commit that referenced this pull request
Oct 30, 2025… keys (#7688) ## Context Part of **Configuration Inversion (Step 2)** - Stack progress: 1. [#7548](#7548) - Add GitLab step and JSON configuration file 2. **→ [#7688](#7688) - [Config registry] Reorganize keys, split config keys and platform keys(this PR)** 3. [#7698](#7698) - Generate ConfigurationKeys with source generator 4. [#7689](#7689) - Aliases handling and analyzers 5. [#7697](#7697) - Replace manual ConfigurationKeys by generated ones in the whole solution ## Summary Separates platform-specific environment variables into `PlatformKeys` class and adds Roslyn analyzers to enforce proper usage of configuration keys throughout the codebase. ## Changes **Platform Keys Separation:** - Created `PlatformKeys` class for external platform environment variables (AWS, Azure, GCP, Kubernetes, CLR Profiler), those that don't start with DD_ and come from other platforms. - Updated all references throughout codebase to use `PlatformKeys` for platform-specific variables - Keeps `ConfigurationKeys` focused only on Datadog configuration **Roslyn Analyzers:** - **DD0007**: Platform key analyzer - Enforces use of `PlatformKeys` for external platform environment variables - **DD0008**: ConfigurationBuilder analyzer - Enforces use of `ConfigurationKeys` or `PlatformKeys` constants in `ConfigurationBuilder.WithKeys()` calls - Prevents hardcoded strings and ensures type safety **Configuration Mapping:** - Added `configuration_keys_mapping.json` to map environment variable names to their `ConfigurationKeys` constant names, so that we avoid renaming hundreds of variables according to automatically source generated deducted ones. - Enables preservation of original constant names during refactoring - Supports source generator integration **Code Updates:** - Updated `TracerSettings`, test files, and profiler code to use `PlatformKeys` - Replaced manual `ConfigurationKeys` with generated version from `supported-configurations.json` - Added missing `DD_TRACE_LOG_PATH` to configuration registry ## Motivation Ensures clean separation between Datadog and platform configuration by: - Preventing accidental mixing of platform and Datadog keys - Enforcing compile-time validation of configuration key usage - Making it clear which keys are owned by Datadog vs external platforms - Catching configuration errors at build time instead of runtime ## Validation - Roslyn analyzers run on every build - All existing tests pass with updated key references - Analyzers report clear diagnostic messages for violations ## Related Work This PR builds on the configuration registry foundation established in #7548: - **Commit `2639cf0`** added `supported-configurations.json` with ~280 keys, aliases, deprecations, and GitLab CI validation - **Commit `70e632e`** added [supported-configurations-docs.yaml](cci:7://file:///Users/anna.yafi/go/src/github.com/DataDog/dd-trace-dotnet3/tracer/src/Datadog.Trace/Configuration/supported-configurations-docs.yaml:0:0-0:0) with XML documentation for automated doc generation
anna-git added a commit that referenced this pull request
Nov 26, 2025…#7698) ## Context Part of **Configuration Inversion (Step 3)** - Stack progress: 1. [#7548](#7548) - Add GitLab step and JSON configuration file 2. [#7688](#7688) - Cleanup configuration / platform keys + analyzers 3. **→ [#7698](#7698) - Generate ConfigurationKeys with source generator (this PR)** 4. [#7689](#7689) - Aliases handling via source generator 5. [#7697](#7697) - Replace manual ConfigurationKeys by generated ones in the whole solution ## Summary Adds source generator to auto-generate `ConfigurationKeys` from `supported-configurations.json` with name mapping to preserve existing constant names. ## Changes **Source Generator:** - [ConfigurationKeysGenerator](cci:7://file:///var/folders/4j/q1f1tvc503g_thnq6p79cp500000gp/T/SourceGeneratedDocuments/D0E9C9C62821E2CB6FADE080/Datadog.Trace.SourceGenerators/ConfigurationKeysGenerator:0:0-0:0) reads `supported-configurations.json` and [supported-configurations-docs.yaml](cci:7://file:///Users/anna.yafi/go/src/github.com/DataDog/dd-trace-dotnet3/tracer/src/Datadog.Trace/Configuration/supported-configurations-docs.yaml:0:0-0:0) - Auto-generates `ConfigurationKeys` with nested product classes (AppSec, CIVisibility, OpenTelemetry, etc.) - Generates XML documentation and `[Obsolete]` attributes from JSON **Configuration Mapping:** - `configuration_keys_mapping.json` maps env vars to original constant names (e.g., `DD_TRACE_ENABLED` → `TraceEnabled`) - Avoids refactoring hundreds of references across codebase **Updates:** - Added missing `DD_TRACE_ACTIVITY_LISTENER_ENABLED` key - Committed temporary [ConfigurationKeys2.g.cs](cci:7://file:///Users/anna.yafi/go/src/github.com/DataDog/dd-trace-dotnet3/tracer/src/Datadog.Trace/Generated/net461/Datadog.Trace.SourceGenerators/ConfigurationKeysGenerator/ConfigurationKeys2.g.cs:0:0-0:0) files for validation ## Motivation Makes `supported-configurations.json` the single source of truth while preserving existing constant names to avoid massive refactoring. ## Test Coverage - Added `ConfigurationKeysGeneratorTests.cs` with comprehensive unit tests - Tests validate generator output, XML documentation generation, and obsolete attribute application - Tests cover nested product classes, deprecations, and YAML documentation integration - All existing tests pass without modification ## Related Work Builds on #7548 (configuration registry) and #7688 (PlatformKeys separation + analyzers).
This was referenced
Dec 4, 2025anna-git added a commit that referenced this pull request
Dec 9, 2025…any string key (#7689) ## Context Part of **Configuration Inversion (Step 4)** - Stack progress: 1. [#7548](#7548) - Add GitLab step and JSON configuration file 2. [#7688](#7688) - Cleanup configuration / platform keys + analyzers 3. [#7698](#7698) - Source generator for ConfigurationKeys 4. **→ [#7689](#7689) - Aliases handling and analyzers (this PR)** 5. [#7931](#7931) - Replace manual ConfigurationKeys with generated version 6. #[7932](#7932 Forbid use of System.Environment methods and adapt everywhere ## Summary Adds source generator for configuration key aliases, integrates alias resolution into `ConfigurationBuilder`, and adds Roslyn analyzers to enforce proper configuration key usage. ## Changes **Alias Source Generator:** - `ConfigKeyAliasesSwitcherGenerator` reads aliases from `supported-configurations.json` - Auto-generates switch statements to resolve primary keys from aliases - Generated for all target frameworks (net461, netstandard2.0, netcoreapp3.1, net6.0) **ConfigurationBuilder Improvements:** - Integrated alias resolution directly into `ConfigurationBuilder.WithKeys()` - Removed manual fallback overloads throughout codebase - Added `GetKeyWithAlias()` method to `IConfigurationSource` interface **IntegrationSettings Special Handling:** - Updated to use pattern-based key construction (e.g., `DD_TRACE_{INTEGRATION}_ENABLED`) - Simplified configuration reading by leveraging alias system **Roslyn Analyzers:** - **DD0007**: `PlatformKeysAnalyzer` - Enforces use of `PlatformKeys` for external platform environment variables - **DD0008**: `ConfigurationBuilderWithKeysAnalyzer` - Enforces use of [ConfigurationKeys](cci:2://file:///Users/anna.yafi/go/src/github.com/DataDog/dd-trace-dotnet3/tracer/src/Datadog.Trace/Configuration/ConfigurationKeys.Logging.cs:9:0-58:1)/`PlatformKeys` constants in `ConfigurationBuilder.WithKeys()` calls **Special Cases:** - Fixed `DatadogLoggingFactory` to handle deprecated `DD_TRACE_LOG_PATH` with proper analyzer suppression - Updated configuration tests to work with alias resolution ## Motivation Completes configuration inversion by: - Auto-generating alias resolution from `supported-configurations.json` - Eliminating manual fallback chains - Enforcing compile-time validation of configuration keys - Preventing hardcoded strings and typos ## Test Coverage - Added `ConfigKeyAliasesSwitcherGeneratorTests` with comprehensive test coverage - Added `PlatformKeysAnalyzerTests` with 146 lines of tests - Added `ConfigurationBuilderWithKeysAnalyzerTests` with 538 lines of tests - Updated existing configuration tests for alias support - All tests pass with new alias system ## Related Work Builds on #7548 (configuration registry), #7688 (PlatformKeys separation), and #7698 (source generator for ConfigurationKeys).
anna-git added a commit that referenced this pull request
Dec 10, 2025…nes in the whole solution (#7931) (This PR is actually #7697 that accidentally got closed) ## Context Part of **Configuration Inversion (Step 5)** - Stack progress: 1. [#7548](#7548) - Add GitLab step and JSON configuration file 2. [#7688](#7688) - Cleanup configuration / platform keys + source generator 3. [#7698](#7698) - Aliases handling via source generator 4. [#7689](#7689) - Analyzers for platform and ConfigurationBuilder 5. **→ [#7931](#7697) - Replace manual ConfigurationKeys by generated ones in the whole solution (this PR)** 6. [#7932](#7932) - Forbid use of System.Environment methods and adapt everywhere 7. [#7937](#7937) - Integration names to generated keys ## Summary of changes Fixed the [ConfigurationKeysGenerator](cci:2://file:///Users/anna.yafi/go/src/github.com/DataDog/dd-trace-dotnet3/tracer/src/Datadog.Trace.SourceGenerators/Configuration/ConfigurationKeysGenerator.cs:21:0-958:1) to properly read and apply the [configuration_keys_mapping.json](cci:7://file:///Users/anna.yafi/go/src/github.com/DataDog/dd-trace-dotnet3/tracer/src/Datadog.Trace/Configuration/configuration_keys_mapping.json:0:0-0:0) file, and extracted common file header comments to a reusable constant. **Key changes:** - Fixed JSON array extraction in [ParseMappingFile](cci:1://file:///Users/anna.yafi/go/src/github.com/DataDog/dd-trace-dotnet3/tracer/src/Datadog.Trace.SourceGenerators/Configuration/ConfigurationKeysGenerator.cs:350:4-524:5) method - the generator was incorrectly trying to extract the `"mappings"` field as an object instead of an array - Extracted configuration generator comments to `Constants.ConfigurationGeneratorComment` for reuse across multiple generators - Updated both [ConfigurationKeysGenerator](cci:2://file:///Users/anna.yafi/go/src/github.com/DataDog/dd-trace-dotnet3/tracer/src/Datadog.Trace.SourceGenerators/Configuration/ConfigurationKeysGenerator.cs:21:0-958:1) and [ConfigKeyAliasesSwitcherGenerator](cci:2://file:///Users/anna.yafi/go/src/github.com/DataDog/dd-trace-dotnet3/tracer/src/Datadog.Trace.SourceGenerators/Configuration/ConfigKeyAliasesSwitcherGenerator.cs:24:0-340:1) to use the shared constant - Added documentation as to how to add a key now ## Reason for change The [configuration_keys_mapping.json](cci:7://file:///Users/anna.yafi/go/src/github.com/DataDog/dd-trace-dotnet3/tracer/src/Datadog.Trace/Configuration/configuration_keys_mapping.json:0:0-0:0) file was being ignored during code generation, causing the generator to produce incorrect constant names. This meant that any manual edits to constant names in the mapping file were not being respected, and the generated code would use auto-generated names instead of the explicitly mapped ones. Additionally, the file header comments explaining that files are auto-generated were duplicated across generators, violating DRY principles. ## Implementation details 1. **Fixed array extraction logic**: Replaced the call to [JsonReader.ExtractJsonObjectSection()](cci:1://file:///Users/anna.yafi/go/src/github.com/DataDog/dd-trace-dotnet3/tracer/src/Datadog.Trace.SourceGenerators/Configuration/JsonReader.cs:16:4-91:5) with custom array extraction code that: - Finds the `"mappings":` key in the JSON - Locates the opening `[` bracket - Tracks bracket nesting to find the matching closing `]` - Extracts the complete array content 2. **Resolved variable scope issue**: Reused existing `inString` and `escapeNext` variables from the outer scope instead of redeclaring them, fixing compilation errors. 3. **Centralized header comments**: Created `Constants.ConfigurationGeneratorComment` containing the standardized auto-generation notice and updated both generators to use it. ## Test coverage - Verified the generator builds successfully without errors - The mapping file is now properly parsed and applied during code generation - Generated constant names now match the mappings defined in [configuration_keys_mapping.json](cci:7://file:///Users/anna.yafi/go/src/github.com/DataDog/dd-trace-dotnet3/tracer/src/Datadog.Trace/Configuration/configuration_keys_mapping.json:0:0-0:0) ## Other details This fix ensures that the explicit naming conventions defined in [configuration_keys_mapping.json](cci:7://file:///Users/anna.yafi/go/src/github.com/DataDog/dd-trace-dotnet3/tracer/src/Datadog.Trace/Configuration/configuration_keys_mapping.json:0:0-0:0) are respected, maintaining consistency with the existing codebase and preventing future confusion when constant names don't match their expected values. --------- Co-authored-by: Andrew Lock <andrew.lock@datadoghq.com>
anna-git added a commit that referenced this pull request
Jan 20, 2026…apt everywhere (#7932) ## Context Part of **Configuration Inversion (Step 6)** - Stack progress: 1. [#7548](#7548) - Add GitLab step and JSON configuration file 2. [#7688](#7688) - Cleanup configuration / platform keys + source generator 3. [#7698](#7698) - Aliases handling via source generator 4. [#7689](#7689) - Analyzers for platform and ConfigurationBuilder 5. [#7931](#7931) - Replace manual ConfigurationKeys by generated ones in the whole solution 6. **→ [#7932](#7932) - Forbid use of System.Environment methods and adapt everywhere (this PR)** 7. [#7937](#7937) - Integration names to generated keys I'll update the PR summary to mention the YAML documentation file: ## Summary of changes Banned direct `System.Environment.GetEnvironmentVariable()` usage and migrated all environment variable access to use `EnvironmentHelpers` with strongly-typed [ConfigurationKeys](cci:2://file:///Users/anna.yafi/go/src/github.com/DataDog/dd-trace-dotnet3/tracer/src/Datadog.Trace/Generated/net461/Datadog.Trace.SourceGenerators/ConfigurationKeysGenerator/ConfigurationKeys.g.cs:15:0-737:1) and `PlatformKeys` constants. ### Key changes: - **Banned `System.Environment.GetEnvironmentVariable()`** via `BannedApiAnalyzers` - **Added `EnvironmentGetEnvironmentVariableAnalyzer` (DD0009)** to enforce [ConfigurationKeys](cci:2://file:///Users/anna.yafi/go/src/github.com/DataDog/dd-trace-dotnet3/tracer/src/Datadog.Trace/Generated/net461/Datadog.Trace.SourceGenerators/ConfigurationKeysGenerator/ConfigurationKeys.g.cs:15:0-737:1)/`PlatformKeys` usage only - **Nested `PlatformKeys` by category** (Ci, Aws, AzureAppService, ServiceFabric, DotNet) - **Migrated ~50+ files** across CI Visibility, AWS Lambda, AppSec, Telemetry, Profiler, and Agent components - **Added missing keys** to [supported-configurations.json](cci:7://file:///Users/anna.yafi/go/src/github.com/DataDog/dd-trace-dotnet3/tracer/src/Datadog.Trace/Configuration/supported-configurations.json:0:0-0:0) and their documentation to [supported-configurations-docs.yaml](cci:7://file:///Users/anna.yafi/go/src/github.com/DataDog/dd-trace-dotnet3/tracer/src/Datadog.Trace/Configuration/supported-configurations-docs.yaml:0:0-0:0) ## Reason for change Direct `System.Environment.GetEnvironmentVariable()` calls with string literals are error-prone. Centralizing through `EnvironmentHelpers` with strongly-typed constants provides compile-time validation, discoverability, and refactoring safety. ## Implementation details 1. **Banned API enforcement** - Added `BannedSymbols.txt` and configured `.editorconfig` to treat RS0030 as error (vendored code excluded) 2. **Custom analyzer** - DD0009 validates all `EnvironmentHelpers` calls accept only [ConfigurationKeys](cci:2://file:///Users/anna.yafi/go/src/github.com/DataDog/dd-trace-dotnet3/tracer/src/Datadog.Trace/Generated/net461/Datadog.Trace.SourceGenerators/ConfigurationKeysGenerator/ConfigurationKeys.g.cs:15:0-737:1)/`PlatformKeys` constants, rejecting hardcoded strings 3. **PlatformKeys organization** - Nested by category for better discoverability 4. **EnvironmentHelpers refactoring** - Added overloads for nested class constants, maintained backward compatibility 5. **Configuration documentation** - Added missing configuration keys to [supported-configurations.json](cci:7://file:///Users/anna.yafi/go/src/github.com/DataDog/dd-trace-dotnet3/tracer/src/Datadog.Trace/Configuration/supported-configurations.json:0:0-0:0) and comprehensive XML documentation to [supported-configurations-docs.yaml](cci:7://file:///Users/anna.yafi/go/src/github.com/DataDog/dd-trace-dotnet3/tracer/src/Datadog.Trace/Configuration/supported-configurations-docs.yaml:0:0-0:0), which the source generator uses to generate XML doc comments in [ConfigurationKeys](cci:2://file:///Users/anna.yafi/go/src/github.com/DataDog/dd-trace-dotnet3/tracer/src/Datadog.Trace/Generated/net461/Datadog.Trace.SourceGenerators/ConfigurationKeysGenerator/ConfigurationKeys.g.cs:15:0-737:1) classes ## Test coverage - ✅ Comprehensive analyzer tests covering valid/invalid scenarios - ✅ All existing unit tests pass - ✅ Banned API analyzer correctly flags direct `System.Environment` usage ## Other details +2,600/-940 lines. No breaking changes, negligible performance impact.
anna-git added a commit that referenced this pull request
Jan 27, 2026## Context Part of **Configuration Inversion (Step 6)** - Stack progress: 1. [#7548](#7548) - Add GitLab step and JSON configuration file 2. [#7688](#7688) - Cleanup configuration / platform keys + source generator 3. [#7698](#7698) - Aliases handling via source generator 4. [#7689](#7689) - Analyzers for platform and ConfigurationBuilder 5. [#7931](#7931) - Replace manual ConfigurationKeys by generated ones in the whole solution 6. [#7932](#7932) - Forbid use of System.Environment methods and adapt everywhere 7. **→ [#7937](#7937) - Integration names to generated keys (this PR)** ## Summary of changes Extends the `EnumExtensionsGenerator` to generate [IntegrationNameToKeys](cci:2://file:///Users/anna.yafi/go/src/github.com/DataDog/dd-trace-dotnet3/tracer/src/Datadog.Trace/Generated/net6.0/Datadog.Trace.SourceGenerators/EnumExtensionsGenerator/IntegrationNameToKeys.g.cs:14:4-392:5) class for the `IntegrationId` enum, providing centralized configuration key mapping for integrations including enabled, analytics enabled, and analytics sample rate keys. Also fixes AdoNet instrumentation compilation error. ## Reason for change - Eliminates code duplication in configuration key generation across the codebase - Provides compile-time safety for integration configuration key access - Centralizes key pattern definitions with proper documentation - Ensures consistency across all integration key formats (canonical, mixed case, short aliases) ## Implementation details - **Source Generator**: Extended `EnumExtensionsGenerator` with [GenerateIntegrationNameToKeys](cci:1://file:///Users/anna.yafi/go/src/github.com/DataDog/dd-trace-dotnet3/tracer/src/Datadog.Trace.SourceGenerators/EnumExtensions/Sources.cs:82:4-178:5) method that generates: - `AllIntegrationEnabledKeys` array containing all enabled configuration keys - [GetIntegrationEnabledKeys(string)](cci:1://file:///Users/anna.yafi/go/src/github.com/DataDog/dd-trace-dotnet3/tracer/src/Datadog.Trace/Generated/net6.0/Datadog.Trace.SourceGenerators/EnumExtensionsGenerator/IntegrationNameToKeys.g.cs:102:8-197:9) - returns enabled keys for an integration - [GetIntegrationAnalyticsEnabledKeys(string)](cci:1://file:///Users/anna.yafi/go/src/github.com/DataDog/dd-trace-dotnet3/tracer/src/Datadog.Trace/Generated/net6.0/Datadog.Trace.SourceGenerators/EnumExtensionsGenerator/IntegrationNameToKeys.g.cs:198:8-294:9) - returns analytics enabled keys (marked obsolete) - [GetIntegrationAnalyticsSampleRateKeys(string)](cci:1://file:///Users/anna.yafi/go/src/github.com/DataDog/dd-trace-dotnet3/tracer/src/Datadog.Trace/Generated/net6.0/Datadog.Trace.SourceGenerators/EnumExtensionsGenerator/IntegrationNameToKeys.g.cs:295:8-391:9) - returns analytics sample rate keys (marked obsolete) - Key pattern constants (`IntegrationEnabledKeyPattern`, `AnalyticsEnabledKeyPattern`, `AnalyticsSampleRateKeyPattern`) - **Configuration**: Updated [ConfigurationBuilder.WithIntegrationAnalyticsKey](cci:1://file:///Users/anna.yafi/go/src/github.com/DataDog/dd-trace-dotnet3/tracer/src/Datadog.Trace/Configuration/ConfigurationSources/Telemetry/ConfigurationBuilder.cs:32:4-42:5) and [WithIntegrationAnalyticsSampleRateKey](cci:1://file:///Users/anna.yafi/go/src/github.com/DataDog/dd-trace-dotnet3/tracer/src/Datadog.Trace/Configuration/ConfigurationSources/Telemetry/ConfigurationBuilder.cs:44:4-54:5) to use generated methods ## Test coverage - Added [CanGenerateIntegrationNameToKeysForIntegrationId](cci:1://file:///Users/anna.yafi/go/src/github.com/DataDog/dd-trace-dotnet3/tracer/test/Datadog.Trace.SourceGenerators.Tests/EnumExtensionsGeneratorTests.cs:411:4-527:5) - verifies complete generated output with all three methods - Added [DoesNotGenerateIntegrationNameToKeysForNonIntegrationIdEnum](cci:1://file:///Users/anna.yafi/go/src/github.com/DataDog/dd-trace-dotnet3/tracer/test/Datadog.Trace.SourceGenerators.Tests/EnumExtensionsGeneratorTests.cs:529:4-552:5) - ensures generation only for `IntegrationId` enum - Added [IntegrationNameToKeysGeneratesCorrectKeyFormats](cci:1://file:///Users/anna.yafi/go/src/github.com/DataDog/dd-trace-dotnet3/tracer/test/Datadog.Trace.SourceGenerators.Tests/EnumExtensionsGeneratorTests.cs:554:4-670:5) - validates key formats for various integration names - All tests verify obsolete attributes, key patterns, and switch expression logic
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters