feat(config): recursively convert parsed dicts to typed dataclasses in loader by MikeGoldsmith · Pull Request #5269 · open-telemetry/opentelemetry-python

MikeGoldsmith · 2026-06-03T09:55:50Z

Description

Closes the gap between load_config_file() and the factory functions: YAML/JSON config → SDK objects now works end-to-end through the typed model tree.

Previously, the loader's _dict_to_model did OpenTelemetryConfiguration(**data) which only constructed the top-level dataclass — nested fields stayed as raw dicts. This meant factory functions like create_tracer_provider(config: TracerProviderConfig) would break trying to access config.sampler as an attribute when it was actually a dict.

Approach

Added _dict_to_dataclass in a new _conversion.py module. It walks each field's type annotation via typing.get_type_hints and recursively converts:

Nested dicts → typed dataclass instances (e.g. dict → TracerProvider → SpanProcessor → BatchSpanProcessor → ...)
Lists of dicts → lists of typed dataclasses (list[SpanProcessor])
String/value → Enum members (e.g. log_level: info → SeverityNumber.info)
Unknown keys → captured by the @_additional_properties decorator (so user-defined plugin names still flow through)

Optional[X] / X | None is unwrapped before checking the inner type. ClassVar fields are skipped (the additional_properties annotation on decorated classes is correctly ignored).

Verified end-to-end

yaml_data = '''
file_format: '1.0-rc.1'
tracer_provider:
  processors:
    - simple:
        exporter:
          console: {}
  sampler:
    parent_based:
      root:
        trace_id_ratio_based: {ratio: 0.5}
  limits:
    attribute_count_limit: 10
'''
config = _dict_to_model(yaml.safe_load(yaml_data))
provider = create_tracer_provider(config.tracer_provider)
# → TracerProvider with ParentBased sampler, 10 attribute limit, etc.

User-defined plugins continue to work — unknown sampler/propagator/exporter names land in additional_properties and are loaded via entry points.

Tests

11 new tests in test_conversion.py covering: flat dicts, nested dataclasses, lists, optionals, missing fields, unknown keys (additional_properties), enum coercion, primitive pass-through.

Closes #5127

Adds `_dict_to_dataclass` in `_conversion.py` which walks each field's type annotation and converts: - nested dicts → typed dataclass instances - lists of dicts → lists of typed dataclasses - string/value → Enum members (e.g. log_level: info) - unknown keys → routed to the @_additional_properties decorator The loader's `_dict_to_model` now produces a fully-typed OpenTelemetryConfiguration tree end-to-end. Factory functions can rely on typed attribute access (config.tracer_provider.processors[0].batch .exporter.otlp_http.endpoint) instead of failing on raw dicts. This closes the gap between load_config_file() and the factory functions — YAML/JSON config → SDK objects now works end-to-end. Closes open-telemetry#5127 Assisted-by: Claude Opus 4.6

- Use TypeVar for _dict_to_dataclass return — callers now get the correct type instead of Any - Use collections.abc.Mapping for input (more permissive than dict) - Add explicit is_dataclass check at entry — raises TypeError with a descriptive message instead of failing later in dataclasses.fields Assisted-by: Claude Opus 4.6

Astroid 3.x (used by pylint 3.x) follows typing.get_type_hints into Python 3.14's annotationlib, which contains t-string literals it can't parse and crashes with AttributeError on 'visit_templatestr'. Wrapping the call in a helper that returns dict[str, Any] stops the inference at the declared return type. Assisted-by: Claude Opus 4.7

Same effect as the prior helper — declaring the local as ``dict[str, Any]`` stops astroid's inference at the annotation rather than tracing into the typing internals. Assisted-by: Claude Opus 4.7

… codespell Replace the bespoke _Level enum (which violated pylint's invalid-name on lowercase members) with the real ExemplarFilter enum from models.py — the generated models use lowercase values verbatim from the JSON schema, so using one of them avoids fighting the linter and exercises the same code path with real data shapes. Add 'astroid' to codespell's ignore-words-list; the prior commit's explanatory comment mentions the library by name and codespell flagged it as a misspelling of 'asteroid'. Assisted-by: Claude Opus 4.7

DylanRussell · 2026-06-03T15:35:55Z

Looks good.. should we have a full e2e test like the one you described in your comment ? That seems useful

The conversion module has unit tests that exercise _dict_to_dataclass in isolation, but nothing verified the full pipeline: load a real YAML file, get back fully-typed nested dataclasses, and feed the result into a downstream factory function. Adds two checks built on a representative nested fixture (tracer provider with a parent-based / trace-id-ratio sampler and a batch processor with console exporter): - nested fields (sampler, processors[*].batch) come back as the expected typed dataclasses, not raw dicts - the typed result is accepted by ``create_tracer_provider`` and produces an SDK ``TracerProvider`` This is the integration coverage requested in PR review feedback; the inline example in the PR description is now an actual regression test. Assisted-by: Claude Opus 4.7

Reach into the SDK private fields the same way other tests in this file already do (pylint protected-access disabled at the class level on the similar test_meter_provider.py) so we confirm the YAML actually flowed through into the right Sampler/SpanProcessor/Exporter structure rather than just landing as some TracerProvider instance. Addresses Dylan's nit on open-telemetry#5269. Assisted-by: Claude Opus 4.7 (1M context)

…rsion' into mike/config-recursive-dict-conversion

Resolve conflict in test_loader.py by keeping main's end-to-end factory assertions from open-telemetry#5269 alongside the PR branch's loader integration tests.

…pen-telemetry#5270) * recursively convert parsed dicts to typed dataclasses in loader Adds `_dict_to_dataclass` in `_conversion.py` which walks each field's type annotation and converts: - nested dicts → typed dataclass instances - lists of dicts → lists of typed dataclasses - string/value → Enum members (e.g. log_level: info) - unknown keys → routed to the @_additional_properties decorator The loader's `_dict_to_model` now produces a fully-typed OpenTelemetryConfiguration tree end-to-end. Factory functions can rely on typed attribute access (config.tracer_provider.processors[0].batch .exporter.otlp_http.endpoint) instead of failing on raw dicts. This closes the gap between load_config_file() and the factory functions — YAML/JSON config → SDK objects now works end-to-end. Closes open-telemetry#5127 Assisted-by: Claude Opus 4.6 * rename changelog fragment to PR open-telemetry#5269 * tighten typing on conversion module - Use TypeVar for _dict_to_dataclass return — callers now get the correct type instead of Any - Use collections.abc.Mapping for input (more permissive than dict) - Add explicit is_dataclass check at entry — raises TypeError with a descriptive message instead of failing later in dataclasses.fields Assisted-by: Claude Opus 4.6 * isolate typing.get_type_hints call to placate astroid 3.x on py3.14 Astroid 3.x (used by pylint 3.x) follows typing.get_type_hints into Python 3.14's annotationlib, which contains t-string literals it can't parse and crashes with AttributeError on 'visit_templatestr'. Wrapping the call in a helper that returns dict[str, Any] stops the inference at the declared return type. Assisted-by: Claude Opus 4.7 * inline the typing.get_type_hints wrap Same effect as the prior helper — declaring the local as ``dict[str, Any]`` stops astroid's inference at the annotation rather than tracing into the typing internals. Assisted-by: Claude Opus 4.7 * add configure_sdk orchestrator for declarative config Single entry point that takes a parsed OpenTelemetryConfiguration, builds the resource, and applies the tracer/meter/logger providers and propagator globally. Honors the top-level disabled flag — when true, no globals are touched. The orchestrator is a thin composition of the existing per-signal configure_* factories; the deeper unification with the env-var path (see open-telemetry#5126) is left for follow-up. Refs open-telemetry#3631 Refs open-telemetry#5126 Assisted-by: Claude Opus 4.7 * rename changelog fragment to PR open-telemetry#5270 Assisted-by: Claude Opus 4.7 * use ExemplarFilter for enum coercion test fixture; allow 'astroid' in codespell Replace the bespoke _Level enum (which violated pylint's invalid-name on lowercase members) with the real ExemplarFilter enum from models.py — the generated models use lowercase values verbatim from the JSON schema, so using one of them avoids fighting the linter and exercises the same code path with real data shapes. Add 'astroid' to codespell's ignore-words-list; the prior commit's explanatory comment mentions the library by name and codespell flagged it as a misspelling of 'asteroid'. Assisted-by: Claude Opus 4.7 * fix lint on test_sdk.py: hoist import, disable no-self-use Move ``SdkTracerProvider`` import to module top (ruff PLC0415 / pylint C0415) and add explicit ``# pylint: disable=no-self-use`` on the three mock-only tests that intentionally do not touch ``self``. Assisted-by: Claude Opus 4.7 * remove extra blank line after imports (ruff I001) Assisted-by: Claude Opus 4.7 * add end-to-end loader tests covering YAML -> typed config -> factory The conversion module has unit tests that exercise _dict_to_dataclass in isolation, but nothing verified the full pipeline: load a real YAML file, get back fully-typed nested dataclasses, and feed the result into a downstream factory function. Adds two checks built on a representative nested fixture (tracer provider with a parent-based / trace-id-ratio sampler and a batch processor with console exporter): - nested fields (sampler, processors[*].batch) come back as the expected typed dataclasses, not raw dicts - the typed result is accepted by ``create_tracer_provider`` and produces an SDK ``TracerProvider`` This is the integration coverage requested in PR review feedback; the inline example in the PR description is now an actual regression test. Assisted-by: Claude Opus 4.7 * address review feedback on configure_sdk - log a warning rather than info when called with disabled=true; the caller asked for setup and got a no-op, so the noise is warranted - drop test_disabled_false_runs_setup; test_calls_each_signal_with_resource already covers the disabled=False path with stricter assertions Assisted-by: Claude Opus 4.7 (1M context) * accept PathLike in load_config_file callers with pathlib.Path no longer have to coerce to str at the boundary. Path(file_path) and everything downstream already handle PathLike, so this is a signature-only change. Assisted-by: Claude Opus 4.7 (1M context) * drop Union import from config conversion module Use types.UnionType and typing.Union for optional unwrapping instead of importing Union directly, matching modern | syntax in type hints.

@patch

…telemetry#5271) * recursively convert parsed dicts to typed dataclasses in loader Adds `_dict_to_dataclass` in `_conversion.py` which walks each field's type annotation and converts: - nested dicts → typed dataclass instances - lists of dicts → lists of typed dataclasses - string/value → Enum members (e.g. log_level: info) - unknown keys → routed to the @_additional_properties decorator The loader's `_dict_to_model` now produces a fully-typed OpenTelemetryConfiguration tree end-to-end. Factory functions can rely on typed attribute access (config.tracer_provider.processors[0].batch .exporter.otlp_http.endpoint) instead of failing on raw dicts. This closes the gap between load_config_file() and the factory functions — YAML/JSON config → SDK objects now works end-to-end. Closes open-telemetry#5127 Assisted-by: Claude Opus 4.6 * rename changelog fragment to PR open-telemetry#5269 * tighten typing on conversion module - Use TypeVar for _dict_to_dataclass return — callers now get the correct type instead of Any - Use collections.abc.Mapping for input (more permissive than dict) - Add explicit is_dataclass check at entry — raises TypeError with a descriptive message instead of failing later in dataclasses.fields Assisted-by: Claude Opus 4.6 * isolate typing.get_type_hints call to placate astroid 3.x on py3.14 Astroid 3.x (used by pylint 3.x) follows typing.get_type_hints into Python 3.14's annotationlib, which contains t-string literals it can't parse and crashes with AttributeError on 'visit_templatestr'. Wrapping the call in a helper that returns dict[str, Any] stops the inference at the declared return type. Assisted-by: Claude Opus 4.7 * inline the typing.get_type_hints wrap Same effect as the prior helper — declaring the local as ``dict[str, Any]`` stops astroid's inference at the annotation rather than tracing into the typing internals. Assisted-by: Claude Opus 4.7 * add configure_sdk orchestrator for declarative config Single entry point that takes a parsed OpenTelemetryConfiguration, builds the resource, and applies the tracer/meter/logger providers and propagator globally. Honors the top-level disabled flag — when true, no globals are touched. The orchestrator is a thin composition of the existing per-signal configure_* factories; the deeper unification with the env-var path (see open-telemetry#5126) is left for follow-up. Refs open-telemetry#3631 Refs open-telemetry#5126 Assisted-by: Claude Opus 4.7 * rename changelog fragment to PR open-telemetry#5270 Assisted-by: Claude Opus 4.7 * honor OTEL_CONFIG_FILE in the SDK configurator When the environment variable is set, route the SDK through the declarative config path — load the file via load_config_file() and apply it via configure_sdk() — in place of the env-var-based _initialize_components(). Other OTEL_* vars are ignored (per spec v1.0.0: when a config file is given, it is the sole source of truth). Kwargs passed to _OTelSDKConfigurator._configure are ignored with a warning when the file path is set, so distros that inject kwargs via super() see a clear signal rather than silent drops. The file-loader imports (pyyaml, jsonschema) stay lazy so installs without the file-configuration extras are not affected. Refs open-telemetry#3631 Assisted-by: Claude Opus 4.7 * rename changelog fragment to PR open-telemetry#5271 Assisted-by: Claude Opus 4.7 * use ExemplarFilter for enum coercion test fixture; allow 'astroid' in codespell Replace the bespoke _Level enum (which violated pylint's invalid-name on lowercase members) with the real ExemplarFilter enum from models.py — the generated models use lowercase values verbatim from the JSON schema, so using one of them avoids fighting the linter and exercises the same code path with real data shapes. Add 'astroid' to codespell's ignore-words-list; the prior commit's explanatory comment mentions the library by name and codespell flagged it as a misspelling of 'asteroid'. Assisted-by: Claude Opus 4.7 * fix lint on test_sdk.py: hoist import, disable no-self-use Move ``SdkTracerProvider`` import to module top (ruff PLC0415 / pylint C0415) and add explicit ``# pylint: disable=no-self-use`` on the three mock-only tests that intentionally do not touch ``self``. Assisted-by: Claude Opus 4.7 * silence pylint/ruff on intentional lazy imports The configure_sdk / load_config_file imports inside ``_configure`` are deliberately deferred so that the SDK does not pull in the optional file-configuration extras (pyyaml, jsonschema) unless ``OTEL_CONFIG_FILE`` is actually set. Annotate with the corresponding pylint and ruff suppressions; the existing comment already explains why. Assisted-by: Claude Opus 4.7 * remove extra blank line after imports (ruff I001) Assisted-by: Claude Opus 4.7 * collapse multi-line @patch decorators (ruff format) Assisted-by: Claude Opus 4.7 * add end-to-end loader tests covering YAML -> typed config -> factory The conversion module has unit tests that exercise _dict_to_dataclass in isolation, but nothing verified the full pipeline: load a real YAML file, get back fully-typed nested dataclasses, and feed the result into a downstream factory function. Adds two checks built on a representative nested fixture (tracer provider with a parent-based / trace-id-ratio sampler and a batch processor with console exporter): - nested fields (sampler, processors[*].batch) come back as the expected typed dataclasses, not raw dicts - the typed result is accepted by ``create_tracer_provider`` and produces an SDK ``TracerProvider`` This is the integration coverage requested in PR review feedback; the inline example in the PR description is now an actual regression test. Assisted-by: Claude Opus 4.7 * address review feedback on OTEL_CONFIG_FILE routing Use a walrus operator in _configure, simplify singleton reset to tearDown only, and hoist no-self-use pylint disable to file scope. * tighten OTEL_CONFIG_FILE docstring (review feedback from herin049) The previous wording overstated the env-var contract by implying all ``OTEL_*`` variables are ignored when ``OTEL_CONFIG_FILE`` is set. That's only true for spec-defined variables with schema equivalents: * resource detectors enabled in the config can still read env vars at runtime (e.g. ``OTEL_RESOURCE_ATTRIBUTES``, ``OTEL_SERVICE_NAME``) * ``${env:VAR}`` substitutions inside the file remain in effect Reword to be precise about both. Assisted-by: Claude Opus 4.7

MikeGoldsmith requested a review from a team as a code owner June 3, 2026 09:55

rename changelog fragment to PR open-telemetry#5269

582c37f

otelbot-python Bot added this to Python PR digest Jun 3, 2026

MikeGoldsmith mentioned this pull request Jun 3, 2026

Add declarative configuration prototype #3631

Open

36 tasks

MikeGoldsmith added 3 commits June 3, 2026 11:16

inline the typing.get_type_hints wrap

131378c

Same effect as the prior helper — declaring the local as ``dict[str, Any]`` stops astroid's inference at the annotation rather than tracing into the typing internals. Assisted-by: Claude Opus 4.7

MikeGoldsmith moved this to Ready for review in Python PR digest Jun 3, 2026

This was referenced Jun 3, 2026

feat(config): add configure_sdk orchestrator for declarative config #5270

Merged

feat(config): support OTEL_CONFIG_FILE in the SDK configurator #5271

Merged

MikeGoldsmith marked this pull request as draft June 3, 2026 14:29

DylanRussell approved these changes Jun 3, 2026

View reviewed changes

github-project-automation Bot moved this from Ready for review to Approved PRs in Python PR digest Jun 3, 2026

MikeGoldsmith marked this pull request as ready for review June 4, 2026 18:37

MikeGoldsmith mentioned this pull request Jun 5, 2026

feat(config): add public opentelemetry.sdk.configuration namespace #5276

Open

11 tasks

MikeGoldsmith added 2 commits June 5, 2026 14:38

Merge branch 'main' into mike/config-recursive-dict-conversion

f6e0eac

herin049 reviewed Jun 6, 2026

View reviewed changes

Comment thread opentelemetry-sdk/src/opentelemetry/sdk/_configuration/_conversion.py

Comment thread opentelemetry-sdk/src/opentelemetry/sdk/_configuration/_conversion.py

Comment thread opentelemetry-sdk/src/opentelemetry/sdk/_configuration/_conversion.py

DylanRussell reviewed Jun 8, 2026

View reviewed changes

Comment thread opentelemetry-sdk/tests/_configuration/file/test_loader.py

DylanRussell approved these changes Jun 8, 2026

View reviewed changes

MikeGoldsmith added 3 commits June 9, 2026 13:10

Merge remote-tracking branch 'origin/mike/config-recursive-dict-conve…

4d24140

…rsion' into mike/config-recursive-dict-conversion

Merge branch 'main' into mike/config-recursive-dict-conversion

7e49de9

herin049 approved these changes Jun 9, 2026

View reviewed changes

Merge branch 'main' into mike/config-recursive-dict-conversion

1e2b5b3

lzchen approved these changes Jun 10, 2026

View reviewed changes

lzchen added this pull request to the merge queue Jun 10, 2026

Merged via the queue into open-telemetry:main with commit 15954ef Jun 10, 2026
479 checks passed

github-project-automation Bot moved this from Approved PRs to Done in Python PR digest Jun 10, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(config): recursively convert parsed dicts to typed dataclasses in loader#5269

feat(config): recursively convert parsed dicts to typed dataclasses in loader#5269
lzchen merged 12 commits into
open-telemetry:mainfrom
MikeGoldsmith:mike/config-recursive-dict-conversion

MikeGoldsmith commented Jun 3, 2026

Uh oh!

DylanRussell commented Jun 3, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

MikeGoldsmith commented Jun 3, 2026

Description

Approach

Verified end-to-end

Tests

Uh oh!

DylanRussell commented Jun 3, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants