Skip to content

Python: Consolidate OTel GenAI Semantic Conventions verisioning #7657

Description

@TaoChenOSU

Agent Framework observability doesn't maintain a versioning support according to the OTel GenAI semantic conventions: https://github.com/open-telemetry/semantic-conventions/blob/v1.41.1/docs/gen-ai/

The stable release is v1.36.0 and everything above is considered experimental.

Agent Framework emits spans and events from a mixed of versions. For example, inputs and outputs are emitted as events (defined in v1.36.0 and is removed in future versions) and also as span attributes (defined in versions above v1.36.0).

The current behavior creates confusion for monitor backends on what to be expected from MAF.

To improve the situation, MAF will:

  1. Continue to support v1.36.0 as the stable semantic conventions.
  2. Support the latest experimental conventions gated by a flag: OTEL_SEMCONV_STABILITY_OPT_IN, which will be default to [gen_ai_latest_experimental] to minimize breaking changes to customers, because most of us already depend on the newer convention versions.

To avoid breaking customers completely, we can add another flag to control the input and output events which will be on by default.

For opt-in events, create flags to control those events.

The existing MAF env vars ENABLE_INSTRUMENTATION and ENABLE_SENSITIVE_DATA will remain valid.

Metadata

Metadata

Labels

observabilityUsage: [Issues, PRs], Target: observability related featurespythonUsage: [Issues, PRs], Target: Python

Type

Projects

Status
No status

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions