Skip to content

docs(telemetry): Phase 4 — Privacy statement + README section#417

Merged
timenick merged 3 commits into
mainfrom
zhiwang/telemetry-phase4
Apr 29, 2026
Merged

docs(telemetry): Phase 4 — Privacy statement + README section#417
timenick merged 3 commits into
mainfrom
zhiwang/telemetry-phase4

Conversation

@timenick

Copy link
Copy Markdown
Collaborator

Summary

PR 4 of 4 of the ModelKit telemetry rollout. User-facing documentation only:

  • docs/Privacy.md (new): canonical privacy statement — what the three event types (ModelKitHeartbeat / ModelKitAction / ModelKitError) carry, common context attributes, the explicit NOT-collected list, the consent model (first-run prompt + %USERPROFILE%\.winml\config.json), CI auto-disable env vars, the failure cache file location, and the dev-install no-op invariant.
  • README.md: short Data / Telemetry section summarizing the consent model and linking to docs/Privacy.md.

Paths in both docs reflect Phase 3 reality (.winml, not .modelkit) and the persistent failure cache wired in PR #399. The dev-only MODELKIT_TELEMETRY_CACHE_DIR override is intentionally omitted from the end-user docs.

Plan reference

@timenick timenick requested a review from a team as a code owner April 29, 2026 04:21
@DingmaomaoBJTU

Copy link
Copy Markdown
Collaborator

Code review

Found 2 issues:

  1. The "Cache directory" section describes a file-based retry mechanism that does not exist. _encode_cache_entry / _decode_cache_entry exist in utils.py but are never called — there is no code that writes, reads, or clears a cache file. Events that fail to send are lost on process exit (the BatchLogRecordProcessor requeues in-memory only). The section should be removed or replaced with an accurate description of failure behavior.

https://github.com/microsoft/WinML-ModelKit/blob/f48919011f082fd89e5dd1423f3dd889ceaeed74/docs/Privacy.md#L122-L133

  1. "Delete the file" is listed under both "Opt out" (line 107) and "Re-show the prompt on next run" (line 109) — but those are opposite outcomes. The code confirms only the second is correct: when the config file is absent, _read_stored_consent() returns None and resolve_consent() falls through to the interactive prompt, which defaults to enabled. Deleting the file does not opt out; it re-prompts with accept-as-default. The same conflation appears in README.md line 445. The "(or delete the file)" parenthetical in the "Opt out" row should be removed.

https://github.com/microsoft/WinML-ModelKit/blob/f48919011f082fd89e5dd1423f3dd889ceaeed74/docs/Privacy.md#L105-L110

https://github.com/microsoft/WinML-ModelKit/blob/f48919011f082fd89e5dd1423f3dd889ceaeed74/README.md#L444-L448

🤖 Generated with Claude Code

- If this code review was useful, please react with 👍. Otherwise, react with 👎.

Deleting %USERPROFILE%\.winml\config.json does not opt out — it falls
through to the first-run prompt, which defaults to accept. Only setting
telemetry.consent to "disabled" actually opts out.
@timenick

timenick commented Apr 29, 2026

Copy link
Copy Markdown
Collaborator Author

Thanks for the review!

Issue 1 — the cache is actually wired up; I think the grep missed library/exporter.py:

  • OneCollectorLogExporter.__init__ defaults to _PersistentCache() (exporter.py:51)
  • export() calls self._cache.drain() on first call (exporter.py:95) and self._cache.append() on POST failure (exporter.py:97, 100)
  • _PersistentCache.append/drain call _encode_cache_entry / _decode_cache_entry (_cache.py:93, 118)
  • telemetry.py:71 calls _PersistentCache().clear() on opt-out

Failed envelopes persist to %USERPROFILE%\.winml\telemetry\modelkit.cache and drain on the next run. The exporter writes to disk before returning LogRecordExportResult.FAILURE, so the BatchLogRecordProcessor in-memory requeue is a second line of defense, not the only one. The "Cache directory" section matches the implementation.

Issue 2 — agreed, deleting the file falls through to the prompt and defaults to accept. Fixed in b166ff0: removed the misleading "(or delete the file)" from both Privacy.md (Opt-out row) and README.md (opt-out bullet).

🤖 Generated with Claude Code

@timenick timenick enabled auto-merge (squash) April 29, 2026 06:16
@timenick timenick merged commit 52d312b into main Apr 29, 2026
9 checks passed
@timenick timenick deleted the zhiwang/telemetry-phase4 branch April 29, 2026 06:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants