Skip to content

Python: add feature-usage User-Agent telemetry - #7420

Merged
Eduard van Valkenburg (eavanvalkenburg) merged 8 commits into
microsoft:mainfrom
eavanvalkenburg:python-feature-usage-telemetry
Jul 30, 2026
Merged

Python: add feature-usage User-Agent telemetry#7420
Eduard van Valkenburg (eavanvalkenburg) merged 8 commits into
microsoft:mainfrom
eavanvalkenburg:python-feature-usage-telemetry

Conversation

@eavanvalkenburg

@eavanvalkenburg Eduard van Valkenburg (eavanvalkenburg) commented Jul 30, 2026

Copy link
Copy Markdown
Member

Motivation & Context

Agent Framework's User-Agent identifies the Python SDK, but it does not show
which framework capabilities have been observed together in a process. ADR-0033
defines a transparent, first-party-only feature mask so product and support
analysis can answer coarse adoption and co-occurrence questions without adding a
new telemetry pipeline or turning the signal into per-call tracking.

Description & Review Guide

  • What are the major changes? Implements the Python v1 feature-usage
    mechanism: a thread-safe, process-global 128-bit monotonic mask; package-local
    FeatureIndex declarations and activation marks for all 63 assigned Python
    registry rows; a strict repository validator for exact registry parity,
    complete coverage, valid range, and zero overlap; and the dedicated
    AGENT_FRAMEWORK_FEATURE_MASK_DISABLED opt-out. Approved Azure OpenAI and
    Foundry request hooks append (feat=v1.<hex>) only for reviewed HTTPS origins,
    refresh it at request time, and remove it on unapproved redirect hops. OpenAI
    hooks retain SDK HTTP defaults and cleanup behavior, while Foundry policies
    preserve Azure SDK custom hooks. The base User-Agent now resolves its version
    from the installed agent-framework-core distribution.
  • What is the impact of these changes? Eligible first-party requests can carry
    a bounded feature token representing capabilities observed at least once in
    the current process. The bits are binary and non-countable: repeated requests
    do not represent additional feature invocations or request-level attribution.
    Third-party/custom origins and caller-owned transports remain unchanged.
  • What do you want reviewers to focus on? Activation timing, approved-origin
    and redirect boundaries, process-global privacy semantics, transport lifecycle
    preservation, and the registry completeness test.

Related Issue

Fixes #7421. Contributes the Python implementation for #7240. No other open PR
implements this issue.

Contribution Checklist

  • The code builds clean without any errors or warnings
  • All unit tests pass, and I have added new tests where possible
  • The PR follows the Contribution Guidelines
  • This PR is linked to an issue and there is no other open PR for this issue (see Related Issue above).
  • This is not a breaking change. If it is a breaking change, add the breaking change label (or add "[BREAKING]" to the title prefix, before or after any language prefix) — a workflow keeps the label and title prefix in sync automatically.

Copilot AI review requested due to automatic review settings July 30, 2026 06:27
@agent-framework-automation agent-framework-automation Bot added documentation Usage: [Issues, PRs], Target: documentation in the code base and learn docs python Usage: [Issues, PRs], Target: Python lab Usage: [Issues, PRs], Target: lab packages labels Jul 30, 2026
@github-actions

github-actions Bot commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Python Test Coverage

Python Test Coverage Report •
FileStmtsMissCoverMissing
packages/a2a/agent_framework_a2a
   _a2a_executor.py960100% 
   _agent.py4042593%159–160, 174–175, 313, 550, 555, 557, 611–612, 642–643, 647, 798, 814, 826, 848, 869, 908, 924, 934, 945, 952–953, 1004
   _feature_usage.py30100% 
packages/ag-ui/agent_framework_ag_ui
   _agent.py47295%73, 133
   _client.py153795%88, 101–105, 265
   _feature_usage.py30100% 
   _workflow.py2071692%66, 129, 134, 173, 212–213, 296, 340, 342, 344–346, 348, 366–368
packages/anthropic/agent_framework_anthropic
   _chat_client.py5363593%470, 473, 559, 681, 693, 781, 815, 905, 944–945, 1023, 1025, 1055–1056, 1140–1142, 1146–1148, 1152–1155, 1190, 1306, 1316, 1368, 1516–1517, 1534, 1547, 1560, 1585–1586
   _feature_usage.py30100% 
packages/azure-ai-search/agent_framework_azure_ai_search
   _context_provider.py3921995%98–99, 124–127, 538, 617–618, 746–747, 800–801, 845, 850, 855, 949–950, 998
   _feature_usage.py30100% 
packages/azure-contentunderstanding/agent_framework_azure_contentunderstanding
   _context_provider.py2624184%81, 84, 202–205, 309–310, 312, 316–317, 320, 324, 409, 547, 551, 605, 655–656, 679, 681–684, 849, 853, 879–884, 886–891, 899, 908–909
   _feature_usage.py30100% 
packages/azure-cosmos-memory/agent_framework_azure_cosmos_memory
   _context_provider.py1370100% 
   _feature_usage.py30100% 
packages/azure-cosmos/agent_framework_azure_cosmos
   _checkpoint_storage.py138199%431
   _feature_usage.py30100% 
   _history_provider.py129596%166–168, 266, 273
packages/azurefunctions/agent_framework_azurefunctions
   _app.py58813077%84–85, 88–91, 320, 378, 438–440, 442, 462, 466, 469, 471, 473, 497, 500–503, 537–539, 541, 547–548, 555, 557, 572–577, 590, 592, 607–608, 610–611, 616–618, 620–623, 627, 632–635, 639, 645, 698, 735, 741, 744, 747, 892–893, 1001, 1009–1010, 1030–1032, 1038–1040, 1046–1048, 1081–1082, 1142–1143, 1192–1193, 1198, 1280, 1283, 1292–1294, 1296–1298, 1300, 1302, 1313–1316, 1318, 1320–1321, 1323, 1330–1331, 1333–1334, 1336–1337, 1339, 1343, 1353–1355, 1357–1358, 1360–1362, 1369, 1371–1372, 1374, 1395, 1400, 1412, 1484, 1574, 1589–1592, 1617
   _feature_usage.py30100% 
packages/bedrock/agent_framework_bedrock
   _chat_client.py4511896%437, 506, 589–590, 602–604, 628, 636, 797, 801–802, 805, 823, 847, 859, 863, 899
   _embedding_client.py821285%131–142
   _feature_usage.py30100% 
packages/chatkit/agent_framework_chatkit
   _converter.py1380100% 
   _feature_usage.py30100% 
   _streaming.py300100% 
packages/claude/agent_framework_claude
   _agent.py3312891%420–421, 425, 437, 445–447, 449–450, 480–482, 501, 505, 507, 511, 520, 566, 569, 609, 614–615, 689, 775, 802–805
   _feature_usage.py30100% 
packages/copilotstudio/agent_framework_copilotstudio
   _agent.py98495%176, 184, 254, 287
   _feature_usage.py30100% 
packages/core/agent_framework
   _agents.py4494989%541, 553, 608, 1125, 1174, 1245–1249, 1348, 1378, 1415, 1448–1449, 1454, 1461, 1513, 1541, 1554, 1606, 1608, 1617–1622, 1627, 1629, 1635–1636, 1643, 1645–1646, 1654–1655, 1658–1660, 1670–1675, 1679, 1684, 1686
   _compaction.py8566692%138–139, 147, 174, 265–266, 284–285, 302, 337, 351, 358, 373–374, 428, 435, 451, 501, 537, 539, 559, 601, 657, 663, 665, 684, 728–733, 745, 829, 831, 846, 891, 953, 1079, 1085–1089, 1092–1094, 1100, 1119, 1232, 1234, 1236, 1273, 1280, 1285, 1300, 1314–1315, 1318–1319, 1421, 1444, 1456, 1553, 1580, 1664
   _mcp.py138211491%254, 260, 369, 388, 609, 688–689, 803, 828, 879, 994, 997, 1007, 1011, 1058–1059, 1064, 1071–1072, 1079, 1084–1085, 1092–1093, 1097, 1102–1103, 1112, 1119–1120, 1138, 1151, 1175–1176, 1195–1198, 1200–1201, 1205, 1231, 1265–1267, 1269, 1323–1325, 1384–1385, 1668, 1709–1710, 1723, 1726, 1735–1736, 1741–1742, 1748, 1802–1803, 1823–1824, 1833–1834, 1839–1840, 1846, 1939, 1942, 1969, 1992–1996, 2019–2021, 2026, 2030–2031, 2133, 2140, 2142, 2213, 2228–2229, 2236–2237, 2242–2243, 2248, 2252, 2267, 2329, 2512, 2514, 2536, 2538–2541, 2554–2555, 2599, 2661, 3100–3101, 3338–3339, 3357
   _sessions.py5504192%131–133, 135–136, 153–154, 156–158, 236–237, 370, 701–705, 720, 750, 787–788, 802, 804, 823, 825, 908, 914, 948, 1019, 1023, 1033, 1174, 1191, 1325, 1339–1340, 1364, 1386, 1396, 1438
   _skills.py10973996%315, 585, 1103, 1118, 1120–1121, 1487–1488, 1730, 1759, 2271, 2461, 2970–2971, 3068, 3076, 3081, 3084, 3089, 3109, 3121, 3126, 3221, 3229, 3234, 3237, 3242, 3262, 3271, 3276, 3542–3543, 4130, 4419–4420, 4447–4448, 4455–4456
   _telemetry.py106694%94, 109–111, 168, 206
packages/core/agent_framework/_harness
   _agent.py132496%202, 625–626, 628
   _background_agents.py2602092%170, 195–196, 221–223, 226–227, 230–231, 385, 430, 433, 458, 467, 471, 530–533
   _file_access.py6365092%94, 204, 235, 298, 408, 410, 422, 426, 454, 467–468, 475–482, 486, 490, 856–857, 882, 886, 941–943, 965–968, 1007, 1009, 1066, 1068, 1473–1474, 1485–1486, 1498–1499, 1513–1514, 1537–1538, 1553, 1558–1559, 1588
   _memory.py77611784%79–80, 96, 103, 155, 168, 175–177, 200, 220, 229–233, 296, 302, 337, 418, 426, 428, 432, 435, 439, 534, 546, 564–565, 576–577, 588, 703, 720, 729, 744, 772, 775–777, 787, 809–812, 861, 871, 874, 876, 896, 906, 911, 914, 926, 994, 996, 998, 1000, 1002, 1004, 1074, 1091, 1119, 1148, 1156–1158, 1160, 1215, 1241–1244, 1250, 1405, 1408, 1419–1420, 1424, 1427–1429, 1435, 1439, 1444, 1448, 1453, 1457, 1465–1466, 1517, 1520–1523, 1530–1531, 1559, 1563–1571, 1588, 1614–1615, 1618–1619, 1625, 1627, 1632, 1637, 1643
   _mode.py100199%97
   _todo.py3092791%88, 90, 92, 94, 140, 153, 155, 161–162, 173, 175, 208, 211, 217, 220, 360, 371–372, 383, 385, 394, 408, 412, 414, 509, 535, 567
   _tool_approval.py3564188%69, 72, 77, 114, 131, 134, 137, 194–195, 214, 243, 251, 262, 280–285, 297, 311, 314, 334, 385, 408, 410–411, 443, 453, 468, 470–471, 473–474, 521–523, 561–562, 612, 621
packages/core/agent_framework/_workflows
   _workflow_builder.py2111393%226, 503–507, 509, 612, 627, 656, 672, 680, 715
packages/declarative/agent_framework_declarative
   _feature_usage.py40100% 
   _loader.py2645878%586, 588, 617–627, 669–672, 675–677, 680, 682–684, 687, 695, 712–717, 769–774, 792–797, 805, 807, 812, 815–818, 821–822, 825–826, 840–841, 871
packages/declarative/agent_framework_declarative/_workflows
   _factory.py158298%517, 572
packages/durabletask/agent_framework_durabletask
   _feature_usage.py30100% 
   _shim.py48491%50, 124, 146, 168
   _worker.py1221587%271, 326, 343–344, 346–348, 385, 387, 392, 407, 411–412, 416–417
packages/foundry/agent_framework_foundry
   _agent.py2683487%124, 127, 251, 256, 265, 369, 477–478, 502, 505, 516–528, 532–533, 731–732, 735, 761, 771, 793, 881, 886, 890
   _chat_client.py2432191%111, 113–115, 119–120, 124, 220, 259, 315, 320, 371, 583, 587, 681, 688–691, 738, 820
   _embedding_client.py1191289%155, 176–179, 183, 187, 238, 248, 259, 268, 294
   _feature_usage.py24195%53
   _foundry_evals.py341897%473–474, 509–510, 667, 672, 862, 930
   _memory_provider.py970100% 
packages/foundry_hosting/agent_framework_foundry_hosting
   _feature_usage.py40100% 
   _invocations.py480100% 
   _responses.py8729788%189–192, 279, 284, 308, 363–364, 378, 381–382, 413–414, 451–452, 462, 504, 536–538, 541–542, 553, 596, 610, 685, 688, 694, 696, 768, 855–856, 974, 1204, 1217, 1708–1710, 1712, 1759–1760, 1762–1763, 1765–1766, 1768–1769, 1774, 1783, 1786–1788, 1790, 1800, 1804, 1817, 1908–1909, 1924–1925, 1927, 1932–1936, 1938, 1945–1946, 1948–1949, 1955, 1957–1961, 1968, 1974, 1996, 2002, 2008, 2010, 2012–2015, 2023, 2025, 2086–2088, 2102–2103, 2106–2107
   _toolbox.py990100% 
packages/foundry_local/agent_framework_foundry_local
   _feature_usage.py30100% 
   _foundry_local_client.py48589%201, 205–208
packages/gemini/agent_framework_gemini
   _chat_client.py4711995%404, 723, 744–745, 763–764, 773–774, 777–778, 810, 817, 942, 953, 1142–1143, 1147, 1158–1159
   _feature_usage.py30100% 
packages/github_copilot/agent_framework_github_copilot
   _agent.py4312294%70–71, 113, 122–124, 128, 549, 564–565, 644, 657, 794, 798, 967, 1001–1002, 1041, 1044, 1181, 1255, 1273
   _feature_usage.py30100% 
packages/hosting-a2a/agent_framework_hosting_a2a
   _conversion.py207995%116–118, 124, 149–150, 160–161, 366
   _feature_usage.py30100% 
packages/hosting-mcp/agent_framework_hosting_mcp
   _agent_tool.py69888%74, 76, 84, 100, 155, 159, 167, 171
   _feature_usage.py30100% 
   _workflow_tool.py63395%97, 111, 144
packages/hosting-responses/agent_framework_hosting_responses
   _feature_usage.py30100% 
   _parsing.py48820657%100, 123, 255, 259, 274–277, 279–284, 286–289, 293–303, 312–316, 347–349, 351–355, 359–362, 364–367, 369–372, 386–417, 481, 495, 515, 518–521, 523, 542–549, 551, 567–569, 583, 595, 607, 622–629, 631, 642–643, 657, 666–668, 678, 683, 692, 696, 700–701, 706–709, 716, 720–728, 732–737, 741–748, 752–756, 760–762, 766–768, 772–774, 779, 781, 786–792, 798–801, 809–813, 819, 821–823, 828–834, 838–841, 922
packages/hosting-telegram/agent_framework_hosting_telegram
   _feature_usage.py30100% 
   _parsing.py158596%82, 124, 203, 211, 243
   _rendering.py600100% 
packages/hosting/agent_framework_hosting
   _feature_usage.py30100% 
   _state.py1261092%217, 230–232, 250, 312, 343, 356–358
packages/hyperlight/agent_framework_hyperlight
   _feature_usage.py30100% 
   _provider.py45393%55, 59, 67
packages/mem0/agent_framework_mem0
   _context_provider.py131993%140, 151, 154–155, 160, 170, 178, 181, 189
   _feature_usage.py30100% 
packages/mistral/agent_framework_mistral
   _embedding_client.py75297%31, 37
   _feature_usage.py30100% 
packages/monty/agent_framework_monty
   _feature_usage.py30100% 
   _provider.py370100% 
packages/ollama/agent_framework_ollama
   _chat_client.py231996%404, 406, 417, 421–422, 437, 448, 527, 553
   _embedding_client.py56198%152
   _feature_usage.py30100% 
packages/openai/agent_framework_openai
   _chat_client.py138712391%313, 326, 680–686, 695–698, 704–708, 716, 760–765, 769–772, 774–776, 783–785, 788, 845, 853, 876, 1093, 1152, 1154, 1156, 1158, 1224, 1238, 1318, 1328, 1333, 1376, 1488–1489, 1504, 1777, 1884, 1889–1890, 1973, 1983, 2010, 2016, 2026, 2032, 2037, 2043, 2048–2049, 2129, 2173, 2176–2179, 2193, 2203–2204, 2216, 2258, 2323, 2340, 2343, 2370–2372, 2411, 2428, 2431, 2493, 2500, 2537–2538, 2573, 2611–2612, 2630–2631, 2803–2804, 2822, 2908–2916, 3094, 3109, 3159, 3169–3171, 3198–3200, 3210–3211, 3217, 3232, 3365–3366
   _chat_completion_client.py4272095%488, 582, 589–590, 594, 619, 794, 876, 878, 883, 886, 993, 1010, 1031, 1039, 1063, 1076, 1109, 1132, 1453
   _embedding_client.py81396%240, 252, 288
   _feature_usage.py250100% 
   _shared.py1651690%245, 264–266, 278, 288, 301, 307, 330, 334, 368–369, 388, 407–408, 410
packages/orchestrations/agent_framework_orchestrations
   _concurrent.py1502484%61, 70–71, 99–100, 105, 123, 128, 133–134, 155, 165, 172, 246, 262, 265, 322, 352, 354–355, 357, 362, 375, 379
   _feature_usage.py70100% 
   _group_chat.py3357079%185, 354, 361, 394, 405–406, 412, 417, 438, 442, 455–456, 469, 484–485, 487, 503, 530–535, 537, 571–574, 576, 581–585, 679, 682, 721, 724, 727, 730, 738, 750–751, 753–754, 756–757, 759, 764, 767, 776, 782, 826–827, 831–832, 846–847, 849–850, 881–882, 948, 967, 975, 980–982, 989, 999
   _handoff.py3425284%114–115, 117, 170–180, 182, 184, 186, 191, 322, 347, 374, 400, 464, 507, 515, 519–520, 551–553, 558–560, 690, 693, 706, 768, 773, 780, 790, 792, 811, 813, 895–896, 928–929, 1042, 1049, 1121–1122, 1124
   _magentic.py5948685%74–83, 88, 92–103, 268, 303, 364, 373, 375, 417, 434, 443–444, 446–448, 450, 461, 615, 617, 657, 707, 743–745, 747, 826–829, 920, 926, 932, 974, 1012, 1044, 1061, 1072, 1129–1130, 1134–1136, 1160, 1184–1185, 1198, 1218, 1241, 1286–1287, 1325–1326, 1495, 1498, 1507, 1510, 1515, 1566–1567, 1608–1609, 1657, 1688, 1746, 1760, 1771
   _orchestration_request_info.py66198%163
   _sequential.py86693%58, 134, 145, 151, 199, 227
   _workflow_builder.py50100% 
packages/purview/agent_framework_purview
   _client.py133298%75–76
   _feature_usage.py30100% 
packages/redis/agent_framework_redis
   _context_provider.py185796%198, 271–272, 279, 284–285, 287
   _feature_usage.py30100% 
   _history_provider.py63198%196
packages/tools/agent_framework_tools
   _feature_usage.py30100% 
packages/tools/agent_framework_tools/shell
   _docker.py2351294%137, 145–146, 431–432, 435, 651–655, 709
   _tool.py107496%160, 257–258, 313
TOTAL46577448390% 

Python Unit Test Overview

Tests Skipped Failures Errors Time
9584 34 💤 0 ❌ 0 🔥 2m 33s ⏱️

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated Code Review

Reviewers: 5 | Confidence: 68%

✓ Correctness

No actionable issues found in this dimension.

✓ Security Reliability

No actionable issues found in this dimension.

✓ Test Coverage

Test coverage is uneven: orchestrations sequential/magentic builders and purview process_content have explicit feature-marking assertions, but several other instrumented call sites lack corresponding tests. The tools/shell package has two production files instrumented with no feature-marking tests in the diff. The comprehensive registry validation test provides structural coverage (every FeatureIndex is referenced) but does not verify runtime marking behavior at each call site.

✓ Failure Modes

No actionable issues found in this dimension.

✓ Design Approach

The new feature-mask plumbing is mostly consistent, but the Foundry OpenAI integration now unconditionally swaps in package-owned HTTP clients on both the chat-client and agent paths. That conflicts with the PR’s stated requirement that caller-owned transports remain unchanged, and it can silently discard custom retry/auth/event-hook behavior supplied through a caller-provided project/OpenAI client.

Suggestions

  • The tools/shell package adds mark_feature_used(FeatureIndex.TOOLS_SHELL) in both _docker.py:447 and _tool.py:246, but no tests in the diff verify these calls. Consider adding a test (similar to the pattern in other packages) that patches mark_feature_used and asserts it is called with FeatureIndex.TOOLS_SHELL when run() is invoked.

Automated review by eavanvalkenburg's agents

Comment thread python/packages/foundry/agent_framework_foundry/_chat_client.py
Comment thread python/packages/foundry/agent_framework_foundry/_agent.py

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR implements ADR-0033/SPEC-004 “feature-usage” telemetry for the Python SDK by maintaining a process-global monotonic 128-bit feature mask, stamping it into the Agent Framework User-Agent only on approved first-party request destinations, and adding per-package feature index ownership/activation plus registry parity validation.

Changes:

  • Adds core feature-mask support (mark_feature_used, token formatting, opt-out env var) and switches the base Agent Framework User-Agent version source to the installed agent-framework-core distribution.
  • Introduces per-package FeatureIndex declarations and marks “first meaningful activation” across many packages (clients/providers/orchestrations/tools/etc.), with expanded test coverage.
  • Adds destination-aware User-Agent stamping for Azure OpenAI (httpx hook) and Foundry (Azure pipeline policy + OpenAI client hook), plus documentation updates.

Reviewed changes

Copilot reviewed 151 out of 151 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
python/samples/README.md Documents env vars for User-Agent/feature token controls.
python/packages/tools/agent_framework_tools/shell/_tool.py Marks shell tool feature usage.
python/packages/tools/agent_framework_tools/shell/_docker.py Marks shell tool feature usage (docker impl).
python/packages/tools/agent_framework_tools/_feature_usage.py Declares tools package feature index.
python/packages/redis/tests/test_providers.py Adds assertions for Redis feature marking.
python/packages/redis/agent_framework_redis/_history_provider.py Marks Redis provider usage in history operations.
python/packages/redis/agent_framework_redis/_feature_usage.py Declares Redis package feature index.
python/packages/redis/agent_framework_redis/_context_provider.py Marks Redis provider usage in context operations.
python/packages/purview/tests/purview/test_purview_client.py Adds assertions for Purview feature marking.
python/packages/purview/agent_framework_purview/_feature_usage.py Declares Purview package feature index.
python/packages/purview/agent_framework_purview/_client.py Marks Purview client feature usage.
python/packages/orchestrations/tests/test_sequential.py Validates orchestration vs core workflow attribution.
python/packages/orchestrations/tests/test_magentic.py Validates Magentic orchestration feature marking.
python/packages/orchestrations/agent_framework_orchestrations/_workflow_builder.py Adds orchestration-specific WorkflowBuilder override to suppress core workflow bit.
python/packages/orchestrations/agent_framework_orchestrations/_sequential.py Marks sequential orchestration feature usage.
python/packages/orchestrations/agent_framework_orchestrations/_orchestration_request_info.py Uses orchestration builder override.
python/packages/orchestrations/agent_framework_orchestrations/_magentic.py Marks Magentic orchestration feature usage.
python/packages/orchestrations/agent_framework_orchestrations/_handoff.py Marks handoff orchestration feature usage.
python/packages/orchestrations/agent_framework_orchestrations/_group_chat.py Marks group chat orchestration feature usage.
python/packages/orchestrations/agent_framework_orchestrations/_feature_usage.py Declares orchestration feature indices.
python/packages/orchestrations/agent_framework_orchestrations/_concurrent.py Marks concurrent orchestration feature usage.
python/packages/openai/tests/openai/test_openai_shared.py Tests Azure OpenAI feature-stamping hook + client cleanup behavior.
python/packages/openai/tests/openai/test_openai_embedding_client.py Tests OpenAI embedding feature marking.
python/packages/openai/tests/openai/test_openai_chat_completion_client.py Tests OpenAI streaming feature marking timing.
python/packages/openai/agent_framework_openai/_shared.py Injects feature-usage-aware http client for Azure OpenAI clients.
python/packages/openai/agent_framework_openai/_feature_usage.py Adds OpenAI feature index + approved-origin stamping client.
python/packages/openai/agent_framework_openai/_embedding_client.py Marks OpenAI embedding usage.
python/packages/openai/agent_framework_openai/_chat_completion_client.py Marks OpenAI chat-completion usage (incl. streaming timing).
python/packages/openai/agent_framework_openai/_chat_client.py Marks OpenAI Responses client usage, including continuation flows.
python/packages/ollama/tests/ollama/test_ollama_embedding_client.py Adds assertions for Ollama feature marking.
python/packages/ollama/agent_framework_ollama/_feature_usage.py Declares Ollama package feature index.
python/packages/ollama/agent_framework_ollama/_embedding_client.py Marks Ollama embedding usage.
python/packages/ollama/agent_framework_ollama/_chat_client.py Marks Ollama chat usage.
python/packages/monty/agent_framework_monty/_provider.py Marks Monty provider usage.
python/packages/monty/agent_framework_monty/_feature_usage.py Declares Monty package feature index.
python/packages/mistral/tests/mistral/test_mistral_embedding_client.py Adds assertions for Mistral feature marking.
python/packages/mistral/agent_framework_mistral/_feature_usage.py Declares Mistral package feature index.
python/packages/mistral/agent_framework_mistral/_embedding_client.py Marks Mistral embedding usage.
python/packages/mem0/tests/test_mem0_context_provider.py Adds assertions for Mem0 feature marking.
python/packages/mem0/agent_framework_mem0/_feature_usage.py Declares Mem0 package feature index.
python/packages/mem0/agent_framework_mem0/_context_provider.py Marks Mem0 provider usage.
python/packages/lab/tau2/agent_framework_lab_tau2/runner.py Marks Lab usage in tau2 runner.
python/packages/lab/pyproject.toml Adds shared lab-common package + pyright paths.
python/packages/lab/lightning/agent_framework_lab_lightning/init.py Marks Lab usage in lightning tracer init.
python/packages/lab/gaia/agent_framework_lab_gaia/gaia.py Marks Lab usage across GAIA entry points.
python/packages/lab/common/agent_framework_lab_common/_feature_usage.py Declares Lab common feature index.
python/packages/lab/common/agent_framework_lab_common/init.py Adds package initializer.
python/packages/hyperlight/agent_framework_hyperlight/_provider.py Marks Hyperlight provider usage.
python/packages/hyperlight/agent_framework_hyperlight/_feature_usage.py Declares Hyperlight package feature index.
python/packages/hosting/agent_framework_hosting/_state.py Marks hosting state usage on initialization.
python/packages/hosting/agent_framework_hosting/_feature_usage.py Declares hosting package feature index.
python/packages/hosting-telegram/agent_framework_hosting_telegram/_rendering.py Marks Telegram hosting feature usage during rendering.
python/packages/hosting-telegram/agent_framework_hosting_telegram/_parsing.py Marks Telegram hosting feature usage during parsing.
python/packages/hosting-telegram/agent_framework_hosting_telegram/_feature_usage.py Declares Telegram hosting feature index.
python/packages/hosting-responses/agent_framework_hosting_responses/_parsing.py Marks Responses hosting feature usage.
python/packages/hosting-responses/agent_framework_hosting_responses/_feature_usage.py Declares Responses hosting feature index.
python/packages/hosting-mcp/agent_framework_hosting_mcp/_workflow_tool.py Marks MCP hosting usage on tool operations.
python/packages/hosting-mcp/agent_framework_hosting_mcp/_feature_usage.py Declares MCP hosting feature index.
python/packages/hosting-mcp/agent_framework_hosting_mcp/_agent_tool.py Marks MCP hosting usage on tool operations.
python/packages/hosting-a2a/agent_framework_hosting_a2a/_feature_usage.py Declares A2A hosting feature index.
python/packages/hosting-a2a/agent_framework_hosting_a2a/_conversion.py Marks A2A hosting usage in conversions.
python/packages/github_copilot/tests/test_github_copilot_agent.py Adds assertions for GitHub Copilot agent feature marking.
python/packages/github_copilot/agent_framework_github_copilot/_feature_usage.py Declares GitHub Copilot package feature index.
python/packages/github_copilot/agent_framework_github_copilot/_agent.py Marks GitHub Copilot agent usage.
python/packages/gemini/tests/test_gemini_client.py Adds assertions for Gemini feature marking.
python/packages/gemini/agent_framework_gemini/_feature_usage.py Declares Gemini package feature index.
python/packages/gemini/agent_framework_gemini/_chat_client.py Marks Gemini chat usage.
python/packages/foundry/tests/foundry/test_foundry_memory_provider.py Updates expectations around Foundry client policy wiring.
python/packages/foundry/tests/foundry/test_foundry_embedding_client.py Updates expectations around Foundry client policy wiring.
python/packages/foundry/tests/foundry/test_foundry_chat_client.py Adds tests for Foundry feature policy refresh + redirect stripping.
python/packages/foundry/tests/foundry/test_foundry_agent.py Adds tests for Foundry/OpenAI feature-bit separation + policy wiring.
python/packages/foundry/agent_framework_foundry/_memory_provider.py Adds Foundry policy/hook wiring and marks Foundry memory usage.
python/packages/foundry/agent_framework_foundry/_foundry_evals.py Wires feature-usage-aware OpenAI client for evals and marks eval usage.
python/packages/foundry/agent_framework_foundry/_feature_usage.py Declares Foundry feature indices + policy/http-client factories.
python/packages/foundry/agent_framework_foundry/_embedding_client.py Wires Foundry policy and marks embedding usage.
python/packages/foundry/agent_framework_foundry/_chat_client.py Wires Foundry policy/http-client and sets Foundry chat feature index.
python/packages/foundry/agent_framework_foundry/_agent.py Wires Foundry policy/http-client and sets Foundry agent feature index.
python/packages/foundry_local/tests/test_foundry_local_client.py Adds assertion for Foundry Local feature index ownership.
python/packages/foundry_local/agent_framework_foundry_local/_foundry_local_client.py Sets Foundry Local feature index.
python/packages/foundry_local/agent_framework_foundry_local/_feature_usage.py Declares Foundry Local feature index.
python/packages/foundry_hosting/tests/test_toolbox.py Adds assertions for toolbox feature marking + index ownership.
python/packages/foundry_hosting/agent_framework_foundry_hosting/_toolbox.py Marks toolbox usage on connect.
python/packages/foundry_hosting/agent_framework_foundry_hosting/_responses.py Marks Foundry hosting usage.
python/packages/foundry_hosting/agent_framework_foundry_hosting/_invocations.py Marks Foundry hosting usage.
python/packages/foundry_hosting/agent_framework_foundry_hosting/_feature_usage.py Declares Foundry hosting feature indices.
python/packages/durabletask/agent_framework_durabletask/_worker.py Marks Durable Task usage when worker starts.
python/packages/durabletask/agent_framework_durabletask/_shim.py Marks Durable Task usage on shim run.
python/packages/durabletask/agent_framework_durabletask/_feature_usage.py Declares Durable Task feature index.
python/packages/devui/agent_framework_devui/_feature_usage.py Declares DevUI feature index.
python/packages/devui/agent_framework_devui/init.py Marks DevUI usage when starting server.
python/packages/declarative/tests/test_workflow_factory.py Adds assertions for declarative workflow feature marking.
python/packages/declarative/tests/test_declarative_loader.py Adds assertions for declarative agent feature marking.
python/packages/declarative/agent_framework_declarative/_workflows/_factory.py Marks declarative workflow usage on successful creation.
python/packages/declarative/agent_framework_declarative/_loader.py Marks declarative agent usage on successful creation.
python/packages/declarative/agent_framework_declarative/_feature_usage.py Declares declarative feature indices.
python/packages/core/tests/core/test_telemetry.py Adds comprehensive feature-mask + registry validator tests.
python/packages/core/README.md Documents telemetry controls (mask-only vs full UA disable).
python/packages/core/AGENTS.md Documents _telemetry.py module responsibility.
python/packages/core/agent_framework/_workflows/_workflow_builder.py Adds core workflow feature marking via _FEATURE_USAGE_INDEX.
python/packages/core/agent_framework/_telemetry.py Implements feature mask, token formatting, env var controls, and token (un)stamping helpers.
python/packages/core/agent_framework/_skills.py Marks skills provider and skills sources feature usage.
python/packages/core/agent_framework/_sessions.py Marks in-memory and file history provider usage.
python/packages/core/agent_framework/_mcp.py Marks MCP usage on connect.
python/packages/core/agent_framework/_harness/_tool_approval.py Marks tool-approval harness usage.
python/packages/core/agent_framework/_harness/_todo.py Marks todo harness usage.
python/packages/core/agent_framework/_harness/_mode.py Marks agent mode provider usage.
python/packages/core/agent_framework/_harness/_memory.py Marks memory provider usage.
python/packages/core/agent_framework/_harness/_file_access.py Marks file access provider usage.
python/packages/core/agent_framework/_harness/_background_agents.py Marks background agents provider usage.
python/packages/core/agent_framework/_harness/_agent.py Marks harness agent creation usage.
python/packages/core/agent_framework/_compaction.py Marks compaction provider usage.
python/packages/core/agent_framework/_agents.py Marks core agent usage on run.
python/packages/core/agent_framework/init.py Resolves version from installed agent-framework-core distribution.
python/packages/copilotstudio/tests/test_copilot_agent.py Adds assertions for Copilot Studio feature marking.
python/packages/copilotstudio/agent_framework_copilotstudio/_feature_usage.py Declares Copilot Studio feature index.
python/packages/copilotstudio/agent_framework_copilotstudio/_agent.py Marks Copilot Studio usage on run/stream.
python/packages/claude/tests/test_claude_agent.py Adds assertions for Claude feature marking.
python/packages/claude/agent_framework_claude/_feature_usage.py Declares Claude feature index.
python/packages/claude/agent_framework_claude/_agent.py Marks Claude usage on query/receive path.
python/packages/chatkit/agent_framework_chatkit/_streaming.py Marks ChatKit usage on streaming conversion.
python/packages/chatkit/agent_framework_chatkit/_feature_usage.py Declares ChatKit feature index.
python/packages/chatkit/agent_framework_chatkit/_converter.py Marks ChatKit usage on message conversion.
python/packages/bedrock/tests/test_bedrock_client.py Adds assertions for Bedrock feature marking.
python/packages/bedrock/agent_framework_bedrock/_feature_usage.py Declares Bedrock feature index.
python/packages/bedrock/agent_framework_bedrock/_embedding_client.py Marks Bedrock embedding usage.
python/packages/bedrock/agent_framework_bedrock/_chat_client.py Marks Bedrock chat usage.
python/packages/azurefunctions/agent_framework_azurefunctions/_feature_usage.py Declares Azure Functions feature index.
python/packages/azurefunctions/agent_framework_azurefunctions/_app.py Marks Azure Functions app usage on init.
python/packages/azure-cosmos/tests/test_cosmos_history_provider.py Adds assertions for Cosmos feature marking.
python/packages/azure-cosmos/agent_framework_azure_cosmos/_history_provider.py Marks Cosmos history provider usage.
python/packages/azure-cosmos/agent_framework_azure_cosmos/_feature_usage.py Declares Cosmos feature index.
python/packages/azure-cosmos/agent_framework_azure_cosmos/_checkpoint_storage.py Marks Cosmos checkpoint storage usage.
python/packages/azure-cosmos-memory/tests/test_context_provider.py Adds assertions for Cosmos memory feature marking.
python/packages/azure-cosmos-memory/agent_framework_azure_cosmos_memory/_feature_usage.py Declares Cosmos memory feature index.
python/packages/azure-cosmos-memory/agent_framework_azure_cosmos_memory/_context_provider.py Marks Cosmos memory provider usage.
python/packages/azure-contentunderstanding/tests/cu/test_context_provider.py Adds assertions for Content Understanding feature marking.
python/packages/azure-contentunderstanding/agent_framework_azure_contentunderstanding/_feature_usage.py Declares Content Understanding feature index.
python/packages/azure-contentunderstanding/agent_framework_azure_contentunderstanding/_context_provider.py Marks Content Understanding provider usage.
python/packages/azure-ai-search/tests/test_aisearch_context_provider.py Adds assertions for Azure AI Search feature marking.
python/packages/azure-ai-search/agent_framework_azure_ai_search/_feature_usage.py Declares Azure AI Search feature index.
python/packages/azure-ai-search/agent_framework_azure_ai_search/_context_provider.py Marks Azure AI Search provider usage.
python/packages/anthropic/tests/test_anthropic_client.py Adds assertions for Anthropic feature marking.
python/packages/anthropic/agent_framework_anthropic/_feature_usage.py Declares Anthropic feature index.
python/packages/anthropic/agent_framework_anthropic/_chat_client.py Marks Anthropic chat usage.
python/packages/ag-ui/agent_framework_ag_ui/_workflow.py Marks AG-UI workflow usage.
python/packages/ag-ui/agent_framework_ag_ui/_feature_usage.py Declares AG-UI feature index.
python/packages/ag-ui/agent_framework_ag_ui/_client.py Marks AG-UI client streaming usage.
python/packages/ag-ui/agent_framework_ag_ui/_agent.py Marks AG-UI agent usage.
python/packages/a2a/agent_framework_a2a/_feature_usage.py Declares A2A feature index.
python/packages/a2a/agent_framework_a2a/_agent.py Marks A2A usage in agent mapping path.
python/packages/a2a/agent_framework_a2a/_a2a_executor.py Marks A2A usage in executor path.

Comment thread python/packages/foundry/agent_framework_foundry/_agent.py
@eavanvalkenburg

Copy link
Copy Markdown
Member Author

Addressed the automated review summary in 6187492: caller-provided Foundry project clients now retain their transport, conversation creation reuses the existing OpenAI client, and LocalShellTool plus DockerShellTool have explicit activation-time TOOLS_SHELL marking assertions.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated Code Review

Reviewers: 5 | Confidence: 62%

✓ Correctness

No actionable issues found in this dimension.

✓ Security Reliability

No actionable issues found in this dimension.

✓ Test Coverage

The PR adds comprehensive feature-usage telemetry marks across ~20 packages. Core telemetry logic is well-tested (mask accumulation, threading, disable controls, registry parity). Most leaf-package marks have dedicated tests. However, several packages show mark_feature_used calls without corresponding test assertions visible in the diff: AG-UI (3 call sites), A2A (2 call sites), ChatKit (2 call sites), DevUI, Azure Functions, DurableTask (2 call sites), Bedrock embedding client, and most core harness/session/skills providers. While 53+ additional test files are noted as omitted from the diff, the coverage pattern across the visible packages suggests some of these may genuinely lack unit-level mark verification. The PR adds feature-usage marking across ~25 packages. Most packages with visible test changes have appropriate test assertions verifying mark_feature_used is called with the correct FeatureIndex. The structural registry-parity test in test_telemetry.py provides broad declarative coverage but does not verify runtime call-site behavior. The new extra_headers pass-through in the ChatCompletion client also lacks direct test coverage.

✓ Failure Modes

No actionable issues found in this dimension.

✓ Design Approach

I found one non-blocking design concern: some hosting feature bits are marked during object construction rather than at first meaningful use, which can over-report capabilities that were instantiated during startup but never actually exercised. The rest of the chunk is consistent with the PR’s feature-usage approach and the approved-origin transport handling.

Suggestions

  • The AG-UI package adds mark_feature_used(FeatureIndex.AG_UI) in three distinct entry points (_agent.py:144, _client.py:400, _workflow.py:300). If these lack dedicated tests (not visible in this diff chunk), consider adding at least one test per entry point to match the pattern used by other packages.

Automated review by eavanvalkenburg's agents

@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to a conflict with the base branch Jul 30, 2026
Add the 128-bit feature accumulator, package-local indexes, activation markers, and destination-scoped User-Agent emission for the initial Python implementation slice.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 346bf168-b668-4c4a-a8db-a67282ee5e5f
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 346bf168-b668-4c4a-a8db-a67282ee5e5f
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 346bf168-b668-4c4a-a8db-a67282ee5e5f
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 346bf168-b668-4c4a-a8db-a67282ee5e5f
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 346bf168-b668-4c4a-a8db-a67282ee5e5f
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 346bf168-b668-4c4a-a8db-a67282ee5e5f
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 346bf168-b668-4c4a-a8db-a67282ee5e5f
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 346bf168-b668-4c4a-a8db-a67282ee5e5f
Merged via the queue into microsoft:main with commit b64a2e2 Jul 30, 2026
49 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Usage: [Issues, PRs], Target: documentation in the code base and learn docs lab Usage: [Issues, PRs], Target: lab packages python Usage: [Issues, PRs], Target: Python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Python: [Feature]: Python feature-usage bitmask

4 participants