docs(comparison): audit competitive matrix vs latest vendor docs + distilled snapshots

docs/06_vendor_docs/README.md

@@ -9,6 +9,7 @@ Engine-, provider-, or vendor-specific references, adapter notes, version quirks
 | [`github-branch-protection.md`](github-branch-protection.md) | Two procedures (modern Rulesets + classic Branch Protection) for requiring CI status checks before merge to `main`; the three exact check names for the `relyloop` repo; verification + gotchas | `infra_foundation` plan §7.5 manual handoff #3; every operator who needs to update branch rules |
 | [`solr-10/`](solr-10/) | Apache Solr 10.0 ref-guide pages (asciidoc source, tag `releases/solr/10.0.0`) — matches the `solr:10.0` image. Module loading (`SOLR_MODULES`, no `<lib>`), configsets + UPLOAD API, LTR, auth | `infra_adapter_solr` (MVP2 Solr adapter + Compose infra) |
 | [`solr-9/`](solr-9/) | Apache Solr 9.9 ref-guide pages (asciidoc source, tag `releases/solr/9.9.0`) for cross-version comparison | `infra_adapter_solr` — confirm which behaviours differ between 9.x and 10 |
+| [`relevance-tools/`](relevance-tools/) | Distilled capability snapshots (with upstream URLs + access dates) of the adjacent/competing relevance tools — OpenSearch SRW + Relevance Agent, Quepid, RRE, Chorus, Elasticsearch native, Splainer. Competitive-landscape references, not integration targets | Evidence base for [`docs/07_research/comparison.md`](../07_research/comparison.md); refresh when a tool ships a release that flips a comparison cell |
 
 ### Solr docs detail
 

docs/06_vendor_docs/relevance-tools/README.md

@@ -0,0 +1,32 @@
+# Adjacent relevance tools — distilled snapshots
+
+Distilled capability snapshots of the OSS/commercial relevance-tuning tools
+that overlap RelyLoop's surface. These are the **evidence base** for the
+competitive matrix in
+[`docs/07_research/comparison.md`](../../07_research/comparison.md): each note
+records what the tool does, the version/status, the license/tier, the upstream
+URLs, and the **access date** the claims were verified against.
+
+These are *competitive-landscape* references, distinct from the
+engine-integration references at the top level (`solr-9/`, `solr-10/`) — RelyLoop
+does not integrate with these tools; it is compared against them.
+
+## Index
+
+| Note | Tool(s) | Backs comparison columns |
+|---|---|---|
+| [`opensearch-relevance.md`](opensearch-relevance.md) | OpenSearch Search Relevance Workbench (SRW) + OpenSearch Relevance Agent | "OpenSearch SRW", "OpenSearch Relevance Agent" |
+| [`quepid.md`](quepid.md) | Quepid (OpenSource Connections / o19s) | "Quepid" |
+| [`rated-ranking-evaluator.md`](rated-ranking-evaluator.md) | Rated Ranking Evaluator (Sease) | "RRE" |
+| [`chorus.md`](chorus.md) | Chorus (Querqy / o19s) | "Chorus" |
+| [`elasticsearch-native.md`](elasticsearch-native.md) | Elasticsearch native (`_rank_eval`, LTR, deprecated Behavioral Analytics / Search Applications) | "Elasticsearch (native)" |
+| [`splainer.md`](splainer.md) | Splainer (o19s) | "Splainer" |
+
+## Maintenance
+
+Refresh a note when the tool ships a release that changes a comparison cell.
+Always update the **Access date** line and re-verify the upstream URLs when you
+touch a note, then propagate any cell change into `comparison.md` (and bump its
+"Last updated" line). Corrections preferred over new claims.
+
+**Last audited:** 2026-05-31.

docs/06_vendor_docs/relevance-tools/chorus.md

@@ -0,0 +1,39 @@
+# Chorus
+
+**Vendor:** Querqy / OpenSource Connections community (Apache 2.0).
+**Access date:** 2026-05-31.
+
+A **reference integration stack** for e-commerce search — a way to stand up a
+realistic relevance stack (search engine + Querqy rules + SMUI + UBI +
+Quepid/RRE) rather than a single tuning tool.
+
+**Capabilities (verified 2026-05-31):**
+
+- **Engine support — three editions:** Solr (the original/primary,
+  [`querqy/chorus`](https://github.com/querqy/chorus)), **Elasticsearch**
+  ([`querqy/chorus-elasticsearch-edition`](https://github.com/querqy/chorus-elasticsearch-edition)),
+  and **OpenSearch**
+  ([`o19s/chorus-opensearch-edition`](https://github.com/o19s/chorus-opensearch-edition)).
+  The ES/OpenSearch editions trail the Solr original in completeness (e.g. SMUI
+  was built for Solr; **Querqy was removed from the OpenSearch edition** until
+  updated for recent OpenSearch versions). → **This corrects the earlier
+  comparison claim of "Solr (primary) + OpenSearch (partial)," which omitted
+  the Elasticsearch edition.**
+- **UBI** — Chorus (esp. the OpenSearch edition) is a reference **UBI showcase**;
+  UBI itself has plugins for OpenSearch, Elasticsearch, and Solr.
+- **Evaluation / A/B** — via the bundled tools (Quepid/RRE), not a native Chorus
+  feature.
+- **Optimization** — none of its own. No automated sweep, no Bayesian search,
+  no LLM-driven loop, no apply path. It's an assembly of components.
+
+**Why it matters vs RelyLoop:** Chorus shows how the OSS relevance pieces fit
+together across engines, but the *optimization* is left to the human using the
+bundled tools — there is no autonomous loop or Git-PR apply path.
+
+**Upstream URLs:**
+
+- Solr (primary) — https://github.com/querqy/chorus
+- Elasticsearch edition — https://github.com/querqy/chorus-elasticsearch-edition
+- OpenSearch edition — https://github.com/o19s/chorus-opensearch-edition
+- Querqy umbrella — https://querqy.org/
+- UBI — https://www.ubisearch.dev/

docs/06_vendor_docs/relevance-tools/elasticsearch-native.md

@@ -0,0 +1,39 @@
+# Elasticsearch (native relevance tooling)
+
+**Vendor:** Elastic. **License:** Elastic License 2.0 + SSPL (not OSI open
+source); feature gating by subscription tier.
+**Access date:** 2026-05-31. **Latest major reviewed:** Elasticsearch 9.x.
+
+What Elasticsearch ships *natively* for relevance evaluation/tuning — distinct
+from third-party tools that run against it.
+
+**Capabilities (verified 2026-05-31):**
+
+- **`_rank_eval` API** — an evaluation **primitive** (computes MRR, precision,
+  DCG/nDCG over a query set + rated docs). Available in the **Basic** tier. It
+  is an API, not a UI/workbench, and it is **not deprecated** in 9.x.
+- **No native SRW/RelyLoop equivalent.** Elastic **deprecated** its
+  higher-level **Behavioral Analytics** and **Search Applications** in
+  **9.0.0** (Behavioral Analytics emits deprecation warnings; **App Search is
+  discontinued** with Stack 9.0 and Enterprise Search ships no new majors past
+  8.x). Search Applications + Behavioral Analytics required a paid tier
+  (trial/Platinum/Enterprise) even before deprecation.
+- **Learning to Rank (LTR)** — Elasticsearch supports **model inference** for
+  LTR (GBDT/LambdaMART; training happens outside ES). LTR + ML inference are
+  **paid-tier** features (Platinum or higher), not Basic.
+- **Subscription tiers (2026):** Free/Basic, Gold (Elastic Cloud Hosted only),
+  Platinum, Enterprise.
+- **Optimization / apply path** — none native. No automated sweep, no Bayesian
+  search, no Git-PR apply.
+
+**Why it matters vs RelyLoop:** ES gives you the evaluation primitive
+(`_rank_eval`) but no optimizer, no workbench (the higher-level products are
+deprecated), and its advanced ranking (LTR) is paywalled. RelyLoop runs
+against ES (Apache-2.0) and adds the loop + apply path on top.
+
+**Upstream URLs:**
+
+- 9.0 release notes (deprecations) — https://www.elastic.co/guide/en/elastic-stack/9.0/release-notes-elasticsearch-9.0.0.html
+- `_rank_eval` — https://www.elastic.co/docs/reference/elasticsearch/rest-apis/search-rank-eval
+- LTR — https://www.elastic.co/docs/solutions/search/ranking/learning-to-rank-ltr
+- Subscriptions / tiers — https://www.elastic.co/subscriptions

docs/06_vendor_docs/relevance-tools/opensearch-relevance.md

@@ -0,0 +1,82 @@
+# OpenSearch relevance tooling (SRW + Relevance Agent)
+
+**Vendor:** OpenSearch project (Apache 2.0).
+**Access date:** 2026-05-31. **Latest version reviewed:** OpenSearch 3.6.
+
+Two distinct, related products ship under the OpenSearch umbrella. RelyLoop is
+compared against both.
+
+---
+
+## OpenSearch Search Relevance Workbench (SRW)
+
+A Dashboards-based workbench for evaluating and comparing search
+configurations. The closest competitor to RelyLoop.
+
+**Capabilities (verified 2026-05-31):**
+
+- **Query sets, judgments, experiments** — sampling test queries, managing
+  relevance ratings, and comparing configurations with metrics. New metrics
+  added through 3.5/3.6: Recall@K, MRR, DCG@K.
+- **LLM-as-judge** — GA in 3.5; customizable prompts; requires a configured
+  LLM connector.
+- **UBI-derived judgments** — COEC click model, GA.
+- **Scheduled experiments** — nightly/weekly cadence, GA in 3.5.
+- **Multi-datasource** — run experiments across multiple sources, added 3.6.
+- **Hybrid Search Optimizer** — a **grid search**, not Bayesian. The "Global"
+  optimizer tests **66 combinations** = `{2 normalization × 3 combination ×
+  11 weight steps}`, over **hybrid lexical/neural weights only** — not field
+  boosts or broader query-time parameters. A separate "Dynamic" optimizer
+  (ML regression / random forest to predict per-query "neuralness") and the
+  grid optimizer both live in an **external o19s notebook repository**, not as
+  built-in OpenSearch features. Bayesian optimization is listed as
+  **"not yet implemented"** in that repo and is tracked in
+  [RFC #934](https://github.com/opensearch-project/neural-search/issues/934).
+- **Apply path** — none. The SRW RFC scopes the product to evaluation and
+  analysis (query sets / judgments / experimentation); it does not describe
+  applying winning configs to production, promoting settings, or managing
+  rollouts. No PR/Git apply mechanism.
+
+**Why it matters vs RelyLoop:** SRW is the most feature-complete competitor,
+but (1) its optimizer is a 66-cell grid over hybrid weights, versus RelyLoop's
+Optuna/TPE Bayesian search over the full query-time space; (2) it has no
+apply path; (3) it is OpenSearch-only.
+
+**Upstream URLs:**
+
+- Workbench docs — https://docs.opensearch.org/latest/search-plugins/search-relevance/using-search-relevance-workbench/
+- Hybrid search optimizer docs — https://docs.opensearch.org/latest/search-plugins/search-relevance/optimize-hybrid-search/
+- Optimization blog — https://opensearch.org/blog/hybrid-search-optimization/
+- External optimizer notebooks (o19s) — https://github.com/o19s/opensearch-hybrid-search-optimization
+- Judgments (COEC) — https://docs.opensearch.org/latest/search-plugins/search-relevance/judgments/
+- SRW scope RFC #17735 — https://github.com/opensearch-project/OpenSearch/issues/17735
+- Hybrid optimizer RFC #934 — https://github.com/opensearch-project/neural-search/issues/934
+- Repo — https://github.com/opensearch-project/search-relevance
+- 3.5 / 3.6 release notes — https://github.com/opensearch-project/opensearch-build/blob/main/release-notes/opensearch-release-notes-3.5.0.md · https://github.com/opensearch-project/opensearch-build/blob/main/release-notes/opensearch-release-notes-3.6.0.md
+
+---
+
+## OpenSearch Relevance Agent
+
+An **experimental** (3.6) conversational, agent-driven tuning system in
+OpenSearch Dashboards. A separate product from SRW.
+
+**Capabilities (verified 2026-05-31):**
+
+- A **multi-agent, hypothesis-driven** workflow: a User-Behavior-Analysis agent
+  identifies relevance gaps (from UBI data or query patterns), a
+  Hypothesis-Generator agent proposes tuning strategies, and an Evaluator agent
+  validates them via **offline evaluation on historical data**.
+- Produces **query-DSL-level optimizations** — refining search fields,
+  adjusting weights, tuning boost functions. Human-in-the-loop ("you remain the
+  ultimate decision-maker").
+- **Does not** run large automated parameter sweeps or Bayesian optimization,
+  and **does not** apply changes to production (online/interleaving tests are
+  listed as future work).
+
+**Why it matters vs RelyLoop:** it's conversational like RelyLoop's chat agent,
+but it recommends/validates DSL changes one hypothesis at a time rather than
+running thousands of Bayesian trials, and it is OpenSearch-only with no
+Git-PR apply path.
+
+**Upstream URL:** https://opensearch.org/blog/introducing-opensearch-relevance-agent-ai-powered-search-tuning/

docs/06_vendor_docs/relevance-tools/quepid.md

@@ -0,0 +1,39 @@
+# Quepid
+
+**Vendor:** OpenSource Connections / o19s (Apache 2.0).
+**Access date:** 2026-05-31.
+
+The de-facto open-source **interactive relevance workbench**: create test
+cases ("cases"/"books"), gather human judgments, compute metrics (nDCG, etc.),
+and watch them move as you tweak the query configuration by hand.
+
+**Capabilities (verified 2026-05-31):**
+
+- **Engine support** — OpenSearch, Elasticsearch, Solr, plus Vectara, Algolia,
+  and custom Search APIs. (Diagnostics are powered by the shared `splainer-search`
+  library.)
+- **LLM-as-judge — built into the OSS core.** Quepid ships an **"AI Judge"**
+  ("LLM as a Judge") feature, added in **v8.0.0 (2025-02-14)** in the
+  open-source repo (not a paid/hosted-only feature). You set an LLM key, supply
+  a customizable judgment prompt, and the AI Judge rates query/document pairs.
+  Later releases added Ollama support and encrypted key storage (8.2.0). →
+  **This corrects the earlier comparison claim that Quepid's LLM-judge was a
+  "community plugin, not in the OSS core."**
+- **A/B comparison** — yes, but **manual/human-driven**: a person changes the
+  query config and compares cases. No automated parameter sweep.
+- **Multi-cluster** — yes (multiple cases/configs).
+- **Optimization** — none. Quepid does not run automated parameter sweeps or
+  Bayesian optimization, and does not write/apply search configs (it manages
+  judgments and measurement, not config deployment).
+
+**Why it matters vs RelyLoop:** Quepid is the strongest tool for human-rated
+judgment management and manual A/B, and it now applies LLMs to *judging*. But
+its LLM use is for judging, not for *driving an optimization loop* — there is
+no autonomous sweep and no apply path.
+
+**Upstream URLs:**
+
+- Repo — https://github.com/o19s/quepid
+- Changelog (AI Judge debuts v8.0.0) — https://github.com/o19s/quepid/blob/main/CHANGELOG.md
+- Product site — https://www.quepidapp.com/
+- User manual — https://quepid-docs.dev.o19s.com/

docs/06_vendor_docs/relevance-tools/rated-ranking-evaluator.md

@@ -0,0 +1,32 @@
+# Rated Ranking Evaluator (RRE)
+
+**Vendor:** Sease Ltd (Apache 2.0).
+**Access date:** 2026-05-31.
+
+An **offline search-quality evaluation** library/toolkit for Apache Solr and
+Elasticsearch. Aimed at the search engineer's CI workflow.
+
+**Capabilities (verified 2026-05-31):**
+
+- **Engine support** — Apache Solr + Elasticsearch.
+- **How you run it** — primarily a **Java library invoked via a Maven plugin**
+  (set up a Maven project, import RRE, run the evaluation); there is also an
+  RRE Server (web console). A standalone **CLI is in development / not part of
+  the current release** — so "CLI-driven" overstates it; "Maven-plugin / CI-driven"
+  is accurate.
+- **A/B comparison / evaluation** — yes, offline; compares rankings across
+  configurations/versions and computes IR metrics.
+- **Scheduled / unattended** — via CI/cron driving the Maven build (not a
+  built-in scheduler).
+- **Optimization** — none. RRE evaluates and reports; it does not run automated
+  parameter sweeps, is not LLM-driven, and has no apply path.
+
+**Why it matters vs RelyLoop:** RRE is a mature offline evaluation harness for
+the Solr/ES world, but it's a measurement tool, not an optimizer — no Bayesian
+search, no LLM, no Git-PR apply.
+
+**Upstream URLs:**
+
+- Repo — https://github.com/SeaseLtd/rated-ranking-evaluator
+- Wiki — https://github.com/SeaseLtd/rated-ranking-evaluator/wiki
+- Overview — https://sease.io/2021/01/offline-search-quality-evaluation-rated-ranking-evaluator-rre.html

docs/06_vendor_docs/relevance-tools/splainer.md

@@ -0,0 +1,29 @@
+# Splainer
+
+**Vendor:** OpenSource Connections / o19s (Apache 2.0).
+**Access date:** 2026-05-31.
+
+A **relevance diagnostics sandbox** — paste a search URL with all its query
+params and Splainer shows the results alongside parsed, summarized `explain`
+information so you can understand *why* documents rank where they do. A
+debugging tool, not an optimizer.
+
+**Capabilities (verified 2026-05-31):**
+
+- **Engine support** — Solr, Elasticsearch, and **OpenSearch** (with
+  experimental support for others, e.g. Vectara). → **This corrects the earlier
+  comparison claim of "Solr + ES," which omitted OpenSearch.** The underlying
+  `splainer-search` library also powers Quepid's diagnostics.
+- **What it does** — interactive explain/score breakdown and query tweaking in a
+  sandbox.
+- **What it doesn't** — no judgments/metrics workflow, no automated A/B at
+  scale, no parameter sweep, no LLM, no scheduling, no apply path.
+
+**Why it matters vs RelyLoop:** Splainer is the "why did this rank?" microscope;
+it's complementary to RelyLoop, not a competitor on optimization or apply.
+
+**Upstream URLs:**
+
+- Sandbox app — https://splainer.io/
+- `splainer-search` library — https://github.com/o19s/splainer-search
+- Splainer app repo — https://github.com/o19s/splainer