Within Variance
— QSR +0.7pp week-over-week
n = 348 queries
PASS Logging artifact
PASS Decomposition completeness
PASS Trust gate
Hypothesis Evaluation (priority order)
Instrumentation / logging anomalyruled out
Connector / data pipeline changenot evaluated
Query understanding regressionruled out
Algorithm / model changenot evaluated
Experiment ramp / de-rampnot evaluated
AI feature effectmatched
Seasonal / externalruled out
User behavior shiftruled out
Search Quality Success for Customer Cohort FPS measured at
71.2% this week vs 70.5% last week
(+0.7pp). The movement is driven by a
+2.4pp increase in AI Trigger Rate (35.8% → 38.2%),
offset by a small Click Quality dip (-0.7pp) — consistent with the expected
inverse co-movement pattern when AI answers increase.
Sample size is modest (n=348); consider monitoring for another week
before drawing firm conclusions.
QSR Week-over-Week — Customer Cohort FPS
2026-02-10 → 2026-02-24
| Period | Queries | QSR | Click | AI Trigger | AI Success | Δ QSR |
|---|---|---|---|---|---|---|
| This week | 348 | 71.2% | 65.4% | 38.2% | 82.1% | +0.7pp |
| Last week | 347 | 70.5% | 66.1% | 35.8% | 80.9% | — |
Week-over-Week Change by Component
Δ percentage points
QSR (net)
+0.7
AI Trigger Rate
+2.4
AI Success Rate
+1.2
Click Quality
-0.7
Zero Result Rate
0.0
-5pp
0
+5pp
Expected co-movement
Pattern: AI Trigger ↑ and Click Quality ↓ is the expected inverse co-movement — more AI answers means fewer clicks. Net QSR is positive.
QSR Daily Trend — This Week vs Last Week
14d window
75%
70%
65%
MonTueWedThuFriSatSun
This week (71.2% avg, n=348)
Last week (70.5% avg, n=347)
QSR by Tenant Tier — Decomposition
contribution to blended Δ
| Segment | Queries | QSR | Δ QSR | Contribution |
|---|---|---|---|---|
| Enterprise | 89 | 73.8% | +2.1pp |
68%
|
| Premium | 124 | 71.0% | +0.4pp |
22%
|
| Standard | 135 | 69.5% | +0.1pp |
10%
|
Decomposition: Enterprise tier (n=89) drives 68% of the blended QSR movement at +2.1pp. Premium and Standard show small changes at +0.4pp and +0.1pp respectively.
Using search_query_relevance_metrics_enriched
(~695 queries; 2026-02-03 → 2026-02-24), QSR is measured as
max(click_component, ai_trigger × ai_success),
computed per-query then averaged across all queries in the period.
- This week = 2026-02-17 to 2026-02-24
- Last week = 2026-02-10 to 2026-02-17
- Filtered: searchExperience = 'fullPageSearch', is_hello = 0
Query used
View permalink ↗
SELECT
day,
COUNT(DISTINCT queryId) AS total_queries,
AVG(search_quality_success_value) AS qsr,
AVG(click_quality_value) AS click_comp,
AVG(ai_trigger) AS ai_trigger_rate
FROM production.search_metrics_enriched
WHERE day >= '2026-02-10'
AND day <= '2026-02-24'
AND searchExperience = 'fullPageSearch'
AND is_hello = 0
GROUP BY day ORDER BY day
day,
COUNT(DISTINCT queryId) AS total_queries,
AVG(search_quality_success_value) AS qsr,
AVG(click_quality_value) AS click_comp,
AVG(ai_trigger) AS ai_trigger_rate
FROM production.search_metrics_enriched
WHERE day >= '2026-02-10'
AND day <= '2026-02-24'
AND searchExperience = 'fullPageSearch'
AND is_hello = 0
GROUP BY day ORDER BY day
Phase 0: Classification
4 steps · 1.1s
›
Reasoning
Classify query intent and identify target metric
Phase 1: Schema Discovery
2 steps · 0.0s
›
Knowledge
metric_definitions.yaml
← data/knowledge/metric_definitions.yaml
Phase 1: Data Quality
3 steps · 11.9s
›
Knowledge
corrections.yaml
← knowledge/corrections/corrections.yaml
SQL Query
Data quality gate — check latest day and row counts
11.9s · 3 rows
SELECT day, COUNT(DISTINCT queryId) AS total
FROM production.search_metrics_enriched
WHERE day >= '2026-02-22' AND day <= '2026-02-25'
GROUP BY day ORDER BY day
FROM production.search_metrics_enriched
WHERE day >= '2026-02-22' AND day <= '2026-02-25'
GROUP BY day ORDER BY day
Reasoning
Phase 1: Data Quality — outcome
Phase Output (JSON)
Phase 2: Trend Analysis
3 steps · 42.4s
›
Phase 3: Product Decomposition
2 steps · 12.8s
IN PROGRESS
Phase 4: Synthesis
pending