Files

T

saqib mirandClaude Sonnet 5 4a159725c2 docs(015-reporting-dashboards): plan, research, data model, contract, quickstart

Documents the exact Prisma query per dashboard figure, the one new
durable table this feature needs (ErrorCodeLookup — 014's own equivalent
metric is process-lifetime, unusable for a historical report), the
"no data -> null, never NaN" convention, and why the AI dashboard's
confidence distribution deliberately uses the system-default threshold
rather than resolving a per-diagnosis policy (AIDiagnosis has no
reliable FK back to which policy applied).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

2026-09-09 11:14:29 +05:30

2.8 KiB

Raw Blame History

Quickstart: Reporting and Analytics Dashboards

Manual verification steps for each user story, against a running instance backed by real Postgres/Redis, logged in as an ADMIN.

Scenario 1 — Management dashboard (User Story 1)

Create several tickets within a known date range: some reaching AI_RESOLVED/RESOLVED via an AI session, some escalated to a human and resolved via resolutionsService.record, some left open.
Let one ticket's SLA run complete on time and another breach (via the existing breach sweep).
GET /admin/reports/management?from=<range start>&to=<range end>.
Expected: totalCases, aiResolved, humanEscalated, resolved, open all match what was actually created; slaCompliance.met/.breached match the two SLA outcomes; averageResponseSeconds/averageResolutionSeconds are non-null and plausible.
Request the same endpoint for a date range with no activity at all.
Expected: every count is 0, every rate/average is null, not an error.

Scenario 2 — Product dashboard (User Story 2)

Create tickets for two distinct products in the same range, one with a categorized problem.
Look up a known error code for one product several times, a different code once.
GET /admin/reports/product/:externalProductId for each product.
Expected: each product's supportVolume/problemsByCategory/aiResolutionRate reflect only its own tickets; topErrors ranks the more-frequently-looked-up code first.
Request the endpoint for a nonexistent externalProductId.
Expected: 404 NOT_FOUND, not an empty 200.

Scenario 3 — Support dashboard (User Story 3)

Assign several tickets across two agents (some via the real orchestration flow).
Let one ticket's SLA run sit within REPORTING_SLA_RISK_THRESHOLD_MINUTES of its resolution due date without breaching.
GET /admin/reports/support.
Expected: workloadByAgent matches each agent's real current open-assignment count; slaAtRisk counts exactly the near-due run, distinct from slaBreached.

Scenario 4 — AI dashboard (User Story 4)

Run AI sessions to a mix of terminal outcomes (resolved, escalated), with some tool invocations succeeding and others failing, and diagnoses spanning a range of confidence values.
GET /admin/reports/ai.
Expected: aiResolutionRate/humanHandoffRate reflect the real outcome mix; confidenceDistribution buckets match decideConfidenceBand's own classification of each diagnosis's stored confidence against the system-default thresholds; toolInvocations reflects the real success/failure counts.

What "done" looks like

All four scenarios pass against a real Postgres/Redis, every figure independently verified against hand-computed expected values, and no route is reachable by a non-admin session.

2.8 KiB Raw Blame History