support_backend/specs/015-reporting-dashboards/spec.md

# Feature Specification: Reporting and Analytics Dashboards

**Feature Branch**: `015-reporting-dashboards`

**Created**: 2026-09-09

**Status**: Draft

**Input**: User description: "Reporting and analytics dashboards: real, read-only aggregation endpoints backing the four dashboards named in docs/09-testing-observability-cicd.md (Management, Product, Support, AI) — wiring up the pre-scaffolded but never-implemented platform/reports module into actual database-backed aggregation queries, admin-gated, with a date-range filter."

## User Scenarios & Testing *(mandatory)*

### User Story 1 - Management sees organization-wide support health (Priority: P1)

An admin or team lead opens a single view showing how support is doing overall for a chosen
period: how many cases came in, how many were resolved (by AI vs. by a human), how many are
still open, whether SLA commitments are being met, and how escalation is trending.

**Why this priority**: This is the one dashboard covering the whole roadmap's own top-level
success criteria (`docs/10-implementation-roadmap.md`'s checklist) in one place — the first
thing anyone asks about a support operation is "how are we doing," and today there is no way to
answer that except querying the database by hand.

**Independent Test**: Can be fully tested by creating a known set of tickets in various terminal
states (AI-resolved, human-resolved, still open) plus a mix of met/breached SLA runs and
escalations within a chosen date range, then requesting the Management dashboard for that range
and confirming every figure matches what was actually created.

**Acceptance Scenarios**:

1. **Given** a mix of tickets created within a chosen date range — some AI-resolved, some
   human-resolved, some still open — **When** the Management dashboard is requested for that
   range, **Then** total cases, AI-resolved count, human-escalated count, resolved count, and
   open count all match the actual data exactly.
2. **Given** SLA runs that completed on time and others that breached within the range,
   **When** the dashboard is requested, **Then** SLA compliance (a rate) and SLA breach count
   both reflect the real outcomes.
3. **Given** some tickets have a recorded first agent response and a resolution timestamp,
   **When** the dashboard is requested, **Then** average response time and average resolution
   time are computed only from tickets that actually reached those milestones within the range
   (a still-open ticket contributes to "open count" but never a fabricated resolution time).
4. **Given** a date range with zero activity, **When** the dashboard is requested, **Then** every
   count is zero and every average is reported as "no data" rather than a computed zero or a
   division-by-zero error.

---

### User Story 2 - See support broken down by product (Priority: P1)

An admin viewing support data for a specific product (or comparing products) sees volume,
problem-type breakdown, which problems recur most, how well AI is resolving that product's
issues versus escalating them, and which error codes come up most often.

**Why this priority**: SupportHub serves multiple SaaS products (Constitution Principle I); a
number that isn't broken out by product hides which integration actually needs attention — this
is as fundamental as the Management view, just sliced differently.

**Independent Test**: Can be fully tested by creating tickets/problems/error-code lookups across
two distinct products within a date range, requesting the Product dashboard for each product,
and confirming each one's figures include only its own product's data.

**Acceptance Scenarios**:

1. **Given** tickets exist for two different products in the same date range, **When** the
   Product dashboard is requested scoped to one product, **Then** support volume and every other
   figure reflect only that product's tickets, never the other product's.
2. **Given** problems in several categories for one product, **When** the dashboard is
   requested, **Then** the problem-type breakdown and "recurring problems" ranking both reflect
   the real category distribution, most-frequent first.
3. **Given** a mix of AI-resolved and human-escalated tickets for one product, **When** the
   dashboard is requested, **Then** AI resolution rate and human escalation rate are both
   computed as a percentage of that product's own total, not the platform-wide total.
4. **Given** several known-error-code lookups for one product, some codes looked up more than
   others, **When** the dashboard is requested, **Then** "top errors" lists those codes ranked by
   lookup frequency.

---

### User Story 3 - Support sees team workload and performance (Priority: P2)

An admin or team lead sees how much work is currently assigned across agents, which tickets are
at SLA risk, how much escalation is happening, and how quickly the team is responding to and
resolving tickets.

**Why this priority**: This view is about ongoing operational load, not historical trend — useful
for day-to-day team management, but the organization can already see whether it's healthy
overall from User Story 1 without this one; P2 reflects that it adds an operational lens rather
than a new class of information.

**Independent Test**: Can be fully tested by assigning several tickets to known agents (some
close to SLA breach, some not), then requesting the Support dashboard and confirming workload
per agent and the SLA-risk count both match reality.

**Acceptance Scenarios**:

1. **Given** several tickets are currently assigned across two agents, **When** the Support
   dashboard is requested, **Then** each agent's current open-assignment count matches what was
   actually assigned to them (not a stale count from a previous, now-unassigned period).
2. **Given** a ticket's SLA run is running and past a configurable risk threshold of its
   resolution due date (but not yet breached), **When** the dashboard is requested, **Then** it
   is counted as "at risk," distinct from both "on track" and "breached."
3. **Given** response and resolution durations for several resolved tickets in the period,
   **When** the dashboard is requested, **Then** response-performance and resolution-performance
   figures are computed only from tickets that actually reached those milestones.

---

### User Story 4 - See how well the AI is performing (Priority: P2)

An admin sees, for a chosen period, how often the AI resolves issues on its own versus escalating
them, how often its attempted troubleshooting fails outright, how often it finds relevant
knowledge, how confident its diagnoses tend to be, how reliably its tools succeed, and how often
it ultimately hands off to a human.

**Why this priority**: This is the dashboard that validates the AI-first design's core premise
(Constitution Principle IV) is actually working in practice — valuable, but a narrower audience
than the org-wide and per-product views above, hence P2.

**Independent Test**: Can be fully tested by running several AI sessions to different terminal
outcomes (resolved, escalated, escalated-after-failed-troubleshooting) with a mix of tool
successes/failures and confidence levels recorded, then requesting the AI dashboard and
confirming every figure matches the real session data.

**Acceptance Scenarios**:

1. **Given** a mix of AI sessions ending resolved vs. escalated in the period, **When** the AI
   dashboard is requested, **Then** AI resolution rate and human-handoff rate both reflect the
   real outcome mix as percentages of total sessions.
2. **Given** some AI tool invocations succeeded and others failed in the period, **When** the
   dashboard is requested, **Then** tool success/failure figures reflect the real invocation
   outcomes.
3. **Given** diagnoses were recorded with a range of confidence values, **When** the dashboard is
   requested, **Then** the confidence distribution groups them into the same proceed/ask/escalate
   bands the AI support module's own confidence-policy service already classifies each diagnosis
   into (005-ai-support), not a newly-invented scheme.
4. **Given** some AI sessions' knowledge-retrieval step found matching entries and others found
   none, **When** the dashboard is requested, **Then** knowledge-match rate reflects the real
   match/no-match mix.

---

### Edge Cases

- What happens when no `from`/`to` date range is given? Defaults to a reasonable trailing window
  (see Assumptions) rather than scanning the entire history unbounded on every request.
- What happens when `from` is after `to`? Rejected as a validation error, not silently swapped or
  silently returning empty data.
- What happens when a requested `productId` (Product dashboard) doesn't exist? Rejected with a
  clear not-found error, not an empty-but-200 response that looks like "this product has zero
  activity."
- What happens when an average would divide by zero (no tickets reached that milestone in the
  range)? Reported as an explicit "no data" value, never `NaN`, `null` silently coerced to `0`,
  or a thrown error.
- What happens when a ticket's SLA run was paused for part of the period? SLA-risk/compliance
  figures use the run's own already-durable due dates (008-sla-escalation's pause/resume
  already accounts for paused time) rather than this feature re-deriving elapsed time itself.
- Who can see these dashboards? Same admin-only gate as every other admin configuration/reporting
  surface introduced since 010-identity-auth — no new role is introduced.

## Requirements *(mandatory)*

### Functional Requirements

- **FR-001**: System MUST provide a Management dashboard summarizing, for a given date range:
  total cases created, cases resolved by AI, cases escalated to a human, total resolved
  (either path), total still open, SLA compliance rate, SLA breach count, escalation count,
  average first-response time, and average resolution time.
- **FR-002**: System MUST provide a Product dashboard summarizing, for a given date range and a
  specific product: support volume, a breakdown by problem category, a ranked list of the most
  recurring problem categories, AI resolution rate, human escalation rate, and a ranked list of
  the most frequently looked-up error codes.
- **FR-003**: System MUST provide a Support dashboard summarizing, for a given date range:
  current per-agent open-assignment workload, count of tickets at SLA risk (past a configurable
  risk threshold of their resolution due date but not yet breached), count of tickets already
  breached, escalation count, average response performance, and average resolution performance.
- **FR-004**: System MUST provide an AI dashboard summarizing, for a given date range: AI
  resolution rate, rate of sessions that escalated after at least one failed troubleshooting
  attempt, knowledge-match rate, a distribution of diagnosis confidence across the existing
  proceed/ask/escalate bands, tool invocation success/failure counts, and human-handoff rate.
- **FR-005**: Every dashboard endpoint MUST accept an optional `from`/`to` date range; when
  omitted, it MUST default to a documented trailing window rather than scanning unbounded
  history.
- **FR-006**: The Product dashboard MUST require a valid `productId` and MUST reject an unknown
  one with a clear not-found error rather than returning an empty-but-successful response.
- **FR-007**: Every rate/average figure MUST be computed only from tickets/sessions/runs that
  actually reached the relevant milestone within the range; a metric with no qualifying data MUST
  be reported as an explicit "no data" value, never a computed `0`, `null`, or `NaN`.
- **FR-008**: All four dashboard endpoints MUST be admin-gated, consistent with every other
  admin-only reporting/configuration surface in this codebase.
- **FR-009**: This feature MUST NOT alter the meaning or shape of any existing endpoint, event, or
  table — nearly every figure is derived read-only from data already durably recorded by the
  modules that own it (003 ticketing, 005 AI support, 007 orchestration, 008 SLA/escalation, 009
  problem resolution). The one exception is FR-011: a small new durable record needed only
  because no existing table can answer "which error codes are looked up most" historically.
- **FR-011**: System MUST durably record each known-error-code lookup (product, error code,
  timestamp) at the point it already happens (the existing error-code lookup call site) so the
  Product dashboard's "top errors" ranking (FR-002) can be computed historically — the
  equivalent live, in-process counter this project already exposes on `/metrics` (014-full-
  observability) is process-lifetime and reset on every restart, unusable for a historical
  dashboard.
- **FR-010**: This feature is backend-only; presenting these figures in a UI is a separate,
  explicitly out-of-scope follow-on (see Assumptions).

### Key Entities

- **Dashboard response**: A read-only, computed JSON summary for one of the four dashboards over
  a requested date range (and, for the Product dashboard, one product) — never itself persisted;
  recomputed fresh on every request from existing durable records.
- **Date range**: An inclusive `from`/`to` pair (calendar dates or timestamps) scoping every
  aggregation query; not a stored entity, a request parameter.
- **Error code lookup record** (new, FR-011): a durable, append-only audit row — which product,
  which error code, when — written at the existing lookup call site; exists solely so "top
  errors" can be computed over a historical range, never read or written anywhere else.
- **Confidence band**: The existing proceed/ask/escalate classification 005-ai-support already
  applies to a diagnosis's confidence score — reused here for the AI dashboard's distribution, not
  redefined.

## Success Criteria *(mandatory)*

### Measurable Outcomes

- **SC-001**: For any chosen date range, every figure on all four dashboards can be independently
  verified against the underlying ticket/session/SLA-run/escalation-event records and matches
  exactly — no discrepancy between what a dashboard reports and what actually happened.
- **SC-002**: An admin can answer "how is support doing right now" (Management), "how is this
  specific product doing" (Product), "who's overloaded and what's at risk" (Support), and "is the
  AI actually helping" (AI) each from a single request, with no manual database query needed.
- **SC-003**: A dashboard request for a period with no matching activity returns clean, explicit
  "no data" results in well under a second — never an error, a stall, or a misleading zero.

## Assumptions

- **Presentation is out of scope for this feature.** The user's own explicit direction was to
  build the backend aggregation capability first (the established pattern this project has
  followed for every prior feature that touched both repos — identity/auth, the agent ticket
  queue, and full observability were each built backend-first). A `supporthub-web` dashboard UI
  consuming these endpoints is a natural, separate follow-on, not bundled into this spec.
- The default trailing window when no date range is given is the last 30 days, matching common
  reporting-dashboard convention; CONFIGURABLE via the same admin-config env-driven pattern this
  project already uses for every other business-policy value (Constitution Principle II), not
  hardcoded as a magic number in application logic.
- "SLA risk" needs a threshold (how close to the due date counts as "at risk") that the business
  has not specified — CONFIGURABLE, not invented as a hardcoded percentage, consistent with
  `docs/10-implementation-roadmap.md`'s own "never hardcode a placeholder value and ship it as
  final" instruction.
- These endpoints compute their figures synchronously, on request, directly from the existing
  tables — no new pre-aggregation table, no scheduled batch job, and no use of the pre-scaffolded
  `ANALYTICS` queue (`src/jobs/analytics`), which remains an inert stub outside this feature's
  scope. Live query performance at current data volumes is assumed adequate; a future feature can
  introduce pre-aggregation if and when it's actually needed (load/concurrency testing, a
  separate not-yet-started Phase 11 sub-area, is where that question would be validated).
- "Top errors"/"recurring problems" rankings return a bounded top-N list (CONFIGURABLE limit,
  defaulting to 10) rather than the full distribution, matching how a dashboard is actually
  consumed.
- Dashboard responses are computed fresh per request (no caching layer) — acceptable given the
  assumed data volumes and consistent with not prematurely optimizing ahead of the load-testing
  phase.