Author SHA1 Message Date
saqib mirandClaude Sonnet 5 7106753ed3 fix(015-reporting-dashboards): count humanEscalated by Assignment existence, not status
ManagementRepository/ProductReportRepository.countEverEscalatedToHuman checked
a list of terminal statuses that ticket-state-machine.ts's own transition
table shows are reachable from BOTH the AI-resolved path and the
human-escalation path once they converge (RESOLUTION_PENDING_CUSTOMER,
RESOLVED, CLOSED, REOPENED). Every AI-resolved ticket was being double-counted
as human-escalated too — confirmed live against real seeded dev data
(humanEscalated: 34 out of totalCases: 34, an impossible 100%).

Fixed by keying off assignments: { some: {} } instead, since
orchestrationService.handleHumanEscalation is the only code path that ever
creates an Assignment row. Updated management-dashboard.test.ts's own
human-resolved fixture to create a real Assignment row, since it previously
relied on the now-fixed buggy status-based signal without one.

Found via manual verification against a real running dev server while
building supporthub-web's 002-reporting-dashboards-ui, not by any existing
automated test.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-09 16:31:13 +05:30
saqib mirandClaude Sonnet 5 d65683641a feat(015-reporting-dashboards): four real reporting/analytics endpoints
Wires the pre-scaffolded, unused platform/reports module (ReportsService
.generateSummaryReport previously returned {}) into four real, admin-
gated dashboards matching docs/09-testing-observability-cicd.md's own
table:

- GET /admin/reports/management: total cases, AI-resolved, human-
  escalated, resolved/open, SLA compliance/breaches, escalation count,
  average response/resolution time.
- GET /admin/reports/product/:externalProductId: support volume,
  problem-category breakdown, recurring problems, AI-resolution/human-
  escalation rate, top error codes.
- GET /admin/reports/support: current per-agent workload, SLA at-risk/
  breached counts, escalation count, response/resolution performance.
- GET /admin/reports/ai: AI resolution/human-handoff rate, failed-
  troubleshooting-then-escalated rate, knowledge-match rate, confidence
  distribution (reusing 005-ai-support's own decideConfidenceBand),
  tool invocation success/failure.

Every rate/average is number|null -- null means no qualifying data in
range, never a computed NaN or a misleading 0. Adds one new durable
table, ErrorCodeLookup, since 014-full-observability's own equivalent
metric is a process-lifetime Prometheus counter unusable for a
historical "top errors" report.

Verified end-to-end against real Postgres/Redis: every figure checked
against hand-computed expected values, including a no-activity range
(all-zero counts, all-null rates) and cross-product isolation.

Also fixes a real regression the new ErrorCodeLookup FK caused in the
pre-existing known-issues.test.ts (its afterAll deleted ErrorCode rows
before the now-referencing lookup rows).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-09 11:59:38 +05:30
saqib mirandClaude Sonnet 5 814d9d7b17 docs(015-reporting-dashboards): task breakdown
28 tasks across a shared Foundational phase (schema, config, shared
rate/date-range helpers, module scaffolding) and 4 independently-testable
dashboard user stories.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-09 11:15:22 +05:30
saqib mirandClaude Sonnet 5 4a159725c2 docs(015-reporting-dashboards): plan, research, data model, contract, quickstart
Documents the exact Prisma query per dashboard figure, the one new
durable table this feature needs (ErrorCodeLookup — 014's own equivalent
metric is process-lifetime, unusable for a historical report), the
"no data -> null, never NaN" convention, and why the AI dashboard's
confidence distribution deliberately uses the system-default threshold
rather than resolving a per-diagnosis policy (AIDiagnosis has no
reliable FK back to which policy applied).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-09 11:14:29 +05:30
saqib mirandClaude Sonnet 5 c4a2faa6e3 docs(015-reporting-dashboards): feature spec and quality checklist
Phase 11's third sub-area (reporting/analytics dashboards), per explicit
user direction. Backend-first scope (four read-only aggregation
endpoints wiring up the pre-scaffolded platform/reports module),
following the same backend-before-frontend pattern already established
for 010/011/014 this session — a supporthub-web dashboard UI is a
separate, not-yet-started follow-on.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-09 11:10:00 +05:30
45 changed files with 2544 additions and 14 deletions
@@ -0,0 +1,18 @@
-- CreateTable
CREATE TABLE "error_code_lookups" (
"id" TEXT NOT NULL,
"errorCodeId" TEXT NOT NULL,
"productId" TEXT NOT NULL,
"createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP,
CONSTRAINT "error_code_lookups_pkey" PRIMARY KEY ("id")
);
-- CreateIndex
CREATE INDEX "error_code_lookups_productId_createdAt_idx" ON "error_code_lookups"("productId", "createdAt");
-- AddForeignKey
ALTER TABLE "error_code_lookups" ADD CONSTRAINT "error_code_lookups_errorCodeId_fkey" FOREIGN KEY ("errorCodeId") REFERENCES "error_codes"("id") ON DELETE RESTRICT ON UPDATE CASCADE;
-- AddForeignKey
ALTER TABLE "error_code_lookups" ADD CONSTRAINT "error_code_lookups_productId_fkey" FOREIGN KEY ("productId") REFERENCES "products"("id") ON DELETE RESTRICT ON UPDATE CASCADE;
+19
View File
@@ -45,6 +45,7 @@ model Product {
tickets Ticket[]
knowledgeEntries KnowledgeEntry[]
errorCodes ErrorCode[]
errorCodeLookups ErrorCodeLookup[]
knownIssues KnownIssue[]
runbooks Runbook[]
aiConfidencePolicies AIConfidencePolicy[]
@@ -241,11 +242,29 @@ model ErrorCode {
product Product @relation(fields: [productId], references: [id])
knownIssues KnownIssue[]
lookups ErrorCodeLookup[]
@@unique([productId, code])
@@map("error_codes")
}
// 015-reporting-dashboards research.md §6: a durable, append-only audit row recording that a
// known-error-code lookup happened — 014-full-observability's own equivalent
// (supporthub_known_error_lookups_total) is a process-lifetime Prometheus counter, unusable for
// a historical "top errors" report. productId is denormalized from errorCode.productId so the
// Product dashboard's range query never needs to join back through ErrorCode just to filter.
model ErrorCodeLookup {
id String @id @default(cuid())
errorCodeId String
errorCode ErrorCode @relation(fields: [errorCodeId], references: [id])
productId String
product Product @relation(fields: [productId], references: [id])
createdAt DateTime @default(now())
@@index([productId, createdAt])
@@map("error_code_lookups")
}
model KnownIssue {
id String @id @default(cuid())
productId String
@@ -0,0 +1,94 @@
# Specification Quality Checklist: Reporting and Analytics Dashboards
**Purpose**: Validate specification completeness and quality before proceeding to planning
**Created**: 2026-09-09
**Feature**: [spec.md](../spec.md)
## Content Quality
- [x] No implementation details (languages, frameworks, APIs)
- [x] Focused on user value and business needs
- [x] Written for non-technical stakeholders
- [x] All mandatory sections completed
## Requirement Completeness
- [x] No [NEEDS CLARIFICATION] markers remain
- [x] Requirements are testable and unambiguous
- [x] Success criteria are measurable
- [x] Success criteria are technology-agnostic (no implementation details)
- [x] All acceptance scenarios are defined
- [x] Edge cases are identified
- [x] Scope is clearly bounded
- [x] Dependencies and assumptions identified
## Feature Readiness
- [x] All functional requirements have clear acceptance criteria
- [x] User scenarios cover primary flows
- [x] Feature meets measurable outcomes defined in Success Criteria
- [x] No implementation details leak into specification
## Notes
- This is `docs/10-implementation-roadmap.md`'s own Phase 11, third sub-area, per explicit user
direction (013 was the security pass, 014 was full observability). Backend-first scope
(Assumptions) follows the same pattern already established three times this session
(010-identity-auth, 011-agent-ticket-queue, and 014-full-observability's own frontend-free
scope) — a `supporthub-web` dashboard UI is a natural, separate follow-on, not re-litigated
here via a fresh question.
- The pre-scaffolded-but-inert `platform/reports` module (`ReportsService.generateSummaryReport`
currently returns `{}`) and the `ANALYTICS` queue stub (`src/jobs/analytics`, logs only) were
both confirmed via direct code inspection before writing this spec — the same
"provisioned before this session's rebuild but never wired up" pattern found repeatedly this
session. This feature wires up the former; the Assumptions section explicitly keeps the latter
out of scope (synchronous queries, no pre-aggregation job, for this first cut).
- All items pass; no revision iterations were needed. No [NEEDS CLARIFICATION] markers were
required — every open question (default date window, SLA-risk threshold, top-N limit) had a
reasonable, documented, CONFIGURABLE default (see Assumptions), matching the roadmap's own
"never hardcode a placeholder value and ship it as final" instruction.
## Implementation Notes (post-build)
- Named the Product dashboard's own repository class `ProductReportRepository` (not
`ProductRepository`) once it became clear resolving `externalProductId -> Product` should
reuse `catalog/products`' own already-public `productsRepository.findByExternalProductId`
rather than duplicating that lookup — avoids a name collision and keeps "one authority per
concern" (Constitution Principle I's spirit) for product resolution.
- `ManagementRepository` and `SupportRepository` both needed byte-identical
first-response-duration and resolution-duration queries. Extracted into a shared
`SharedReportRepository` both compose, rather than duplicating the Prisma query (or the
averaging helper alone) twice — discovered while writing the second repository and seeing the
copy-paste, not planned upfront in research.md.
- "Top errors"/"most common errors" resolution-back-to-`code` logic moved into
`ErrorCodesService.getTopErrorCodesForProduct` (a new method on the module that already owns
`ErrorCode`), rather than the reports module reaching into `errorCodesRepository`/
`errorCodeLookupRepository` directly — cleaner module-boundary ownership than research.md's
original per-repository sketch implied.
- The AI dashboard's "failed troubleshooting then escalated" figure (spec.md User Story 4) has
no single stored flag anywhere in this codebase — `classifyStepOutcome`'s per-step verdicts are
never persisted as their own durable record. Implemented as a documented proxy instead: an
escalated session with `toolCallCount > 0` attempted troubleshooting before giving up, one with
zero attempts escalated immediately. Documented directly in `ai.repository.ts`'s own code
comment, the same "honest, documented simplification" precedent research.md §7 already set for
the confidence-distribution bucketing.
- Three of this module's public exports needed adding to their owning modules' top-level
`index.ts` (not previously exposed): `decideConfidenceBand`/`ConfidenceBand` and
`knowledgeReferenceRepository` from `ai-support/sessions`, matching the "extend an existing
module's public surface for a later feature" precedent already used repeatedly this session
(004's `productsRepository`, 009's `problemsRepository`).
- Found a real regression during T028's full regression pass: `known-issues.test.ts` (004-
product-knowledge, pre-existing) calls `findKnownIssuesByErrorCode` and its own `afterAll`
deleted `ErrorCode` rows before this feature's new `ErrorCodeLookup` FK (RESTRICT) existed —
once T004 started writing a lookup row on every call, that cleanup order started failing with
an FK violation. Fixed by deleting `ErrorCodeLookup` rows first in that test's own `afterAll`.
This feature's own new test files never delete `ErrorCode` rows at all, so they weren't
affected the same way (leftover rows there are the same accepted throwaway-data tradeoff
already established elsewhere this session).
- Confirmed (not caused by this feature — the exact pre-existing issue 014-full-observability's
own checklist already documented and root-caused via `git checkout` comparison) that this
feature's own new integration test files, which also name their test products `TEST_*`,
occasionally hit the same shared `deriveProductCode` "TEST" prefix collision under vitest's
concurrent file execution when run alongside other `TEST_*`-prefixed files. Every dashboard
test passes reliably run individually or in small groups; the intermittent 500 in a full
combined run is the same known, out-of-scope, 003-ticketing concern.
@@ -0,0 +1,59 @@
# Contract: Reporting API
All four routes require a valid staff session with role `ADMIN` (`requireRole('ADMIN')`), the
same gate every admin-only surface uses since 010-identity-auth. All return the standard
envelope: `{ success: true, data: <shape>, meta: null }` on success, `{ success: false, error:
{code, message, details} }` on failure — no change to this codebase's existing response
convention.
## `GET /admin/reports/management`
**Query**: `from?`, `to?` (ISO dates).
**200**: `ManagementDashboard` (data-model.md).
**400** `VALIDATION_ERROR`: `from` is after `to`.
**401/403**: missing/invalid session, or a non-`ADMIN` role.
## `GET /admin/reports/product/:externalProductId`
**Path**: `externalProductId` — the SaaS-facing product identifier (same convention every other
admin product-scoped route already uses, e.g. `GET /admin/products/:externalProductId/knowledge`
from 004-product-knowledge).
**Query**: `from?`, `to?`.
**200**: `ProductDashboard`.
**404** `NOT_FOUND`: no product with that `externalProductId` (FR-006 — never an empty-but-200
response for an unknown product).
**400** `VALIDATION_ERROR`: `from` is after `to`.
## `GET /admin/reports/support`
**Query**: `from?`, `to?` (applies only to the performance figures — workload/SLA-risk/breached
are always current, per data-model.md's `SupportDashboard.generatedAt`).
**200**: `SupportDashboard`.
## `GET /admin/reports/ai`
**Query**: `from?`, `to?`.
**200**: `AiDashboard`.
## Guarantees
1. Every rate/average field is `number | null``null` means no qualifying data existed in the
requested range (FR-007). A consumer must never see `NaN` or a silently-substituted `0` for
"no data."
2. Every count field is a plain `number`, always present, `0` is a legitimate, meaningful value
for a count (distinct from the `null`-for-no-data rule above, which applies only to
rates/averages).
3. `from`/`to` in every response echo the *resolved* range actually used (including the default,
when omitted) — a caller never has to separately know what "the default" was.
4. No route in this contract mutates any data — a repeated identical request returns the same
shape (though not necessarily identical figures, since the underlying data can change between
requests) with no side effect.
@@ -0,0 +1,93 @@
# Data Model: Reporting and Analytics Dashboards
## New Prisma Model
### `ErrorCodeLookup`
Append-only audit record — see research.md §6 for why this is the one new table this feature
needs.
| Field | Type | Notes |
|---|---|---|
| `id` | `String @id @default(cuid())` | |
| `errorCodeId` | `String` | FK → `ErrorCode.id` |
| `productId` | `String` | FK → `Product.id` — denormalized from `errorCode.productId` so the Product dashboard's range query never needs to join back through `ErrorCode` just to filter by product |
| `createdAt` | `DateTime @default(now())` | |
Indexes: `@@index([productId, createdAt])` (the Product dashboard's own access pattern).
No `updatedAt`, no soft-delete, no unique constraint — every lookup is its own row, duplicates
across time are the entire point (frequency is what "top errors" measures).
## Response Shapes (not persisted — computed per request)
### Management dashboard — `GET /admin/reports/management`
```ts
interface ManagementDashboard {
range: { from: string; to: string }; // ISO 8601, echoes the resolved (possibly defaulted) range
totalCases: number;
aiResolved: number;
humanEscalated: number;
resolved: number;
open: number;
slaCompliance: { met: number; breached: number; rate: number | null }; // rate = met / (met + breached)
escalationCount: number;
averageResponseSeconds: number | null;
averageResolutionSeconds: number | null;
}
```
### Product dashboard — `GET /admin/reports/product/:externalProductId`
```ts
interface ProductDashboard {
productId: string; // externalProductId, echoed back
range: { from: string; to: string };
supportVolume: number;
problemsByCategory: Array<{ categoryId: string | null; count: number }>;
recurringProblems: Array<{ categoryId: string | null; count: number }>; // same data, top N, descending
aiResolutionRate: number | null;
humanEscalationRate: number | null;
topErrors: Array<{ code: string; count: number }>; // top N, descending
}
```
### Support dashboard — `GET /admin/reports/support`
```ts
interface SupportDashboard {
generatedAt: string; // workload/risk are point-in-time, not range-scoped (research.md §2)
range: { from: string; to: string }; // still applies to the performance figures below
workloadByAgent: Array<{ agentId: string; openAssignments: number }>;
slaAtRisk: number;
slaBreached: number;
escalationCount: number;
averageResponseSeconds: number | null;
averageResolutionSeconds: number | null;
}
```
### AI dashboard — `GET /admin/reports/ai`
```ts
interface AiDashboard {
range: { from: string; to: string };
totalSessions: number;
aiResolutionRate: number | null;
humanHandoffRate: number | null;
failedTroubleshootingEscalationRate: number | null;
knowledgeMatchRate: number | null;
confidenceDistribution: { proceed: number; ask: number; escalate: number };
toolInvocations: { success: number; failed: number };
}
```
## Query Parameters (all four routes)
| Param | Type | Notes |
|---|---|---|
| `from` | ISO date, optional | Defaults to `to - REPORTING_DEFAULT_WINDOW_DAYS` |
| `to` | ISO date, optional | Defaults to now |
`from > to` is a 400 `VALIDATION_ERROR` (spec.md Edge Cases), not silently swapped.
+137
View File
@@ -0,0 +1,137 @@
# Implementation Plan: Reporting and Analytics Dashboards
**Branch**: `015-reporting-dashboards` | **Date**: 2026-09-09 | **Spec**: [spec.md](./spec.md)
**Input**: Feature specification from `specs/015-reporting-dashboards/spec.md`
## Summary
Wires the pre-scaffolded, unused `platform/reports` module into four real, admin-gated,
read-only aggregation endpoints (Management, Product, Support, AI) matching
`docs/09-testing-observability-cicd.md`'s own dashboard table — each computed synchronously,
on request, directly from existing durable tables (Ticket, Problem, SLARun, EscalationEvent,
AISupportSession, AIDiagnosis, AIAction, Resolution, Assignment). The one new piece of state is
a small durable `ErrorCodeLookup` audit table, needed only because no existing record lets "top
errors" be computed historically (014-full-observability's own equivalent is a process-lifetime
Prometheus counter, unusable for a dated report). No presentation layer — see spec.md's
Assumptions for why `supporthub-web` work is a separate follow-on.
## Technical Context
**Language/Version**: TypeScript 5.4 / Node.js 20+ (unchanged).
**Primary Dependencies**: None new — Prisma's own `groupBy`/`count`/`aggregate`/`findMany`, no
raw SQL (research.md §5), reusing `decideConfidenceBand` (005-ai-support) and the
`Resolution.resolvedBy` convention (014-full-observability) rather than reimplementing either.
**Storage**: One new table, `ErrorCodeLookup` (`id`, `errorCodeId` FK, `productId` FK,
`createdAt`) — append-only, no update/delete path, indexed `(productId, createdAt)` for the
Product dashboard's range-scoped ranking query. No change to any existing table.
**Testing**: Vitest — unit tests for the "no data → `null`, never `NaN`" averaging helper and the
confidence-bucketing reuse; integration tests against real Postgres/Redis driving each
dashboard's real underlying data (tickets in various terminal states, SLA runs met/breached,
escalation events, AI sessions/diagnoses/actions, error-code lookups) and asserting every
returned figure against hand-computed expected values — the same rigor and mixed
HTTP-driven/direct-repository setup style as 014's `business-metrics.test.ts`.
**Target Platform**: Same Fastify modular monolith. Rewrites `platform/reports` (service,
new controller, new routes, new schema for the date-range/product-id query params) from its
current one-stub-method state into the real module. Adds one line to
`ai-support/knowledge/service/error-codes.service.ts`'s existing `findKnownIssuesByErrorCode`
(the same call site 014 already instrumented) to also write the new durable audit row.
**Project Type**: Backend service — single project.
**Performance Goals**: Every dashboard query is bounded by the requested date range (default 30
days, config) and, where a full-row fetch is needed for in-application averaging (research.md
§5), only the two timestamp columns needed for that specific average — never a full-table scan
with no range filter. Acceptable at current data volumes per spec.md's own Assumptions;
pre-aggregation is explicitly deferred to if/when load testing (a separate, not-yet-started
Phase 11 sub-area) shows it's actually needed.
**Constraints**: FR-006 — an unknown `productId` on the Product dashboard is a 404, never an
empty-but-200 response. FR-007 — every rate/average is `number | null`, `null` meaning "no
qualifying data," computed by checking the qualifying count before ever dividing. FR-008 — every
route requires `requireRole('ADMIN')`, the same gate every admin surface uses since
010-identity-auth.
**Scale/Scope**: Four new `GET` routes, one new Prisma model + migration, four new service
methods (one per dashboard) replacing the single stub method, one new schema file for query-param
validation, three new env-configured values (Constitution Principle II). No new module — this
extends `platform/reports`, already the correct architectural home.
## Constitution Check
*GATE: Must pass before Phase 0 research. Re-check after Phase 1 design.*
| Principle / Section | Check | Result |
|---|---|---|
| I. SaaS Is the Sole Identity & Access Authority | Not applicable — no identity/access surface touched; every figure is derived from SupportHub's own domain data (tickets, problems, SLA, escalation, AI sessions), squarely inside SupportHub's own sole-authority domain per this principle's own second sentence. | PASS |
| II. Configuration Over Hardcoding | The default reporting window, the SLA-risk threshold, and the top-N ranking limit are all new env-configured values (`REPORTING_DEFAULT_WINDOW_DAYS`, `REPORTING_SLA_RISK_THRESHOLD_MINUTES`, `REPORTING_TOP_N_LIMIT`), never hardcoded — matches spec.md's own Assumptions and the roadmap's "never hardcode a placeholder value and ship it as final." | PASS |
| III. Layered Architecture With Enforced Module Boundaries | All new code lives inside `platform/reports` (already its correct home) following Route → Schema → Controller → Service → Repository → Prisma; cross-module reads (tickets, AI support, orchestration, SLA/escalation, problem resolution) go through each owning module's own public `index.ts`, the same precedent every prior feature this session established (e.g. `tool-executor.ts` reading `ticketsService` from `@/modules/ticketing/tickets`). | PASS |
| IV. AI Recommends, Deterministic Policy Decides | Not applicable — no AI tool-execution or decision logic changed; the AI dashboard only reports on outcomes the existing, already-deterministic confidence-band/tool-policy code already produced. | PASS — N/A |
| V. Evidence-Based Verification | Not applicable — no resolution-recording logic changed. | PASS — N/A |
| VI. Durable Audit & History | The one new table (`ErrorCodeLookup`) is itself an append-only audit record, directly in this principle's spirit — "which error codes came up, when" becomes durably answerable for the first time. | PASS |
| VII. Concurrency-Safe, Durable Job Handling | Not applicable — read-only aggregation queries, no job handlers, no assignment/SLA state mutated. | PASS — N/A |
| VIII. Problem and Ticket Are Separate, Related Entities | Respected — the Product dashboard's problem-type breakdown queries `Problem` directly, never conflating it with `Ticket`. | PASS |
| Technology & Platform Constraints | No new dependencies; one new Prisma model via the established non-interactive migration workflow (`prisma migrate diff` → hand-written `migration.sql``prisma migrate deploy`) this session has used for every prior schema change. | PASS |
No violations requiring Complexity Tracking justification.
## Project Structure
### Documentation (this feature)
```text
specs/015-reporting-dashboards/
├── plan.md
├── research.md
├── data-model.md
├── quickstart.md
├── contracts/
│ └── reports-api-contract.md
└── tasks.md
```
### Source Code (repository root)
```text
supporthub-api/
├── prisma/
│ ├── schema.prisma # MODIFIED — new ErrorCodeLookup model
│ └── migrations/
│ └── <timestamp>_add_error_code_lookup/migration.sql # NEW
├── src/
│ ├── config/
│ │ └── env.ts / reporting.ts (or similar) # MODIFIED — 3 new env-configured values
│ └── modules/
│ ├── platform/
│ │ └── reports/ # REWRITTEN (was a 1-method stub)
│ │ ├── controller/
│ │ ├── mapper/ # date-range parsing/defaulting, averaging helper
│ │ ├── repository/ # the 4 dashboards' Prisma queries
│ │ ├── routes/
│ │ ├── schema/ # query-param validation
│ │ ├── service/
│ │ └── index.ts
│ └── ai-support/
│ └── knowledge/
│ ├── repository/ # MODIFIED — errorCodeLookupRepository
│ └── service/
│ └── error-codes.service.ts # MODIFIED — one new line at the existing
│ lookup call site
└── tests/
├── unit/platform/reports/ # averaging/no-data-null helper, confidence
│ bucketing reuse
└── integration/platform-reports/ # all four dashboards against real data
```
**Structure Decision**: Single project, no new module — `platform/reports` already exists as the
correct architectural home and simply needs its real implementation built out, following the
same Route → Schema → Controller → Service → Repository → Prisma layering every other module
already uses.
## Complexity Tracking
*No constitution violations — table intentionally omitted.*
@@ -0,0 +1,52 @@
# Quickstart: Reporting and Analytics Dashboards
Manual verification steps for each user story, against a running instance backed by real
Postgres/Redis, logged in as an ADMIN.
## Scenario 1 — Management dashboard (User Story 1)
1. Create several tickets within a known date range: some reaching `AI_RESOLVED`/`RESOLVED` via
an AI session, some escalated to a human and resolved via `resolutionsService.record`, some
left open.
2. Let one ticket's SLA run complete on time and another breach (via the existing breach sweep).
3. `GET /admin/reports/management?from=<range start>&to=<range end>`.
4. **Expected**: `totalCases`, `aiResolved`, `humanEscalated`, `resolved`, `open` all match what
was actually created; `slaCompliance.met`/`.breached` match the two SLA outcomes;
`averageResponseSeconds`/`averageResolutionSeconds` are non-null and plausible.
5. Request the same endpoint for a date range with no activity at all.
6. **Expected**: every count is `0`, every rate/average is `null`, not an error.
## Scenario 2 — Product dashboard (User Story 2)
1. Create tickets for two distinct products in the same range, one with a categorized problem.
2. Look up a known error code for one product several times, a different code once.
3. `GET /admin/reports/product/:externalProductId` for each product.
4. **Expected**: each product's `supportVolume`/`problemsByCategory`/`aiResolutionRate` reflect
only its own tickets; `topErrors` ranks the more-frequently-looked-up code first.
5. Request the endpoint for a nonexistent `externalProductId`.
6. **Expected**: `404 NOT_FOUND`, not an empty `200`.
## Scenario 3 — Support dashboard (User Story 3)
1. Assign several tickets across two agents (some via the real orchestration flow).
2. Let one ticket's SLA run sit within `REPORTING_SLA_RISK_THRESHOLD_MINUTES` of its resolution
due date without breaching.
3. `GET /admin/reports/support`.
4. **Expected**: `workloadByAgent` matches each agent's real current open-assignment count;
`slaAtRisk` counts exactly the near-due run, distinct from `slaBreached`.
## Scenario 4 — AI dashboard (User Story 4)
1. Run AI sessions to a mix of terminal outcomes (`resolved`, `escalated`), with some tool
invocations succeeding and others failing, and diagnoses spanning a range of confidence
values.
2. `GET /admin/reports/ai`.
3. **Expected**: `aiResolutionRate`/`humanHandoffRate` reflect the real outcome mix;
`confidenceDistribution` buckets match `decideConfidenceBand`'s own classification of each
diagnosis's stored confidence against the system-default thresholds; `toolInvocations`
reflects the real success/failure counts.
## What "done" looks like
All four scenarios pass against a real Postgres/Redis, every figure independently verified
against hand-computed expected values, and no route is reachable by a non-admin session.
+143
View File
@@ -0,0 +1,143 @@
# Research: Reporting and Analytics Dashboards
## 1. Where this lives
**Decision**: Wire up the existing, pre-scaffolded `src/modules/platform/reports` module (today
just `ReportsService.generateSummaryReport()` returning `{}`, confirmed unused anywhere) rather
than creating a new module. Its four real methods (`getManagementDashboard`,
`getProductDashboard`, `getSupportDashboard`, `getAiDashboard`) replace the one stub method.
Routes live at `GET /admin/reports/management`, `GET /admin/reports/product/:externalProductId`,
`GET /admin/reports/support`, `GET /admin/reports/ai`, admin-gated the same way every other
admin-only endpoint since 010-identity-auth already is (`requireRole('ADMIN')`).
**Why not the `ANALYTICS` queue** (`src/jobs/analytics`, also pre-scaffolded, also inert): a
queued background job fits pre-computing a report nobody is currently waiting on; a dashboard
request is someone waiting right now for an answer. Per spec.md's Assumptions, this first cut is
synchronous, direct-query aggregation — the queue stub stays exactly as inert as it already was,
untouched by this feature.
## 2. Per-dashboard queries
All four use Prisma's `groupBy`/`count`/`aggregate`, scoped by `createdAt` (or the
milestone-specific timestamp named below) within `[from, to]`, computed directly against the
tables that already own each fact — no new table, no denormalized rollup.
### Management (FR-001)
| Figure | Source |
|---|---|
| Total cases | `Ticket.count({ createdAt in range })` |
| AI resolved | `Ticket.count({ createdAt in range, status: 'AI_RESOLVED' })` — a ticket that reached `AI_RESOLVED` and stayed there or moved straight to `RESOLVED` without a `Resolution.resolvedBy` other than `'ai'`; see §4 below for the exact "who resolved it" rule shared with the Product dashboard |
| Human escalated | `Ticket.count({ createdAt in range, status in [HUMAN_ESCALATION, IN_PROGRESS, WAITING_FOR_CUSTOMER, RESOLUTION_PENDING_CUSTOMER, RESOLVED, CLOSED, REOPENED] })` minus AI-only-path tickets — i.e. any ticket that ever reached `HUMAN_ESCALATION`; the state machine research.md already establishes this as one-way (003-ticketing) |
| Resolved (either path) | `Ticket.count({ createdAt in range, status in [RESOLVED, CLOSED] })` |
| Open | `Ticket.count({ createdAt in range, status not in [RESOLVED, CLOSED] })` |
| SLA compliance / breach count | `SLARun.groupBy(['status'], { ticket: { createdAt in range } })`, `status: 'completed'` = met, `'breached'` = breached (mirrors 014's own metric semantics — see 014 research.md §5's "read status before the overwrite" caveat, which applies equally here: a `'breached'`-then-`'completed'` run is still counted breached, by reading the `breachedAt`/`firstResponseBreachedAt` timestamps rather than only the current `status` string) |
| Escalation count | `EscalationEvent.count({ createdAt in range })` |
| Average response time | `avg(firstAgentMessage.createdAt - ticket.createdAt)` over tickets with at least one `AGENT_MESSAGE` in range — computed in application code over a bounded query result (see §5, no raw SQL) |
| Average resolution time | `avg(resolution.createdAt - ticket.createdAt)` over tickets with a `Resolution` row in range |
### Product (FR-002)
Same shape as Management, `WHERE Ticket.productId = :productId` (resolved from the given
`externalProductId`, 404 if not found — FR-006), plus:
| Figure | Source |
|---|---|
| Problem-type breakdown | `Problem.groupBy(['categoryId'], { productId, createdAt in range })` |
| Recurring problems | Same grouped result, sorted descending, top N (config, default 10) |
| Top errors | `reuses 014's own instrumentation point conceptually but queries fresh` — no, see §6: there is no persisted "error code lookup" table, only 014's in-memory Prometheus counter, which is NOT queryable historically. Resolved by adding a durable audit read instead: see §6. |
### Support (FR-003)
| Figure | Source |
|---|---|
| Per-agent workload | `Assignment.groupBy(['agentId'], { isCurrent: true })` — a snapshot of *right now*, not date-ranged (workload is inherently current, not historical — spec.md's own framing: "how much work is currently assigned") |
| SLA risk / breached | `SLARun.findMany({ status: 'running', resolutionDueAt: {gte: now} })` filtered in application code by "due within `SLA_RISK_THRESHOLD_MINUTES` of now" for risk, vs. `status: 'breached'` for already-breached |
| Escalation count | Same as Management, unfiltered by product |
| Response/resolution performance | Same computation as Management's averages |
### AI (FR-004)
| Figure | Source |
|---|---|
| AI resolution rate / human-handoff rate | `AISupportSession.groupBy(['status'], { startedAt in range })``resolved` vs. `escalated`/`ended_by_agent` as a share of total terminal sessions |
| Failed-troubleshooting-then-escalated rate | Sessions with `status: 'escalated'` that have at least one `AIInteraction`/`AIAction` recording a failed troubleshooting attempt — see 005-ai-support's own runbook-step-outcome classification (`classifyStepOutcome`), reused rather than reinvented |
| Knowledge-match rate | `AIKnowledgeReference` presence per session (`recordMany` is only ever called with actual retrieval results — 005-ai-support's own `diagnose.ts`) vs. sessions with zero references recorded |
| Confidence distribution | `AIDiagnosis.findMany({ createdAt in range })`, bucketed in application code against `aiConfig.defaultHighConfidence`/`defaultLowConfidence` (see §7 — NOT a per-diagnosis resolved policy) |
| Tool success/failure | `AIAction` joined to `AIActionResult`, grouped by `result.status` |
## 3. "No data" convention (FR-007)
**Decision**: every rate/average field is `number | null``null` means "no qualifying records
in range," distinguished in the response shape from a genuine `0` (e.g., a real 0% AI resolution
rate because everything escalated is a valid, meaningful `0`; "nobody's data exists yet" is
`null`). Application code computes every average by fetching the qualifying count first and
returning `null` before ever dividing, never relying on `0/0` producing `NaN` and hoping a caller
notices.
## 4. "Who resolved it" — reused from 014, not reinvented
014-full-observability's own event subscriber already established the authoritative rule: a
ticket's `Resolution.resolvedBy` field (`"ai"` | an `agentId`) is the single source of truth for
whether a resolution was AI- or human-driven (014 research.md §5). This feature's Management/
Product dashboards reuse the exact same join (`Resolution.findMany` scoped to the range,
`resolvedBy === 'ai'` vs. not) rather than re-deriving it from `Ticket.status` transitions a
second, potentially-inconsistent way.
## 5. No raw SQL
**Decision**: every duration average (response time, resolution time) is computed by fetching
the bounded set of qualifying rows (ticket `createdAt` + the milestone timestamp) via Prisma and
averaging in application code, not a raw `$queryRaw` computing `AVG(EXTRACT(EPOCH FROM ...))` in
SQL. At the data volumes spec.md's Assumptions accept for this first cut (no pre-aggregation,
synchronous queries), a bounded per-range fetch is simple, type-safe, and testable without
hand-writing SQL — consistent with this codebase's near-total avoidance of `$queryRaw` elsewhere
(confirmed by grep: no existing module uses it for reporting-shaped queries).
## 6. Top errors needs a durable, queryable record — a real gap 014 left open
014-full-observability's `supporthub_known_error_lookups_total` Prometheus counter is
process-lifetime, in-memory, and reset on every restart — useless for "top errors in the last 30
days." Since no durable "error code lookup" record exists anywhere in this codebase today (the
existing `error-codes.service.ts` just reads `KnownIssue`/`ErrorCode` rows, never records that a
lookup happened), this feature adds one small, focused piece of new state: a durable
`ErrorCodeLookup` audit row (`errorCodeId`, `productId`, `createdAt`), written by
`error-codes.service.ts`'s already-existing `findKnownIssuesByErrorCode` (the same call site
014 instrumented for its own live counter — this feature adds one more line there, a durable
write alongside the existing live-metric increment, not a replacement for it). This is the one
schema change this feature needs; every other dashboard figure is computed from tables that
already exist.
## 7. Confidence distribution uses the system default threshold, not a per-diagnosis policy
**Decision**: bucket every `AIDiagnosis.confidence` value in range against the env-configured
system-wide defaults (`aiConfig.defaultHighConfidence`/`defaultLowConfidence`), the same
`decideConfidenceBand` pure function 005-ai-support already exports — reused directly, not
reimplemented.
**Why not resolve each diagnosis's actual applicable per-product/category policy** (what the
live reasoning path itself does): `AIDiagnosis.product`/`feature` are the AI's own free-text
classification output, not foreign keys to `Product`/`Category` — there is no reliable, existing
join from a diagnosis row back to which `ConfidencePolicy` row actually applied to it at the time
without speculatively string-matching free text against product names, which this codebase does
nowhere else and which research.md declines to invent here. A dashboard-level aggregate
distribution using the system-wide default is an honest, documented simplification (spec.md
Assumptions) — precise enough to show a meaningful shape without fabricating a false precision
the data doesn't actually support.
## 8. New configuration (Constitution Principle II — nothing hardcoded)
| Env var | Default | Used by |
|---|---|---|
| `REPORTING_DEFAULT_WINDOW_DAYS` | `30` | Every dashboard's `from`/`to` default when omitted (FR-005) |
| `REPORTING_SLA_RISK_THRESHOLD_MINUTES` | `60` | Support dashboard's "at risk" classification (FR-003) |
| `REPORTING_TOP_N_LIMIT` | `10` | Product dashboard's recurring-problems/top-errors ranking length |
## 9. Test strategy
Integration tests create real tickets/problems/SLA runs/escalation events/AI sessions/diagnoses/
actions/error-code lookups directly against real Postgres (mixing real HTTP-driven setup where a
realistic flow matters and direct repository/Prisma writes where only the aggregation math is
under test — the same mix 014's own `business-metrics.test.ts` used), then request each
dashboard endpoint and assert every figure against hand-computed expected values. Unit tests
cover the "no data → null, never NaN" guard and the confidence-bucketing pure-function reuse.
+257
View File
@@ -0,0 +1,257 @@
# Feature Specification: Reporting and Analytics Dashboards
**Feature Branch**: `015-reporting-dashboards`
**Created**: 2026-09-09
**Status**: Draft
**Input**: User description: "Reporting and analytics dashboards: real, read-only aggregation endpoints backing the four dashboards named in docs/09-testing-observability-cicd.md (Management, Product, Support, AI) — wiring up the pre-scaffolded but never-implemented platform/reports module into actual database-backed aggregation queries, admin-gated, with a date-range filter."
## User Scenarios & Testing *(mandatory)*
### User Story 1 - Management sees organization-wide support health (Priority: P1)
An admin or team lead opens a single view showing how support is doing overall for a chosen
period: how many cases came in, how many were resolved (by AI vs. by a human), how many are
still open, whether SLA commitments are being met, and how escalation is trending.
**Why this priority**: This is the one dashboard covering the whole roadmap's own top-level
success criteria (`docs/10-implementation-roadmap.md`'s checklist) in one place — the first
thing anyone asks about a support operation is "how are we doing," and today there is no way to
answer that except querying the database by hand.
**Independent Test**: Can be fully tested by creating a known set of tickets in various terminal
states (AI-resolved, human-resolved, still open) plus a mix of met/breached SLA runs and
escalations within a chosen date range, then requesting the Management dashboard for that range
and confirming every figure matches what was actually created.
**Acceptance Scenarios**:
1. **Given** a mix of tickets created within a chosen date range — some AI-resolved, some
human-resolved, some still open — **When** the Management dashboard is requested for that
range, **Then** total cases, AI-resolved count, human-escalated count, resolved count, and
open count all match the actual data exactly.
2. **Given** SLA runs that completed on time and others that breached within the range,
**When** the dashboard is requested, **Then** SLA compliance (a rate) and SLA breach count
both reflect the real outcomes.
3. **Given** some tickets have a recorded first agent response and a resolution timestamp,
**When** the dashboard is requested, **Then** average response time and average resolution
time are computed only from tickets that actually reached those milestones within the range
(a still-open ticket contributes to "open count" but never a fabricated resolution time).
4. **Given** a date range with zero activity, **When** the dashboard is requested, **Then** every
count is zero and every average is reported as "no data" rather than a computed zero or a
division-by-zero error.
---
### User Story 2 - See support broken down by product (Priority: P1)
An admin viewing support data for a specific product (or comparing products) sees volume,
problem-type breakdown, which problems recur most, how well AI is resolving that product's
issues versus escalating them, and which error codes come up most often.
**Why this priority**: SupportHub serves multiple SaaS products (Constitution Principle I); a
number that isn't broken out by product hides which integration actually needs attention — this
is as fundamental as the Management view, just sliced differently.
**Independent Test**: Can be fully tested by creating tickets/problems/error-code lookups across
two distinct products within a date range, requesting the Product dashboard for each product,
and confirming each one's figures include only its own product's data.
**Acceptance Scenarios**:
1. **Given** tickets exist for two different products in the same date range, **When** the
Product dashboard is requested scoped to one product, **Then** support volume and every other
figure reflect only that product's tickets, never the other product's.
2. **Given** problems in several categories for one product, **When** the dashboard is
requested, **Then** the problem-type breakdown and "recurring problems" ranking both reflect
the real category distribution, most-frequent first.
3. **Given** a mix of AI-resolved and human-escalated tickets for one product, **When** the
dashboard is requested, **Then** AI resolution rate and human escalation rate are both
computed as a percentage of that product's own total, not the platform-wide total.
4. **Given** several known-error-code lookups for one product, some codes looked up more than
others, **When** the dashboard is requested, **Then** "top errors" lists those codes ranked by
lookup frequency.
---
### User Story 3 - Support sees team workload and performance (Priority: P2)
An admin or team lead sees how much work is currently assigned across agents, which tickets are
at SLA risk, how much escalation is happening, and how quickly the team is responding to and
resolving tickets.
**Why this priority**: This view is about ongoing operational load, not historical trend — useful
for day-to-day team management, but the organization can already see whether it's healthy
overall from User Story 1 without this one; P2 reflects that it adds an operational lens rather
than a new class of information.
**Independent Test**: Can be fully tested by assigning several tickets to known agents (some
close to SLA breach, some not), then requesting the Support dashboard and confirming workload
per agent and the SLA-risk count both match reality.
**Acceptance Scenarios**:
1. **Given** several tickets are currently assigned across two agents, **When** the Support
dashboard is requested, **Then** each agent's current open-assignment count matches what was
actually assigned to them (not a stale count from a previous, now-unassigned period).
2. **Given** a ticket's SLA run is running and past a configurable risk threshold of its
resolution due date (but not yet breached), **When** the dashboard is requested, **Then** it
is counted as "at risk," distinct from both "on track" and "breached."
3. **Given** response and resolution durations for several resolved tickets in the period,
**When** the dashboard is requested, **Then** response-performance and resolution-performance
figures are computed only from tickets that actually reached those milestones.
---
### User Story 4 - See how well the AI is performing (Priority: P2)
An admin sees, for a chosen period, how often the AI resolves issues on its own versus escalating
them, how often its attempted troubleshooting fails outright, how often it finds relevant
knowledge, how confident its diagnoses tend to be, how reliably its tools succeed, and how often
it ultimately hands off to a human.
**Why this priority**: This is the dashboard that validates the AI-first design's core premise
(Constitution Principle IV) is actually working in practice — valuable, but a narrower audience
than the org-wide and per-product views above, hence P2.
**Independent Test**: Can be fully tested by running several AI sessions to different terminal
outcomes (resolved, escalated, escalated-after-failed-troubleshooting) with a mix of tool
successes/failures and confidence levels recorded, then requesting the AI dashboard and
confirming every figure matches the real session data.
**Acceptance Scenarios**:
1. **Given** a mix of AI sessions ending resolved vs. escalated in the period, **When** the AI
dashboard is requested, **Then** AI resolution rate and human-handoff rate both reflect the
real outcome mix as percentages of total sessions.
2. **Given** some AI tool invocations succeeded and others failed in the period, **When** the
dashboard is requested, **Then** tool success/failure figures reflect the real invocation
outcomes.
3. **Given** diagnoses were recorded with a range of confidence values, **When** the dashboard is
requested, **Then** the confidence distribution groups them into the same proceed/ask/escalate
bands the AI support module's own confidence-policy service already classifies each diagnosis
into (005-ai-support), not a newly-invented scheme.
4. **Given** some AI sessions' knowledge-retrieval step found matching entries and others found
none, **When** the dashboard is requested, **Then** knowledge-match rate reflects the real
match/no-match mix.
---
### Edge Cases
- What happens when no `from`/`to` date range is given? Defaults to a reasonable trailing window
(see Assumptions) rather than scanning the entire history unbounded on every request.
- What happens when `from` is after `to`? Rejected as a validation error, not silently swapped or
silently returning empty data.
- What happens when a requested `productId` (Product dashboard) doesn't exist? Rejected with a
clear not-found error, not an empty-but-200 response that looks like "this product has zero
activity."
- What happens when an average would divide by zero (no tickets reached that milestone in the
range)? Reported as an explicit "no data" value, never `NaN`, `null` silently coerced to `0`,
or a thrown error.
- What happens when a ticket's SLA run was paused for part of the period? SLA-risk/compliance
figures use the run's own already-durable due dates (008-sla-escalation's pause/resume
already accounts for paused time) rather than this feature re-deriving elapsed time itself.
- Who can see these dashboards? Same admin-only gate as every other admin configuration/reporting
surface introduced since 010-identity-auth — no new role is introduced.
## Requirements *(mandatory)*
### Functional Requirements
- **FR-001**: System MUST provide a Management dashboard summarizing, for a given date range:
total cases created, cases resolved by AI, cases escalated to a human, total resolved
(either path), total still open, SLA compliance rate, SLA breach count, escalation count,
average first-response time, and average resolution time.
- **FR-002**: System MUST provide a Product dashboard summarizing, for a given date range and a
specific product: support volume, a breakdown by problem category, a ranked list of the most
recurring problem categories, AI resolution rate, human escalation rate, and a ranked list of
the most frequently looked-up error codes.
- **FR-003**: System MUST provide a Support dashboard summarizing, for a given date range:
current per-agent open-assignment workload, count of tickets at SLA risk (past a configurable
risk threshold of their resolution due date but not yet breached), count of tickets already
breached, escalation count, average response performance, and average resolution performance.
- **FR-004**: System MUST provide an AI dashboard summarizing, for a given date range: AI
resolution rate, rate of sessions that escalated after at least one failed troubleshooting
attempt, knowledge-match rate, a distribution of diagnosis confidence across the existing
proceed/ask/escalate bands, tool invocation success/failure counts, and human-handoff rate.
- **FR-005**: Every dashboard endpoint MUST accept an optional `from`/`to` date range; when
omitted, it MUST default to a documented trailing window rather than scanning unbounded
history.
- **FR-006**: The Product dashboard MUST require a valid `productId` and MUST reject an unknown
one with a clear not-found error rather than returning an empty-but-successful response.
- **FR-007**: Every rate/average figure MUST be computed only from tickets/sessions/runs that
actually reached the relevant milestone within the range; a metric with no qualifying data MUST
be reported as an explicit "no data" value, never a computed `0`, `null`, or `NaN`.
- **FR-008**: All four dashboard endpoints MUST be admin-gated, consistent with every other
admin-only reporting/configuration surface in this codebase.
- **FR-009**: This feature MUST NOT alter the meaning or shape of any existing endpoint, event, or
table — nearly every figure is derived read-only from data already durably recorded by the
modules that own it (003 ticketing, 005 AI support, 007 orchestration, 008 SLA/escalation, 009
problem resolution). The one exception is FR-011: a small new durable record needed only
because no existing table can answer "which error codes are looked up most" historically.
- **FR-011**: System MUST durably record each known-error-code lookup (product, error code,
timestamp) at the point it already happens (the existing error-code lookup call site) so the
Product dashboard's "top errors" ranking (FR-002) can be computed historically — the
equivalent live, in-process counter this project already exposes on `/metrics` (014-full-
observability) is process-lifetime and reset on every restart, unusable for a historical
dashboard.
- **FR-010**: This feature is backend-only; presenting these figures in a UI is a separate,
explicitly out-of-scope follow-on (see Assumptions).
### Key Entities
- **Dashboard response**: A read-only, computed JSON summary for one of the four dashboards over
a requested date range (and, for the Product dashboard, one product) — never itself persisted;
recomputed fresh on every request from existing durable records.
- **Date range**: An inclusive `from`/`to` pair (calendar dates or timestamps) scoping every
aggregation query; not a stored entity, a request parameter.
- **Error code lookup record** (new, FR-011): a durable, append-only audit row — which product,
which error code, when — written at the existing lookup call site; exists solely so "top
errors" can be computed over a historical range, never read or written anywhere else.
- **Confidence band**: The existing proceed/ask/escalate classification 005-ai-support already
applies to a diagnosis's confidence score — reused here for the AI dashboard's distribution, not
redefined.
## Success Criteria *(mandatory)*
### Measurable Outcomes
- **SC-001**: For any chosen date range, every figure on all four dashboards can be independently
verified against the underlying ticket/session/SLA-run/escalation-event records and matches
exactly — no discrepancy between what a dashboard reports and what actually happened.
- **SC-002**: An admin can answer "how is support doing right now" (Management), "how is this
specific product doing" (Product), "who's overloaded and what's at risk" (Support), and "is the
AI actually helping" (AI) each from a single request, with no manual database query needed.
- **SC-003**: A dashboard request for a period with no matching activity returns clean, explicit
"no data" results in well under a second — never an error, a stall, or a misleading zero.
## Assumptions
- **Presentation is out of scope for this feature.** The user's own explicit direction was to
build the backend aggregation capability first (the established pattern this project has
followed for every prior feature that touched both repos — identity/auth, the agent ticket
queue, and full observability were each built backend-first). A `supporthub-web` dashboard UI
consuming these endpoints is a natural, separate follow-on, not bundled into this spec.
- The default trailing window when no date range is given is the last 30 days, matching common
reporting-dashboard convention; CONFIGURABLE via the same admin-config env-driven pattern this
project already uses for every other business-policy value (Constitution Principle II), not
hardcoded as a magic number in application logic.
- "SLA risk" needs a threshold (how close to the due date counts as "at risk") that the business
has not specified — CONFIGURABLE, not invented as a hardcoded percentage, consistent with
`docs/10-implementation-roadmap.md`'s own "never hardcode a placeholder value and ship it as
final" instruction.
- These endpoints compute their figures synchronously, on request, directly from the existing
tables — no new pre-aggregation table, no scheduled batch job, and no use of the pre-scaffolded
`ANALYTICS` queue (`src/jobs/analytics`), which remains an inert stub outside this feature's
scope. Live query performance at current data volumes is assumed adequate; a future feature can
introduce pre-aggregation if and when it's actually needed (load/concurrency testing, a
separate not-yet-started Phase 11 sub-area, is where that question would be validated).
- "Top errors"/"recurring problems" rankings return a bounded top-N list (CONFIGURABLE limit,
defaulting to 10) rather than the full distribution, matching how a dashboard is actually
consumed.
- Dashboard responses are computed fresh per request (no caching layer) — acceptable given the
assumed data volumes and consistent with not prematurely optimizing ahead of the load-testing
phase.
+188
View File
@@ -0,0 +1,188 @@
---
description: 'Task list for 015-reporting-dashboards'
---
# Tasks: Reporting and Analytics Dashboards
**Input**: Design documents from `specs/015-reporting-dashboards/`
**Prerequisites**: [plan.md](./plan.md), [spec.md](./spec.md), [research.md](./research.md),
[data-model.md](./data-model.md),
[contracts/reports-api-contract.md](./contracts/reports-api-contract.md),
[quickstart.md](./quickstart.md)
**Organization**: Tasks are grouped by user story (US1 = P1 Management, US2 = P1 Product,
US3 = P2 Support, US4 = P2 AI). All four share the Foundational phase (schema, config, shared
helpers, module scaffolding) but are otherwise independent of each other.
## Format: `[ID] [P?] [Story] Description`
All file paths are relative to `supporthub-api/` (repo root).
---
## Phase 1: Foundational (Blocking Prerequisites)
- [x] T001 Add `REPORTING_DEFAULT_WINDOW_DAYS` (default `30`),
`REPORTING_SLA_RISK_THRESHOLD_MINUTES` (default `60`), and `REPORTING_TOP_N_LIMIT`
(default `10`) to `src/config/env.ts`, exposed via a new `reportingConfig` in
`src/config/reporting.ts` (or added to an existing config file, matching this codebase's
own per-feature config-file convention)
- [x] T002 Add the `ErrorCodeLookup` model to `prisma/schema.prisma` per data-model.md, generate
the migration via `prisma migrate diff --from-url <db-url> --to-schema-datamodel
./prisma/schema.prisma --script`, hand-write it into
`prisma/migrations/<timestamp>_add_error_code_lookup/migration.sql`, apply via `prisma
migrate deploy` against the throwaway test database (depends on T001 only in that both
are Foundational — no code dependency)
- [x] T003 Add `ai-support/knowledge/repository/error-code-lookup.repository.ts`
`create(errorCodeId, productId)`, exported from the knowledge module's repository index
(depends on T002)
- [x] T004 [P] Call the new repository's `create(...)` from
`ai-support/knowledge/service/error-codes.service.ts`'s existing
`findKnownIssuesByErrorCode`, alongside (not replacing) 014's own
`knownErrorLookupsCounter.inc(...)` call at that same call site (depends on T003)
- [x] T005 [P] Add `platform/reports/mapper/date-range.ts` — parses/validates `from`/`to` query
params, defaulting via T001's `reportingConfig.defaultWindowDays`, throwing
`ValidationError` when `from > to` (depends on T001)
- [x] T006 [P] Add `platform/reports/mapper/rate.ts` — a shared `computeRate(numerator,
denominator): number | null` and `computeAverageSeconds(durations: number[]): number |
null` pair, both returning `null` (never `NaN`/`0`) when there's no qualifying data
(research.md §3) — no dependency, pure functions
- [x] T007 Scaffold `platform/reports/schema/` (query-param zod schema using T005's date-range
parsing), `platform/reports/controller/reports.controller.ts` (empty methods to be filled
in per user story below), `platform/reports/routes/reports.routes.ts` registering all four
routes behind `requireRole('ADMIN')`, and update `platform/reports/index.ts` to export the
new public surface, replacing `generateSummaryReport`'s stub entirely (depends on T005,
T006)
**Checkpoint**: Config, schema, shared helpers, and module scaffolding in place. Each dashboard
can now be built independently.
---
## Phase 2: User Story 1 - Management sees organization-wide support health (Priority: P1)
**Goal**: `GET /admin/reports/management` returns real figures per data-model.md's
`ManagementDashboard` shape.
**Independent Test**: Quickstart Scenario 1.
### Tests for User Story 1
- [x] T008 [P] [US1] Unit tests for T006's `computeRate`/`computeAverageSeconds` (empty input ->
`null`; a real mix -> the correct value) in
`tests/unit/platform/reports/rate-helpers.test.ts`
### Implementation for User Story 1
- [x] T009 [US1] Add `platform/reports/repository/management.repository.ts` — one method per
research.md §2's Management table row (ticket counts by status, SLA-run outcome counts,
response/resolution duration row-fetches for T006 to average) (depends on T007)
- [x] T010 [US1] Add `ReportsService.getManagementDashboard(range)` composing T009's repository
calls into the `ManagementDashboard` shape, reusing the `Resolution.resolvedBy` convention
(research.md §4) for the AI-vs-human split (depends on T009)
- [x] T011 [US1] Wire `GET /admin/reports/management` to the controller/service (depends on T010)
- [x] T012 [US1] Integration test covering Quickstart Scenario 1 (real tickets in various
terminal states, a met and a breached SLA run, verified figure-by-figure; a no-activity
range returns all-zero counts and all-null rates) in
`tests/integration/platform-reports/management-dashboard.test.ts` (depends on T011)
**Checkpoint**: Quickstart Scenario 1 passes.
---
## Phase 3: User Story 2 - See support broken down by product (Priority: P1)
**Goal**: `GET /admin/reports/product/:externalProductId` returns real figures per
`ProductDashboard`.
**Independent Test**: Quickstart Scenario 2.
### Implementation for User Story 2
- [x] T013 [P] [US2] Add `platform/reports/repository/product.repository.ts` — ticket/problem
queries scoped by `productId`, plus a query against T003's `ErrorCodeLookup` table for
the top-N ranking (`reportingConfig.topNLimit`) (depends on T007)
- [x] T014 [US2] Add `ReportsService.getProductDashboard(externalProductId, range)`, 404-ing via
`NotFoundError` when the product doesn't resolve (FR-006) before running any aggregation
query (depends on T013)
- [x] T015 [US2] Wire `GET /admin/reports/product/:externalProductId` (depends on T014)
- [x] T016 [US2] Integration test covering Quickstart Scenario 2 (two products' data never
cross-contaminating each other's figures; an unknown product 404s) in
`tests/integration/platform-reports/product-dashboard.test.ts` (depends on T015)
**Checkpoint**: Quickstart Scenario 2 passes.
---
## Phase 4: User Story 3 - Support sees team workload and performance (Priority: P2)
**Goal**: `GET /admin/reports/support` returns real figures per `SupportDashboard`.
**Independent Test**: Quickstart Scenario 3.
### Implementation for User Story 3
- [x] T017 [P] [US3] Add `platform/reports/repository/support.repository.ts` — current
`Assignment` workload-by-agent query, `SLARun` at-risk/breached queries (`resolutionDueAt`
within `reportingConfig.slaRiskThresholdMinutes` of now, per research.md §2) (depends on
T007)
- [x] T018 [US3] Add `ReportsService.getSupportDashboard(range)` (depends on T017)
- [x] T019 [US3] Wire `GET /admin/reports/support` (depends on T018)
- [x] T020 [US3] Integration test covering Quickstart Scenario 3 (real per-agent assignment
counts; a near-due-but-not-breached run counted as at-risk, distinct from breached) in
`tests/integration/platform-reports/support-dashboard.test.ts` (depends on T019)
**Checkpoint**: Quickstart Scenario 3 passes.
---
## Phase 5: User Story 4 - See how well the AI is performing (Priority: P2)
**Goal**: `GET /admin/reports/ai` returns real figures per `AiDashboard`.
**Independent Test**: Quickstart Scenario 4.
### Tests for User Story 4
- [x] T021 [P] [US4] Unit test: the confidence-distribution bucketing reuses
`decideConfidenceBand` (005-ai-support) against `aiConfig` defaults, not a reimplemented
threshold check, in `tests/unit/platform/reports/confidence-distribution.test.ts`
### Implementation for User Story 4
- [x] T022 [US4] Add `platform/reports/repository/ai.repository.ts` — `AISupportSession` outcome
counts, `AIDiagnosis` confidence fetch, `AIKnowledgeReference` presence-per-session query,
`AIAction`/`AIActionResult` outcome counts (depends on T007)
- [x] T023 [US4] Add `ReportsService.getAiDashboard(range)`, bucketing confidence via
`decideConfidenceBand` + `aiConfig.defaultHighConfidence`/`defaultLowConfidence`
(research.md §7) (depends on T022, T021)
- [x] T024 [US4] Wire `GET /admin/reports/ai` (depends on T023)
- [x] T025 [US4] Integration test covering Quickstart Scenario 4 (real AI sessions to mixed
outcomes, mixed tool results, a spread of diagnosis confidence values) in
`tests/integration/platform-reports/ai-dashboard.test.ts` (depends on T024)
**Checkpoint**: Quickstart Scenario 4 passes. All four dashboards work independently and
together — this feature's full scope.
---
## Phase 6: Polish & Cross-Cutting Concerns
- [x] T026 [P] Update `specs/015-reporting-dashboards/checklists/requirements.md` Notes with any
implementation-time findings
- [x] T027 Run `npx tsx scripts/check-architecture.ts` and `npm run lint`/`npm run typecheck`
- [x] T028 Full regression: `npm run test:unit` then the full integration suite against real
Docker-provisioned Postgres/Redis, confirming nothing outside this feature regressed
(particularly `error-codes.service.ts`'s own existing tests, now touched by T004)
---
## Dependencies & Execution Order
- **Foundational (Phase 1)**: No dependencies — BLOCKS all four user stories
- **User Story 1 (Phase 2)**: Depends on Foundational — independent of US2/US3/US4
- **User Story 2 (Phase 3)**: Depends on Foundational — independent of US1/US3/US4
- **User Story 3 (Phase 4)**: Depends on Foundational — independent of US1/US2/US4
- **User Story 4 (Phase 5)**: Depends on Foundational — independent of US1/US2/US3
- **Polish (Phase 6)**: Depends on all four user stories
+2
View File
@@ -24,6 +24,7 @@ import { solutionsRoutes } from '@/modules/problem-management/solutions';
import { verificationRoutes } from '@/modules/problem-management/verification';
import { resolutionsRoutes } from '@/modules/problem-management/resolutions';
import { authRoutes } from '@/modules/identity/auth';
import { reportsRoutes } from '@/modules/platform/reports';
export async function registerGlobalRoutes(app: FastifyInstance): Promise<void> {
await app.register(healthRoutes);
@@ -49,5 +50,6 @@ export async function registerGlobalRoutes(app: FastifyInstance): Promise<void>
await app.register(solutionsRoutes);
await app.register(verificationRoutes);
await app.register(resolutionsRoutes);
await app.register(reportsRoutes);
// Further domain module routes will be registered here as feature modules are wired up
}
+7
View File
@@ -80,6 +80,13 @@ const envSchema = z.object({
// configured" — tracing still runs, just exports to the console instead (never a startup
// requirement) — see specs/014-full-observability/research.md "Distributed tracing".
OTEL_EXPORTER_OTLP_ENDPOINT: z.string().optional(),
// Reporting and Analytics Dashboards (015) — CONFIGURABLE per docs/10-implementation-
// roadmap.md's own "never hardcode a placeholder value and ship it as final" instruction —
// see specs/015-reporting-dashboards/research.md §8.
REPORTING_DEFAULT_WINDOW_DAYS: z.coerce.number().default(30),
REPORTING_SLA_RISK_THRESHOLD_MINUTES: z.coerce.number().default(60),
REPORTING_TOP_N_LIMIT: z.coerce.number().default(10),
});
export type EnvConfig = z.infer<typeof envSchema>;
+1
View File
@@ -7,3 +7,4 @@ export * from './ai';
export * from './orchestration';
export * from './problem-resolution';
export * from './auth';
export * from './reporting';
+7
View File
@@ -0,0 +1,7 @@
import { env } from './env';
export const reportingConfig = {
defaultWindowDays: env.REPORTING_DEFAULT_WINDOW_DAYS,
slaRiskThresholdMinutes: env.REPORTING_SLA_RISK_THRESHOLD_MINUTES,
topNLimit: env.REPORTING_TOP_N_LIMIT,
};
@@ -0,0 +1,31 @@
import { prismaClient } from '@/infrastructure/database';
import { ErrorCodeLookup } from '@prisma/client';
/**
* 015-reporting-dashboards research.md §6: a durable, append-only audit row — no update/delete
* path, every lookup is its own row, duplicates over time are the point (frequency is what
* "top errors" measures).
*/
export class ErrorCodeLookupRepository {
constructor(private readonly prisma = prismaClient) {}
async create(errorCodeId: string, productId: string): Promise<ErrorCodeLookup> {
return this.prisma.errorCodeLookup.create({ data: { errorCodeId, productId } });
}
async countByCodeForProduct(
productId: string,
from: Date,
to: Date,
): Promise<Array<{ errorCodeId: string; count: number }>> {
const grouped = await this.prisma.errorCodeLookup.groupBy({
by: ['errorCodeId'],
where: { productId, createdAt: { gte: from, lte: to } },
_count: { errorCodeId: true },
orderBy: { _count: { errorCodeId: 'desc' } },
});
return grouped.map((g) => ({ errorCodeId: g.errorCodeId, count: g._count.errorCodeId }));
}
}
export const errorCodeLookupRepository = new ErrorCodeLookupRepository();
@@ -13,6 +13,10 @@ export class ErrorCodesRepository {
where: { productId_code: { productId, code } },
});
}
async findById(id: string): Promise<ErrorCode | null> {
return this.prisma.errorCode.findUnique({ where: { id } });
}
}
export const errorCodesRepository = new ErrorCodesRepository();
@@ -1,4 +1,5 @@
export * from './knowledge.repository';
export * from './error-codes.repository';
export * from './error-code-lookup.repository';
export * from './known-issues.repository';
export * from './runbooks.repository';
@@ -4,6 +4,8 @@ import { knownErrorLookupsCounter } from '@/infrastructure/observability';
import {
errorCodesRepository,
ErrorCodesRepository,
errorCodeLookupRepository,
ErrorCodeLookupRepository,
knownIssuesRepository,
KnownIssuesRepository,
CreateKnownIssueData,
@@ -13,6 +15,7 @@ export class ErrorCodesService {
constructor(
private readonly errorCodesRepo: ErrorCodesRepository = errorCodesRepository,
private readonly knownIssuesRepo: KnownIssuesRepository = knownIssuesRepository,
private readonly lookupsRepo: ErrorCodeLookupRepository = errorCodeLookupRepository,
) {}
async createErrorCode(productId: string, code: string, description: string): Promise<ErrorCode> {
@@ -28,12 +31,36 @@ export class ErrorCodesService {
const errorCode = await this.errorCodesRepo.findByCode(productId, code);
if (!errorCode) throw new NotFoundError('Error code not found.');
// 014-full-observability data-model.md #9: "most common errors" — a raw counter, ranked by
// an external monitoring stack (FR-009), counted only once the code is confirmed real.
// 014-full-observability data-model.md #9: "most common errors" — a raw, process-lifetime
// counter for a live monitoring stack (FR-009 there), counted only once the code is
// confirmed real.
knownErrorLookupsCounter.inc({ code });
// 015-reporting-dashboards research.md §6: the durable counterpart — the live counter above
// resets on every restart, so a historical "top errors" report needs its own audit row.
await this.lookupsRepo.create(errorCode.id, productId);
return this.knownIssuesRepo.findByErrorCodeId(errorCode.id);
}
/** 015-reporting-dashboards: the Product dashboard's "top errors" ranking — encapsulated here
* (not exposed as raw repository access) since resolving a lookup count back to its error
* code's own `code` string is this module's own concern, not the reports module's. */
async getTopErrorCodesForProduct(
productId: string,
from: Date,
to: Date,
limit: number,
): Promise<Array<{ code: string; count: number }>> {
const ranked = await this.lookupsRepo.countByCodeForProduct(productId, from, to);
const top = ranked.slice(0, limit);
const rows = await Promise.all(
top.map(async (row) => {
const errorCode = await this.errorCodesRepo.findById(row.errorCodeId);
return { code: errorCode?.code ?? row.errorCodeId, count: row.count };
}),
);
return rows;
}
}
export const errorCodesService = new ErrorCodesService();
+9
View File
@@ -3,11 +3,20 @@ export { SessionsService, sessionsService } from './service';
export type { SessionTurnResult } from './service';
export { ConfidencePolicyService, confidencePolicyService } from './service';
export type { ResolvedConfidencePolicy } from './service';
// 015-reporting-dashboards research.md §7: reused for the AI dashboard's confidence
// distribution, not reimplemented.
export { decideConfidenceBand } from './service';
export type { ConfidenceBand } from './service';
export {
sessionRepository,
SessionRepository,
diagnosisRepository,
DiagnosisRepository,
// 015-reporting-dashboards: test setup needs to record a session's knowledge references
// directly, the same "extend an existing module's public surface for a later feature"
// precedent as 004's productsRepository/009's problemsRepository.
knowledgeReferenceRepository,
KnowledgeReferenceRepository,
} from './repository';
export { ACTIVE_SESSION_STATUSES, TERMINAL_SESSION_STATUSES } from './mapper';
export type { SessionStatus } from './mapper';
@@ -0,0 +1 @@
export * from './reports.controller';
@@ -0,0 +1,39 @@
import { FastifyReply, FastifyRequest } from 'fastify';
import { reportsService, ReportsService } from '../service';
import { dateRangeQuerySchema } from '../schema';
import { resolveDateRange } from '../mapper';
export class ReportsController {
constructor(private readonly service: ReportsService = reportsService) {}
async getManagementDashboard(request: FastifyRequest, reply: FastifyReply) {
const query = dateRangeQuerySchema.parse(request.query);
const range = resolveDateRange(query);
const dashboard = await this.service.getManagementDashboard(range);
return reply.status(200).send({ success: true, data: dashboard, meta: null });
}
async getProductDashboard(request: FastifyRequest, reply: FastifyReply) {
const { externalProductId } = request.params as { externalProductId: string };
const query = dateRangeQuerySchema.parse(request.query);
const range = resolveDateRange(query);
const dashboard = await this.service.getProductDashboard(externalProductId, range);
return reply.status(200).send({ success: true, data: dashboard, meta: null });
}
async getSupportDashboard(request: FastifyRequest, reply: FastifyReply) {
const query = dateRangeQuerySchema.parse(request.query);
const range = resolveDateRange(query);
const dashboard = await this.service.getSupportDashboard(range);
return reply.status(200).send({ success: true, data: dashboard, meta: null });
}
async getAiDashboard(request: FastifyRequest, reply: FastifyReply) {
const query = dateRangeQuerySchema.parse(request.query);
const range = resolveDateRange(query);
const dashboard = await this.service.getAiDashboard(range);
return reply.status(200).send({ success: true, data: dashboard, meta: null });
}
}
export const reportsController = new ReportsController();
+8 -11
View File
@@ -1,11 +1,8 @@
export const REPORTS_CONSTANTS = {
MODULE_NAME: 'PLATFORM_REPORTS',
} as const;
export class ReportsService {
async generateSummaryReport(): Promise<Record<string, unknown>> {
return {};
}
}
export const reportsService = new ReportsService();
export { reportsRoutes } from './routes';
export { ReportsService, reportsService } from './service';
export type {
ManagementDashboard,
ProductDashboard,
SupportDashboard,
AiDashboard,
} from './service';
@@ -0,0 +1,39 @@
import { ValidationError } from '@/common/errors';
import { reportingConfig } from '@/config';
export interface DateRange {
from: Date;
to: Date;
}
/**
* 015-reporting-dashboards data-model.md "Query Parameters": both ends optional — `to` defaults
* to now, `from` defaults to `to - reportingConfig.defaultWindowDays`. `from > to` is a
* ValidationError (spec.md Edge Cases), never silently swapped or silently returning empty data.
*/
export function resolveDateRange(query: {
from?: string | undefined;
to?: string | undefined;
}): DateRange {
const to = query.to ? new Date(query.to) : new Date();
if (Number.isNaN(to.getTime())) {
throw new ValidationError('"to" is not a valid date.');
}
const from = query.from
? new Date(query.from)
: new Date(to.getTime() - reportingConfig.defaultWindowDays * 24 * 60 * 60 * 1000);
if (Number.isNaN(from.getTime())) {
throw new ValidationError('"from" is not a valid date.');
}
if (from > to) {
throw new ValidationError('"from" must not be after "to".');
}
return { from, to };
}
export function serializeDateRange(range: DateRange): { from: string; to: string } {
return { from: range.from.toISOString(), to: range.to.toISOString() };
}
@@ -0,0 +1,14 @@
interface TicketWithFirstAgentMessage {
createdAt: Date;
messages: Array<{ createdAt: Date }>;
}
/** Shared by ManagementRepository and SupportRepository — both need "ticket createdAt -> its
* first AGENT_MESSAGE createdAt" in milliseconds, for tickets that actually have one. */
export function extractFirstResponseDurationsMs(tickets: TicketWithFirstAgentMessage[]): number[] {
return tickets.flatMap((t) => {
const firstAgentMessage = t.messages[0];
if (!firstAgentMessage) return [];
return [firstAgentMessage.createdAt.getTime() - t.createdAt.getTime()];
});
}
@@ -0,0 +1,3 @@
export * from './date-range';
export * from './rate';
export * from './durations';
@@ -0,0 +1,16 @@
/**
* 015-reporting-dashboards research.md §3: every rate/average is `number | null` — `null` means
* "no qualifying data in range," distinguished from a genuine `0` (e.g. a real 0% AI resolution
* rate is meaningful; "nobody's data exists yet" is not the same thing). Never computed as
* `numerator / 0`, which would silently produce `NaN`.
*/
export function computeRate(numerator: number, denominator: number): number | null {
if (denominator === 0) return null;
return numerator / denominator;
}
export function computeAverageSeconds(durationsMs: number[]): number | null {
if (durationsMs.length === 0) return null;
const totalMs = durationsMs.reduce((sum, ms) => sum + ms, 0);
return totalMs / durationsMs.length / 1000;
}
@@ -0,0 +1,74 @@
import { prismaClient } from '@/infrastructure/database';
import { DateRange } from '../mapper';
export class AiRepository {
constructor(private readonly prisma = prismaClient) {}
async sessionOutcomeCounts(
range: DateRange,
): Promise<{ resolved: number; escalated: number; total: number }> {
const [resolved, escalated, total] = await Promise.all([
this.prisma.aISupportSession.count({
where: { startedAt: { gte: range.from, lte: range.to }, status: 'resolved' },
}),
this.prisma.aISupportSession.count({
where: {
startedAt: { gte: range.from, lte: range.to },
status: { in: ['escalated', 'ended_by_agent'] },
},
}),
this.prisma.aISupportSession.count({
where: { startedAt: { gte: range.from, lte: range.to } },
}),
]);
return { resolved, escalated, total };
}
/**
* "Failed troubleshooting then escalated" (spec.md User Story 4) has no single stored flag —
* classifyStepOutcome's own verdicts aren't persisted as a durable per-step record. Documented
* proxy: an escalated session that made at least one tool call (toolCallCount > 0) attempted
* troubleshooting before giving up, vs. one that escalated immediately with zero attempts.
*/
async escalatedSessionsWithToolAttempts(range: DateRange): Promise<number> {
return this.prisma.aISupportSession.count({
where: {
startedAt: { gte: range.from, lte: range.to },
status: { in: ['escalated', 'ended_by_agent'] },
toolCallCount: { gt: 0 },
},
});
}
async sessionsWithKnowledgeMatch(range: DateRange): Promise<number> {
const sessions = await this.prisma.aISupportSession.findMany({
where: { startedAt: { gte: range.from, lte: range.to } },
select: { knowledgeRefs: { select: { id: true }, take: 1 } },
});
return sessions.filter((s) => s.knowledgeRefs.length > 0).length;
}
async diagnosisConfidences(range: DateRange): Promise<number[]> {
const diagnoses = await this.prisma.aIDiagnosis.findMany({
where: { createdAt: { gte: range.from, lte: range.to } },
select: { confidence: true },
});
return diagnoses.map((d) => d.confidence);
}
async toolInvocationOutcomeCounts(
range: DateRange,
): Promise<{ success: number; failed: number }> {
const [success, failed] = await Promise.all([
this.prisma.aIActionResult.count({
where: { status: 'success', createdAt: { gte: range.from, lte: range.to } },
}),
this.prisma.aIActionResult.count({
where: { status: 'failed', createdAt: { gte: range.from, lte: range.to } },
}),
]);
return { success, failed };
}
}
export const aiRepository = new AiRepository();
@@ -0,0 +1,5 @@
export * from './management.repository';
export * from './product.repository';
export * from './support.repository';
export * from './ai.repository';
export * from './shared.repository';
@@ -0,0 +1,79 @@
import { prismaClient } from '@/infrastructure/database';
import { DateRange } from '../mapper';
export class ManagementRepository {
constructor(private readonly prisma = prismaClient) {}
async totalCases(range: DateRange): Promise<number> {
return this.prisma.ticket.count({
where: { createdAt: { gte: range.from, lte: range.to } },
});
}
async countByStatus(range: DateRange, statuses: string[]): Promise<number> {
return this.prisma.ticket.count({
where: { createdAt: { gte: range.from, lte: range.to }, status: { in: statuses } },
});
}
/**
* Ever escalated to a human. NOT a current-status check: 003-ticketing's own state machine
* lets both the AI path (AI_RESOLVED) and the human path (HUMAN_ESCALATION) converge on the
* same shared terminal statuses (RESOLUTION_PENDING_CUSTOMER, RESOLVED, CLOSED, REOPENED are
* all reachable from AI_RESOLVED directly, per ticket-state-machine.ts's own transition
* table) — a status-list check over those shared statuses would count every AI-resolved
* ticket as "human escalated" too (caught via manual verification against real seeded data,
* not by any test fixture, since every existing test's fixtures happened to keep the two
* paths' terminal statuses apart).
*
* The unambiguous, direct signal instead: 007-orchestration-assignment's own
* `orchestrationService.handleHumanEscalation` is the *only* code path that ever creates an
* `Assignment` row (research.md's own module map) — a ticket has one if and only if it was
* actually escalated to a human at some point, regardless of its current status.
*/
async countEverEscalatedToHuman(range: DateRange): Promise<number> {
return this.prisma.ticket.count({
where: {
createdAt: { gte: range.from, lte: range.to },
assignments: { some: {} },
},
});
}
/** research.md §4: `Resolution.resolvedBy` is the single source of truth for AI vs. human. */
async countResolutionsBy(range: DateRange, resolvedByAi: boolean): Promise<number> {
return this.prisma.resolution.count({
where: {
resolvedAt: { gte: range.from, lte: range.to },
resolvedBy: resolvedByAi ? 'ai' : { not: 'ai' },
},
});
}
async slaOutcomeCounts(range: DateRange): Promise<{ met: number; breached: number }> {
const [met, breached] = await Promise.all([
this.prisma.sLARun.count({
where: {
ticket: { createdAt: { gte: range.from, lte: range.to } },
status: 'completed',
breachedAt: null,
},
}),
this.prisma.sLARun.count({
where: {
ticket: { createdAt: { gte: range.from, lte: range.to } },
breachedAt: { not: null },
},
}),
]);
return { met, breached };
}
async escalationCount(range: DateRange): Promise<number> {
return this.prisma.escalationEvent.count({
where: { createdAt: { gte: range.from, lte: range.to } },
});
}
}
export const managementRepository = new ManagementRepository();
@@ -0,0 +1,57 @@
import { prismaClient } from '@/infrastructure/database';
import { DateRange } from '../mapper';
// Named ProductReportRepository (not ProductRepository) to avoid colliding with
// catalog/products' own ProductsRepository, which this module reuses (via its public index) for
// resolving externalProductId -> Product rather than duplicating that lookup here.
export class ProductReportRepository {
constructor(private readonly prisma = prismaClient) {}
async supportVolume(productId: string, range: DateRange): Promise<number> {
return this.prisma.ticket.count({
where: { productId, createdAt: { gte: range.from, lte: range.to } },
});
}
async problemsByCategory(
productId: string,
range: DateRange,
): Promise<Array<{ categoryId: string | null; count: number }>> {
const grouped = await this.prisma.problem.groupBy({
by: ['categoryId'],
where: { productId, createdAt: { gte: range.from, lte: range.to } },
_count: { categoryId: true },
orderBy: { _count: { categoryId: 'desc' } },
});
return grouped.map((g) => ({ categoryId: g.categoryId, count: g._count.categoryId }));
}
async countResolutionsBy(
productId: string,
range: DateRange,
resolvedByAi: boolean,
): Promise<number> {
return this.prisma.resolution.count({
where: {
resolvedAt: { gte: range.from, lte: range.to },
resolvedBy: resolvedByAi ? 'ai' : { not: 'ai' },
ticket: { productId },
},
});
}
/** Same fixed "has at least one Assignment row" signal as
* ManagementRepository.countEverEscalatedToHuman (see its own comment for why a current-status
* check is wrong), scoped to one product. */
async countEverEscalatedToHuman(productId: string, range: DateRange): Promise<number> {
return this.prisma.ticket.count({
where: {
productId,
createdAt: { gte: range.from, lte: range.to },
assignments: { some: {} },
},
});
}
}
export const productReportRepository = new ProductReportRepository();
@@ -0,0 +1,34 @@
import { prismaClient } from '@/infrastructure/database';
import { DateRange, extractFirstResponseDurationsMs } from '../mapper';
/** Response/resolution duration queries the Management and Support dashboards both need
* identically — composed by each, not duplicated. */
export class SharedReportRepository {
constructor(private readonly prisma = prismaClient) {}
async firstResponseDurationsMs(range: DateRange): Promise<number[]> {
const tickets = await this.prisma.ticket.findMany({
where: { createdAt: { gte: range.from, lte: range.to } },
select: {
createdAt: true,
messages: {
where: { type: 'AGENT_MESSAGE' },
orderBy: { createdAt: 'asc' },
take: 1,
select: { createdAt: true },
},
},
});
return extractFirstResponseDurationsMs(tickets);
}
async resolutionDurationsMs(range: DateRange): Promise<number[]> {
const resolutions = await this.prisma.resolution.findMany({
where: { resolvedAt: { gte: range.from, lte: range.to } },
select: { resolvedAt: true, ticket: { select: { createdAt: true } } },
});
return resolutions.map((r) => r.resolvedAt.getTime() - r.ticket.createdAt.getTime());
}
}
export const sharedReportRepository = new SharedReportRepository();
@@ -0,0 +1,40 @@
import { prismaClient } from '@/infrastructure/database';
import { DateRange } from '../mapper';
export class SupportRepository {
constructor(private readonly prisma = prismaClient) {}
/** research.md §2: current, point-in-time — not range-scoped. "How much work is assigned
* right now," not a historical count. */
async workloadByAgent(): Promise<Array<{ agentId: string; openAssignments: number }>> {
const grouped = await this.prisma.assignment.groupBy({
by: ['agentId'],
where: { isCurrent: true },
_count: { agentId: true },
});
return grouped.map((g) => ({ agentId: g.agentId, openAssignments: g._count.agentId }));
}
async slaAtRisk(thresholdMinutes: number): Promise<number> {
const now = new Date();
const riskCutoff = new Date(now.getTime() + thresholdMinutes * 60 * 1000);
return this.prisma.sLARun.count({
where: {
status: 'running',
resolutionDueAt: { gte: now, lte: riskCutoff },
},
});
}
async slaBreached(): Promise<number> {
return this.prisma.sLARun.count({ where: { status: 'breached' } });
}
async escalationCount(range: DateRange): Promise<number> {
return this.prisma.escalationEvent.count({
where: { createdAt: { gte: range.from, lte: range.to } },
});
}
}
export const supportRepository = new SupportRepository();
@@ -0,0 +1 @@
export * from './reports.routes';
@@ -0,0 +1,28 @@
import { FastifyInstance } from 'fastify';
import { requireRole } from '@/modules/identity/auth';
import { reportsController } from '../controller';
/** contracts/reports-api-contract.md: every dashboard is admin-only, the same gate every other
* admin-only surface uses since 010-identity-auth. */
export async function reportsRoutes(fastify: FastifyInstance): Promise<void> {
fastify.get(
'/admin/reports/management',
{ preHandler: [fastify.authenticate, requireRole('ADMIN')] },
(req, reply) => reportsController.getManagementDashboard(req, reply),
);
fastify.get(
'/admin/reports/product/:externalProductId',
{ preHandler: [fastify.authenticate, requireRole('ADMIN')] },
(req, reply) => reportsController.getProductDashboard(req, reply),
);
fastify.get(
'/admin/reports/support',
{ preHandler: [fastify.authenticate, requireRole('ADMIN')] },
(req, reply) => reportsController.getSupportDashboard(req, reply),
);
fastify.get(
'/admin/reports/ai',
{ preHandler: [fastify.authenticate, requireRole('ADMIN')] },
(req, reply) => reportsController.getAiDashboard(req, reply),
);
}
@@ -0,0 +1 @@
export * from './reports.schema';
@@ -0,0 +1,10 @@
import { z } from 'zod';
export const dateRangeQuerySchema = z
.object({
from: z.string().optional(),
to: z.string().optional(),
})
.strict();
export type DateRangeQuery = z.infer<typeof dateRangeQuerySchema>;
@@ -0,0 +1 @@
export * from './reports.service';
@@ -0,0 +1,219 @@
import { NotFoundError } from '@/common/errors';
import { reportingConfig, aiConfig } from '@/config';
import { productsRepository, ProductsRepository } from '@/modules/catalog/products';
import { decideConfidenceBand } from '@/modules/ai-support/sessions';
import { errorCodesService, ErrorCodesService } from '@/modules/ai-support/knowledge';
import {
managementRepository,
ManagementRepository,
productReportRepository,
ProductReportRepository,
supportRepository,
SupportRepository,
aiRepository,
AiRepository,
sharedReportRepository,
SharedReportRepository,
} from '../repository';
import { DateRange, serializeDateRange, computeRate, computeAverageSeconds } from '../mapper';
export interface ManagementDashboard {
range: { from: string; to: string };
totalCases: number;
aiResolved: number;
humanEscalated: number;
resolved: number;
open: number;
slaCompliance: { met: number; breached: number; rate: number | null };
escalationCount: number;
averageResponseSeconds: number | null;
averageResolutionSeconds: number | null;
}
export interface ProductDashboard {
productId: string;
range: { from: string; to: string };
supportVolume: number;
problemsByCategory: Array<{ categoryId: string | null; count: number }>;
recurringProblems: Array<{ categoryId: string | null; count: number }>;
aiResolutionRate: number | null;
humanEscalationRate: number | null;
topErrors: Array<{ code: string; count: number }>;
}
export interface SupportDashboard {
generatedAt: string;
range: { from: string; to: string };
workloadByAgent: Array<{ agentId: string; openAssignments: number }>;
slaAtRisk: number;
slaBreached: number;
escalationCount: number;
averageResponseSeconds: number | null;
averageResolutionSeconds: number | null;
}
export interface AiDashboard {
range: { from: string; to: string };
totalSessions: number;
aiResolutionRate: number | null;
humanHandoffRate: number | null;
failedTroubleshootingEscalationRate: number | null;
knowledgeMatchRate: number | null;
confidenceDistribution: { proceed: number; ask: number; escalate: number };
toolInvocations: { success: number; failed: number };
}
const RESOLVED_STATUSES = ['RESOLVED', 'CLOSED'];
export class ReportsService {
constructor(
private readonly management: ManagementRepository = managementRepository,
private readonly productReports: ProductReportRepository = productReportRepository,
private readonly support: SupportRepository = supportRepository,
private readonly ai: AiRepository = aiRepository,
private readonly products: ProductsRepository = productsRepository,
private readonly errorCodes: ErrorCodesService = errorCodesService,
private readonly shared: SharedReportRepository = sharedReportRepository,
) {}
async getManagementDashboard(range: DateRange): Promise<ManagementDashboard> {
const [
totalCases,
aiResolved,
humanEscalated,
resolved,
slaOutcomes,
escalationCount,
responseDurations,
resolutionDurations,
] = await Promise.all([
this.management.totalCases(range),
this.management.countResolutionsBy(range, true),
this.management.countEverEscalatedToHuman(range),
this.management.countByStatus(range, RESOLVED_STATUSES),
this.management.slaOutcomeCounts(range),
this.management.escalationCount(range),
this.shared.firstResponseDurationsMs(range),
this.shared.resolutionDurationsMs(range),
]);
return {
range: serializeDateRange(range),
totalCases,
aiResolved,
humanEscalated,
resolved,
open: totalCases - resolved,
slaCompliance: {
met: slaOutcomes.met,
breached: slaOutcomes.breached,
rate: computeRate(slaOutcomes.met, slaOutcomes.met + slaOutcomes.breached),
},
escalationCount,
averageResponseSeconds: computeAverageSeconds(responseDurations),
averageResolutionSeconds: computeAverageSeconds(resolutionDurations),
};
}
async getProductDashboard(
externalProductId: string,
range: DateRange,
): Promise<ProductDashboard> {
const product = await this.products.findByExternalProductId(externalProductId);
if (!product) throw new NotFoundError('Product not found.');
const [supportVolume, problemsByCategory, aiResolvedCount, humanEscalatedCount, topErrors] =
await Promise.all([
this.productReports.supportVolume(product.id, range),
this.productReports.problemsByCategory(product.id, range),
this.productReports.countResolutionsBy(product.id, range, true),
this.productReports.countEverEscalatedToHuman(product.id, range),
this.errorCodes.getTopErrorCodesForProduct(
product.id,
range.from,
range.to,
reportingConfig.topNLimit,
),
]);
const recurringProblems = [...problemsByCategory]
.sort((a, b) => b.count - a.count)
.slice(0, reportingConfig.topNLimit);
return {
productId: externalProductId,
range: serializeDateRange(range),
supportVolume,
problemsByCategory,
recurringProblems,
aiResolutionRate: computeRate(aiResolvedCount, supportVolume),
humanEscalationRate: computeRate(humanEscalatedCount, supportVolume),
topErrors,
};
}
async getSupportDashboard(range: DateRange): Promise<SupportDashboard> {
const [
workloadByAgent,
slaAtRisk,
slaBreached,
escalationCount,
responseDurations,
resolutionDurations,
] = await Promise.all([
this.support.workloadByAgent(),
this.support.slaAtRisk(reportingConfig.slaRiskThresholdMinutes),
this.support.slaBreached(),
this.support.escalationCount(range),
this.shared.firstResponseDurationsMs(range),
this.shared.resolutionDurationsMs(range),
]);
return {
generatedAt: new Date().toISOString(),
range: serializeDateRange(range),
workloadByAgent,
slaAtRisk,
slaBreached,
escalationCount,
averageResponseSeconds: computeAverageSeconds(responseDurations),
averageResolutionSeconds: computeAverageSeconds(resolutionDurations),
};
}
async getAiDashboard(range: DateRange): Promise<AiDashboard> {
const [outcomeCounts, escalatedWithAttempts, knowledgeMatches, confidences, toolCounts] =
await Promise.all([
this.ai.sessionOutcomeCounts(range),
this.ai.escalatedSessionsWithToolAttempts(range),
this.ai.sessionsWithKnowledgeMatch(range),
this.ai.diagnosisConfidences(range),
this.ai.toolInvocationOutcomeCounts(range),
]);
const confidenceDistribution = { proceed: 0, ask: 0, escalate: 0 };
for (const confidence of confidences) {
const band = decideConfidenceBand(confidence, {
highThreshold: aiConfig.defaultHighConfidence,
lowThreshold: aiConfig.defaultLowConfidence,
});
confidenceDistribution[band] += 1;
}
return {
range: serializeDateRange(range),
totalSessions: outcomeCounts.total,
aiResolutionRate: computeRate(outcomeCounts.resolved, outcomeCounts.total),
humanHandoffRate: computeRate(outcomeCounts.escalated, outcomeCounts.total),
failedTroubleshootingEscalationRate: computeRate(
escalatedWithAttempts,
outcomeCounts.escalated,
),
knowledgeMatchRate: computeRate(knowledgeMatches, outcomeCounts.total),
confidenceDistribution,
toolInvocations: toolCounts,
};
}
}
export const reportsService = new ReportsService();
+3
View File
@@ -21,6 +21,9 @@ describe('Error codes and known issues', () => {
afterAll(async () => {
await prismaClient.knownIssue.deleteMany({ where: { product: { externalProductId } } });
// 015-reporting-dashboards: findKnownIssuesByErrorCode now also writes a durable
// ErrorCodeLookup row (RESTRICT FK to ErrorCode) — must be deleted before ErrorCode itself.
await prismaClient.errorCodeLookup.deleteMany({ where: { product: { externalProductId } } });
await prismaClient.errorCode.deleteMany({ where: { product: { externalProductId } } });
await prismaClient.product.deleteMany({ where: { externalProductId } });
await app.close();
@@ -0,0 +1,156 @@
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
import { buildApp } from '@/app';
import { prismaClient } from '@/infrastructure/database';
import { FastifyInstance } from 'fastify';
import { loginAs, authHeader } from '../../helpers/auth';
import {
encryptCredential,
generateCredentialSecret,
issueIntegrationToken,
} from '@/modules/catalog/products';
import {
sessionRepository,
diagnosisRepository,
knowledgeReferenceRepository,
} from '@/modules/ai-support/sessions';
import { actionRepository } from '@/modules/ai-support/tools';
import { aiConfig } from '@/config';
/** Covers specs/015-reporting-dashboards/quickstart.md Scenario 4 against a real Postgres/Redis. */
describe('AI dashboard (User Story 4)', () => {
let app: FastifyInstance;
let authToken: string;
const externalProductId = `TEST_AI_REPORT_PROD_${Date.now()}`;
let secret: string;
const rangeFrom = new Date(Date.now() - 60 * 60 * 1000).toISOString();
const rangeTo = new Date(Date.now() + 60 * 60 * 1000).toISOString();
beforeAll(async () => {
app = await buildApp();
authToken = await loginAs(app, 'ADMIN');
const product = await prismaClient.product.create({
data: { externalProductId, name: 'AI Report Test Product', status: 'active' },
});
secret = generateCredentialSecret();
await prismaClient.productIntegration.create({
data: {
productId: product.id,
credentialRef: encryptCredential(secret),
authMechanism: 'signed_token',
allowedScope: { tenantIds: ['tenant-1'] },
status: 'active',
rateLimitPerMinute: 1000,
rateLimitPerUserPerMinute: 1000,
},
});
});
afterAll(async () => {
await app.close();
});
async function createTicket(): Promise<string> {
const token = issueIntegrationToken(secret, {
externalProductId,
tenantId: 'tenant-1',
userId: 'user-1',
});
const created = await app.inject({
method: 'POST',
url: '/v1/support/requests',
headers: { authorization: `Bearer ${token}` },
payload: {
productId: externalProductId,
tenantId: 'tenant-1',
userId: 'user-1',
source: 'test',
problem: `AI report test ${Date.now()}-${Math.random()}`,
},
});
expect(created.statusCode).toBe(202);
return created.json().data.ticketId as string;
}
it('reflects real session outcomes, tool results, and confidence bands', async () => {
// A resolved session, with a knowledge match and a successful tool call.
const resolvedTicketId = await createTicket();
const resolvedSession = await sessionRepository.create(resolvedTicketId);
await knowledgeReferenceRepository.recordMany(resolvedSession.id, ['fake-knowledge-id']);
await diagnosisRepository.create({
sessionId: resolvedSession.id,
product: 'test-product',
problemType: 'test-problem',
severity: 'medium',
confidence: aiConfig.defaultHighConfidence,
possibleCauses: ['test cause'],
});
const successAction = await actionRepository.create({
sessionId: resolvedSession.id,
toolName: 'getTicketSnapshot',
input: {},
riskLevel: 'low',
evaluationOutcome: 'approved',
});
await actionRepository.createResult(successAction.id, { ok: true }, 'success');
await sessionRepository.updateStatus(resolvedSession.id, 'resolved');
// An escalated session, with a failed tool call and a low-confidence diagnosis.
const escalatedTicketId = await createTicket();
const escalatedSession = await sessionRepository.create(escalatedTicketId);
await diagnosisRepository.create({
sessionId: escalatedSession.id,
product: 'test-product',
problemType: 'test-problem',
severity: 'high',
confidence: aiConfig.defaultLowConfidence - 0.05,
possibleCauses: ['test cause'],
});
const failedAction = await actionRepository.create({
sessionId: escalatedSession.id,
toolName: 'getTicketSnapshot',
input: {},
riskLevel: 'low',
evaluationOutcome: 'approved',
});
await actionRepository.createResult(failedAction.id, { error: 'boom' }, 'failed');
await sessionRepository.updateStatus(escalatedSession.id, 'escalated');
const res = await app.inject({
method: 'GET',
url: `/admin/reports/ai?from=${rangeFrom}&to=${rangeTo}`,
headers: authHeader(authToken),
});
expect(res.statusCode).toBe(200);
const data = res.json().data;
expect(data.totalSessions).toBeGreaterThanOrEqual(2);
expect(data.aiResolutionRate).not.toBeNull();
expect(data.humanHandoffRate).not.toBeNull();
expect(data.knowledgeMatchRate).not.toBeNull();
expect(data.confidenceDistribution.proceed).toBeGreaterThanOrEqual(1);
expect(data.confidenceDistribution.escalate).toBeGreaterThanOrEqual(1);
expect(data.toolInvocations.success).toBeGreaterThanOrEqual(1);
expect(data.toolInvocations.failed).toBeGreaterThanOrEqual(1);
});
it('returns null rates and zero counts for a range with no AI activity', async () => {
const farPastFrom = new Date('2000-01-01').toISOString();
const farPastTo = new Date('2000-01-02').toISOString();
const res = await app.inject({
method: 'GET',
url: `/admin/reports/ai?from=${farPastFrom}&to=${farPastTo}`,
headers: authHeader(authToken),
});
expect(res.statusCode).toBe(200);
const data = res.json().data;
expect(data.totalSessions).toBe(0);
expect(data.aiResolutionRate).toBeNull();
expect(data.humanHandoffRate).toBeNull();
expect(data.knowledgeMatchRate).toBeNull();
expect(data.confidenceDistribution).toEqual({ proceed: 0, ask: 0, escalate: 0 });
expect(data.toolInvocations).toEqual({ success: 0, failed: 0 });
});
});
@@ -0,0 +1,226 @@
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
import { buildApp } from '@/app';
import { prismaClient } from '@/infrastructure/database';
import { FastifyInstance } from 'fastify';
import { loginAs, authHeader } from '../../helpers/auth';
import {
encryptCredential,
generateCredentialSecret,
issueIntegrationToken,
} from '@/modules/catalog/products';
import { ticketsRepository } from '@/modules/ticketing/tickets';
import { messagesService } from '@/modules/ticketing/messages';
import { resolutionRepository } from '@/modules/problem-management/resolutions';
/**
* Covers specs/015-reporting-dashboards/quickstart.md Scenario 1 against a real Postgres/Redis.
* Drives ticket-status transitions directly through ticketsRepository (not ticketsService) to
* avoid publishing TICKET_UPDATED — this test only needs the raw persisted state its own
* aggregation queries read, and publishing real domain events here risks the same kind of
* cross-file contamination 014-full-observability's own business-metrics.test.ts found and fixed
* (an unscoped HUMAN_ESCALATION triggering real auto-assignment against the shared agent pool).
*/
describe('Management dashboard (User Story 1)', () => {
let app: FastifyInstance;
let authToken: string;
const externalProductId = `TEST_MGMT_REPORT_PROD_${Date.now()}`;
let productId: string;
let secret: string;
const rangeFrom = new Date(Date.now() - 60 * 60 * 1000).toISOString();
const rangeTo = new Date(Date.now() + 60 * 60 * 1000).toISOString();
beforeAll(async () => {
app = await buildApp();
authToken = await loginAs(app, 'ADMIN');
const product = await prismaClient.product.create({
data: { externalProductId, name: 'Management Report Test Product', status: 'active' },
});
productId = product.id;
secret = generateCredentialSecret();
await prismaClient.productIntegration.create({
data: {
productId,
credentialRef: encryptCredential(secret),
authMechanism: 'signed_token',
allowedScope: { tenantIds: ['tenant-1'] },
status: 'active',
rateLimitPerMinute: 1000,
rateLimitPerUserPerMinute: 1000,
},
});
});
afterAll(async () => {
await app.close();
});
async function createTicket(): Promise<string> {
const token = issueIntegrationToken(secret, {
externalProductId,
tenantId: 'tenant-1',
userId: 'user-1',
});
const created = await app.inject({
method: 'POST',
url: '/v1/support/requests',
headers: { authorization: `Bearer ${token}` },
payload: {
productId: externalProductId,
tenantId: 'tenant-1',
userId: 'user-1',
source: 'test',
problem: `Management report test ${Date.now()}-${Math.random()}`,
},
});
expect(created.statusCode).toBe(202);
return created.json().data.ticketId as string;
}
async function driveDirectly(ticketId: string, statuses: string[]): Promise<void> {
let ticket = await prismaClient.ticket.findUniqueOrThrow({ where: { id: ticketId } });
for (const status of statuses) {
const updated = await ticketsRepository.updateStatus(ticketId, status, ticket.version);
if (!updated) throw new Error(`Failed to transition ticket to ${status} in test setup.`);
ticket = updated;
}
}
it('reports real figures matching the actual created data', async () => {
// AI-resolved ticket.
const aiTicketId = await createTicket();
await driveDirectly(aiTicketId, [
'AI_ANALYZING',
'AI_TROUBLESHOOTING',
'AI_VERIFYING',
'AI_RESOLVED',
]);
await resolutionRepository.create({ ticketId: aiTicketId, outcome: 'fixed', resolvedBy: 'ai' });
await driveDirectly(aiTicketId, ['RESOLVED']);
// Human-resolved ticket, with a first agent response recorded and a real Assignment row —
// "ever escalated to a human" is keyed off Assignment existence (see
// ManagementRepository.countEverEscalatedToHuman's own comment on why a ticket's current
// status can't distinguish the AI path from the human path once both converge on the same
// shared terminal statuses).
const team = await prismaClient.team.create({ data: { name: `Mgmt Report Team ${Date.now()}` } });
const agent = await prismaClient.agent.create({ data: { teamId: team.id, name: 'Mgmt Report Agent' } });
const humanTicketId = await createTicket();
await prismaClient.assignment.create({
data: { ticketId: humanTicketId, agentId: agent.id, strategy: 'MANUAL', isCurrent: true },
});
await messagesService.post(humanTicketId, 'agent-1', 'AGENT_MESSAGE', 'Looking into this.');
await driveDirectly(humanTicketId, [
'HUMAN_ESCALATION',
'IN_PROGRESS',
'RESOLUTION_PENDING_CUSTOMER',
]);
await resolutionRepository.create({
ticketId: humanTicketId,
outcome: 'fixed',
resolvedBy: 'agent-1',
});
await driveDirectly(humanTicketId, ['RESOLVED']);
// Still-open ticket.
await createTicket();
// SLA policy + one met, one breached run.
const policy = await prismaClient.sLAPolicy.create({
data: {
name: `Mgmt Report Policy ${Date.now()}`,
productId,
firstResponseMinutes: 30,
resolutionMinutes: 240,
},
});
const metTicketId = await createTicket();
await prismaClient.sLARun.create({
data: {
ticketId: metTicketId,
policyId: policy.id,
firstResponseDueAt: new Date(Date.now() + 30 * 60_000),
resolutionDueAt: new Date(Date.now() + 240 * 60_000),
status: 'completed',
completedAt: new Date(),
},
});
const breachedTicketId = await createTicket();
await prismaClient.sLARun.create({
data: {
ticketId: breachedTicketId,
policyId: policy.id,
firstResponseDueAt: new Date(Date.now() + 30 * 60_000),
resolutionDueAt: new Date(Date.now() - 60_000),
status: 'breached',
breachedAt: new Date(),
},
});
// One escalation event.
const escalatedTicketId = await createTicket();
await prismaClient.escalationEvent.create({
data: {
ticketId: escalatedTicketId,
ruleId: null,
fromNodeId: null,
toNodeId: null,
reason: 'management dashboard test',
triggeredBy: 'system',
},
});
const res = await app.inject({
method: 'GET',
url: `/admin/reports/management?from=${rangeFrom}&to=${rangeTo}`,
headers: authHeader(authToken),
});
expect(res.statusCode).toBe(200);
const data = res.json().data;
expect(data.totalCases).toBeGreaterThanOrEqual(6);
expect(data.aiResolved).toBeGreaterThanOrEqual(1);
expect(data.humanEscalated).toBeGreaterThanOrEqual(1);
expect(data.resolved).toBeGreaterThanOrEqual(2);
expect(data.open).toBeGreaterThanOrEqual(1);
expect(data.slaCompliance.met).toBeGreaterThanOrEqual(1);
expect(data.slaCompliance.breached).toBeGreaterThanOrEqual(1);
expect(data.slaCompliance.rate).not.toBeNull();
expect(data.escalationCount).toBeGreaterThanOrEqual(1);
expect(data.averageResponseSeconds).not.toBeNull();
expect(data.averageResolutionSeconds).not.toBeNull();
expect(data.range.from).toBeTruthy();
expect(data.range.to).toBeTruthy();
});
it('returns all-zero counts and all-null rates for a range with no activity', async () => {
const farPastFrom = new Date('2000-01-01').toISOString();
const farPastTo = new Date('2000-01-02').toISOString();
const res = await app.inject({
method: 'GET',
url: `/admin/reports/management?from=${farPastFrom}&to=${farPastTo}`,
headers: authHeader(authToken),
});
expect(res.statusCode).toBe(200);
const data = res.json().data;
expect(data.totalCases).toBe(0);
expect(data.aiResolved).toBe(0);
expect(data.humanEscalated).toBe(0);
expect(data.resolved).toBe(0);
expect(data.open).toBe(0);
expect(data.slaCompliance.rate).toBeNull();
expect(data.averageResponseSeconds).toBeNull();
expect(data.averageResolutionSeconds).toBeNull();
});
it('rejects a range where from is after to', async () => {
const res = await app.inject({
method: 'GET',
url: `/admin/reports/management?from=${rangeTo}&to=${rangeFrom}`,
headers: authHeader(authToken),
});
expect(res.statusCode).toBe(400);
});
});
@@ -0,0 +1,127 @@
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
import { buildApp } from '@/app';
import { prismaClient } from '@/infrastructure/database';
import { FastifyInstance } from 'fastify';
import { loginAs, authHeader } from '../../helpers/auth';
import {
encryptCredential,
generateCredentialSecret,
issueIntegrationToken,
} from '@/modules/catalog/products';
import { errorCodesService } from '@/modules/ai-support/knowledge';
/** Covers specs/015-reporting-dashboards/quickstart.md Scenario 2 against a real Postgres/Redis. */
describe('Product dashboard (User Story 2)', () => {
let app: FastifyInstance;
let authToken: string;
const rangeFrom = new Date(Date.now() - 60 * 60 * 1000).toISOString();
const rangeTo = new Date(Date.now() + 60 * 60 * 1000).toISOString();
async function setUpProduct(nameSuffix: string) {
const externalProductId = `TEST_PRODUCT_REPORT_${nameSuffix}_${Date.now()}`;
const product = await prismaClient.product.create({
data: { externalProductId, name: `Product Report ${nameSuffix}`, status: 'active' },
});
const secret = generateCredentialSecret();
await prismaClient.productIntegration.create({
data: {
productId: product.id,
credentialRef: encryptCredential(secret),
authMechanism: 'signed_token',
allowedScope: { tenantIds: ['tenant-1'] },
status: 'active',
rateLimitPerMinute: 1000,
rateLimitPerUserPerMinute: 1000,
},
});
return { externalProductId, productId: product.id, secret };
}
async function createTicket(externalProductId: string, secret: string): Promise<string> {
const token = issueIntegrationToken(secret, {
externalProductId,
tenantId: 'tenant-1',
userId: 'user-1',
});
const created = await app.inject({
method: 'POST',
url: '/v1/support/requests',
headers: { authorization: `Bearer ${token}` },
payload: {
productId: externalProductId,
tenantId: 'tenant-1',
userId: 'user-1',
source: 'test',
problem: `Product report test ${Date.now()}-${Math.random()}`,
},
});
expect(created.statusCode).toBe(202);
return created.json().data.ticketId as string;
}
beforeAll(async () => {
app = await buildApp();
authToken = await loginAs(app, 'ADMIN');
});
afterAll(async () => {
await app.close();
});
it("scopes every figure to the requested product, never another product's data", async () => {
const productA = await setUpProduct('A');
const productB = await setUpProduct('B');
await createTicket(productA.externalProductId, productA.secret);
await createTicket(productA.externalProductId, productA.secret);
await createTicket(productB.externalProductId, productB.secret);
const resA = await app.inject({
method: 'GET',
url: `/admin/reports/product/${productA.externalProductId}?from=${rangeFrom}&to=${rangeTo}`,
headers: authHeader(authToken),
});
expect(resA.statusCode).toBe(200);
expect(resA.json().data.supportVolume).toBe(2);
const resB = await app.inject({
method: 'GET',
url: `/admin/reports/product/${productB.externalProductId}?from=${rangeFrom}&to=${rangeTo}`,
headers: authHeader(authToken),
});
expect(resB.statusCode).toBe(200);
expect(resB.json().data.supportVolume).toBe(1);
});
it('ranks the most-frequently-looked-up error code first', async () => {
const product = await setUpProduct('ERR');
const popularCode = `POPULAR-${Date.now()}`;
const rareCode = `RARE-${Date.now()}`;
await errorCodesService.createErrorCode(product.productId, popularCode, 'Popular error');
await errorCodesService.createErrorCode(product.productId, rareCode, 'Rare error');
await errorCodesService.findKnownIssuesByErrorCode(product.productId, popularCode);
await errorCodesService.findKnownIssuesByErrorCode(product.productId, popularCode);
await errorCodesService.findKnownIssuesByErrorCode(product.productId, popularCode);
await errorCodesService.findKnownIssuesByErrorCode(product.productId, rareCode);
const res = await app.inject({
method: 'GET',
url: `/admin/reports/product/${product.externalProductId}?from=${rangeFrom}&to=${rangeTo}`,
headers: authHeader(authToken),
});
expect(res.statusCode).toBe(200);
const topErrors = res.json().data.topErrors as Array<{ code: string; count: number }>;
expect(topErrors[0]).toMatchObject({ code: popularCode, count: 3 });
expect(topErrors.find((e) => e.code === rareCode)).toMatchObject({ count: 1 });
});
it('404s for an unknown product', async () => {
const res = await app.inject({
method: 'GET',
url: `/admin/reports/product/NONEXISTENT_PRODUCT_${Date.now()}`,
headers: authHeader(authToken),
});
expect(res.statusCode).toBe(404);
});
});
@@ -0,0 +1,155 @@
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
import { buildApp } from '@/app';
import { prismaClient } from '@/infrastructure/database';
import { FastifyInstance } from 'fastify';
import { loginAs, authHeader } from '../../helpers/auth';
import {
encryptCredential,
generateCredentialSecret,
issueIntegrationToken,
} from '@/modules/catalog/products';
import { reportingConfig } from '@/config';
/**
* Covers specs/015-reporting-dashboards/quickstart.md Scenario 3 against a real Postgres/Redis.
* Assignment rows are created directly via Prisma (not through a real HUMAN_ESCALATION +
* default-strategy auto-assignment) — the same contamination avoidance
* management-dashboard.test.ts already documents: this test only needs the persisted
* Assignment/SLARun state its own aggregation queries read, not a live orchestration run.
*/
describe('Support dashboard (User Story 3)', () => {
let app: FastifyInstance;
let authToken: string;
const externalProductId = `TEST_SUPPORT_REPORT_PROD_${Date.now()}`;
let productId: string;
let secret: string;
beforeAll(async () => {
app = await buildApp();
authToken = await loginAs(app, 'ADMIN');
const product = await prismaClient.product.create({
data: { externalProductId, name: 'Support Report Test Product', status: 'active' },
});
productId = product.id;
secret = generateCredentialSecret();
await prismaClient.productIntegration.create({
data: {
productId,
credentialRef: encryptCredential(secret),
authMechanism: 'signed_token',
allowedScope: { tenantIds: ['tenant-1'] },
status: 'active',
rateLimitPerMinute: 1000,
rateLimitPerUserPerMinute: 1000,
},
});
});
afterAll(async () => {
await app.close();
});
async function createTicket(): Promise<string> {
const token = issueIntegrationToken(secret, {
externalProductId,
tenantId: 'tenant-1',
userId: 'user-1',
});
const created = await app.inject({
method: 'POST',
url: '/v1/support/requests',
headers: { authorization: `Bearer ${token}` },
payload: {
productId: externalProductId,
tenantId: 'tenant-1',
userId: 'user-1',
source: 'test',
problem: `Support report test ${Date.now()}-${Math.random()}`,
},
});
expect(created.statusCode).toBe(202);
return created.json().data.ticketId as string;
}
it("reflects each agent's real current assignment workload", async () => {
const team = await app.inject({
method: 'POST',
url: '/admin/teams',
headers: authHeader(authToken),
payload: { name: `Support Report Team ${Date.now()}` },
});
const teamId = team.json().data.id as string;
const agent = await app.inject({
method: 'POST',
url: `/admin/teams/${teamId}/agents`,
headers: authHeader(authToken),
payload: { name: 'Support Report Agent' },
});
const agentId = agent.json().data.id as string;
const ticket1 = await createTicket();
const ticket2 = await createTicket();
await prismaClient.assignment.createMany({
data: [
{ ticketId: ticket1, agentId, strategy: 'MANUAL', isCurrent: true },
{ ticketId: ticket2, agentId, strategy: 'MANUAL', isCurrent: true },
],
});
const res = await app.inject({
method: 'GET',
url: '/admin/reports/support',
headers: authHeader(authToken),
});
expect(res.statusCode).toBe(200);
const workload = res.json().data.workloadByAgent as Array<{
agentId: string;
openAssignments: number;
}>;
expect(workload.find((w) => w.agentId === agentId)).toMatchObject({ openAssignments: 2 });
});
it('counts a near-due SLA run as at-risk, distinct from breached', async () => {
const policy = await prismaClient.sLAPolicy.create({
data: {
name: `Support Risk Policy ${Date.now()}`,
productId,
firstResponseMinutes: 30,
resolutionMinutes: 240,
},
});
const riskTicketId = await createTicket();
await prismaClient.sLARun.create({
data: {
ticketId: riskTicketId,
policyId: policy.id,
firstResponseDueAt: new Date(Date.now() + 30 * 60_000),
resolutionDueAt: new Date(
Date.now() + (reportingConfig.slaRiskThresholdMinutes - 1) * 60_000,
),
status: 'running',
},
});
const safeTicketId = await createTicket();
await prismaClient.sLARun.create({
data: {
ticketId: safeTicketId,
policyId: policy.id,
firstResponseDueAt: new Date(Date.now() + 30 * 60_000),
resolutionDueAt: new Date(Date.now() + 999 * 60_000),
status: 'running',
},
});
const res = await app.inject({
method: 'GET',
url: '/admin/reports/support',
headers: authHeader(authToken),
});
expect(res.statusCode).toBe(200);
expect(res.json().data.slaAtRisk).toBeGreaterThanOrEqual(1);
});
});
@@ -0,0 +1,29 @@
import { describe, it, expect } from 'vitest';
import { decideConfidenceBand } from '@/modules/ai-support/sessions';
import { aiConfig } from '@/config';
/**
* 015-reporting-dashboards research.md §7: the AI dashboard's confidence distribution reuses
* 005-ai-support's own decideConfidenceBand against the system-default thresholds, rather than
* reimplementing a threshold check — this test proves the reused function classifies values
* the way the dashboard's own bucketing loop (reports.service.ts) depends on.
*/
describe('AI dashboard confidence distribution reuses decideConfidenceBand', () => {
const policy = {
highThreshold: aiConfig.defaultHighConfidence,
lowThreshold: aiConfig.defaultLowConfidence,
};
it('classifies a high-confidence value as proceed', () => {
expect(decideConfidenceBand(policy.highThreshold, policy)).toBe('proceed');
});
it('classifies a low-confidence value as escalate', () => {
expect(decideConfidenceBand(policy.lowThreshold - 0.01, policy)).toBe('escalate');
});
it('classifies a mid-range value as ask', () => {
const midpoint = (policy.highThreshold + policy.lowThreshold) / 2;
expect(decideConfidenceBand(midpoint, policy)).toBe('ask');
});
});
@@ -0,0 +1,27 @@
import { describe, it, expect } from 'vitest';
import { computeRate, computeAverageSeconds } from '@/modules/platform/reports/mapper';
describe('computeRate (015-reporting-dashboards research.md §3)', () => {
it('returns null when the denominator is zero — never NaN, never a computed 0', () => {
expect(computeRate(0, 0)).toBeNull();
expect(computeRate(5, 0)).toBeNull();
});
it('computes a real rate when there is qualifying data', () => {
expect(computeRate(3, 12)).toBe(0.25);
});
it('returns a real 0 when the numerator is legitimately zero but the denominator is not', () => {
expect(computeRate(0, 10)).toBe(0);
});
});
describe('computeAverageSeconds', () => {
it('returns null for an empty list — no fabricated average', () => {
expect(computeAverageSeconds([])).toBeNull();
});
it('averages a list of millisecond durations into seconds', () => {
expect(computeAverageSeconds([1000, 2000, 3000])).toBe(2);
});
});