feat(015-reporting-dashboards): four real reporting/analytics endpoints

Wires the pre-scaffolded, unused platform/reports module (ReportsService
.generateSummaryReport previously returned {}) into four real, admin-
gated dashboards matching docs/09-testing-observability-cicd.md's own
table:

- GET /admin/reports/management: total cases, AI-resolved, human-
  escalated, resolved/open, SLA compliance/breaches, escalation count,
  average response/resolution time.
- GET /admin/reports/product/:externalProductId: support volume,
  problem-category breakdown, recurring problems, AI-resolution/human-
  escalation rate, top error codes.
- GET /admin/reports/support: current per-agent workload, SLA at-risk/
  breached counts, escalation count, response/resolution performance.
- GET /admin/reports/ai: AI resolution/human-handoff rate, failed-
  troubleshooting-then-escalated rate, knowledge-match rate, confidence
  distribution (reusing 005-ai-support's own decideConfidenceBand),
  tool invocation success/failure.

Every rate/average is number|null -- null means no qualifying data in
range, never a computed NaN or a misleading 0. Adds one new durable
table, ErrorCodeLookup, since 014-full-observability's own equivalent
metric is a process-lifetime Prometheus counter unusable for a
historical "top errors" report.

Verified end-to-end against real Postgres/Redis: every figure checked
against hand-computed expected values, including a no-activity range
(all-zero counts, all-null rates) and cross-product isolation.

Also fixes a real regression the new ErrorCodeLookup FK caused in the
pre-existing known-issues.test.ts (its afterAll deleted ErrorCode rows
before the now-referencing lookup rows).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
saqib mir
2026-09-09 11:59:38 +05:30
co-authored by Claude Sonnet 5
parent 814d9d7b17
commit d65683641a
39 changed files with 1596 additions and 47 deletions
@@ -0,0 +1,18 @@
-- CreateTable
CREATE TABLE "error_code_lookups" (
"id" TEXT NOT NULL,
"errorCodeId" TEXT NOT NULL,
"productId" TEXT NOT NULL,
"createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP,
CONSTRAINT "error_code_lookups_pkey" PRIMARY KEY ("id")
);
-- CreateIndex
CREATE INDEX "error_code_lookups_productId_createdAt_idx" ON "error_code_lookups"("productId", "createdAt");
-- AddForeignKey
ALTER TABLE "error_code_lookups" ADD CONSTRAINT "error_code_lookups_errorCodeId_fkey" FOREIGN KEY ("errorCodeId") REFERENCES "error_codes"("id") ON DELETE RESTRICT ON UPDATE CASCADE;
-- AddForeignKey
ALTER TABLE "error_code_lookups" ADD CONSTRAINT "error_code_lookups_productId_fkey" FOREIGN KEY ("productId") REFERENCES "products"("id") ON DELETE RESTRICT ON UPDATE CASCADE;
+20 -1
View File
@@ -45,6 +45,7 @@ model Product {
tickets Ticket[]
knowledgeEntries KnowledgeEntry[]
errorCodes ErrorCode[]
errorCodeLookups ErrorCodeLookup[]
knownIssues KnownIssue[]
runbooks Runbook[]
aiConfidencePolicies AIConfidencePolicy[]
@@ -239,13 +240,31 @@ model ErrorCode {
productId String
description String
product Product @relation(fields: [productId], references: [id])
product Product @relation(fields: [productId], references: [id])
knownIssues KnownIssue[]
lookups ErrorCodeLookup[]
@@unique([productId, code])
@@map("error_codes")
}
// 015-reporting-dashboards research.md §6: a durable, append-only audit row recording that a
// known-error-code lookup happened — 014-full-observability's own equivalent
// (supporthub_known_error_lookups_total) is a process-lifetime Prometheus counter, unusable for
// a historical "top errors" report. productId is denormalized from errorCode.productId so the
// Product dashboard's range query never needs to join back through ErrorCode just to filter.
model ErrorCodeLookup {
id String @id @default(cuid())
errorCodeId String
errorCode ErrorCode @relation(fields: [errorCodeId], references: [id])
productId String
product Product @relation(fields: [productId], references: [id])
createdAt DateTime @default(now())
@@index([productId, createdAt])
@@map("error_code_lookups")
}
model KnownIssue {
id String @id @default(cuid())
productId String
@@ -47,3 +47,48 @@
required — every open question (default date window, SLA-risk threshold, top-N limit) had a
reasonable, documented, CONFIGURABLE default (see Assumptions), matching the roadmap's own
"never hardcode a placeholder value and ship it as final" instruction.
## Implementation Notes (post-build)
- Named the Product dashboard's own repository class `ProductReportRepository` (not
`ProductRepository`) once it became clear resolving `externalProductId -> Product` should
reuse `catalog/products`' own already-public `productsRepository.findByExternalProductId`
rather than duplicating that lookup — avoids a name collision and keeps "one authority per
concern" (Constitution Principle I's spirit) for product resolution.
- `ManagementRepository` and `SupportRepository` both needed byte-identical
first-response-duration and resolution-duration queries. Extracted into a shared
`SharedReportRepository` both compose, rather than duplicating the Prisma query (or the
averaging helper alone) twice — discovered while writing the second repository and seeing the
copy-paste, not planned upfront in research.md.
- "Top errors"/"most common errors" resolution-back-to-`code` logic moved into
`ErrorCodesService.getTopErrorCodesForProduct` (a new method on the module that already owns
`ErrorCode`), rather than the reports module reaching into `errorCodesRepository`/
`errorCodeLookupRepository` directly — cleaner module-boundary ownership than research.md's
original per-repository sketch implied.
- The AI dashboard's "failed troubleshooting then escalated" figure (spec.md User Story 4) has
no single stored flag anywhere in this codebase — `classifyStepOutcome`'s per-step verdicts are
never persisted as their own durable record. Implemented as a documented proxy instead: an
escalated session with `toolCallCount > 0` attempted troubleshooting before giving up, one with
zero attempts escalated immediately. Documented directly in `ai.repository.ts`'s own code
comment, the same "honest, documented simplification" precedent research.md §7 already set for
the confidence-distribution bucketing.
- Three of this module's public exports needed adding to their owning modules' top-level
`index.ts` (not previously exposed): `decideConfidenceBand`/`ConfidenceBand` and
`knowledgeReferenceRepository` from `ai-support/sessions`, matching the "extend an existing
module's public surface for a later feature" precedent already used repeatedly this session
(004's `productsRepository`, 009's `problemsRepository`).
- Found a real regression during T028's full regression pass: `known-issues.test.ts` (004-
product-knowledge, pre-existing) calls `findKnownIssuesByErrorCode` and its own `afterAll`
deleted `ErrorCode` rows before this feature's new `ErrorCodeLookup` FK (RESTRICT) existed —
once T004 started writing a lookup row on every call, that cleanup order started failing with
an FK violation. Fixed by deleting `ErrorCodeLookup` rows first in that test's own `afterAll`.
This feature's own new test files never delete `ErrorCode` rows at all, so they weren't
affected the same way (leftover rows there are the same accepted throwaway-data tradeoff
already established elsewhere this session).
- Confirmed (not caused by this feature — the exact pre-existing issue 014-full-observability's
own checklist already documented and root-caused via `git checkout` comparison) that this
feature's own new integration test files, which also name their test products `TEST_*`,
occasionally hit the same shared `deriveProductCode` "TEST" prefix collision under vitest's
concurrent file execution when run alongside other `TEST_*`-prefixed files. Every dashboard
test passes reliably run individually or in small groups; the intermittent 500 in a full
combined run is the same known, out-of-scope, 003-ticketing concern.
+33 -33
View File
@@ -1,5 +1,5 @@
---
description: "Task list for 015-reporting-dashboards"
description: 'Task list for 015-reporting-dashboards'
---
# Tasks: Reporting and Analytics Dashboards
@@ -23,32 +23,32 @@ All file paths are relative to `supporthub-api/` (repo root).
## Phase 1: Foundational (Blocking Prerequisites)
- [ ] T001 Add `REPORTING_DEFAULT_WINDOW_DAYS` (default `30`),
- [x] T001 Add `REPORTING_DEFAULT_WINDOW_DAYS` (default `30`),
`REPORTING_SLA_RISK_THRESHOLD_MINUTES` (default `60`), and `REPORTING_TOP_N_LIMIT`
(default `10`) to `src/config/env.ts`, exposed via a new `reportingConfig` in
`src/config/reporting.ts` (or added to an existing config file, matching this codebase's
own per-feature config-file convention)
- [ ] T002 Add the `ErrorCodeLookup` model to `prisma/schema.prisma` per data-model.md, generate
- [x] T002 Add the `ErrorCodeLookup` model to `prisma/schema.prisma` per data-model.md, generate
the migration via `prisma migrate diff --from-url <db-url> --to-schema-datamodel
./prisma/schema.prisma --script`, hand-write it into
./prisma/schema.prisma --script`, hand-write it into
`prisma/migrations/<timestamp>_add_error_code_lookup/migration.sql`, apply via `prisma
migrate deploy` against the throwaway test database (depends on T001 only in that both
migrate deploy` against the throwaway test database (depends on T001 only in that both
are Foundational — no code dependency)
- [ ] T003 Add `ai-support/knowledge/repository/error-code-lookup.repository.ts`
- [x] T003 Add `ai-support/knowledge/repository/error-code-lookup.repository.ts`
`create(errorCodeId, productId)`, exported from the knowledge module's repository index
(depends on T002)
- [ ] T004 [P] Call the new repository's `create(...)` from
- [x] T004 [P] Call the new repository's `create(...)` from
`ai-support/knowledge/service/error-codes.service.ts`'s existing
`findKnownIssuesByErrorCode`, alongside (not replacing) 014's own
`knownErrorLookupsCounter.inc(...)` call at that same call site (depends on T003)
- [ ] T005 [P] Add `platform/reports/mapper/date-range.ts` — parses/validates `from`/`to` query
- [x] T005 [P] Add `platform/reports/mapper/date-range.ts` — parses/validates `from`/`to` query
params, defaulting via T001's `reportingConfig.defaultWindowDays`, throwing
`ValidationError` when `from > to` (depends on T001)
- [ ] T006 [P] Add `platform/reports/mapper/rate.ts` — a shared `computeRate(numerator,
denominator): number | null` and `computeAverageSeconds(durations: number[]): number |
null` pair, both returning `null` (never `NaN`/`0`) when there's no qualifying data
- [x] T006 [P] Add `platform/reports/mapper/rate.ts` — a shared `computeRate(numerator,
denominator): number | null` and `computeAverageSeconds(durations: number[]): number |
null` pair, both returning `null` (never `NaN`/`0`) when there's no qualifying data
(research.md §3) — no dependency, pure functions
- [ ] T007 Scaffold `platform/reports/schema/` (query-param zod schema using T005's date-range
- [x] T007 Scaffold `platform/reports/schema/` (query-param zod schema using T005's date-range
parsing), `platform/reports/controller/reports.controller.ts` (empty methods to be filled
in per user story below), `platform/reports/routes/reports.routes.ts` registering all four
routes behind `requireRole('ADMIN')`, and update `platform/reports/index.ts` to export the
@@ -69,20 +69,20 @@ can now be built independently.
### Tests for User Story 1
- [ ] T008 [P] [US1] Unit tests for T006's `computeRate`/`computeAverageSeconds` (empty input ->
- [x] T008 [P] [US1] Unit tests for T006's `computeRate`/`computeAverageSeconds` (empty input ->
`null`; a real mix -> the correct value) in
`tests/unit/platform/reports/rate-helpers.test.ts`
### Implementation for User Story 1
- [ ] T009 [US1] Add `platform/reports/repository/management.repository.ts` — one method per
- [x] T009 [US1] Add `platform/reports/repository/management.repository.ts` — one method per
research.md §2's Management table row (ticket counts by status, SLA-run outcome counts,
response/resolution duration row-fetches for T006 to average) (depends on T007)
- [ ] T010 [US1] Add `ReportsService.getManagementDashboard(range)` composing T009's repository
- [x] T010 [US1] Add `ReportsService.getManagementDashboard(range)` composing T009's repository
calls into the `ManagementDashboard` shape, reusing the `Resolution.resolvedBy` convention
(research.md §4) for the AI-vs-human split (depends on T009)
- [ ] T011 [US1] Wire `GET /admin/reports/management` to the controller/service (depends on T010)
- [ ] T012 [US1] Integration test covering Quickstart Scenario 1 (real tickets in various
- [x] T011 [US1] Wire `GET /admin/reports/management` to the controller/service (depends on T010)
- [x] T012 [US1] Integration test covering Quickstart Scenario 1 (real tickets in various
terminal states, a met and a breached SLA run, verified figure-by-figure; a no-activity
range returns all-zero counts and all-null rates) in
`tests/integration/platform-reports/management-dashboard.test.ts` (depends on T011)
@@ -100,14 +100,14 @@ can now be built independently.
### Implementation for User Story 2
- [ ] T013 [P] [US2] Add `platform/reports/repository/product.repository.ts` — ticket/problem
- [x] T013 [P] [US2] Add `platform/reports/repository/product.repository.ts` — ticket/problem
queries scoped by `productId`, plus a query against T003's `ErrorCodeLookup` table for
the top-N ranking (`reportingConfig.topNLimit`) (depends on T007)
- [ ] T014 [US2] Add `ReportsService.getProductDashboard(externalProductId, range)`, 404-ing via
- [x] T014 [US2] Add `ReportsService.getProductDashboard(externalProductId, range)`, 404-ing via
`NotFoundError` when the product doesn't resolve (FR-006) before running any aggregation
query (depends on T013)
- [ ] T015 [US2] Wire `GET /admin/reports/product/:externalProductId` (depends on T014)
- [ ] T016 [US2] Integration test covering Quickstart Scenario 2 (two products' data never
- [x] T015 [US2] Wire `GET /admin/reports/product/:externalProductId` (depends on T014)
- [x] T016 [US2] Integration test covering Quickstart Scenario 2 (two products' data never
cross-contaminating each other's figures; an unknown product 404s) in
`tests/integration/platform-reports/product-dashboard.test.ts` (depends on T015)
@@ -123,13 +123,13 @@ can now be built independently.
### Implementation for User Story 3
- [ ] T017 [P] [US3] Add `platform/reports/repository/support.repository.ts` — current
- [x] T017 [P] [US3] Add `platform/reports/repository/support.repository.ts` — current
`Assignment` workload-by-agent query, `SLARun` at-risk/breached queries (`resolutionDueAt`
within `reportingConfig.slaRiskThresholdMinutes` of now, per research.md §2) (depends on
T007)
- [ ] T018 [US3] Add `ReportsService.getSupportDashboard(range)` (depends on T017)
- [ ] T019 [US3] Wire `GET /admin/reports/support` (depends on T018)
- [ ] T020 [US3] Integration test covering Quickstart Scenario 3 (real per-agent assignment
- [x] T018 [US3] Add `ReportsService.getSupportDashboard(range)` (depends on T017)
- [x] T019 [US3] Wire `GET /admin/reports/support` (depends on T018)
- [x] T020 [US3] Integration test covering Quickstart Scenario 3 (real per-agent assignment
counts; a near-due-but-not-breached run counted as at-risk, distinct from breached) in
`tests/integration/platform-reports/support-dashboard.test.ts` (depends on T019)
@@ -145,20 +145,20 @@ can now be built independently.
### Tests for User Story 4
- [ ] T021 [P] [US4] Unit test: the confidence-distribution bucketing reuses
- [x] T021 [P] [US4] Unit test: the confidence-distribution bucketing reuses
`decideConfidenceBand` (005-ai-support) against `aiConfig` defaults, not a reimplemented
threshold check, in `tests/unit/platform/reports/confidence-distribution.test.ts`
### Implementation for User Story 4
- [ ] T022 [US4] Add `platform/reports/repository/ai.repository.ts` — `AISupportSession` outcome
- [x] T022 [US4] Add `platform/reports/repository/ai.repository.ts` — `AISupportSession` outcome
counts, `AIDiagnosis` confidence fetch, `AIKnowledgeReference` presence-per-session query,
`AIAction`/`AIActionResult` outcome counts (depends on T007)
- [ ] T023 [US4] Add `ReportsService.getAiDashboard(range)`, bucketing confidence via
- [x] T023 [US4] Add `ReportsService.getAiDashboard(range)`, bucketing confidence via
`decideConfidenceBand` + `aiConfig.defaultHighConfidence`/`defaultLowConfidence`
(research.md §7) (depends on T022, T021)
- [ ] T024 [US4] Wire `GET /admin/reports/ai` (depends on T023)
- [ ] T025 [US4] Integration test covering Quickstart Scenario 4 (real AI sessions to mixed
- [x] T024 [US4] Wire `GET /admin/reports/ai` (depends on T023)
- [x] T025 [US4] Integration test covering Quickstart Scenario 4 (real AI sessions to mixed
outcomes, mixed tool results, a spread of diagnosis confidence values) in
`tests/integration/platform-reports/ai-dashboard.test.ts` (depends on T024)
@@ -169,10 +169,10 @@ together — this feature's full scope.
## Phase 6: Polish & Cross-Cutting Concerns
- [ ] T026 [P] Update `specs/015-reporting-dashboards/checklists/requirements.md` Notes with any
- [x] T026 [P] Update `specs/015-reporting-dashboards/checklists/requirements.md` Notes with any
implementation-time findings
- [ ] T027 Run `npx tsx scripts/check-architecture.ts` and `npm run lint`/`npm run typecheck`
- [ ] T028 Full regression: `npm run test:unit` then the full integration suite against real
- [x] T027 Run `npx tsx scripts/check-architecture.ts` and `npm run lint`/`npm run typecheck`
- [x] T028 Full regression: `npm run test:unit` then the full integration suite against real
Docker-provisioned Postgres/Redis, confirming nothing outside this feature regressed
(particularly `error-codes.service.ts`'s own existing tests, now touched by T004)
+2
View File
@@ -24,6 +24,7 @@ import { solutionsRoutes } from '@/modules/problem-management/solutions';
import { verificationRoutes } from '@/modules/problem-management/verification';
import { resolutionsRoutes } from '@/modules/problem-management/resolutions';
import { authRoutes } from '@/modules/identity/auth';
import { reportsRoutes } from '@/modules/platform/reports';
export async function registerGlobalRoutes(app: FastifyInstance): Promise<void> {
await app.register(healthRoutes);
@@ -49,5 +50,6 @@ export async function registerGlobalRoutes(app: FastifyInstance): Promise<void>
await app.register(solutionsRoutes);
await app.register(verificationRoutes);
await app.register(resolutionsRoutes);
await app.register(reportsRoutes);
// Further domain module routes will be registered here as feature modules are wired up
}
+7
View File
@@ -80,6 +80,13 @@ const envSchema = z.object({
// configured" — tracing still runs, just exports to the console instead (never a startup
// requirement) — see specs/014-full-observability/research.md "Distributed tracing".
OTEL_EXPORTER_OTLP_ENDPOINT: z.string().optional(),
// Reporting and Analytics Dashboards (015) — CONFIGURABLE per docs/10-implementation-
// roadmap.md's own "never hardcode a placeholder value and ship it as final" instruction —
// see specs/015-reporting-dashboards/research.md §8.
REPORTING_DEFAULT_WINDOW_DAYS: z.coerce.number().default(30),
REPORTING_SLA_RISK_THRESHOLD_MINUTES: z.coerce.number().default(60),
REPORTING_TOP_N_LIMIT: z.coerce.number().default(10),
});
export type EnvConfig = z.infer<typeof envSchema>;
+1
View File
@@ -7,3 +7,4 @@ export * from './ai';
export * from './orchestration';
export * from './problem-resolution';
export * from './auth';
export * from './reporting';
+7
View File
@@ -0,0 +1,7 @@
import { env } from './env';
export const reportingConfig = {
defaultWindowDays: env.REPORTING_DEFAULT_WINDOW_DAYS,
slaRiskThresholdMinutes: env.REPORTING_SLA_RISK_THRESHOLD_MINUTES,
topNLimit: env.REPORTING_TOP_N_LIMIT,
};
@@ -0,0 +1,31 @@
import { prismaClient } from '@/infrastructure/database';
import { ErrorCodeLookup } from '@prisma/client';
/**
* 015-reporting-dashboards research.md §6: a durable, append-only audit row — no update/delete
* path, every lookup is its own row, duplicates over time are the point (frequency is what
* "top errors" measures).
*/
export class ErrorCodeLookupRepository {
constructor(private readonly prisma = prismaClient) {}
async create(errorCodeId: string, productId: string): Promise<ErrorCodeLookup> {
return this.prisma.errorCodeLookup.create({ data: { errorCodeId, productId } });
}
async countByCodeForProduct(
productId: string,
from: Date,
to: Date,
): Promise<Array<{ errorCodeId: string; count: number }>> {
const grouped = await this.prisma.errorCodeLookup.groupBy({
by: ['errorCodeId'],
where: { productId, createdAt: { gte: from, lte: to } },
_count: { errorCodeId: true },
orderBy: { _count: { errorCodeId: 'desc' } },
});
return grouped.map((g) => ({ errorCodeId: g.errorCodeId, count: g._count.errorCodeId }));
}
}
export const errorCodeLookupRepository = new ErrorCodeLookupRepository();
@@ -13,6 +13,10 @@ export class ErrorCodesRepository {
where: { productId_code: { productId, code } },
});
}
async findById(id: string): Promise<ErrorCode | null> {
return this.prisma.errorCode.findUnique({ where: { id } });
}
}
export const errorCodesRepository = new ErrorCodesRepository();
@@ -1,4 +1,5 @@
export * from './knowledge.repository';
export * from './error-codes.repository';
export * from './error-code-lookup.repository';
export * from './known-issues.repository';
export * from './runbooks.repository';
@@ -4,6 +4,8 @@ import { knownErrorLookupsCounter } from '@/infrastructure/observability';
import {
errorCodesRepository,
ErrorCodesRepository,
errorCodeLookupRepository,
ErrorCodeLookupRepository,
knownIssuesRepository,
KnownIssuesRepository,
CreateKnownIssueData,
@@ -13,6 +15,7 @@ export class ErrorCodesService {
constructor(
private readonly errorCodesRepo: ErrorCodesRepository = errorCodesRepository,
private readonly knownIssuesRepo: KnownIssuesRepository = knownIssuesRepository,
private readonly lookupsRepo: ErrorCodeLookupRepository = errorCodeLookupRepository,
) {}
async createErrorCode(productId: string, code: string, description: string): Promise<ErrorCode> {
@@ -28,12 +31,36 @@ export class ErrorCodesService {
const errorCode = await this.errorCodesRepo.findByCode(productId, code);
if (!errorCode) throw new NotFoundError('Error code not found.');
// 014-full-observability data-model.md #9: "most common errors" — a raw counter, ranked by
// an external monitoring stack (FR-009), counted only once the code is confirmed real.
// 014-full-observability data-model.md #9: "most common errors" — a raw, process-lifetime
// counter for a live monitoring stack (FR-009 there), counted only once the code is
// confirmed real.
knownErrorLookupsCounter.inc({ code });
// 015-reporting-dashboards research.md §6: the durable counterpart — the live counter above
// resets on every restart, so a historical "top errors" report needs its own audit row.
await this.lookupsRepo.create(errorCode.id, productId);
return this.knownIssuesRepo.findByErrorCodeId(errorCode.id);
}
/** 015-reporting-dashboards: the Product dashboard's "top errors" ranking — encapsulated here
* (not exposed as raw repository access) since resolving a lookup count back to its error
* code's own `code` string is this module's own concern, not the reports module's. */
async getTopErrorCodesForProduct(
productId: string,
from: Date,
to: Date,
limit: number,
): Promise<Array<{ code: string; count: number }>> {
const ranked = await this.lookupsRepo.countByCodeForProduct(productId, from, to);
const top = ranked.slice(0, limit);
const rows = await Promise.all(
top.map(async (row) => {
const errorCode = await this.errorCodesRepo.findById(row.errorCodeId);
return { code: errorCode?.code ?? row.errorCodeId, count: row.count };
}),
);
return rows;
}
}
export const errorCodesService = new ErrorCodesService();
+9
View File
@@ -3,11 +3,20 @@ export { SessionsService, sessionsService } from './service';
export type { SessionTurnResult } from './service';
export { ConfidencePolicyService, confidencePolicyService } from './service';
export type { ResolvedConfidencePolicy } from './service';
// 015-reporting-dashboards research.md §7: reused for the AI dashboard's confidence
// distribution, not reimplemented.
export { decideConfidenceBand } from './service';
export type { ConfidenceBand } from './service';
export {
sessionRepository,
SessionRepository,
diagnosisRepository,
DiagnosisRepository,
// 015-reporting-dashboards: test setup needs to record a session's knowledge references
// directly, the same "extend an existing module's public surface for a later feature"
// precedent as 004's productsRepository/009's problemsRepository.
knowledgeReferenceRepository,
KnowledgeReferenceRepository,
} from './repository';
export { ACTIVE_SESSION_STATUSES, TERMINAL_SESSION_STATUSES } from './mapper';
export type { SessionStatus } from './mapper';
@@ -0,0 +1 @@
export * from './reports.controller';
@@ -0,0 +1,39 @@
import { FastifyReply, FastifyRequest } from 'fastify';
import { reportsService, ReportsService } from '../service';
import { dateRangeQuerySchema } from '../schema';
import { resolveDateRange } from '../mapper';
export class ReportsController {
constructor(private readonly service: ReportsService = reportsService) {}
async getManagementDashboard(request: FastifyRequest, reply: FastifyReply) {
const query = dateRangeQuerySchema.parse(request.query);
const range = resolveDateRange(query);
const dashboard = await this.service.getManagementDashboard(range);
return reply.status(200).send({ success: true, data: dashboard, meta: null });
}
async getProductDashboard(request: FastifyRequest, reply: FastifyReply) {
const { externalProductId } = request.params as { externalProductId: string };
const query = dateRangeQuerySchema.parse(request.query);
const range = resolveDateRange(query);
const dashboard = await this.service.getProductDashboard(externalProductId, range);
return reply.status(200).send({ success: true, data: dashboard, meta: null });
}
async getSupportDashboard(request: FastifyRequest, reply: FastifyReply) {
const query = dateRangeQuerySchema.parse(request.query);
const range = resolveDateRange(query);
const dashboard = await this.service.getSupportDashboard(range);
return reply.status(200).send({ success: true, data: dashboard, meta: null });
}
async getAiDashboard(request: FastifyRequest, reply: FastifyReply) {
const query = dateRangeQuerySchema.parse(request.query);
const range = resolveDateRange(query);
const dashboard = await this.service.getAiDashboard(range);
return reply.status(200).send({ success: true, data: dashboard, meta: null });
}
}
export const reportsController = new ReportsController();
+8 -11
View File
@@ -1,11 +1,8 @@
export const REPORTS_CONSTANTS = {
MODULE_NAME: 'PLATFORM_REPORTS',
} as const;
export class ReportsService {
async generateSummaryReport(): Promise<Record<string, unknown>> {
return {};
}
}
export const reportsService = new ReportsService();
export { reportsRoutes } from './routes';
export { ReportsService, reportsService } from './service';
export type {
ManagementDashboard,
ProductDashboard,
SupportDashboard,
AiDashboard,
} from './service';
@@ -0,0 +1,39 @@
import { ValidationError } from '@/common/errors';
import { reportingConfig } from '@/config';
export interface DateRange {
from: Date;
to: Date;
}
/**
* 015-reporting-dashboards data-model.md "Query Parameters": both ends optional — `to` defaults
* to now, `from` defaults to `to - reportingConfig.defaultWindowDays`. `from > to` is a
* ValidationError (spec.md Edge Cases), never silently swapped or silently returning empty data.
*/
export function resolveDateRange(query: {
from?: string | undefined;
to?: string | undefined;
}): DateRange {
const to = query.to ? new Date(query.to) : new Date();
if (Number.isNaN(to.getTime())) {
throw new ValidationError('"to" is not a valid date.');
}
const from = query.from
? new Date(query.from)
: new Date(to.getTime() - reportingConfig.defaultWindowDays * 24 * 60 * 60 * 1000);
if (Number.isNaN(from.getTime())) {
throw new ValidationError('"from" is not a valid date.');
}
if (from > to) {
throw new ValidationError('"from" must not be after "to".');
}
return { from, to };
}
export function serializeDateRange(range: DateRange): { from: string; to: string } {
return { from: range.from.toISOString(), to: range.to.toISOString() };
}
@@ -0,0 +1,14 @@
interface TicketWithFirstAgentMessage {
createdAt: Date;
messages: Array<{ createdAt: Date }>;
}
/** Shared by ManagementRepository and SupportRepository — both need "ticket createdAt -> its
* first AGENT_MESSAGE createdAt" in milliseconds, for tickets that actually have one. */
export function extractFirstResponseDurationsMs(tickets: TicketWithFirstAgentMessage[]): number[] {
return tickets.flatMap((t) => {
const firstAgentMessage = t.messages[0];
if (!firstAgentMessage) return [];
return [firstAgentMessage.createdAt.getTime() - t.createdAt.getTime()];
});
}
@@ -0,0 +1,3 @@
export * from './date-range';
export * from './rate';
export * from './durations';
@@ -0,0 +1,16 @@
/**
* 015-reporting-dashboards research.md §3: every rate/average is `number | null` — `null` means
* "no qualifying data in range," distinguished from a genuine `0` (e.g. a real 0% AI resolution
* rate is meaningful; "nobody's data exists yet" is not the same thing). Never computed as
* `numerator / 0`, which would silently produce `NaN`.
*/
export function computeRate(numerator: number, denominator: number): number | null {
if (denominator === 0) return null;
return numerator / denominator;
}
export function computeAverageSeconds(durationsMs: number[]): number | null {
if (durationsMs.length === 0) return null;
const totalMs = durationsMs.reduce((sum, ms) => sum + ms, 0);
return totalMs / durationsMs.length / 1000;
}
@@ -0,0 +1,74 @@
import { prismaClient } from '@/infrastructure/database';
import { DateRange } from '../mapper';
export class AiRepository {
constructor(private readonly prisma = prismaClient) {}
async sessionOutcomeCounts(
range: DateRange,
): Promise<{ resolved: number; escalated: number; total: number }> {
const [resolved, escalated, total] = await Promise.all([
this.prisma.aISupportSession.count({
where: { startedAt: { gte: range.from, lte: range.to }, status: 'resolved' },
}),
this.prisma.aISupportSession.count({
where: {
startedAt: { gte: range.from, lte: range.to },
status: { in: ['escalated', 'ended_by_agent'] },
},
}),
this.prisma.aISupportSession.count({
where: { startedAt: { gte: range.from, lte: range.to } },
}),
]);
return { resolved, escalated, total };
}
/**
* "Failed troubleshooting then escalated" (spec.md User Story 4) has no single stored flag —
* classifyStepOutcome's own verdicts aren't persisted as a durable per-step record. Documented
* proxy: an escalated session that made at least one tool call (toolCallCount > 0) attempted
* troubleshooting before giving up, vs. one that escalated immediately with zero attempts.
*/
async escalatedSessionsWithToolAttempts(range: DateRange): Promise<number> {
return this.prisma.aISupportSession.count({
where: {
startedAt: { gte: range.from, lte: range.to },
status: { in: ['escalated', 'ended_by_agent'] },
toolCallCount: { gt: 0 },
},
});
}
async sessionsWithKnowledgeMatch(range: DateRange): Promise<number> {
const sessions = await this.prisma.aISupportSession.findMany({
where: { startedAt: { gte: range.from, lte: range.to } },
select: { knowledgeRefs: { select: { id: true }, take: 1 } },
});
return sessions.filter((s) => s.knowledgeRefs.length > 0).length;
}
async diagnosisConfidences(range: DateRange): Promise<number[]> {
const diagnoses = await this.prisma.aIDiagnosis.findMany({
where: { createdAt: { gte: range.from, lte: range.to } },
select: { confidence: true },
});
return diagnoses.map((d) => d.confidence);
}
async toolInvocationOutcomeCounts(
range: DateRange,
): Promise<{ success: number; failed: number }> {
const [success, failed] = await Promise.all([
this.prisma.aIActionResult.count({
where: { status: 'success', createdAt: { gte: range.from, lte: range.to } },
}),
this.prisma.aIActionResult.count({
where: { status: 'failed', createdAt: { gte: range.from, lte: range.to } },
}),
]);
return { success, failed };
}
}
export const aiRepository = new AiRepository();
@@ -0,0 +1,5 @@
export * from './management.repository';
export * from './product.repository';
export * from './support.repository';
export * from './ai.repository';
export * from './shared.repository';
@@ -0,0 +1,76 @@
import { prismaClient } from '@/infrastructure/database';
import { DateRange } from '../mapper';
export class ManagementRepository {
constructor(private readonly prisma = prismaClient) {}
async totalCases(range: DateRange): Promise<number> {
return this.prisma.ticket.count({
where: { createdAt: { gte: range.from, lte: range.to } },
});
}
async countByStatus(range: DateRange, statuses: string[]): Promise<number> {
return this.prisma.ticket.count({
where: { createdAt: { gte: range.from, lte: range.to }, status: { in: statuses } },
});
}
/** Ever reached HUMAN_ESCALATION — the state machine (003-ticketing) makes this a one-way
* gate, so a ticket currently past it (IN_PROGRESS, WAITING_FOR_CUSTOMER, etc.) still counts. */
async countEverEscalatedToHuman(range: DateRange): Promise<number> {
return this.prisma.ticket.count({
where: {
createdAt: { gte: range.from, lte: range.to },
status: {
in: [
'HUMAN_ESCALATION',
'IN_PROGRESS',
'WAITING_FOR_CUSTOMER',
'RESOLUTION_PENDING_CUSTOMER',
'RESOLVED',
'CLOSED',
'REOPENED',
],
},
},
});
}
/** research.md §4: `Resolution.resolvedBy` is the single source of truth for AI vs. human. */
async countResolutionsBy(range: DateRange, resolvedByAi: boolean): Promise<number> {
return this.prisma.resolution.count({
where: {
resolvedAt: { gte: range.from, lte: range.to },
resolvedBy: resolvedByAi ? 'ai' : { not: 'ai' },
},
});
}
async slaOutcomeCounts(range: DateRange): Promise<{ met: number; breached: number }> {
const [met, breached] = await Promise.all([
this.prisma.sLARun.count({
where: {
ticket: { createdAt: { gte: range.from, lte: range.to } },
status: 'completed',
breachedAt: null,
},
}),
this.prisma.sLARun.count({
where: {
ticket: { createdAt: { gte: range.from, lte: range.to } },
breachedAt: { not: null },
},
}),
]);
return { met, breached };
}
async escalationCount(range: DateRange): Promise<number> {
return this.prisma.escalationEvent.count({
where: { createdAt: { gte: range.from, lte: range.to } },
});
}
}
export const managementRepository = new ManagementRepository();
@@ -0,0 +1,66 @@
import { prismaClient } from '@/infrastructure/database';
import { DateRange } from '../mapper';
// Named ProductReportRepository (not ProductRepository) to avoid colliding with
// catalog/products' own ProductsRepository, which this module reuses (via its public index) for
// resolving externalProductId -> Product rather than duplicating that lookup here.
export class ProductReportRepository {
constructor(private readonly prisma = prismaClient) {}
async supportVolume(productId: string, range: DateRange): Promise<number> {
return this.prisma.ticket.count({
where: { productId, createdAt: { gte: range.from, lte: range.to } },
});
}
async problemsByCategory(
productId: string,
range: DateRange,
): Promise<Array<{ categoryId: string | null; count: number }>> {
const grouped = await this.prisma.problem.groupBy({
by: ['categoryId'],
where: { productId, createdAt: { gte: range.from, lte: range.to } },
_count: { categoryId: true },
orderBy: { _count: { categoryId: 'desc' } },
});
return grouped.map((g) => ({ categoryId: g.categoryId, count: g._count.categoryId }));
}
async countResolutionsBy(
productId: string,
range: DateRange,
resolvedByAi: boolean,
): Promise<number> {
return this.prisma.resolution.count({
where: {
resolvedAt: { gte: range.from, lte: range.to },
resolvedBy: resolvedByAi ? 'ai' : { not: 'ai' },
ticket: { productId },
},
});
}
/** Same "ever reached HUMAN_ESCALATION" one-way-gate logic as
* ManagementRepository.countEverEscalatedToHuman, scoped to one product. */
async countEverEscalatedToHuman(productId: string, range: DateRange): Promise<number> {
return this.prisma.ticket.count({
where: {
productId,
createdAt: { gte: range.from, lte: range.to },
status: {
in: [
'HUMAN_ESCALATION',
'IN_PROGRESS',
'WAITING_FOR_CUSTOMER',
'RESOLUTION_PENDING_CUSTOMER',
'RESOLVED',
'CLOSED',
'REOPENED',
],
},
},
});
}
}
export const productReportRepository = new ProductReportRepository();
@@ -0,0 +1,34 @@
import { prismaClient } from '@/infrastructure/database';
import { DateRange, extractFirstResponseDurationsMs } from '../mapper';
/** Response/resolution duration queries the Management and Support dashboards both need
* identically — composed by each, not duplicated. */
export class SharedReportRepository {
constructor(private readonly prisma = prismaClient) {}
async firstResponseDurationsMs(range: DateRange): Promise<number[]> {
const tickets = await this.prisma.ticket.findMany({
where: { createdAt: { gte: range.from, lte: range.to } },
select: {
createdAt: true,
messages: {
where: { type: 'AGENT_MESSAGE' },
orderBy: { createdAt: 'asc' },
take: 1,
select: { createdAt: true },
},
},
});
return extractFirstResponseDurationsMs(tickets);
}
async resolutionDurationsMs(range: DateRange): Promise<number[]> {
const resolutions = await this.prisma.resolution.findMany({
where: { resolvedAt: { gte: range.from, lte: range.to } },
select: { resolvedAt: true, ticket: { select: { createdAt: true } } },
});
return resolutions.map((r) => r.resolvedAt.getTime() - r.ticket.createdAt.getTime());
}
}
export const sharedReportRepository = new SharedReportRepository();
@@ -0,0 +1,40 @@
import { prismaClient } from '@/infrastructure/database';
import { DateRange } from '../mapper';
export class SupportRepository {
constructor(private readonly prisma = prismaClient) {}
/** research.md §2: current, point-in-time — not range-scoped. "How much work is assigned
* right now," not a historical count. */
async workloadByAgent(): Promise<Array<{ agentId: string; openAssignments: number }>> {
const grouped = await this.prisma.assignment.groupBy({
by: ['agentId'],
where: { isCurrent: true },
_count: { agentId: true },
});
return grouped.map((g) => ({ agentId: g.agentId, openAssignments: g._count.agentId }));
}
async slaAtRisk(thresholdMinutes: number): Promise<number> {
const now = new Date();
const riskCutoff = new Date(now.getTime() + thresholdMinutes * 60 * 1000);
return this.prisma.sLARun.count({
where: {
status: 'running',
resolutionDueAt: { gte: now, lte: riskCutoff },
},
});
}
async slaBreached(): Promise<number> {
return this.prisma.sLARun.count({ where: { status: 'breached' } });
}
async escalationCount(range: DateRange): Promise<number> {
return this.prisma.escalationEvent.count({
where: { createdAt: { gte: range.from, lte: range.to } },
});
}
}
export const supportRepository = new SupportRepository();
@@ -0,0 +1 @@
export * from './reports.routes';
@@ -0,0 +1,28 @@
import { FastifyInstance } from 'fastify';
import { requireRole } from '@/modules/identity/auth';
import { reportsController } from '../controller';
/** contracts/reports-api-contract.md: every dashboard is admin-only, the same gate every other
* admin-only surface uses since 010-identity-auth. */
export async function reportsRoutes(fastify: FastifyInstance): Promise<void> {
fastify.get(
'/admin/reports/management',
{ preHandler: [fastify.authenticate, requireRole('ADMIN')] },
(req, reply) => reportsController.getManagementDashboard(req, reply),
);
fastify.get(
'/admin/reports/product/:externalProductId',
{ preHandler: [fastify.authenticate, requireRole('ADMIN')] },
(req, reply) => reportsController.getProductDashboard(req, reply),
);
fastify.get(
'/admin/reports/support',
{ preHandler: [fastify.authenticate, requireRole('ADMIN')] },
(req, reply) => reportsController.getSupportDashboard(req, reply),
);
fastify.get(
'/admin/reports/ai',
{ preHandler: [fastify.authenticate, requireRole('ADMIN')] },
(req, reply) => reportsController.getAiDashboard(req, reply),
);
}
@@ -0,0 +1 @@
export * from './reports.schema';
@@ -0,0 +1,10 @@
import { z } from 'zod';
export const dateRangeQuerySchema = z
.object({
from: z.string().optional(),
to: z.string().optional(),
})
.strict();
export type DateRangeQuery = z.infer<typeof dateRangeQuerySchema>;
@@ -0,0 +1 @@
export * from './reports.service';
@@ -0,0 +1,219 @@
import { NotFoundError } from '@/common/errors';
import { reportingConfig, aiConfig } from '@/config';
import { productsRepository, ProductsRepository } from '@/modules/catalog/products';
import { decideConfidenceBand } from '@/modules/ai-support/sessions';
import { errorCodesService, ErrorCodesService } from '@/modules/ai-support/knowledge';
import {
managementRepository,
ManagementRepository,
productReportRepository,
ProductReportRepository,
supportRepository,
SupportRepository,
aiRepository,
AiRepository,
sharedReportRepository,
SharedReportRepository,
} from '../repository';
import { DateRange, serializeDateRange, computeRate, computeAverageSeconds } from '../mapper';
export interface ManagementDashboard {
range: { from: string; to: string };
totalCases: number;
aiResolved: number;
humanEscalated: number;
resolved: number;
open: number;
slaCompliance: { met: number; breached: number; rate: number | null };
escalationCount: number;
averageResponseSeconds: number | null;
averageResolutionSeconds: number | null;
}
export interface ProductDashboard {
productId: string;
range: { from: string; to: string };
supportVolume: number;
problemsByCategory: Array<{ categoryId: string | null; count: number }>;
recurringProblems: Array<{ categoryId: string | null; count: number }>;
aiResolutionRate: number | null;
humanEscalationRate: number | null;
topErrors: Array<{ code: string; count: number }>;
}
export interface SupportDashboard {
generatedAt: string;
range: { from: string; to: string };
workloadByAgent: Array<{ agentId: string; openAssignments: number }>;
slaAtRisk: number;
slaBreached: number;
escalationCount: number;
averageResponseSeconds: number | null;
averageResolutionSeconds: number | null;
}
export interface AiDashboard {
range: { from: string; to: string };
totalSessions: number;
aiResolutionRate: number | null;
humanHandoffRate: number | null;
failedTroubleshootingEscalationRate: number | null;
knowledgeMatchRate: number | null;
confidenceDistribution: { proceed: number; ask: number; escalate: number };
toolInvocations: { success: number; failed: number };
}
const RESOLVED_STATUSES = ['RESOLVED', 'CLOSED'];
export class ReportsService {
constructor(
private readonly management: ManagementRepository = managementRepository,
private readonly productReports: ProductReportRepository = productReportRepository,
private readonly support: SupportRepository = supportRepository,
private readonly ai: AiRepository = aiRepository,
private readonly products: ProductsRepository = productsRepository,
private readonly errorCodes: ErrorCodesService = errorCodesService,
private readonly shared: SharedReportRepository = sharedReportRepository,
) {}
async getManagementDashboard(range: DateRange): Promise<ManagementDashboard> {
const [
totalCases,
aiResolved,
humanEscalated,
resolved,
slaOutcomes,
escalationCount,
responseDurations,
resolutionDurations,
] = await Promise.all([
this.management.totalCases(range),
this.management.countResolutionsBy(range, true),
this.management.countEverEscalatedToHuman(range),
this.management.countByStatus(range, RESOLVED_STATUSES),
this.management.slaOutcomeCounts(range),
this.management.escalationCount(range),
this.shared.firstResponseDurationsMs(range),
this.shared.resolutionDurationsMs(range),
]);
return {
range: serializeDateRange(range),
totalCases,
aiResolved,
humanEscalated,
resolved,
open: totalCases - resolved,
slaCompliance: {
met: slaOutcomes.met,
breached: slaOutcomes.breached,
rate: computeRate(slaOutcomes.met, slaOutcomes.met + slaOutcomes.breached),
},
escalationCount,
averageResponseSeconds: computeAverageSeconds(responseDurations),
averageResolutionSeconds: computeAverageSeconds(resolutionDurations),
};
}
async getProductDashboard(
externalProductId: string,
range: DateRange,
): Promise<ProductDashboard> {
const product = await this.products.findByExternalProductId(externalProductId);
if (!product) throw new NotFoundError('Product not found.');
const [supportVolume, problemsByCategory, aiResolvedCount, humanEscalatedCount, topErrors] =
await Promise.all([
this.productReports.supportVolume(product.id, range),
this.productReports.problemsByCategory(product.id, range),
this.productReports.countResolutionsBy(product.id, range, true),
this.productReports.countEverEscalatedToHuman(product.id, range),
this.errorCodes.getTopErrorCodesForProduct(
product.id,
range.from,
range.to,
reportingConfig.topNLimit,
),
]);
const recurringProblems = [...problemsByCategory]
.sort((a, b) => b.count - a.count)
.slice(0, reportingConfig.topNLimit);
return {
productId: externalProductId,
range: serializeDateRange(range),
supportVolume,
problemsByCategory,
recurringProblems,
aiResolutionRate: computeRate(aiResolvedCount, supportVolume),
humanEscalationRate: computeRate(humanEscalatedCount, supportVolume),
topErrors,
};
}
async getSupportDashboard(range: DateRange): Promise<SupportDashboard> {
const [
workloadByAgent,
slaAtRisk,
slaBreached,
escalationCount,
responseDurations,
resolutionDurations,
] = await Promise.all([
this.support.workloadByAgent(),
this.support.slaAtRisk(reportingConfig.slaRiskThresholdMinutes),
this.support.slaBreached(),
this.support.escalationCount(range),
this.shared.firstResponseDurationsMs(range),
this.shared.resolutionDurationsMs(range),
]);
return {
generatedAt: new Date().toISOString(),
range: serializeDateRange(range),
workloadByAgent,
slaAtRisk,
slaBreached,
escalationCount,
averageResponseSeconds: computeAverageSeconds(responseDurations),
averageResolutionSeconds: computeAverageSeconds(resolutionDurations),
};
}
async getAiDashboard(range: DateRange): Promise<AiDashboard> {
const [outcomeCounts, escalatedWithAttempts, knowledgeMatches, confidences, toolCounts] =
await Promise.all([
this.ai.sessionOutcomeCounts(range),
this.ai.escalatedSessionsWithToolAttempts(range),
this.ai.sessionsWithKnowledgeMatch(range),
this.ai.diagnosisConfidences(range),
this.ai.toolInvocationOutcomeCounts(range),
]);
const confidenceDistribution = { proceed: 0, ask: 0, escalate: 0 };
for (const confidence of confidences) {
const band = decideConfidenceBand(confidence, {
highThreshold: aiConfig.defaultHighConfidence,
lowThreshold: aiConfig.defaultLowConfidence,
});
confidenceDistribution[band] += 1;
}
return {
range: serializeDateRange(range),
totalSessions: outcomeCounts.total,
aiResolutionRate: computeRate(outcomeCounts.resolved, outcomeCounts.total),
humanHandoffRate: computeRate(outcomeCounts.escalated, outcomeCounts.total),
failedTroubleshootingEscalationRate: computeRate(
escalatedWithAttempts,
outcomeCounts.escalated,
),
knowledgeMatchRate: computeRate(knowledgeMatches, outcomeCounts.total),
confidenceDistribution,
toolInvocations: toolCounts,
};
}
}
export const reportsService = new ReportsService();
+3
View File
@@ -21,6 +21,9 @@ describe('Error codes and known issues', () => {
afterAll(async () => {
await prismaClient.knownIssue.deleteMany({ where: { product: { externalProductId } } });
// 015-reporting-dashboards: findKnownIssuesByErrorCode now also writes a durable
// ErrorCodeLookup row (RESTRICT FK to ErrorCode) — must be deleted before ErrorCode itself.
await prismaClient.errorCodeLookup.deleteMany({ where: { product: { externalProductId } } });
await prismaClient.errorCode.deleteMany({ where: { product: { externalProductId } } });
await prismaClient.product.deleteMany({ where: { externalProductId } });
await app.close();
@@ -0,0 +1,156 @@
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
import { buildApp } from '@/app';
import { prismaClient } from '@/infrastructure/database';
import { FastifyInstance } from 'fastify';
import { loginAs, authHeader } from '../../helpers/auth';
import {
encryptCredential,
generateCredentialSecret,
issueIntegrationToken,
} from '@/modules/catalog/products';
import {
sessionRepository,
diagnosisRepository,
knowledgeReferenceRepository,
} from '@/modules/ai-support/sessions';
import { actionRepository } from '@/modules/ai-support/tools';
import { aiConfig } from '@/config';
/** Covers specs/015-reporting-dashboards/quickstart.md Scenario 4 against a real Postgres/Redis. */
describe('AI dashboard (User Story 4)', () => {
let app: FastifyInstance;
let authToken: string;
const externalProductId = `TEST_AI_REPORT_PROD_${Date.now()}`;
let secret: string;
const rangeFrom = new Date(Date.now() - 60 * 60 * 1000).toISOString();
const rangeTo = new Date(Date.now() + 60 * 60 * 1000).toISOString();
beforeAll(async () => {
app = await buildApp();
authToken = await loginAs(app, 'ADMIN');
const product = await prismaClient.product.create({
data: { externalProductId, name: 'AI Report Test Product', status: 'active' },
});
secret = generateCredentialSecret();
await prismaClient.productIntegration.create({
data: {
productId: product.id,
credentialRef: encryptCredential(secret),
authMechanism: 'signed_token',
allowedScope: { tenantIds: ['tenant-1'] },
status: 'active',
rateLimitPerMinute: 1000,
rateLimitPerUserPerMinute: 1000,
},
});
});
afterAll(async () => {
await app.close();
});
async function createTicket(): Promise<string> {
const token = issueIntegrationToken(secret, {
externalProductId,
tenantId: 'tenant-1',
userId: 'user-1',
});
const created = await app.inject({
method: 'POST',
url: '/v1/support/requests',
headers: { authorization: `Bearer ${token}` },
payload: {
productId: externalProductId,
tenantId: 'tenant-1',
userId: 'user-1',
source: 'test',
problem: `AI report test ${Date.now()}-${Math.random()}`,
},
});
expect(created.statusCode).toBe(202);
return created.json().data.ticketId as string;
}
it('reflects real session outcomes, tool results, and confidence bands', async () => {
// A resolved session, with a knowledge match and a successful tool call.
const resolvedTicketId = await createTicket();
const resolvedSession = await sessionRepository.create(resolvedTicketId);
await knowledgeReferenceRepository.recordMany(resolvedSession.id, ['fake-knowledge-id']);
await diagnosisRepository.create({
sessionId: resolvedSession.id,
product: 'test-product',
problemType: 'test-problem',
severity: 'medium',
confidence: aiConfig.defaultHighConfidence,
possibleCauses: ['test cause'],
});
const successAction = await actionRepository.create({
sessionId: resolvedSession.id,
toolName: 'getTicketSnapshot',
input: {},
riskLevel: 'low',
evaluationOutcome: 'approved',
});
await actionRepository.createResult(successAction.id, { ok: true }, 'success');
await sessionRepository.updateStatus(resolvedSession.id, 'resolved');
// An escalated session, with a failed tool call and a low-confidence diagnosis.
const escalatedTicketId = await createTicket();
const escalatedSession = await sessionRepository.create(escalatedTicketId);
await diagnosisRepository.create({
sessionId: escalatedSession.id,
product: 'test-product',
problemType: 'test-problem',
severity: 'high',
confidence: aiConfig.defaultLowConfidence - 0.05,
possibleCauses: ['test cause'],
});
const failedAction = await actionRepository.create({
sessionId: escalatedSession.id,
toolName: 'getTicketSnapshot',
input: {},
riskLevel: 'low',
evaluationOutcome: 'approved',
});
await actionRepository.createResult(failedAction.id, { error: 'boom' }, 'failed');
await sessionRepository.updateStatus(escalatedSession.id, 'escalated');
const res = await app.inject({
method: 'GET',
url: `/admin/reports/ai?from=${rangeFrom}&to=${rangeTo}`,
headers: authHeader(authToken),
});
expect(res.statusCode).toBe(200);
const data = res.json().data;
expect(data.totalSessions).toBeGreaterThanOrEqual(2);
expect(data.aiResolutionRate).not.toBeNull();
expect(data.humanHandoffRate).not.toBeNull();
expect(data.knowledgeMatchRate).not.toBeNull();
expect(data.confidenceDistribution.proceed).toBeGreaterThanOrEqual(1);
expect(data.confidenceDistribution.escalate).toBeGreaterThanOrEqual(1);
expect(data.toolInvocations.success).toBeGreaterThanOrEqual(1);
expect(data.toolInvocations.failed).toBeGreaterThanOrEqual(1);
});
it('returns null rates and zero counts for a range with no AI activity', async () => {
const farPastFrom = new Date('2000-01-01').toISOString();
const farPastTo = new Date('2000-01-02').toISOString();
const res = await app.inject({
method: 'GET',
url: `/admin/reports/ai?from=${farPastFrom}&to=${farPastTo}`,
headers: authHeader(authToken),
});
expect(res.statusCode).toBe(200);
const data = res.json().data;
expect(data.totalSessions).toBe(0);
expect(data.aiResolutionRate).toBeNull();
expect(data.humanHandoffRate).toBeNull();
expect(data.knowledgeMatchRate).toBeNull();
expect(data.confidenceDistribution).toEqual({ proceed: 0, ask: 0, escalate: 0 });
expect(data.toolInvocations).toEqual({ success: 0, failed: 0 });
});
});
@@ -0,0 +1,217 @@
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
import { buildApp } from '@/app';
import { prismaClient } from '@/infrastructure/database';
import { FastifyInstance } from 'fastify';
import { loginAs, authHeader } from '../../helpers/auth';
import {
encryptCredential,
generateCredentialSecret,
issueIntegrationToken,
} from '@/modules/catalog/products';
import { ticketsRepository } from '@/modules/ticketing/tickets';
import { messagesService } from '@/modules/ticketing/messages';
import { resolutionRepository } from '@/modules/problem-management/resolutions';
/**
* Covers specs/015-reporting-dashboards/quickstart.md Scenario 1 against a real Postgres/Redis.
* Drives ticket-status transitions directly through ticketsRepository (not ticketsService) to
* avoid publishing TICKET_UPDATED — this test only needs the raw persisted state its own
* aggregation queries read, and publishing real domain events here risks the same kind of
* cross-file contamination 014-full-observability's own business-metrics.test.ts found and fixed
* (an unscoped HUMAN_ESCALATION triggering real auto-assignment against the shared agent pool).
*/
describe('Management dashboard (User Story 1)', () => {
let app: FastifyInstance;
let authToken: string;
const externalProductId = `TEST_MGMT_REPORT_PROD_${Date.now()}`;
let productId: string;
let secret: string;
const rangeFrom = new Date(Date.now() - 60 * 60 * 1000).toISOString();
const rangeTo = new Date(Date.now() + 60 * 60 * 1000).toISOString();
beforeAll(async () => {
app = await buildApp();
authToken = await loginAs(app, 'ADMIN');
const product = await prismaClient.product.create({
data: { externalProductId, name: 'Management Report Test Product', status: 'active' },
});
productId = product.id;
secret = generateCredentialSecret();
await prismaClient.productIntegration.create({
data: {
productId,
credentialRef: encryptCredential(secret),
authMechanism: 'signed_token',
allowedScope: { tenantIds: ['tenant-1'] },
status: 'active',
rateLimitPerMinute: 1000,
rateLimitPerUserPerMinute: 1000,
},
});
});
afterAll(async () => {
await app.close();
});
async function createTicket(): Promise<string> {
const token = issueIntegrationToken(secret, {
externalProductId,
tenantId: 'tenant-1',
userId: 'user-1',
});
const created = await app.inject({
method: 'POST',
url: '/v1/support/requests',
headers: { authorization: `Bearer ${token}` },
payload: {
productId: externalProductId,
tenantId: 'tenant-1',
userId: 'user-1',
source: 'test',
problem: `Management report test ${Date.now()}-${Math.random()}`,
},
});
expect(created.statusCode).toBe(202);
return created.json().data.ticketId as string;
}
async function driveDirectly(ticketId: string, statuses: string[]): Promise<void> {
let ticket = await prismaClient.ticket.findUniqueOrThrow({ where: { id: ticketId } });
for (const status of statuses) {
const updated = await ticketsRepository.updateStatus(ticketId, status, ticket.version);
if (!updated) throw new Error(`Failed to transition ticket to ${status} in test setup.`);
ticket = updated;
}
}
it('reports real figures matching the actual created data', async () => {
// AI-resolved ticket.
const aiTicketId = await createTicket();
await driveDirectly(aiTicketId, [
'AI_ANALYZING',
'AI_TROUBLESHOOTING',
'AI_VERIFYING',
'AI_RESOLVED',
]);
await resolutionRepository.create({ ticketId: aiTicketId, outcome: 'fixed', resolvedBy: 'ai' });
await driveDirectly(aiTicketId, ['RESOLVED']);
// Human-resolved ticket, with a first agent response recorded.
const humanTicketId = await createTicket();
await messagesService.post(humanTicketId, 'agent-1', 'AGENT_MESSAGE', 'Looking into this.');
await driveDirectly(humanTicketId, [
'HUMAN_ESCALATION',
'IN_PROGRESS',
'RESOLUTION_PENDING_CUSTOMER',
]);
await resolutionRepository.create({
ticketId: humanTicketId,
outcome: 'fixed',
resolvedBy: 'agent-1',
});
await driveDirectly(humanTicketId, ['RESOLVED']);
// Still-open ticket.
await createTicket();
// SLA policy + one met, one breached run.
const policy = await prismaClient.sLAPolicy.create({
data: {
name: `Mgmt Report Policy ${Date.now()}`,
productId,
firstResponseMinutes: 30,
resolutionMinutes: 240,
},
});
const metTicketId = await createTicket();
await prismaClient.sLARun.create({
data: {
ticketId: metTicketId,
policyId: policy.id,
firstResponseDueAt: new Date(Date.now() + 30 * 60_000),
resolutionDueAt: new Date(Date.now() + 240 * 60_000),
status: 'completed',
completedAt: new Date(),
},
});
const breachedTicketId = await createTicket();
await prismaClient.sLARun.create({
data: {
ticketId: breachedTicketId,
policyId: policy.id,
firstResponseDueAt: new Date(Date.now() + 30 * 60_000),
resolutionDueAt: new Date(Date.now() - 60_000),
status: 'breached',
breachedAt: new Date(),
},
});
// One escalation event.
const escalatedTicketId = await createTicket();
await prismaClient.escalationEvent.create({
data: {
ticketId: escalatedTicketId,
ruleId: null,
fromNodeId: null,
toNodeId: null,
reason: 'management dashboard test',
triggeredBy: 'system',
},
});
const res = await app.inject({
method: 'GET',
url: `/admin/reports/management?from=${rangeFrom}&to=${rangeTo}`,
headers: authHeader(authToken),
});
expect(res.statusCode).toBe(200);
const data = res.json().data;
expect(data.totalCases).toBeGreaterThanOrEqual(6);
expect(data.aiResolved).toBeGreaterThanOrEqual(1);
expect(data.humanEscalated).toBeGreaterThanOrEqual(1);
expect(data.resolved).toBeGreaterThanOrEqual(2);
expect(data.open).toBeGreaterThanOrEqual(1);
expect(data.slaCompliance.met).toBeGreaterThanOrEqual(1);
expect(data.slaCompliance.breached).toBeGreaterThanOrEqual(1);
expect(data.slaCompliance.rate).not.toBeNull();
expect(data.escalationCount).toBeGreaterThanOrEqual(1);
expect(data.averageResponseSeconds).not.toBeNull();
expect(data.averageResolutionSeconds).not.toBeNull();
expect(data.range.from).toBeTruthy();
expect(data.range.to).toBeTruthy();
});
it('returns all-zero counts and all-null rates for a range with no activity', async () => {
const farPastFrom = new Date('2000-01-01').toISOString();
const farPastTo = new Date('2000-01-02').toISOString();
const res = await app.inject({
method: 'GET',
url: `/admin/reports/management?from=${farPastFrom}&to=${farPastTo}`,
headers: authHeader(authToken),
});
expect(res.statusCode).toBe(200);
const data = res.json().data;
expect(data.totalCases).toBe(0);
expect(data.aiResolved).toBe(0);
expect(data.humanEscalated).toBe(0);
expect(data.resolved).toBe(0);
expect(data.open).toBe(0);
expect(data.slaCompliance.rate).toBeNull();
expect(data.averageResponseSeconds).toBeNull();
expect(data.averageResolutionSeconds).toBeNull();
});
it('rejects a range where from is after to', async () => {
const res = await app.inject({
method: 'GET',
url: `/admin/reports/management?from=${rangeTo}&to=${rangeFrom}`,
headers: authHeader(authToken),
});
expect(res.statusCode).toBe(400);
});
});
@@ -0,0 +1,127 @@
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
import { buildApp } from '@/app';
import { prismaClient } from '@/infrastructure/database';
import { FastifyInstance } from 'fastify';
import { loginAs, authHeader } from '../../helpers/auth';
import {
encryptCredential,
generateCredentialSecret,
issueIntegrationToken,
} from '@/modules/catalog/products';
import { errorCodesService } from '@/modules/ai-support/knowledge';
/** Covers specs/015-reporting-dashboards/quickstart.md Scenario 2 against a real Postgres/Redis. */
describe('Product dashboard (User Story 2)', () => {
let app: FastifyInstance;
let authToken: string;
const rangeFrom = new Date(Date.now() - 60 * 60 * 1000).toISOString();
const rangeTo = new Date(Date.now() + 60 * 60 * 1000).toISOString();
async function setUpProduct(nameSuffix: string) {
const externalProductId = `TEST_PRODUCT_REPORT_${nameSuffix}_${Date.now()}`;
const product = await prismaClient.product.create({
data: { externalProductId, name: `Product Report ${nameSuffix}`, status: 'active' },
});
const secret = generateCredentialSecret();
await prismaClient.productIntegration.create({
data: {
productId: product.id,
credentialRef: encryptCredential(secret),
authMechanism: 'signed_token',
allowedScope: { tenantIds: ['tenant-1'] },
status: 'active',
rateLimitPerMinute: 1000,
rateLimitPerUserPerMinute: 1000,
},
});
return { externalProductId, productId: product.id, secret };
}
async function createTicket(externalProductId: string, secret: string): Promise<string> {
const token = issueIntegrationToken(secret, {
externalProductId,
tenantId: 'tenant-1',
userId: 'user-1',
});
const created = await app.inject({
method: 'POST',
url: '/v1/support/requests',
headers: { authorization: `Bearer ${token}` },
payload: {
productId: externalProductId,
tenantId: 'tenant-1',
userId: 'user-1',
source: 'test',
problem: `Product report test ${Date.now()}-${Math.random()}`,
},
});
expect(created.statusCode).toBe(202);
return created.json().data.ticketId as string;
}
beforeAll(async () => {
app = await buildApp();
authToken = await loginAs(app, 'ADMIN');
});
afterAll(async () => {
await app.close();
});
it("scopes every figure to the requested product, never another product's data", async () => {
const productA = await setUpProduct('A');
const productB = await setUpProduct('B');
await createTicket(productA.externalProductId, productA.secret);
await createTicket(productA.externalProductId, productA.secret);
await createTicket(productB.externalProductId, productB.secret);
const resA = await app.inject({
method: 'GET',
url: `/admin/reports/product/${productA.externalProductId}?from=${rangeFrom}&to=${rangeTo}`,
headers: authHeader(authToken),
});
expect(resA.statusCode).toBe(200);
expect(resA.json().data.supportVolume).toBe(2);
const resB = await app.inject({
method: 'GET',
url: `/admin/reports/product/${productB.externalProductId}?from=${rangeFrom}&to=${rangeTo}`,
headers: authHeader(authToken),
});
expect(resB.statusCode).toBe(200);
expect(resB.json().data.supportVolume).toBe(1);
});
it('ranks the most-frequently-looked-up error code first', async () => {
const product = await setUpProduct('ERR');
const popularCode = `POPULAR-${Date.now()}`;
const rareCode = `RARE-${Date.now()}`;
await errorCodesService.createErrorCode(product.productId, popularCode, 'Popular error');
await errorCodesService.createErrorCode(product.productId, rareCode, 'Rare error');
await errorCodesService.findKnownIssuesByErrorCode(product.productId, popularCode);
await errorCodesService.findKnownIssuesByErrorCode(product.productId, popularCode);
await errorCodesService.findKnownIssuesByErrorCode(product.productId, popularCode);
await errorCodesService.findKnownIssuesByErrorCode(product.productId, rareCode);
const res = await app.inject({
method: 'GET',
url: `/admin/reports/product/${product.externalProductId}?from=${rangeFrom}&to=${rangeTo}`,
headers: authHeader(authToken),
});
expect(res.statusCode).toBe(200);
const topErrors = res.json().data.topErrors as Array<{ code: string; count: number }>;
expect(topErrors[0]).toMatchObject({ code: popularCode, count: 3 });
expect(topErrors.find((e) => e.code === rareCode)).toMatchObject({ count: 1 });
});
it('404s for an unknown product', async () => {
const res = await app.inject({
method: 'GET',
url: `/admin/reports/product/NONEXISTENT_PRODUCT_${Date.now()}`,
headers: authHeader(authToken),
});
expect(res.statusCode).toBe(404);
});
});
@@ -0,0 +1,155 @@
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
import { buildApp } from '@/app';
import { prismaClient } from '@/infrastructure/database';
import { FastifyInstance } from 'fastify';
import { loginAs, authHeader } from '../../helpers/auth';
import {
encryptCredential,
generateCredentialSecret,
issueIntegrationToken,
} from '@/modules/catalog/products';
import { reportingConfig } from '@/config';
/**
* Covers specs/015-reporting-dashboards/quickstart.md Scenario 3 against a real Postgres/Redis.
* Assignment rows are created directly via Prisma (not through a real HUMAN_ESCALATION +
* default-strategy auto-assignment) — the same contamination avoidance
* management-dashboard.test.ts already documents: this test only needs the persisted
* Assignment/SLARun state its own aggregation queries read, not a live orchestration run.
*/
describe('Support dashboard (User Story 3)', () => {
let app: FastifyInstance;
let authToken: string;
const externalProductId = `TEST_SUPPORT_REPORT_PROD_${Date.now()}`;
let productId: string;
let secret: string;
beforeAll(async () => {
app = await buildApp();
authToken = await loginAs(app, 'ADMIN');
const product = await prismaClient.product.create({
data: { externalProductId, name: 'Support Report Test Product', status: 'active' },
});
productId = product.id;
secret = generateCredentialSecret();
await prismaClient.productIntegration.create({
data: {
productId,
credentialRef: encryptCredential(secret),
authMechanism: 'signed_token',
allowedScope: { tenantIds: ['tenant-1'] },
status: 'active',
rateLimitPerMinute: 1000,
rateLimitPerUserPerMinute: 1000,
},
});
});
afterAll(async () => {
await app.close();
});
async function createTicket(): Promise<string> {
const token = issueIntegrationToken(secret, {
externalProductId,
tenantId: 'tenant-1',
userId: 'user-1',
});
const created = await app.inject({
method: 'POST',
url: '/v1/support/requests',
headers: { authorization: `Bearer ${token}` },
payload: {
productId: externalProductId,
tenantId: 'tenant-1',
userId: 'user-1',
source: 'test',
problem: `Support report test ${Date.now()}-${Math.random()}`,
},
});
expect(created.statusCode).toBe(202);
return created.json().data.ticketId as string;
}
it("reflects each agent's real current assignment workload", async () => {
const team = await app.inject({
method: 'POST',
url: '/admin/teams',
headers: authHeader(authToken),
payload: { name: `Support Report Team ${Date.now()}` },
});
const teamId = team.json().data.id as string;
const agent = await app.inject({
method: 'POST',
url: `/admin/teams/${teamId}/agents`,
headers: authHeader(authToken),
payload: { name: 'Support Report Agent' },
});
const agentId = agent.json().data.id as string;
const ticket1 = await createTicket();
const ticket2 = await createTicket();
await prismaClient.assignment.createMany({
data: [
{ ticketId: ticket1, agentId, strategy: 'MANUAL', isCurrent: true },
{ ticketId: ticket2, agentId, strategy: 'MANUAL', isCurrent: true },
],
});
const res = await app.inject({
method: 'GET',
url: '/admin/reports/support',
headers: authHeader(authToken),
});
expect(res.statusCode).toBe(200);
const workload = res.json().data.workloadByAgent as Array<{
agentId: string;
openAssignments: number;
}>;
expect(workload.find((w) => w.agentId === agentId)).toMatchObject({ openAssignments: 2 });
});
it('counts a near-due SLA run as at-risk, distinct from breached', async () => {
const policy = await prismaClient.sLAPolicy.create({
data: {
name: `Support Risk Policy ${Date.now()}`,
productId,
firstResponseMinutes: 30,
resolutionMinutes: 240,
},
});
const riskTicketId = await createTicket();
await prismaClient.sLARun.create({
data: {
ticketId: riskTicketId,
policyId: policy.id,
firstResponseDueAt: new Date(Date.now() + 30 * 60_000),
resolutionDueAt: new Date(
Date.now() + (reportingConfig.slaRiskThresholdMinutes - 1) * 60_000,
),
status: 'running',
},
});
const safeTicketId = await createTicket();
await prismaClient.sLARun.create({
data: {
ticketId: safeTicketId,
policyId: policy.id,
firstResponseDueAt: new Date(Date.now() + 30 * 60_000),
resolutionDueAt: new Date(Date.now() + 999 * 60_000),
status: 'running',
},
});
const res = await app.inject({
method: 'GET',
url: '/admin/reports/support',
headers: authHeader(authToken),
});
expect(res.statusCode).toBe(200);
expect(res.json().data.slaAtRisk).toBeGreaterThanOrEqual(1);
});
});
@@ -0,0 +1,29 @@
import { describe, it, expect } from 'vitest';
import { decideConfidenceBand } from '@/modules/ai-support/sessions';
import { aiConfig } from '@/config';
/**
* 015-reporting-dashboards research.md §7: the AI dashboard's confidence distribution reuses
* 005-ai-support's own decideConfidenceBand against the system-default thresholds, rather than
* reimplementing a threshold check — this test proves the reused function classifies values
* the way the dashboard's own bucketing loop (reports.service.ts) depends on.
*/
describe('AI dashboard confidence distribution reuses decideConfidenceBand', () => {
const policy = {
highThreshold: aiConfig.defaultHighConfidence,
lowThreshold: aiConfig.defaultLowConfidence,
};
it('classifies a high-confidence value as proceed', () => {
expect(decideConfidenceBand(policy.highThreshold, policy)).toBe('proceed');
});
it('classifies a low-confidence value as escalate', () => {
expect(decideConfidenceBand(policy.lowThreshold - 0.01, policy)).toBe('escalate');
});
it('classifies a mid-range value as ask', () => {
const midpoint = (policy.highThreshold + policy.lowThreshold) / 2;
expect(decideConfidenceBand(midpoint, policy)).toBe('ask');
});
});
@@ -0,0 +1,27 @@
import { describe, it, expect } from 'vitest';
import { computeRate, computeAverageSeconds } from '@/modules/platform/reports/mapper';
describe('computeRate (015-reporting-dashboards research.md §3)', () => {
it('returns null when the denominator is zero — never NaN, never a computed 0', () => {
expect(computeRate(0, 0)).toBeNull();
expect(computeRate(5, 0)).toBeNull();
});
it('computes a real rate when there is qualifying data', () => {
expect(computeRate(3, 12)).toBe(0.25);
});
it('returns a real 0 when the numerator is legitimately zero but the denominator is not', () => {
expect(computeRate(0, 10)).toBe(0);
});
});
describe('computeAverageSeconds', () => {
it('returns null for an empty list — no fabricated average', () => {
expect(computeAverageSeconds([])).toBeNull();
});
it('averages a list of millisecond durations into seconds', () => {
expect(computeAverageSeconds([1000, 2000, 3000])).toBe(2);
});
});