support_backend/specs/016-load-concurrency-testing/quickstart.md

# Quickstart: Load and Concurrency Testing

Manual + automated verification steps for each user story, against real Docker-provisioned
Postgres/Redis — this project's standing rule that a concurrency claim is never accepted from
code review alone.

## Prerequisites

- Throwaway test infra up: `supporthub-test-pg` (host port 5433), `supporthub-test-redis` (host
  port 6380) — the same containers `tests/concurrency/round-robin.test.ts` already uses.
- For the load tests (User Story 5) only: a real running instance of the API against the real
  dev infra (`postgres-development`/`redis-development`), reachable at
  `http://localhost:4501`, plus an ADMIN session token for the reporting endpoints.

## Scenario 1 — Assignment double-assignment race (User Story 1)

1. `REDIS_HOST=localhost REDIS_PORT=6380 npx vitest run tests/concurrency/assignment-race.test.ts`
2. The test creates one ticket, then fires >=20 concurrent `assignmentEngine.assignToSpecificNode`
   (or the equivalent orchestration entry point) calls at it against real Postgres.
3. **Expected**: the test itself queries `assignments` directly afterward and asserts exactly
   one row has `is_current = true` for that ticket — not just that one HTTP/service call
   "won." Repeat the run at least 10 times (or use the test's own internal repeat loop) to
   confirm SC-001's "zero exceptions across 10 repeated runs."
4. Before the fix (research.md §1), this test is expected to fail intermittently; after the
   fix, it must pass every time.

## Scenario 2 — SLA pause/resume/sweep race (User Story 2)

1. `REDIS_HOST=localhost REDIS_PORT=6380 npx vitest run tests/concurrency/sla-race.test.ts`
2. The test creates a ticket with an active SLA run, then fires concurrent `pause`/`resume`
   calls and a `runBreachDetectionSweep()` pass against the same run.
3. **Expected**: the run's final DB state (`status`, `pausedAt`, `resumedAt`, `breachedAt`,
   `firstResponseDueAt`, `resolutionDueAt`) is queried directly and asserted internally
   consistent — e.g. never `status: 'paused'` with `pausedAt: null`, never a `breached` run
   silently reverted to `running` by a racing `resume`. Repeat per SC-002.

## Scenario 3 — Escalation idempotency (User Story 3)

1. `REDIS_HOST=localhost REDIS_PORT=6380 npx vitest run tests/concurrency/escalation-idempotency.test.ts`
2. The test creates a ticket eligible for a specific escalation rule, then calls
   `escalationService.handleBreach` (or `fire` via its real trigger path) twice concurrently for
   the identical trigger.
3. **Expected**: exactly one `EscalationEvent` row exists afterward for that `(ticketId,
   ruleId)` pair, and exactly one `Assignment` row resulted from it (cross-checking Scenario 1's
   own guarantee). Repeat per SC-003.

## Scenario 4 — Ticket status optimistic concurrency proof (User Story 4)

1. `REDIS_HOST=localhost REDIS_PORT=6380 npx vitest run tests/concurrency/ticket-status-race.test.ts`
2. The test creates a ticket at a known status/version, then fires >=20 concurrent
   `ticketsRepository.updateStatus` calls all starting from that same version.
3. **Expected**: exactly one call returns the updated ticket; every other call returns `null`
   (stale-version signal); the ticket's final DB status matches the one call that succeeded.
   This is expected to pass on the very first run (spec.md Assumptions) — a failure here would
   mean the existing mechanism has a real gap, not that this quickstart step is wrong.

## Scenario 5 — Load/throughput baseline (User Story 5)

1. Ensure the real dev API is running (`npm run dev` against `.env.development`) and reachable.
2. `npx tsx tests/load/ticket-creation.load.ts`
3. `npx tsx tests/load/ai-support-flow.load.ts`
4. `npx tsx tests/load/admin-reporting.load.ts` (needs an ADMIN token — the script signs in
   itself using the same seeded admin credentials this project's E2E suite already uses)
5. **Expected**: each script prints a report (requests/sec, `p50`/`p90`/`p99` latency, non-2xx
   count, rate-limited count) and writes it to `tests/load/reports/`. There is no pass/fail
   assertion on the numbers themselves (FR-009, `OPEN BUSINESS DECISION`) — the check here is
   that the tooling runs cleanly end-to-end and produces a comparable, re-runnable report, not
   that any specific number is hit.
6. Run the same script twice in a row and confirm the two reports are comparable in shape
   (same fields, plausible numbers) — proving SC-005's "consistent-shape output for comparison
   across runs."

## What "done" looks like

- All four new `tests/concurrency/*.test.ts` files pass consistently (not flakily) against real
  Postgres/Redis, each proving its own user story's guarantee with a direct database assertion,
  not just an HTTP response check.
- Every race the audit found (assignment, SLA, escalation) is fixed in the actual repository
  code per data-model.md, not merely detected and left alone.
- All three `tests/load/*.load.ts` scripts run cleanly against a real running dev server and
  produce a report.
- Full existing quality gate (typecheck, lint, architecture check, full unit + integration
  suite) stays green — these fixes touch shared repositories (`Assignment`, `SLARun`,
  `EscalationEvent`) already exercised by 007-orchestration-assignment's, 008-sla-escalation's,
  012-admin-list-views's, and 015-reporting-dashboards's own existing tests.