support_backend/specs/016-load-concurrency-testing/plan.md

# Implementation Plan: Load and Concurrency Testing

**Branch**: `016-load-concurrency-testing` | **Date**: 2026-09-09 | **Spec**: [spec.md](./spec.md)

**Input**: Feature specification from `/specs/016-load-concurrency-testing/spec.md`

## Summary

Prove — with real, genuinely-concurrent requests against real Postgres/Redis, never mocked
timers — three concurrency guarantees that a prior codebase audit found are NOT currently held
(assignment double-assignment, SLA pause/resume/sweep races, escalation duplicate-event risk),
fix each real race the tests reveal with a minimal, idiomatic DB-level guard consistent with
this codebase's existing patterns, add one new concurrency test proving the existing ticket
optimistic-concurrency guarantee holds under genuine concurrency, and add repeatable
`autocannon`-based HTTP load-test tooling for the three named critical endpoint groups.

## Technical Context

**Language/Version**: TypeScript 5.4 / Node.js >=20

**Primary Dependencies**: Fastify 4.26, Prisma, ioredis/BullMQ, Vitest (existing stack — no new
runtime dependency for the concurrency tests); `autocannon` added as a new devDependency for the
load-test tooling (pure npm package, no external binary, scriptable in TS, matches this
project's existing Node-native toolchain rather than introducing a separate Go binary like k6)

**Storage**: PostgreSQL via Prisma (existing `Assignment`, `SLARun`, `EscalationEvent` models —
one additive schema change per race fix, see data-model.md), Redis (existing, unchanged)

**Testing**: Vitest, run against the existing throwaway Docker Postgres/Redis
(`supporthub-test-pg`/`supporthub-test-redis`) already used by `tests/concurrency/`; load tests
run with `autocannon` against a real running instance of the dev server

**Target Platform**: Linux/Windows server (existing deployment target, unchanged)

**Project Type**: Backend service (existing modular monolith, unchanged)

**Performance Goals**: NEEDS CLARIFICATION resolved in research.md — no business-specified
throughput/latency targets exist yet; FR-009 requires these be marked `OPEN BUSINESS DECISION`
rather than invented, so this feature ships tooling + a baseline report, not a numeric SLA

**Constraints**: Every fix must be additive/backward-compatible (no breaking change to existing
Assignment/SLARun/EscalationEvent consumers — 012-admin-list-views and 015-reporting-dashboards
both already query these tables); every concurrency claim must be proven against real
Docker-provisioned infrastructure per this project's standing verification discipline, never
asserted from code review alone

**Scale/Scope**: 3 real races to prove-and-fix (assignment, SLA, escalation), 1 race to prove
already-safe (ticket status), 3 endpoint groups to load-test (ticket creation, AI support flow,
admin reporting) — entirely within `supporthub-api`, no `supporthub-web` changes

## Constitution Check

*GATE: Must pass before Phase 0 research. Re-check after Phase 1 design.*

| Principle | Check | Status |
|---|---|---|
| I. SaaS Is Sole Identity Authority | N/A — no identity/tenant/product-access logic touched | PASS |
| II. Configuration Over Hardcoding | Load-test pass/fail thresholds are NOT hardcoded — explicitly marked `OPEN BUSINESS DECISION` per FR-009, matching roadmap convention | PASS |
| III. Layered Architecture / Module Boundaries | All three fixes stay inside their owning module (`orchestration/assignments`, `orchestration/sla`, `orchestration/escalation`) — repository-layer changes only, no new cross-module imports | PASS |
| IV. AI Recommends, Policy Decides | N/A — no AI/tool-permission logic touched | PASS |
| V. Evidence-Based Verification | This entire feature IS evidence-based verification — every claimed guarantee must be proven by a real concurrency test against real infra before being considered fixed | PASS (this principle is the feature's own thesis) |
| VI. Durable Audit & History | No audit-log shape changes; EscalationEvent's idempotency fix preserves the existing audit row for the winning attempt, silently no-ops the loser rather than deleting anything | PASS |
| VII. Concurrency-Safe, Durable Job Handling (NON-NEGOTIABLE) | This feature directly implements this principle's own stated requirement ("Assignment and escalation logic MUST be tested under concurrency... job handlers MUST be idempotent") — it is the principle's own overdue test coverage | PASS — this feature exists to close this exact gap |
| VIII. Ticket/Problem Separation | N/A — no Ticket/Problem model changes | PASS |

No violations. No Complexity Tracking entries needed.

## Project Structure

### Documentation (this feature)

```text
specs/016-load-concurrency-testing/
├── plan.md              # This file
├── research.md          # Phase 0 output
├── data-model.md         # Phase 1 output
├── quickstart.md        # Phase 1 output
└── tasks.md             # Phase 2 output (/speckit-tasks — not yet created)
```

No `contracts/` directory: this feature adds no new HTTP endpoints or request/response
contracts — it hardens existing internal behavior and adds test/tooling infrastructure only.

### Source Code (repository root)

```text
prisma/
└── schema.prisma                                   # +1 field (SLARun.version), +2 raw partial
                                                      # unique indexes (migration SQL)

src/modules/orchestration/assignments/
├── repository/assignment.repository.ts              # createAssignment: catch+retry on the new
                                                       # partial-unique-index conflict
└── ...                                               # (engine/service unchanged)

src/modules/orchestration/sla/
├── repository/sla-run.repository.ts                 # update() becomes version-checked; add
│                                                       updateWithVersion(id, expectedVersion, data)
└── service/sla.service.ts                            # pause/resume/complete: read-modify-retry
                                                        loop on version conflict (bounded attempts)

src/modules/orchestration/escalation/
├── repository/escalation-event.repository.ts        # create(): catch the new partial-unique
│                                                       -index conflict, return existing row
└── service/escalation.service.ts                     # fire(): treat a duplicate-conflict as a
                                                        no-op, not an error

tests/concurrency/
├── round-robin.test.ts                               # existing — untouched
├── queue.test.ts                                     # existing — untouched
├── assignment-race.test.ts                           # NEW — User Story 1 / FR-001
├── sla-race.test.ts                                   # NEW — User Story 2 / FR-002
├── escalation-idempotency.test.ts                     # NEW — User Story 3 / FR-003
└── ticket-status-race.test.ts                         # NEW — User Story 4 / FR-004

tests/load/
├── autocannon.config.ts                               # NEW — shared runner + report shape
├── ticket-creation.load.ts                            # NEW — User Story 5 / FR-007, FR-008
├── ai-support-flow.load.ts                             # NEW
└── admin-reporting.load.ts                             # NEW
```

**Structure Decision**: Single backend project (existing `supporthub-api` modular monolith).
Fixes live inside their owning module's existing `repository`/`service` files (Principle III);
new tests live in the existing `tests/concurrency/` directory (already established by
round-robin.test.ts) plus a new `tests/load/` directory for the load-test tooling, mirroring the
existing `tests/{unit,integration,e2e,concurrency}` layout with one new sibling rather than
overloading `tests/concurrency/` with non-correctness-proving load scripts.

## Complexity Tracking

*No violations — table omitted.*