Author SHA1 Message Date
saqibmir da11dbc961 Merge pull request 'development' (#18) from development into main
Reviewed-on: https://gitea.maskantech.in/gitea_admin/support_backend/pulls/18
2026-09-10 09:19:18 +00:00
saqibmir d78bf52182 Merge pull request '016-load-concurrency-testing' (#17) from 016-load-concurrency-testing into development
Reviewed-on: https://gitea.maskantech.in/gitea_admin/support_backend/pulls/17
2026-09-10 07:00:21 +00:00
saqib mirandClaude Sonnet 5 20e5493798 docs(016-load-concurrency-testing): polish — findings, task completion
Documents implementation-time findings in the requirements checklist: all
three suspected races were confirmed real then fixed, the ticket-status
mechanism needed no fix, a real pre-existing test-infrastructure issue
(throwaway DB ticket-code collisions at high accumulated volume) was found
and resolved by resetting the throwaway database and replaying its full
migration history, two full-suite-only integration failures were confirmed
as pre-existing cross-file contamination (not a regression), and the
load-test tooling surfaced a real Anthropic API cost consideration for
ticket creation itself. All 23 tasks marked complete.

Full quality gate green: typecheck, lint, architecture check, full unit
suite (119/119), full integration suite against a freshly reset throwaway
database (122/124 — the 2 failures are the project's own already-accepted
MinIO baseline), and all 6 concurrency test files (11/11).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-10 12:14:16 +05:30
saqib mirandClaude Sonnet 5 8400a8db84 feat(016-load-concurrency-testing): US5 — autocannon load-test tooling
tests/load/autocannon.config.ts is a thin shared wrapper over autocannon's
programmatic API producing this feature's own report shape (throughput,
latency p50/p90/p99, non-2xx, rate-limited count) — printed and written to
tests/load/reports/ (gitignored) for every run. No pass/fail threshold is
applied (FR-009): throughput/latency targets are an OPEN BUSINESS DECISION
per the roadmap's own convention, never invented.

Three scripts cover the named critical endpoint groups: ticket-creation
(pure DB path), admin-reporting (015-reporting-dashboards, pure DB path),
and ai-support-flow (005-ai-support's real Anthropic API calls — clearly
flagged as real, billed cost, run only at a small bounded amount rather than
an open-ended duration). All three were run once at a small scale against the
real dev server to confirm the tooling works end-to-end and cleans up fully
after itself (verified via direct DB checks, not just script exit codes).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-10 11:50:12 +05:30
saqib mirandClaude Sonnet 5 cd8e408d7e test(016-load-concurrency-testing): US4 — prove ticket status optimistic concurrency
tests/concurrency/ticket-status-race.test.ts fires 20 genuinely concurrent
TicketsRepository.updateStatus calls from the same starting version against
real Postgres. Passes on the first run, confirming (rather than assuming)
003-ticketing's existing atomic version-checked updateMany already holds
under real concurrency — no implementation change needed (research.md §4).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-10 11:19:24 +05:30
saqib mirandClaude Sonnet 5 c7aac460b7 test(016-load-concurrency-testing): US3 — prove escalation idempotency holds
tests/concurrency/escalation-idempotency.test.ts fires the same escalation
trigger (escalationService.handleBreach) concurrently more than once for the
same ticket against real Postgres, asserting exactly one EscalationEvent and
one current Assignment result every time — verified across 10 repeated runs
(SC-003). The database-level unique-violation is visibly caught and absorbed
in the logs, confirming the fix (previous commit) actually engages under a
genuine race rather than being untested code.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-10 11:17:31 +05:30
saqib mirandClaude Sonnet 5 93d6fe8b94 fix(016-load-concurrency-testing): US3 — finish escalation idempotency plumbing
Completes the escalation-event repository/service changes from the prior
commit: normalizes the exactOptionalPropertyTypes mismatch in the
findFirst fallback lookup, and updates every existing unit test
(sla-pause-resume, sla-breach-detection, sla-compliance-metric,
escalation-rule-match) to the new SlaRunRepository.updateWithVersion and
EscalationEventRepository.create({event, wasNewlyCreated}) signatures.
Full typecheck/lint/architecture-check clean.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-10 11:13:46 +05:30
saqib mir 794e16f349 create the specf document 2026-09-09 17:39:12 +05:30
saqib mirandClaude Sonnet 5 0e3543bb01 fix(016-load-concurrency-testing): US2 — version-guard SLA pause/resume/sweep
SlaRunRepository.updateWithVersion replaces the old unguarded update(),
mirroring TicketsRepository.updateStatus's exact atomic-updateMany pattern.
pause/resume/complete now retry (bounded) against fresh state on a version
conflict; the breach sweep skips a run that lost the race to a concurrent
pause/resume/complete rather than clobbering it, deferring to the next
scheduled pass. Verified against real Postgres: concurrent pause/resume/sweep
activity against the same run now always leaves it in one
internally-consistent state, across repeated runs.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-09 17:17:09 +05:30
saqib mirandClaude Sonnet 5 58d0a98134 fix(016-load-concurrency-testing): US1 — retry assignment creation on race conflict
AssignmentRepository.createAssignment now catches the
assignments_one_current_per_ticket unique-violation and retries the whole
supersede-then-create transaction (bounded, with jitter) instead of
propagating a raw P2002 to the caller. Verified against real Postgres: before
this fix, 20 genuinely concurrent assignment attempts on the same ticket
reliably threw an unhandled unique-constraint error; after it, exactly one
current assignment results every time across 10 repeated runs.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-09 17:10:10 +05:30
saqib mirandClaude Sonnet 5 1e332143bd build(016-load-concurrency-testing): T001-T002 setup + concurrency-guard migration
Adds autocannon as a devDependency for the load-test tooling (T001), and the
shared schema migration T002 blocks: SLARun.version for optimistic
concurrency, plus two Postgres partial unique indexes
(assignments_one_current_per_ticket, escalation_events_ticket_rule_unique)
guarding against the assignment and escalation races research.md documents.
Applied directly to both the real dev DB and the throwaway test DB.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-09 17:03:37 +05:30
saqib mirandClaude Sonnet 5 2fb5b7ac73 tasks(016-load-concurrency-testing): break down into 23 tasks across 5 stories
Foundational phase (T002) covers the one shared schema migration US1/US2/US3
depend on; US4 (proof-only, no schema change) and US5 (load-test tooling)
have no dependency on it and can proceed independently.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-09 16:57:09 +05:30
saqib mirandClaude Sonnet 5 015ef62b71 plan(016-load-concurrency-testing): design assignment/SLA/escalation race fixes
research.md nails down the exact mechanism for each real race the audit
found: a partial unique index for assignment double-assignment, a
Ticket-style version counter for SLA pause/resume/sweep, and a partial
unique index for escalation-rule idempotency — each traced to the specific
repository/service code that has the gap today. data-model.md and plan.md
carry the resulting schema and repository-contract changes; quickstart.md
defines the real-infra verification steps for each user story.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-09 16:55:32 +05:30
saqib mirandClaude Sonnet 5 9f52d51003 spec(016-load-concurrency-testing): specify concurrency-safety and load testing scope
Five user stories: assignment double-assignment race, SLA pause/resume race,
escalation idempotency, ticket optimistic-concurrency proof, and HTTP
load/throughput testing tooling. Scoped from a targeted audit of existing
concurrency guarantees rather than guesswork — see spec.md's Assumptions.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-09 16:51:00 +05:30
saqibmir 051ee88974 Merge pull request 'development' (#15) from development into main
Reviewed-on: https://gitea.maskantech.in/gitea_admin/support_backend/pulls/15
2026-09-03 11:21:04 +00:00
saqibmir 177488caf8 Merge pull request 'fix' (#14) from 009-problem-resolution into development
Reviewed-on: https://gitea.maskantech.in/gitea_admin/support_backend/pulls/14
2026-09-03 11:20:27 +00:00
MdIrshad1234 5607fdfcd5 Merge branch 'main' of https://gitea.maskantech.in/gitea_admin/support_backend 2026-09-03 16:46:02 +05:30
MdIrshad1234 ca14d45ff1 marge the solve 2026-09-03 16:45:45 +05:30
saqibmir a5650b1089 Merge pull request 'development' (#13) from development into main
Reviewed-on: https://gitea.maskantech.in/gitea_admin/support_backend/pulls/13
2026-09-03 11:12:34 +00:00
saqibmir 7e92a1679b Merge pull request '009-problem-resolution' (#12) from 009-problem-resolution into development
Reviewed-on: https://gitea.maskantech.in/gitea_admin/support_backend/pulls/12
2026-09-03 11:11:24 +00:00
MdIrshad1234 d902f6b26d slove the merge 2026-09-03 14:27:16 +05:30
saqibmir 6a7c6a493e Merge pull request 'development' (#11) from development into main
Reviewed-on: https://gitea.maskantech.in/gitea_admin/support_backend/pulls/11
2026-09-03 08:52:59 +00:00
saqibmir 6beef9584b Merge pull request '008-sla-escalation' (#10) from 008-sla-escalation into development
Reviewed-on: https://gitea.maskantech.in/gitea_admin/support_backend/pulls/10
2026-09-03 08:50:56 +00:00
saqibmir 71520e2423 Merge pull request '007-orchestration-assignment' (#9) from 007-orchestration-assignment into development
Reviewed-on: https://gitea.maskantech.in/gitea_admin/support_backend/pulls/9
2026-09-03 06:41:48 +00:00
maskantech f468dcb6f8 Merge pull request 'main' (#5) from main into development
Reviewed-on: https://gitea.maskantech.in/gitea_admin/support_backend/pulls/5
2026-08-20 07:52:48 +00:00
maskantech fc438d552b Merge pull request 'Update README.md' (#4) from main into development
Reviewed-on: https://gitea.maskantech.in/gitea_admin/support_backend/pulls/4
2026-08-20 07:30:57 +00:00
maskantech 97c954c6fa Merge pull request 'Update README.md' (#3) from main into development
Reviewed-on: https://gitea.maskantech.in/gitea_admin/support_backend/pulls/3
2026-08-20 07:26:08 +00:00
maskantech b44f3d1dd2 Merge pull request 'update compose' (#2) from main into development
Reviewed-on: https://gitea.maskantech.in/gitea_admin/support_backend/pulls/2
2026-08-20 07:17:37 +00:00
maskantech 8ba8cbe4e1 Merge pull request 'main' (#1) from main into development
Reviewed-on: https://gitea.maskantech.in/gitea_admin/support_backend/pulls/1
2026-08-20 06:08:31 +00:00
32 changed files with 3027 additions and 143 deletions
+31
View File
@@ -0,0 +1,31 @@
NODE_ENV=development
PORT=4501
# Nest build
BUILD_COMMAND=npm run build:development
# Database
POSTGRES_HOST=postgres
POSTGRES_DB=support_dev
POSTGRES_USER=support_user
POSTGRES_PASSWORD=z1F3tKF1JNDBQmMq95Up
# DATABASE_URL=postgresql://support_user:z1F3tKF1JNDBQmMq95Up@postgres:5432/support_dev
DATABASE_URL=postgresql://support_user:SupportDev123@localhost:5432/support_dev
# Redis
# REDIS_HOST=redis
# REDIS_PORT=6379
REDIS_HOST=localhost
REDIS_PORT=6379
REDIS_PASSWORD=D7FJ7QDKo5gF9KQAO1GL
# Security & CORS
# JWT_SECRET=super-secret-development-jwt-key-32-chars-long
# CORS_ORIGINS=https://support-dev.maskantech.in
# Security & CORS
JWT_SECRET=super-secret-development-jwt-key-32-chars-long
INTEGRATION_CREDENTIAL_ENCRYPTION_KEY=c2e444fe8cc19eb7465e2f8a05f7384628de7a879fcc812766e093c9182fcd58
CORS_ORIGINS=https://support-dev.maskantech.in
+3
View File
@@ -43,3 +43,6 @@ docker/minio/data/
.env
.env.*
!.env.example
# Load-test run reports — measurement artifacts, not fixtures (016-load-concurrency-testing)
tests/load/reports/
+23 -58
View File
@@ -1,66 +1,31 @@
# SupportHub API
### Development
- docker compose --env-file .env.development -f docker-compose.development.yml up -d --build
### Development (Docker)
- Start all services: `docker compose --env-file .env.development -f docker-compose.development.yml up -d --build`
- Start only database & cache (for local app development): `docker compose --env-file .env.development -f docker-compose.development.yml up -d postgres redis`
### Test
- docker compose --env-file .env.test -f docker-compose.test.yml up --build
### Test (Docker)
- `docker compose --env-file .env.test -f docker-compose.test.yml up --build`
### Production
- docker compose --env-file .env.prod -f docker-compose.prod.yml up --build -d
### Production (Docker)
- `docker compose --env-file .env.prod -f docker-compose.prod.yml up --build -d`
### Stop
- docker compose -f docker-compose.prod.yml down
### Stop / Down
- Stop production: `docker compose -f docker-compose.prod.yml down`
- Stop development: `docker compose -f docker-compose.development.yml down`
- Stop development & wipe volumes: `docker compose --env-file .env.development -f docker-compose.development.yml down -v`
### List Containers
- docker compose --env-file .env.development -f docker-compose.development.yml ps
### List Containers & Logs
- List containers: `docker compose --env-file .env.development -f docker-compose.development.yml ps`
- Follow logs: `docker compose --env-file .env.development -f docker-compose.development.yml logs -f`
### Logs
- docker compose --env-file .env.development -f docker-compose.development.yml logs -f
---
### Local Development (Host)
1. Start database & cache in Docker:
```bash
docker compose --env-file .env.development -f docker-compose.development.yml up -d postgres redis
```
2. Start API server in watch mode:
```bash
npm run dev
```
---
### Database Migrations & Prisma
- **Generate Prisma Client**:
```bash
npm run prisma:generate
```
- **Run / Apply Dev Migrations**:
```bash
npx dotenv-cli -e .env.development -- npm run prisma:migrate
```
- **Deploy Migrations (Production/CI)**:
```bash
npx dotenv-cli -e .env.development -- npm run prisma:deploy
```
- **Push Schema directly (Sync schema without migration files)**:
```bash
npx dotenv-cli -e .env.development -- npx prisma db push
```
---
### Database Migrations
- **Local (using .env.development):**
- Create/apply new migration: `npx prisma migrate dev --name <name>`
- Push schema directly (prototype/sync): `npx prisma db push`
- Deploy pending migrations: `npm run prisma:deploy`
- **Inside Docker Container:**
- `docker exec -it support-api-development npx prisma migrate deploy`
### Database Seeding
- **Seed Database (Roles, Products, Categories, Hierarchy & Demo data)**:
```bash
npx dotenv-cli -e .env.development -- npm run prisma:seed
```
- **Local:**
- `npm run prisma:seed` (or `npx tsx --env-file=.env.development prisma/seed/index.ts`)
- **Inside Docker Container:**
- `docker exec -it support-api-development npm run prisma:seed`
+602 -10
View File
@@ -36,12 +36,14 @@
"zod": "^3.22.4"
},
"devDependencies": {
"@types/autocannon": "^7.12.7",
"@types/bcryptjs": "^2.4.6",
"@types/jsonwebtoken": "^9.0.10",
"@types/luxon": "^3.7.5",
"@types/node": "^20.12.7",
"@typescript-eslint/eslint-plugin": "^7.6.0",
"@typescript-eslint/parser": "^7.6.0",
"autocannon": "^8.0.0",
"eslint": "^8.57.0",
"eslint-config-prettier": "^9.1.0",
"husky": "^9.0.11",
@@ -49,7 +51,7 @@
"prettier": "^3.2.5",
"prisma": "^5.12.1",
"tsc-alias": "^1.9.2",
"tsx": "^4.7.2",
"tsx": "^4.23.13",
"typescript": "^5.4.5",
"vitest": "^1.5.0"
},
@@ -78,6 +80,13 @@
}
}
},
"node_modules/@assemblyscript/loader": {
"version": "0.19.23",
"resolved": "https://registry.npmjs.org/@assemblyscript/loader/-/loader-0.19.23.tgz",
"integrity": "sha512-ulkCYfFbYj01ie1MDOyxv2F6SpRN1TOj7fQxbP07D6HmeR+gr2JLSmINKjga2emB+b1L2KGrFKBTc+e00p54nw==",
"dev": true,
"license": "Apache-2.0"
},
"node_modules/@aws-sdk/checksums": {
"version": "3.1000.28",
"resolved": "https://registry.npmjs.org/@aws-sdk/checksums/-/checksums-3.1000.28.tgz",
@@ -412,6 +421,17 @@
"node": ">=6.9.0"
}
},
"node_modules/@colors/colors": {
"version": "1.5.0",
"resolved": "https://registry.npmjs.org/@colors/colors/-/colors-1.5.0.tgz",
"integrity": "sha512-ooWCrlZP11i8GImSjTHYHLkvFDP48nS4+204nGb1RiX/WXYHmJA2III9/e2DWVabCESdW7hBAEzHRqUn9OUVvQ==",
"dev": true,
"license": "MIT",
"optional": true,
"engines": {
"node": ">=0.1.90"
}
},
"node_modules/@esbuild/aix-ppc64": {
"version": "0.28.2",
"resolved": "https://registry.npmjs.org/@esbuild/aix-ppc64/-/aix-ppc64-0.28.2.tgz",
@@ -1248,6 +1268,16 @@
"node": ">=8"
}
},
"node_modules/@minimistjs/subarg": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/@minimistjs/subarg/-/subarg-1.0.0.tgz",
"integrity": "sha512-Q/ONBiM2zNeYUy0mVSO44mWWKYM3UHuEK43PKIOzJCbvUnPoMH1K+gk3cf1kgnCVJFlWmddahQQCmrmBGlk9jQ==",
"dev": true,
"license": "MIT",
"dependencies": {
"minimist": "^1.1.0"
}
},
"node_modules/@msgpackr-extract/msgpackr-extract-darwin-arm64": {
"version": "3.0.4",
"resolved": "https://registry.npmjs.org/@msgpackr-extract/msgpackr-extract-darwin-arm64/-/msgpackr-extract-darwin-arm64-3.0.4.tgz",
@@ -1614,14 +1644,14 @@
"version": "5.22.0",
"resolved": "https://registry.npmjs.org/@prisma/debug/-/debug-5.22.0.tgz",
"integrity": "sha512-AUt44v3YJeggO2ZU5BkXI7M4hu9BF2zzH2iF2V5pyXT/lRTyWiElZ7It+bRH1EshoMRxHgpYg4VB6rCM+mG5jQ==",
"devOptional": true,
"dev": true,
"license": "Apache-2.0"
},
"node_modules/@prisma/engines": {
"version": "5.22.0",
"resolved": "https://registry.npmjs.org/@prisma/engines/-/engines-5.22.0.tgz",
"integrity": "sha512-UNjfslWhAt06kVL3CjkuYpHAWSO6L4kDCVPegV6itt7nD1kSJavd3vhgAEhjglLJJKEdJ7oIqDJ+yHk6qO8gPA==",
"devOptional": true,
"dev": true,
"hasInstallScript": true,
"license": "Apache-2.0",
"dependencies": {
@@ -1635,14 +1665,14 @@
"version": "5.22.0-44.605197351a3c8bdd595af2d2a9bc3025bca48ea2",
"resolved": "https://registry.npmjs.org/@prisma/engines-version/-/engines-version-5.22.0-44.605197351a3c8bdd595af2d2a9bc3025bca48ea2.tgz",
"integrity": "sha512-2PTmxFR2yHW/eB3uqWtcgRcgAbG1rwG9ZriSvQw+nnb7c4uCr3RAcGMb6/zfE88SKlC1Nj2ziUvc96Z379mHgQ==",
"devOptional": true,
"dev": true,
"license": "Apache-2.0"
},
"node_modules/@prisma/fetch-engine": {
"version": "5.22.0",
"resolved": "https://registry.npmjs.org/@prisma/fetch-engine/-/fetch-engine-5.22.0.tgz",
"integrity": "sha512-bkrD/Mc2fSvkQBV5EpoFcZ87AvOgDxbG99488a5cexp5Ccny+UM6MAe/UFkUC0wLYD9+9befNOqGiIJhhq+HbA==",
"devOptional": true,
"dev": true,
"license": "Apache-2.0",
"dependencies": {
"@prisma/debug": "5.22.0",
@@ -1654,7 +1684,7 @@
"version": "5.22.0",
"resolved": "https://registry.npmjs.org/@prisma/get-platform/-/get-platform-5.22.0.tgz",
"integrity": "sha512-pHhpQdr1UPFpt+zFfnPazhulaZYCUqeIcPpJViYoq9R+D/yw4fjE+CtnsnKzPYm0ddUbeXUzjGVGIRVgPDCk4Q==",
"devOptional": true,
"dev": true,
"license": "Apache-2.0",
"dependencies": {
"@prisma/debug": "5.22.0"
@@ -2104,6 +2134,16 @@
"integrity": "sha512-1bnPQqSxSuc3Ii6MhBysoWCg58j97aUjuCSZrGSmDxNqtytIi0k8utUenAwTZN4V5mXXYGsVUI9zeBqy+jBOSQ==",
"license": "MIT"
},
"node_modules/@types/autocannon": {
"version": "7.12.7",
"resolved": "https://registry.npmjs.org/@types/autocannon/-/autocannon-7.12.7.tgz",
"integrity": "sha512-Pd4nPf7wRpacULa6D/EC9x3CwzFQXwA0z5WFuik/fvJjW44V3WzBTM3jtt8nSBoflUNgswPiMCtgrr1bwnAcMg==",
"dev": true,
"license": "MIT",
"dependencies": {
"@types/node": "*"
}
},
"node_modules/@types/bcryptjs": {
"version": "2.4.6",
"resolved": "https://registry.npmjs.org/@types/bcryptjs/-/bcryptjs-2.4.6.tgz",
@@ -2663,6 +2703,13 @@
"node": "*"
}
},
"node_modules/asynckit": {
"version": "0.4.0",
"resolved": "https://registry.npmjs.org/asynckit/-/asynckit-0.4.0.tgz",
"integrity": "sha512-Oei9OH4tRh0YqU3GxhX79dM/mwVgvbZJaSNaRk+bshkj0S5cfHcgYakreBjrHwatXKbz+IoIdYLxrKim2MjW0Q==",
"dev": true,
"license": "MIT"
},
"node_modules/atomic-sleep": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/atomic-sleep/-/atomic-sleep-1.0.0.tgz",
@@ -2672,6 +2719,41 @@
"node": ">=8.0.0"
}
},
"node_modules/autocannon": {
"version": "8.0.0",
"resolved": "https://registry.npmjs.org/autocannon/-/autocannon-8.0.0.tgz",
"integrity": "sha512-fMMcWc2JPFcUaqHeR6+PbmEpTxCrPZyBUM95oG4w3ngJ8NfBNas/ZXA+pTHXLqJ0UlFVTcy05GC25WxKx/M20A==",
"dev": true,
"license": "MIT",
"dependencies": {
"@minimistjs/subarg": "^1.0.0",
"chalk": "^4.1.0",
"char-spinner": "^1.0.1",
"cli-table3": "^0.6.0",
"color-support": "^1.1.1",
"cross-argv": "^2.0.0",
"form-data": "^4.0.0",
"has-async-hooks": "^1.0.0",
"hdr-histogram-js": "^3.0.0",
"hdr-histogram-percentiles-obj": "^3.0.0",
"http-parser-js": "^0.5.2",
"hyperid": "^3.0.0",
"lodash.chunk": "^4.2.0",
"lodash.clonedeep": "^4.5.0",
"lodash.flatten": "^4.4.0",
"manage-path": "^2.0.0",
"on-net-listen": "^1.1.1",
"pretty-bytes": "^5.4.1",
"progress": "^2.0.3",
"reinterval": "^1.1.0",
"retimer": "^3.0.0",
"semver": "^7.3.2",
"timestring": "^6.0.0"
},
"bin": {
"autocannon": "autocannon.js"
}
},
"node_modules/avvio": {
"version": "8.4.0",
"resolved": "https://registry.npmjs.org/avvio/-/avvio-8.4.0.tgz",
@@ -2829,6 +2911,20 @@
"node": ">=8"
}
},
"node_modules/call-bind-apply-helpers": {
"version": "1.0.2",
"resolved": "https://registry.npmjs.org/call-bind-apply-helpers/-/call-bind-apply-helpers-1.0.2.tgz",
"integrity": "sha512-Sp1ablJ0ivDkSzjcaJdxEunN5/XvksFJ2sMBFfq6x0ryhQV/2b/KwFe21cMpmHtPOSij8K99/wSfoEuTObmuMQ==",
"dev": true,
"license": "MIT",
"dependencies": {
"es-errors": "^1.3.0",
"function-bind": "^1.1.2"
},
"engines": {
"node": ">= 0.4"
}
},
"node_modules/callsites": {
"version": "3.1.0",
"resolved": "https://registry.npmjs.org/callsites/-/callsites-3.1.0.tgz",
@@ -2875,6 +2971,13 @@
"url": "https://github.com/chalk/chalk?sponsor=1"
}
},
"node_modules/char-spinner": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/char-spinner/-/char-spinner-1.0.1.tgz",
"integrity": "sha512-acv43vqJ0+N0rD+Uw3pDHSxP30FHrywu2NO6/wBaHChJIizpDeBUd6NjqhNhy9LGaEAhZAXn46QzmlAvIWd16g==",
"dev": true,
"license": "ISC"
},
"node_modules/check-error": {
"version": "1.0.3",
"resolved": "https://registry.npmjs.org/check-error/-/check-error-1.0.3.tgz",
@@ -2942,6 +3045,54 @@
"url": "https://github.com/sponsors/sindresorhus"
}
},
"node_modules/cli-table3": {
"version": "0.6.5",
"resolved": "https://registry.npmjs.org/cli-table3/-/cli-table3-0.6.5.tgz",
"integrity": "sha512-+W/5efTR7y5HRD7gACw9yQjqMVvEMLBHmboM/kPWam+H+Hmyrgjh6YncVKK122YZkXrLudzTuAukUw9FnMf7IQ==",
"dev": true,
"license": "MIT",
"dependencies": {
"string-width": "^4.2.0"
},
"engines": {
"node": "10.* || >= 12.*"
},
"optionalDependencies": {
"@colors/colors": "1.5.0"
}
},
"node_modules/cli-table3/node_modules/emoji-regex": {
"version": "8.0.0",
"resolved": "https://registry.npmjs.org/emoji-regex/-/emoji-regex-8.0.0.tgz",
"integrity": "sha512-MSjYzcWNOA0ewAHpz0MxpYFvwg6yjy1NG3xteoqz644VCo/RPgnr1/GGt+ic3iJTzQ8Eu3TdM14SawnVUmGE6A==",
"dev": true,
"license": "MIT"
},
"node_modules/cli-table3/node_modules/is-fullwidth-code-point": {
"version": "3.0.0",
"resolved": "https://registry.npmjs.org/is-fullwidth-code-point/-/is-fullwidth-code-point-3.0.0.tgz",
"integrity": "sha512-zymm5+u+sCsSWyD9qNaejV3DFvhCKclKdizYaJUuHA83RLjb7nSuGnddCHGv0hk+KY7BMAlsWeK4Ueg6EV6XQg==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">=8"
}
},
"node_modules/cli-table3/node_modules/string-width": {
"version": "4.2.3",
"resolved": "https://registry.npmjs.org/string-width/-/string-width-4.2.3.tgz",
"integrity": "sha512-wKyQRQpjJ0sIp62ErSZdGsjMJWsap5oRNihHhu6G7JVO/9jIB6UyevL+tXuOqrng8j/cxKTWyWUwvSTriiZz/g==",
"dev": true,
"license": "MIT",
"dependencies": {
"emoji-regex": "^8.0.0",
"is-fullwidth-code-point": "^3.0.0",
"strip-ansi": "^6.0.1"
},
"engines": {
"node": ">=8"
}
},
"node_modules/cli-truncate": {
"version": "4.0.0",
"resolved": "https://registry.npmjs.org/cli-truncate/-/cli-truncate-4.0.0.tgz",
@@ -3040,12 +3191,35 @@
"integrity": "sha512-dOy+3AuW3a2wNbZHIuMZpTcgjGuLU/uBL/ubcZF9OXbDo8ff4O8yVp5Bf0efS8uEoYo5q4Fx7dY9OgQGXgAsQA==",
"license": "MIT"
},
"node_modules/color-support": {
"version": "1.1.3",
"resolved": "https://registry.npmjs.org/color-support/-/color-support-1.1.3.tgz",
"integrity": "sha512-qiBjkpbMLO/HL68y+lh4q0/O1MZFj2RX6X/KmMa3+gJD3z+WwI1ZzDHysvqHGS3mP6mznPckpXmw1nI9cJjyRg==",
"dev": true,
"license": "ISC",
"bin": {
"color-support": "bin.js"
}
},
"node_modules/colorette": {
"version": "2.0.20",
"resolved": "https://registry.npmjs.org/colorette/-/colorette-2.0.20.tgz",
"integrity": "sha512-IfEDxwoWIjkeXL1eXcDiow4UbKjhLdq6/EuSVR9GMN7KVH3r9gQ83e73hsz1Nd1T3ijd5xv1wcWRYO+D6kCI2w==",
"license": "MIT"
},
"node_modules/combined-stream": {
"version": "1.0.8",
"resolved": "https://registry.npmjs.org/combined-stream/-/combined-stream-1.0.8.tgz",
"integrity": "sha512-FQN4MRfuJeHf7cBbBMJFXhKSDq+2kAArBlmRBvcvFE5BB1HZKXtSFASDhdlz9zOYwxh8lDdnvmMOe/+5cdoEdg==",
"dev": true,
"license": "MIT",
"dependencies": {
"delayed-stream": "~1.0.0"
},
"engines": {
"node": ">= 0.8"
}
},
"node_modules/commander": {
"version": "13.1.0",
"resolved": "https://registry.npmjs.org/commander/-/commander-13.1.0.tgz",
@@ -3104,6 +3278,13 @@
"node": ">=12.0.0"
}
},
"node_modules/cross-argv": {
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/cross-argv/-/cross-argv-2.0.0.tgz",
"integrity": "sha512-YIaY9TR5Nxeb8SMdtrU8asWVM4jqJDNDYlKV21LxtYcfNJhp1kEsgSa6qXwXgzN0WQWGODps0+TlGp2xQSHwOg==",
"dev": true,
"license": "MIT"
},
"node_modules/cross-spawn": {
"version": "7.0.6",
"resolved": "https://registry.npmjs.org/cross-spawn/-/cross-spawn-7.0.6.tgz",
@@ -3164,6 +3345,16 @@
"dev": true,
"license": "MIT"
},
"node_modules/delayed-stream": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/delayed-stream/-/delayed-stream-1.0.0.tgz",
"integrity": "sha512-ZySD7Nf91aLB0RxL4KGrKHBXl7Eds1DAmEdcoVawXnLD7SDhpNgtuII2aAkg7a7QS41jxPSZ17p4VdGnMHk3MQ==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">=0.4.0"
}
},
"node_modules/denque": {
"version": "2.1.0",
"resolved": "https://registry.npmjs.org/denque/-/denque-2.1.0.tgz",
@@ -3240,6 +3431,21 @@
"url": "https://dotenvx.com"
}
},
"node_modules/dunder-proto": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/dunder-proto/-/dunder-proto-1.0.1.tgz",
"integrity": "sha512-KIN/nDJBQRcXw0MLVhZE9iQHmG68qAVIBg9CqmUYjmQIhgij9U5MFvrqkUL5FbtyyzZuOeOt0zdeRe4UY7ct+A==",
"dev": true,
"license": "MIT",
"dependencies": {
"call-bind-apply-helpers": "^1.0.1",
"es-errors": "^1.3.0",
"gopd": "^1.2.0"
},
"engines": {
"node": ">= 0.4"
}
},
"node_modules/eastasianwidth": {
"version": "0.2.0",
"resolved": "https://registry.npmjs.org/eastasianwidth/-/eastasianwidth-0.2.0.tgz",
@@ -3283,6 +3489,55 @@
"url": "https://github.com/sponsors/sindresorhus"
}
},
"node_modules/es-define-property": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/es-define-property/-/es-define-property-1.0.1.tgz",
"integrity": "sha512-e3nRfgfUZ4rNGL232gUgX06QNyyez04KdjFrF+LTRoOXmrOgFKDg4BCdsjW8EnT69eqdYGmRpJwiPVYNrCaW3g==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">= 0.4"
}
},
"node_modules/es-errors": {
"version": "1.3.0",
"resolved": "https://registry.npmjs.org/es-errors/-/es-errors-1.3.0.tgz",
"integrity": "sha512-Zf5H2Kxt2xjTvbJvP2ZWLEICxA6j+hAmMzIlypy4xcBg1vKVnx89Wy0GbS+kf5cwCVFFzdCFh2XSCFNULS6csw==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">= 0.4"
}
},
"node_modules/es-object-atoms": {
"version": "1.1.2",
"resolved": "https://registry.npmjs.org/es-object-atoms/-/es-object-atoms-1.1.2.tgz",
"integrity": "sha512-HWcBoN6NileqtSydK2FqHbS/LoDd2pqrnQHLyJzBj4kOp/ky2MWMN694xOfkK8/SnUsW2DH7EfyVlydKCsm1Zw==",
"dev": true,
"license": "MIT",
"dependencies": {
"es-errors": "^1.3.0"
},
"engines": {
"node": ">= 0.4"
}
},
"node_modules/es-set-tostringtag": {
"version": "2.1.0",
"resolved": "https://registry.npmjs.org/es-set-tostringtag/-/es-set-tostringtag-2.1.0.tgz",
"integrity": "sha512-j6vWzfrGVfyXxge+O0x5sh6cvxAog0a/4Rdd2K36zCMV5eJ+/+tOAngRO8cODMNWbVRdVlmGZQL2YS3yR8bIUA==",
"dev": true,
"license": "MIT",
"dependencies": {
"es-errors": "^1.3.0",
"get-intrinsic": "^1.2.6",
"has-tostringtag": "^1.0.2",
"hasown": "^2.0.2"
},
"engines": {
"node": ">= 0.4"
}
},
"node_modules/esbuild": {
"version": "0.28.2",
"resolved": "https://registry.npmjs.org/esbuild/-/esbuild-0.28.2.tgz",
@@ -3980,6 +4235,23 @@
"url": "https://github.com/sponsors/isaacs"
}
},
"node_modules/form-data": {
"version": "4.0.6",
"resolved": "https://registry.npmjs.org/form-data/-/form-data-4.0.6.tgz",
"integrity": "sha512-vKatAh4SlVfgbv+YtmhiRjhEMJsYpsG1Y2rMQtR+SVSbytsSD1YGzDIcrAJmdFec88u/+VoGmxnl+80gL1tRCQ==",
"dev": true,
"license": "MIT",
"dependencies": {
"asynckit": "^0.4.0",
"combined-stream": "^1.0.8",
"es-set-tostringtag": "^2.1.0",
"hasown": "^2.0.4",
"mime-types": "^2.1.35"
},
"engines": {
"node": ">= 6"
}
},
"node_modules/forwarded": {
"version": "0.2.0",
"resolved": "https://registry.npmjs.org/forwarded/-/forwarded-0.2.0.tgz",
@@ -4011,6 +4283,16 @@
"node": "^8.16.0 || ^10.6.0 || >=11.0.0"
}
},
"node_modules/function-bind": {
"version": "1.1.2",
"resolved": "https://registry.npmjs.org/function-bind/-/function-bind-1.1.2.tgz",
"integrity": "sha512-7XHNxH7qX9xG5mIwxkhumTox/MIRNcOgDrxWsMt2pAr23WHp6MrRlN7FBSFpCpr+oVO0F744iUgR82nJMfG2SA==",
"dev": true,
"license": "MIT",
"funding": {
"url": "https://github.com/sponsors/ljharb"
}
},
"node_modules/get-east-asian-width": {
"version": "1.6.0",
"resolved": "https://registry.npmjs.org/get-east-asian-width/-/get-east-asian-width-1.6.0.tgz",
@@ -4034,6 +4316,45 @@
"node": "*"
}
},
"node_modules/get-intrinsic": {
"version": "1.3.0",
"resolved": "https://registry.npmjs.org/get-intrinsic/-/get-intrinsic-1.3.0.tgz",
"integrity": "sha512-9fSjSaos/fRIVIp+xSJlE6lfwhES7LNtKaCBIamHsjr2na1BiABJPo0mOjjz8GJDURarmCPGqaiVg5mfjb98CQ==",
"dev": true,
"license": "MIT",
"dependencies": {
"call-bind-apply-helpers": "^1.0.2",
"es-define-property": "^1.0.1",
"es-errors": "^1.3.0",
"es-object-atoms": "^1.1.1",
"function-bind": "^1.1.2",
"get-proto": "^1.0.1",
"gopd": "^1.2.0",
"has-symbols": "^1.1.0",
"hasown": "^2.0.2",
"math-intrinsics": "^1.1.0"
},
"engines": {
"node": ">= 0.4"
},
"funding": {
"url": "https://github.com/sponsors/ljharb"
}
},
"node_modules/get-proto": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/get-proto/-/get-proto-1.0.1.tgz",
"integrity": "sha512-sTSfBjoXBp89JvIKIefqw7U2CCebsc74kiY6awiGogKtoSGbgjYE/G/+l9sF3MWFPNc9IcoOC4ODfKHfxFmp0g==",
"dev": true,
"license": "MIT",
"dependencies": {
"dunder-proto": "^1.0.1",
"es-object-atoms": "^1.0.0"
},
"engines": {
"node": ">= 0.4"
}
},
"node_modules/get-stream": {
"version": "8.0.1",
"resolved": "https://registry.npmjs.org/get-stream/-/get-stream-8.0.1.tgz",
@@ -4131,6 +4452,19 @@
"url": "https://github.com/sponsors/sindresorhus"
}
},
"node_modules/gopd": {
"version": "1.2.0",
"resolved": "https://registry.npmjs.org/gopd/-/gopd-1.2.0.tgz",
"integrity": "sha512-ZUKRh6/kUFoAiTAtTYPZJ3hw9wNxx+BIBOijnlG9PnrJsCcSjs1wyyD6vJpaYtgnzDrKYRSqf3OO6Rfa93xsRg==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">= 0.4"
},
"funding": {
"url": "https://github.com/sponsors/ljharb"
}
},
"node_modules/graphemer": {
"version": "1.4.0",
"resolved": "https://registry.npmjs.org/graphemer/-/graphemer-1.4.0.tgz",
@@ -4138,6 +4472,13 @@
"dev": true,
"license": "MIT"
},
"node_modules/has-async-hooks": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/has-async-hooks/-/has-async-hooks-1.0.0.tgz",
"integrity": "sha512-YF0VPGjkxr7AyyQQNykX8zK4PvtEDsUJAPqwu06UFz1lb6EvI53sPh5H1kWxg8NXI5LsfRCZ8uX9NkYDZBb/mw==",
"dev": true,
"license": "Apache-2.0"
},
"node_modules/has-flag": {
"version": "4.0.0",
"resolved": "https://registry.npmjs.org/has-flag/-/has-flag-4.0.0.tgz",
@@ -4148,6 +4489,70 @@
"node": ">=8"
}
},
"node_modules/has-symbols": {
"version": "1.1.0",
"resolved": "https://registry.npmjs.org/has-symbols/-/has-symbols-1.1.0.tgz",
"integrity": "sha512-1cDNdwJ2Jaohmb3sg4OmKaMBwuC48sYni5HUw2DvsC8LjGTLK9h+eb1X6RyuOHe4hT0ULCW68iomhjUoKUqlPQ==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">= 0.4"
},
"funding": {
"url": "https://github.com/sponsors/ljharb"
}
},
"node_modules/has-tostringtag": {
"version": "1.0.2",
"resolved": "https://registry.npmjs.org/has-tostringtag/-/has-tostringtag-1.0.2.tgz",
"integrity": "sha512-NqADB8VjPFLM2V0VvHUewwwsw0ZWBaIdgo+ieHtK3hasLz4qeCRjYcqfB6AQrBggRKppKF8L52/VqdVsO47Dlw==",
"dev": true,
"license": "MIT",
"dependencies": {
"has-symbols": "^1.0.3"
},
"engines": {
"node": ">= 0.4"
},
"funding": {
"url": "https://github.com/sponsors/ljharb"
}
},
"node_modules/hasown": {
"version": "2.0.4",
"resolved": "https://registry.npmjs.org/hasown/-/hasown-2.0.4.tgz",
"integrity": "sha512-T2UbfbBEF32wiepXIsMlTW9+dDYC6wMh/t/vYA4tuOMKqWz/n3vr1NFSxQiyP+zk2mXsoMA/i/7qV6LKut1t1A==",
"dev": true,
"license": "MIT",
"dependencies": {
"function-bind": "^1.1.2"
},
"engines": {
"node": ">= 0.4"
}
},
"node_modules/hdr-histogram-js": {
"version": "3.0.1",
"resolved": "https://registry.npmjs.org/hdr-histogram-js/-/hdr-histogram-js-3.0.1.tgz",
"integrity": "sha512-l3GSdZL1Jr1C0kyb461tUjEdrRPZr8Qry7jByltf5JGrA0xvqOSrxRBfcrJqqV/AMEtqqhHhC6w8HW0gn76tRQ==",
"dev": true,
"license": "BSD-2-Clause",
"dependencies": {
"@assemblyscript/loader": "^0.19.21",
"base64-js": "^1.2.0",
"pako": "^1.0.3"
},
"engines": {
"node": ">=14"
}
},
"node_modules/hdr-histogram-percentiles-obj": {
"version": "3.0.0",
"resolved": "https://registry.npmjs.org/hdr-histogram-percentiles-obj/-/hdr-histogram-percentiles-obj-3.0.0.tgz",
"integrity": "sha512-7kIufnBqdsBGcSZLPJwqHT3yhk1QTsSlFsVD3kx5ixH/AlgBs9yM1q6DPhXZ8f8gtdqgh7N7/5btRLpQsS2gHw==",
"dev": true,
"license": "MIT"
},
"node_modules/helmet": {
"version": "7.2.0",
"resolved": "https://registry.npmjs.org/helmet/-/helmet-7.2.0.tgz",
@@ -4179,6 +4584,13 @@
"node": ">= 0.8"
}
},
"node_modules/http-parser-js": {
"version": "0.5.10",
"resolved": "https://registry.npmjs.org/http-parser-js/-/http-parser-js-0.5.10.tgz",
"integrity": "sha512-Pysuw9XpUq5dVc/2SMHpuTY01RFl8fttgcyunjL7eEMhGM3cI4eOmiCycJDVCo/7O7ClfQD3SaI6ftDzqOXYMA==",
"dev": true,
"license": "MIT"
},
"node_modules/human-signals": {
"version": "5.0.0",
"resolved": "https://registry.npmjs.org/human-signals/-/human-signals-5.0.0.tgz",
@@ -4205,6 +4617,43 @@
"url": "https://github.com/sponsors/typicode"
}
},
"node_modules/hyperid": {
"version": "3.3.0",
"resolved": "https://registry.npmjs.org/hyperid/-/hyperid-3.3.0.tgz",
"integrity": "sha512-7qhCVT4MJIoEsNcbhglhdmBKb09QtcmJNiIQGq7js/Khf5FtQQ9bzcAuloeqBeee7XD7JqDeve9KNlQya5tSGQ==",
"dev": true,
"license": "MIT",
"dependencies": {
"buffer": "^5.2.1",
"uuid": "^8.3.2",
"uuid-parse": "^1.1.0"
}
},
"node_modules/hyperid/node_modules/buffer": {
"version": "5.7.1",
"resolved": "https://registry.npmjs.org/buffer/-/buffer-5.7.1.tgz",
"integrity": "sha512-EHcyIPBQ4BSGlvjB16k5KgAJ27CIsHY/2JBmCRReo48y9rQ3MaUzWX3KVlBa4U7MyX02HdVj0K7C3WaB3ju7FQ==",
"dev": true,
"funding": [
{
"type": "github",
"url": "https://github.com/sponsors/feross"
},
{
"type": "patreon",
"url": "https://www.patreon.com/feross"
},
{
"type": "consulting",
"url": "https://feross.org/support"
}
],
"license": "MIT",
"dependencies": {
"base64-js": "^1.3.1",
"ieee754": "^1.1.13"
}
},
"node_modules/ieee754": {
"version": "1.2.1",
"resolved": "https://registry.npmjs.org/ieee754/-/ieee754-1.2.1.tgz",
@@ -4772,6 +5221,27 @@
"url": "https://github.com/sponsors/sindresorhus"
}
},
"node_modules/lodash.chunk": {
"version": "4.2.0",
"resolved": "https://registry.npmjs.org/lodash.chunk/-/lodash.chunk-4.2.0.tgz",
"integrity": "sha512-ZzydJKfUHJwHa+hF5X66zLFCBrWn5GeF28OHEr4WVWtNDXlQ/IjWKPBiikqKo2ne0+v6JgCgJ0GzJp8k8bHC7w==",
"dev": true,
"license": "MIT"
},
"node_modules/lodash.clonedeep": {
"version": "4.5.0",
"resolved": "https://registry.npmjs.org/lodash.clonedeep/-/lodash.clonedeep-4.5.0.tgz",
"integrity": "sha512-H5ZhCF25riFd9uB5UCkVKo61m3S/xZk1x4wA6yp/L3RFP6Z/eHH1ymQcGLo7J3GMPfm0V/7m1tryHuGVxpqEBQ==",
"dev": true,
"license": "MIT"
},
"node_modules/lodash.flatten": {
"version": "4.4.0",
"resolved": "https://registry.npmjs.org/lodash.flatten/-/lodash.flatten-4.4.0.tgz",
"integrity": "sha512-C5N2Z3DgnnKr0LOpv/hKCgKdb7ZZwafIrsesve6lmzvZIRZRGaZ/l6Q8+2W7NaT+ZwO3fFlSCzCzrDCFdJfZ4g==",
"dev": true,
"license": "MIT"
},
"node_modules/lodash.includes": {
"version": "4.3.0",
"resolved": "https://registry.npmjs.org/lodash.includes/-/lodash.includes-4.3.0.tgz",
@@ -4994,6 +5464,23 @@
"@jridgewell/sourcemap-codec": "^1.5.5"
}
},
"node_modules/manage-path": {
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/manage-path/-/manage-path-2.0.0.tgz",
"integrity": "sha512-NJhyB+PJYTpxhxZJ3lecIGgh4kwIY2RAh44XvAz9UlqthlQwtPBf62uBVR8XaD8CRuSjQ6TnZH2lNJkbLPZM2A==",
"dev": true,
"license": "MIT"
},
"node_modules/math-intrinsics": {
"version": "1.1.0",
"resolved": "https://registry.npmjs.org/math-intrinsics/-/math-intrinsics-1.1.0.tgz",
"integrity": "sha512-/IXtbwEk5HTPyEwyKX6hGkYXxM9nbj64B+ilVJnC/R6B0pH5G4V3b0pVbL7DBj4tkhBAppbQUlf6F6Xl9LHu1g==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">= 0.4"
}
},
"node_modules/merge-stream": {
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/merge-stream/-/merge-stream-2.0.0.tgz",
@@ -5037,6 +5524,29 @@
"node": ">=10.0.0"
}
},
"node_modules/mime-db": {
"version": "1.52.0",
"resolved": "https://registry.npmjs.org/mime-db/-/mime-db-1.52.0.tgz",
"integrity": "sha512-sPU4uV7dYlvtWJxwwxHD0PuihVNiE7TyAbQ5SWxDCB9mUYvOgroQOwYQQOKPJ8CIbE+1ETVlOoK1UC2nU3gYvg==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">= 0.6"
}
},
"node_modules/mime-types": {
"version": "2.1.35",
"resolved": "https://registry.npmjs.org/mime-types/-/mime-types-2.1.35.tgz",
"integrity": "sha512-ZDY+bPm5zTTF+YpCrAU9nK0UgICYPT0QtT1NZWFv4s++TNkcgVaT0g6+4R2uI4MjQjzysHB1zxuWL50hzaeXiw==",
"dev": true,
"license": "MIT",
"dependencies": {
"mime-db": "1.52.0"
},
"engines": {
"node": ">= 0.6"
}
},
"node_modules/mimic-fn": {
"version": "4.0.0",
"resolved": "https://registry.npmjs.org/mimic-fn/-/mimic-fn-4.0.0.tgz",
@@ -5277,6 +5787,16 @@
"node": ">=14.0.0"
}
},
"node_modules/on-net-listen": {
"version": "1.1.2",
"resolved": "https://registry.npmjs.org/on-net-listen/-/on-net-listen-1.1.2.tgz",
"integrity": "sha512-y1HRYy8s/RlcBvDUwKXSmkODMdx4KSuIvloCnQYJ2LdBBC1asY4HtfhXwe3UWknLakATZDnbzht2Ijw3M1EqFg==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">=9.4.0 || ^8.9.4"
}
},
"node_modules/once": {
"version": "1.4.0",
"resolved": "https://registry.npmjs.org/once/-/once-1.4.0.tgz",
@@ -5364,6 +5884,13 @@
"integrity": "sha512-UEZIS3/by4OC8vL3P2dTXRETpebLI2NiI5vIrjaD/5UtrkFX/tNbwjTSRAGC/+7CAo2pIcBaRgWmcBBHcsaCIw==",
"license": "BlueOak-1.0.0"
},
"node_modules/pako": {
"version": "1.0.11",
"resolved": "https://registry.npmjs.org/pako/-/pako-1.0.11.tgz",
"integrity": "sha512-4hLB8Py4zZce5s4yd9XzopqwVv/yGNhV1Bl8NTmCq1763HeK2+EwVTv+leGeL13Dnh2wfbqowVPXCIO0z4taYw==",
"dev": true,
"license": "(MIT AND Zlib)"
},
"node_modules/parent-module": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/parent-module/-/parent-module-1.0.1.tgz",
@@ -5682,6 +6209,19 @@
"url": "https://github.com/prettier/prettier?sponsor=1"
}
},
"node_modules/pretty-bytes": {
"version": "5.6.0",
"resolved": "https://registry.npmjs.org/pretty-bytes/-/pretty-bytes-5.6.0.tgz",
"integrity": "sha512-FFw039TmrBqFK8ma/7OL3sDz/VytdtJr044/QUJtH0wK9lb9jLq9tJyIxUwtQJHwar2BqtiA4iCWSwo9JLkzFg==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">=6"
},
"funding": {
"url": "https://github.com/sponsors/sindresorhus"
}
},
"node_modules/pretty-format": {
"version": "29.7.0",
"resolved": "https://registry.npmjs.org/pretty-format/-/pretty-format-29.7.0.tgz",
@@ -5714,7 +6254,7 @@
"version": "5.22.0",
"resolved": "https://registry.npmjs.org/prisma/-/prisma-5.22.0.tgz",
"integrity": "sha512-vtpjW3XuYCSnMsNVBjLMNkTj6OZbudcPPTPYHqX0CJfpcdWciI1dM8uHETwmDxxiqEwCIE6WvXucWUetJgfu/A==",
"devOptional": true,
"dev": true,
"hasInstallScript": true,
"license": "Apache-2.0",
"dependencies": {
@@ -5745,6 +6285,16 @@
"integrity": "sha512-mqn0kFRl0EoqhnL0GQ0veqFHyIN1yig9RHh/InzORTUiZHFRAur+aMtRkELNwGs9aNwKS6tg/An4NYBPGwvtzQ==",
"license": "MIT"
},
"node_modules/progress": {
"version": "2.0.3",
"resolved": "https://registry.npmjs.org/progress/-/progress-2.0.3.tgz",
"integrity": "sha512-7PiHtLll5LdnKIMw100I+8xJXR5gW2QwWYkT6iJva0bXitZKa/XMrSbdmg3r2Xnaidz9Qumd0VPaMrZlF9V9sA==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">=0.4.0"
}
},
"node_modules/prom-client": {
"version": "15.1.3",
"resolved": "https://registry.npmjs.org/prom-client/-/prom-client-15.1.3.tgz",
@@ -5886,6 +6436,13 @@
"node": ">=4"
}
},
"node_modules/reinterval": {
"version": "1.1.0",
"resolved": "https://registry.npmjs.org/reinterval/-/reinterval-1.1.0.tgz",
"integrity": "sha512-QIRet3SYrGp0HUHO88jVskiG6seqUGC5iAG7AwI/BV4ypGcuqk9Du6YQBUOUqm9c8pw1eyLoIaONifRua1lsEQ==",
"dev": true,
"license": "MIT"
},
"node_modules/require-from-string": {
"version": "2.0.2",
"resolved": "https://registry.npmjs.org/require-from-string/-/require-from-string-2.0.2.tgz",
@@ -5957,6 +6514,13 @@
"node": ">=10"
}
},
"node_modules/retimer": {
"version": "3.0.0",
"resolved": "https://registry.npmjs.org/retimer/-/retimer-3.0.0.tgz",
"integrity": "sha512-WKE0j11Pa0ZJI5YIk0nflGI7SQsfl2ljihVy7ogh7DeQSeYAUi0ubZ/yEueGtDfUPk6GH5LRw1hBdLq4IwUBWA==",
"dev": true,
"license": "MIT"
},
"node_modules/reusify": {
"version": "1.1.0",
"resolved": "https://registry.npmjs.org/reusify/-/reusify-1.1.0.tgz",
@@ -6515,6 +7079,16 @@
"real-require": "^0.2.0"
}
},
"node_modules/timestring": {
"version": "6.0.0",
"resolved": "https://registry.npmjs.org/timestring/-/timestring-6.0.0.tgz",
"integrity": "sha512-wMctrWD2HZZLuIlchlkE2dfXJh7J2KDI9Dwl+2abPYg0mswQHfOAyQW3jJg1pY5VfttSINZuKcXoB3FGypVklA==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">=8"
}
},
"node_modules/tinybench": {
"version": "2.9.0",
"resolved": "https://registry.npmjs.org/tinybench/-/tinybench-2.9.0.tgz",
@@ -6631,9 +7205,9 @@
"license": "0BSD"
},
"node_modules/tsx": {
"version": "4.23.12",
"resolved": "https://registry.npmjs.org/tsx/-/tsx-4.23.12.tgz",
"integrity": "sha512-FDf4L4sYzKtzWYhU/Xm0AQFdTjdIxNo9ElTf2mxXM6k8YMHXzYUe4yODVaXP4V9uMFbVg8c0qyBccK2OOxb45Q==",
"version": "4.23.13",
"resolved": "https://registry.npmjs.org/tsx/-/tsx-4.23.13.tgz",
"integrity": "sha512-BL5MGkRln6aDYhb0xbQlEAGw743BaZYWdbWtdJOBriYJboKgUUYCadFp2/FpBBZquBC/ezNBn7wMMPx7FDZUDw==",
"dev": true,
"license": "MIT",
"dependencies": {
@@ -6722,6 +7296,24 @@
"punycode": "^2.1.0"
}
},
"node_modules/uuid": {
"version": "8.3.2",
"resolved": "https://registry.npmjs.org/uuid/-/uuid-8.3.2.tgz",
"integrity": "sha512-+NYs2QeMWy+GWFOEm9xnn6HCDp0l7QBD7ml8zLUmJ+93Q5NF0NocErnwkTkXVFNiX3/fpC6afS8Dhb/gz7R7eg==",
"deprecated": "uuid@10 and below is no longer supported. For ESM codebases, update to uuid@latest. For CommonJS codebases, use uuid@11 (but be aware this version will likely be deprecated in 2028).",
"dev": true,
"license": "MIT",
"bin": {
"uuid": "dist/bin/uuid"
}
},
"node_modules/uuid-parse": {
"version": "1.1.0",
"resolved": "https://registry.npmjs.org/uuid-parse/-/uuid-parse-1.1.0.tgz",
"integrity": "sha512-OdmXxA8rDsQ7YpNVbKSJkNzTw2I+S5WsbMDnCtIWSQaosNAcWtFuI/YK1TjzUI6nbkgiqEyh8gWngfcv8Asd9A==",
"dev": true,
"license": "MIT"
},
"node_modules/vite": {
"version": "5.4.21",
"resolved": "https://registry.npmjs.org/vite/-/vite-5.4.21.tgz",
+3 -1
View File
@@ -73,12 +73,14 @@
"zod": "^3.22.4"
},
"devDependencies": {
"@types/autocannon": "^7.12.7",
"@types/bcryptjs": "^2.4.6",
"@types/jsonwebtoken": "^9.0.10",
"@types/luxon": "^3.7.5",
"@types/node": "^20.12.7",
"@typescript-eslint/eslint-plugin": "^7.6.0",
"@typescript-eslint/parser": "^7.6.0",
"autocannon": "^8.0.0",
"eslint": "^8.57.0",
"eslint-config-prettier": "^9.1.0",
"husky": "^9.0.11",
@@ -86,7 +88,7 @@
"prettier": "^3.2.5",
"prisma": "^5.12.1",
"tsc-alias": "^1.9.2",
"tsx": "^4.7.2",
"tsx": "^4.23.13",
"typescript": "^5.4.5",
"vitest": "^1.5.0"
},
@@ -0,0 +1,13 @@
-- AlterTable
ALTER TABLE "sla_runs" ADD COLUMN "version" INTEGER NOT NULL DEFAULT 0;
-- 016-load-concurrency-testing research.md §1: at most one current assignment per ticket,
-- enforced at the database level (a partial unique index, since Prisma's schema DSL cannot
-- express a WHERE-predicated unique constraint directly).
CREATE UNIQUE INDEX "assignments_one_current_per_ticket" ON "assignments"("ticketId") WHERE "isCurrent" = true;
-- 016-load-concurrency-testing research.md §3: a given escalation rule may fire at most once
-- per ticket over that ticket's lifetime (SLARun.ticketId is already @unique — no reopen-cycle
-- support, so a rule-triggered breach genuinely cannot recur for the same ticket). Manual
-- escalations (rule_id IS NULL) are excluded and remain repeatable.
CREATE UNIQUE INDEX "escalation_events_ticket_rule_unique" ON "escalation_events"("ticketId", "ruleId") WHERE "ruleId" IS NOT NULL;
+5
View File
@@ -595,6 +595,11 @@ model SLARun {
completedAt DateTime?
// 016-load-concurrency-testing: optimistic-concurrency counter, identical convention to
// Ticket.version (003-ticketing) — guards pause/resume/complete/the breach sweep against
// racing each other and silently clobbering this run's state (research.md §2).
version Int @default(0)
@@index([status, resolutionDueAt])
@@index([status, firstResponseDueAt])
@@map("sla_runs")
+1 -1
View File
@@ -2,7 +2,7 @@ import { PrismaClient } from '@prisma/client';
export async function seedCategories(prisma: PrismaClient): Promise<void> {
// eslint-disable-next-line no-console
console.log(' -> Seeding baseline product categories...');
console.log(' Seeding baseline product categories...');
const product = await prisma.product.findUnique({
where: { externalProductId: 'CORE_PLATFORM' },
@@ -0,0 +1,80 @@
# Specification Quality Checklist: Load and Concurrency Testing
**Purpose**: Validate specification completeness and quality before proceeding to planning
**Created**: 2026-09-09
**Feature**: [spec.md](../spec.md)
## Content Quality
- [x] No implementation details (languages, frameworks, APIs)
- [x] Focused on user value and business needs
- [x] Written for non-technical stakeholders
- [x] All mandatory sections completed
## Requirement Completeness
- [x] No [NEEDS CLARIFICATION] markers remain
- [x] Requirements are testable and unambiguous
- [x] Success criteria are measurable
- [x] Success criteria are technology-agnostic (no implementation details)
- [x] All acceptance scenarios are defined
- [x] Edge cases are identified
- [x] Scope is clearly bounded
- [x] Dependencies and assumptions identified
## Feature Readiness
- [x] All functional requirements have clear acceptance criteria
- [x] User scenarios cover primary flows
- [x] Feature meets measurable outcomes defined in Success Criteria
- [x] No implementation details leak into specification
## Notes
- This feature was scoped from a targeted codebase audit (not guesswork) confirming which
concurrency guarantees already exist untested (ticket optimistic concurrency) versus which
have no protection at all today (assignment double-assignment, SLA pause/resume, escalation
idempotency) — see spec.md's own Assumptions section.
- Per this project's own roadmap convention, exact load-test pass/fail thresholds are left as an
explicit `OPEN BUSINESS DECISION` (FR-009) rather than invented — this is intentional, not a
gap requiring [NEEDS CLARIFICATION].
- All items pass; no revision iterations were needed.
## Implementation-time findings
- **All three suspected real races were confirmed real, then fixed.** Before the fix, firing 20
genuinely concurrent assignment attempts at the same ticket reliably threw an unhandled
Postgres unique-constraint error once the new `assignments_one_current_per_ticket` partial
index was in place (proving the race existed even before the retry logic was added) — after
the fix (bounded retry with jitter in `AssignmentRepository.createAssignment`), it holds
consistently across 10 repeated runs. Escalation idempotency was proven the same way: the
database-level unique-violation is visibly caught and absorbed in the logs during the test,
confirming the fix actually engages under a genuine race rather than sitting untested.
- **The ticket-status optimistic-concurrency mechanism (User Story 4) needed no fix** — proven
correct on the first run, exactly as research.md's Assumptions predicted.
- **A real, pre-existing test-infrastructure issue was found and resolved along the way**: the
throwaway integration-test Postgres database had accumulated a very large number of tickets
over this project's long development history, and the ticket-code generator's own
documented "rare race between two concurrent creates" (a read-then-increment sequence number
scoped by code prefix) became a frequent occurrence at that accumulated volume — manifesting
as dozens of unrelated integration-test failures when the full suite ran, unrelated to any
change in this feature. Confirmed by direct reproduction (a debug run showing the literal
`Unique constraint failed on the fields: (code)` error) and by re-running the exact same
suite cleanly (122/124 passing, matching the project's known accepted baseline) after
dropping and recreating the throwaway database and replaying its full migration history
(`prisma migrate deploy`, 12 migrations including this feature's own). This is a test-
infrastructure hygiene finding, not a defect in this feature's own code.
- **Two additional integration-test failures seen only in the full-suite run (never in
isolation)** were confirmed to be pre-existing cross-file contamination inherent to this
suite's shared-database, non-fully-isolated hierarchy/agent scoping (already acknowledged in
comments elsewhere in the suite, e.g. sla-escalation-flow.test.ts's own note about a
wildcard SLA policy leaking across concurrently-running files) — re-running the two affected
files together in isolation passed cleanly (13/13), ruling out this feature's own changes as
the cause.
- The autocannon-based load-test tooling (User Story 5) surfaced a real, non-obvious cost
consideration: ticket creation asynchronously triggers a real, billed Anthropic API call for
that ticket's first AI diagnosis turn (005-ai-support) — this applies to both the
ticket-creation and AI-support-flow load scripts, not only the latter as initially assumed.
All three scripts were run once at a small, explicitly bounded scale (confirmed with the
project owner beforehand) rather than an open-ended duration, specifically to keep this real
cost small and predictable.
@@ -0,0 +1,63 @@
# Data Model: Load and Concurrency Testing
All changes below are additive to existing models — no existing column is removed or
retyped, and no existing consumer (012-admin-list-views, 015-reporting-dashboards) needs any
change, since none of them write to `Assignment`/`SLARun`/`EscalationEvent` directly (all writes
already go through the repositories being changed here).
## `SLARun` (existing model, one new field)
| Field | Type | Notes |
|---|---|---|
| `version` | `Int @default(0)` | NEW. Optimistic-concurrency counter, identical convention to `Ticket.version` (003-ticketing). Incremented on every successful `updateWithVersion` call. |
Migration: additive `ALTER TABLE sla_runs ADD COLUMN version INTEGER NOT NULL DEFAULT 0;`
every existing row defaults to `0`, which is exactly the version any in-flight or future
`updateWithVersion` call expects for a run nobody has updated since this migration ran.
## `Assignment` (existing model, no column change — one new index)
New raw partial unique index (Prisma schema DSL cannot express a partial predicate directly, so
this is added via a raw-SQL migration step, same approach already used elsewhere in this
project for Postgres-specific constraints):
```sql
CREATE UNIQUE INDEX assignments_one_current_per_ticket
ON assignments (ticket_id)
WHERE is_current = true;
```
Enforces at the database level: a ticket may have at most one `Assignment` row with
`isCurrent = true` at any moment, closing the race research.md §1 describes. The existing
non-unique `@@index([ticketId, isCurrent])` is unaffected and stays for the repository's own
`findCurrent` lookup.
## `EscalationEvent` (existing model, no column change — one new index)
```sql
CREATE UNIQUE INDEX escalation_events_ticket_rule_unique
ON escalation_events (ticket_id, rule_id)
WHERE rule_id IS NOT NULL;
```
Enforces at the database level: a given rule may fire at most once per ticket over that
ticket's lifetime (manual escalations, where `rule_id IS NULL`, are explicitly excluded and
remain repeatable). Closes the race research.md §3 describes.
## Repository contract changes
### `SlaRunRepository`
- `update(id, data)`**replaced** by `updateWithVersion(id, expectedVersion, data): Promise<SLARun | null>`, mirroring `TicketsRepository.updateStatus`'s exact shape: an atomic `updateMany({where:{id, version: expectedVersion}, data:{...data, version:{increment:1}}})`, returning the fresh row on success (count === 1) or `null` on a stale-version mismatch. Every existing call site (`pause`, `resume`, `complete`, `runBreachDetectionSweep`) is updated to pass its own last-read `version` and to retry (re-read + recompute + re-call) up to 3 times on a `null` result before giving up silently (matching the sweep's own existing no-throw, best-effort style — these are internal transitions with no HTTP caller waiting on a 409).
### `AssignmentRepository`
- `createAssignment(data)` — same signature and return type; internally catches a Prisma `P2002` on the new `assignments_one_current_per_ticket` index and retries the entire transaction (bounded to 3 attempts) before rethrowing.
### `EscalationEventRepository`
- `create(data)` — same signature; internally catches a Prisma `P2002` on the new `escalation_events_ticket_rule_unique` index and returns the pre-existing row for that `(ticketId, ruleId)` pair (a `findFirst({where:{ticketId, ruleId}})` fallback) instead of throwing, so `EscalationService.fire`'s caller sees a normal `EscalationEvent` either way — a duplicate trigger is invisible to the caller, not an error.
## Test-only entities (not persisted — in-memory test scaffolding)
- **Load test report** (`tests/load/`): `{ endpoint: string; connections: number; durationSec: number; requestsPerSec: number; latencyP50Ms: number; latencyP90Ms: number; latencyP99Ms: number; non2xxCount: number; rateLimitedCount: number }` — printed to console and written as JSON under `tests/load/reports/<endpoint>-<timestamp>.json` (gitignored) for each run, satisfying FR-008's separation of rate-limited responses from genuine failures.
+132
View File
@@ -0,0 +1,132 @@
# Implementation Plan: Load and Concurrency Testing
**Branch**: `016-load-concurrency-testing` | **Date**: 2026-09-09 | **Spec**: [spec.md](./spec.md)
**Input**: Feature specification from `/specs/016-load-concurrency-testing/spec.md`
## Summary
Prove — with real, genuinely-concurrent requests against real Postgres/Redis, never mocked
timers — three concurrency guarantees that a prior codebase audit found are NOT currently held
(assignment double-assignment, SLA pause/resume/sweep races, escalation duplicate-event risk),
fix each real race the tests reveal with a minimal, idiomatic DB-level guard consistent with
this codebase's existing patterns, add one new concurrency test proving the existing ticket
optimistic-concurrency guarantee holds under genuine concurrency, and add repeatable
`autocannon`-based HTTP load-test tooling for the three named critical endpoint groups.
## Technical Context
**Language/Version**: TypeScript 5.4 / Node.js >=20
**Primary Dependencies**: Fastify 4.26, Prisma, ioredis/BullMQ, Vitest (existing stack — no new
runtime dependency for the concurrency tests); `autocannon` added as a new devDependency for the
load-test tooling (pure npm package, no external binary, scriptable in TS, matches this
project's existing Node-native toolchain rather than introducing a separate Go binary like k6)
**Storage**: PostgreSQL via Prisma (existing `Assignment`, `SLARun`, `EscalationEvent` models —
one additive schema change per race fix, see data-model.md), Redis (existing, unchanged)
**Testing**: Vitest, run against the existing throwaway Docker Postgres/Redis
(`supporthub-test-pg`/`supporthub-test-redis`) already used by `tests/concurrency/`; load tests
run with `autocannon` against a real running instance of the dev server
**Target Platform**: Linux/Windows server (existing deployment target, unchanged)
**Project Type**: Backend service (existing modular monolith, unchanged)
**Performance Goals**: NEEDS CLARIFICATION resolved in research.md — no business-specified
throughput/latency targets exist yet; FR-009 requires these be marked `OPEN BUSINESS DECISION`
rather than invented, so this feature ships tooling + a baseline report, not a numeric SLA
**Constraints**: Every fix must be additive/backward-compatible (no breaking change to existing
Assignment/SLARun/EscalationEvent consumers — 012-admin-list-views and 015-reporting-dashboards
both already query these tables); every concurrency claim must be proven against real
Docker-provisioned infrastructure per this project's standing verification discipline, never
asserted from code review alone
**Scale/Scope**: 3 real races to prove-and-fix (assignment, SLA, escalation), 1 race to prove
already-safe (ticket status), 3 endpoint groups to load-test (ticket creation, AI support flow,
admin reporting) — entirely within `supporthub-api`, no `supporthub-web` changes
## Constitution Check
*GATE: Must pass before Phase 0 research. Re-check after Phase 1 design.*
| Principle | Check | Status |
|---|---|---|
| I. SaaS Is Sole Identity Authority | N/A — no identity/tenant/product-access logic touched | PASS |
| II. Configuration Over Hardcoding | Load-test pass/fail thresholds are NOT hardcoded — explicitly marked `OPEN BUSINESS DECISION` per FR-009, matching roadmap convention | PASS |
| III. Layered Architecture / Module Boundaries | All three fixes stay inside their owning module (`orchestration/assignments`, `orchestration/sla`, `orchestration/escalation`) — repository-layer changes only, no new cross-module imports | PASS |
| IV. AI Recommends, Policy Decides | N/A — no AI/tool-permission logic touched | PASS |
| V. Evidence-Based Verification | This entire feature IS evidence-based verification — every claimed guarantee must be proven by a real concurrency test against real infra before being considered fixed | PASS (this principle is the feature's own thesis) |
| VI. Durable Audit & History | No audit-log shape changes; EscalationEvent's idempotency fix preserves the existing audit row for the winning attempt, silently no-ops the loser rather than deleting anything | PASS |
| VII. Concurrency-Safe, Durable Job Handling (NON-NEGOTIABLE) | This feature directly implements this principle's own stated requirement ("Assignment and escalation logic MUST be tested under concurrency... job handlers MUST be idempotent") — it is the principle's own overdue test coverage | PASS — this feature exists to close this exact gap |
| VIII. Ticket/Problem Separation | N/A — no Ticket/Problem model changes | PASS |
No violations. No Complexity Tracking entries needed.
## Project Structure
### Documentation (this feature)
```text
specs/016-load-concurrency-testing/
├── plan.md # This file
├── research.md # Phase 0 output
├── data-model.md # Phase 1 output
├── quickstart.md # Phase 1 output
└── tasks.md # Phase 2 output (/speckit-tasks — not yet created)
```
No `contracts/` directory: this feature adds no new HTTP endpoints or request/response
contracts — it hardens existing internal behavior and adds test/tooling infrastructure only.
### Source Code (repository root)
```text
prisma/
└── schema.prisma # +1 field (SLARun.version), +2 raw partial
# unique indexes (migration SQL)
src/modules/orchestration/assignments/
├── repository/assignment.repository.ts # createAssignment: catch+retry on the new
# partial-unique-index conflict
└── ... # (engine/service unchanged)
src/modules/orchestration/sla/
├── repository/sla-run.repository.ts # update() becomes version-checked; add
│ updateWithVersion(id, expectedVersion, data)
└── service/sla.service.ts # pause/resume/complete: read-modify-retry
loop on version conflict (bounded attempts)
src/modules/orchestration/escalation/
├── repository/escalation-event.repository.ts # create(): catch the new partial-unique
│ -index conflict, return existing row
└── service/escalation.service.ts # fire(): treat a duplicate-conflict as a
no-op, not an error
tests/concurrency/
├── round-robin.test.ts # existing — untouched
├── queue.test.ts # existing — untouched
├── assignment-race.test.ts # NEW — User Story 1 / FR-001
├── sla-race.test.ts # NEW — User Story 2 / FR-002
├── escalation-idempotency.test.ts # NEW — User Story 3 / FR-003
└── ticket-status-race.test.ts # NEW — User Story 4 / FR-004
tests/load/
├── autocannon.config.ts # NEW — shared runner + report shape
├── ticket-creation.load.ts # NEW — User Story 5 / FR-007, FR-008
├── ai-support-flow.load.ts # NEW
└── admin-reporting.load.ts # NEW
```
**Structure Decision**: Single backend project (existing `supporthub-api` modular monolith).
Fixes live inside their owning module's existing `repository`/`service` files (Principle III);
new tests live in the existing `tests/concurrency/` directory (already established by
round-robin.test.ts) plus a new `tests/load/` directory for the load-test tooling, mirroring the
existing `tests/{unit,integration,e2e,concurrency}` layout with one new sibling rather than
overloading `tests/concurrency/` with non-correctness-proving load scripts.
## Complexity Tracking
*No violations — table omitted.*
@@ -0,0 +1,85 @@
# Quickstart: Load and Concurrency Testing
Manual + automated verification steps for each user story, against real Docker-provisioned
Postgres/Redis — this project's standing rule that a concurrency claim is never accepted from
code review alone.
## Prerequisites
- Throwaway test infra up: `supporthub-test-pg` (host port 5433), `supporthub-test-redis` (host
port 6380) — the same containers `tests/concurrency/round-robin.test.ts` already uses.
- For the load tests (User Story 5) only: a real running instance of the API against the real
dev infra (`postgres-development`/`redis-development`), reachable at
`http://localhost:4501`, plus an ADMIN session token for the reporting endpoints.
## Scenario 1 — Assignment double-assignment race (User Story 1)
1. `REDIS_HOST=localhost REDIS_PORT=6380 npx vitest run tests/concurrency/assignment-race.test.ts`
2. The test creates one ticket, then fires >=20 concurrent `assignmentEngine.assignToSpecificNode`
(or the equivalent orchestration entry point) calls at it against real Postgres.
3. **Expected**: the test itself queries `assignments` directly afterward and asserts exactly
one row has `is_current = true` for that ticket — not just that one HTTP/service call
"won." Repeat the run at least 10 times (or use the test's own internal repeat loop) to
confirm SC-001's "zero exceptions across 10 repeated runs."
4. Before the fix (research.md §1), this test is expected to fail intermittently; after the
fix, it must pass every time.
## Scenario 2 — SLA pause/resume/sweep race (User Story 2)
1. `REDIS_HOST=localhost REDIS_PORT=6380 npx vitest run tests/concurrency/sla-race.test.ts`
2. The test creates a ticket with an active SLA run, then fires concurrent `pause`/`resume`
calls and a `runBreachDetectionSweep()` pass against the same run.
3. **Expected**: the run's final DB state (`status`, `pausedAt`, `resumedAt`, `breachedAt`,
`firstResponseDueAt`, `resolutionDueAt`) is queried directly and asserted internally
consistent — e.g. never `status: 'paused'` with `pausedAt: null`, never a `breached` run
silently reverted to `running` by a racing `resume`. Repeat per SC-002.
## Scenario 3 — Escalation idempotency (User Story 3)
1. `REDIS_HOST=localhost REDIS_PORT=6380 npx vitest run tests/concurrency/escalation-idempotency.test.ts`
2. The test creates a ticket eligible for a specific escalation rule, then calls
`escalationService.handleBreach` (or `fire` via its real trigger path) twice concurrently for
the identical trigger.
3. **Expected**: exactly one `EscalationEvent` row exists afterward for that `(ticketId,
ruleId)` pair, and exactly one `Assignment` row resulted from it (cross-checking Scenario 1's
own guarantee). Repeat per SC-003.
## Scenario 4 — Ticket status optimistic concurrency proof (User Story 4)
1. `REDIS_HOST=localhost REDIS_PORT=6380 npx vitest run tests/concurrency/ticket-status-race.test.ts`
2. The test creates a ticket at a known status/version, then fires >=20 concurrent
`ticketsRepository.updateStatus` calls all starting from that same version.
3. **Expected**: exactly one call returns the updated ticket; every other call returns `null`
(stale-version signal); the ticket's final DB status matches the one call that succeeded.
This is expected to pass on the very first run (spec.md Assumptions) — a failure here would
mean the existing mechanism has a real gap, not that this quickstart step is wrong.
## Scenario 5 — Load/throughput baseline (User Story 5)
1. Ensure the real dev API is running (`npm run dev` against `.env.development`) and reachable.
2. `npx tsx tests/load/ticket-creation.load.ts`
3. `npx tsx tests/load/ai-support-flow.load.ts`
4. `npx tsx tests/load/admin-reporting.load.ts` (needs an ADMIN token — the script signs in
itself using the same seeded admin credentials this project's E2E suite already uses)
5. **Expected**: each script prints a report (requests/sec, `p50`/`p90`/`p99` latency, non-2xx
count, rate-limited count) and writes it to `tests/load/reports/`. There is no pass/fail
assertion on the numbers themselves (FR-009, `OPEN BUSINESS DECISION`) — the check here is
that the tooling runs cleanly end-to-end and produces a comparable, re-runnable report, not
that any specific number is hit.
6. Run the same script twice in a row and confirm the two reports are comparable in shape
(same fields, plausible numbers) — proving SC-005's "consistent-shape output for comparison
across runs."
## What "done" looks like
- All four new `tests/concurrency/*.test.ts` files pass consistently (not flakily) against real
Postgres/Redis, each proving its own user story's guarantee with a direct database assertion,
not just an HTTP response check.
- Every race the audit found (assignment, SLA, escalation) is fixed in the actual repository
code per data-model.md, not merely detected and left alone.
- All three `tests/load/*.load.ts` scripts run cleanly against a real running dev server and
produce a report.
- Full existing quality gate (typecheck, lint, architecture check, full unit + integration
suite) stays green — these fixes touch shared repositories (`Assignment`, `SLARun`,
`EscalationEvent`) already exercised by 007-orchestration-assignment's, 008-sla-escalation's,
012-admin-list-views's, and 015-reporting-dashboards's own existing tests.
@@ -0,0 +1,153 @@
# Research: Load and Concurrency Testing
## 1. Assignment double-assignment race
**Decision**: Add a PostgreSQL partial unique index — `CREATE UNIQUE INDEX
assignments_one_current_per_ticket ON assignments (ticket_id) WHERE is_current = true;` — and
change `AssignmentRepository.createAssignment` to catch the resulting unique-violation (Prisma
`P2002`) and retry the whole supersede-then-create transaction (bounded to 3 attempts, matching
this codebase's existing small-bounded-retry convention), rather than surfacing a raw 500.
**Rationale**: `createAssignment`'s existing transaction (`updateMany({isCurrent:false}) +
create({isCurrent:true})`) is correct in isolation but Postgres's default `READ COMMITTED`
isolation lets two concurrent transactions each see "no current row to supersede" and both
successfully `create` their own `isCurrent:true` row — there is no read-modify-write cycle a
version field could guard here (unlike Ticket/SLARun below), because the operation is a
create, not an update, and a create can't be conditioned on "no matching row exists" atomically
without a DB-level constraint. A partial unique index is the standard, minimal Postgres pattern
for "at most one row matching a predicate" and requires no application-level locking. Retrying
on conflict (rather than failing the second caller outright) preserves current behavior for the
common, non-racing case and correctly resolves the race by making the loser's request apply
*after* the winner's, superseding it — exactly the same "last write wins, but exactly once"
semantics `createAssignment`'s own docstring already promises for the non-concurrent case.
**Alternatives considered**:
- *Explicit `SERIALIZABLE` transaction isolation*: would also detect the race (as a
serialization failure) but requires the exact same catch-and-retry handling as the unique
index approach, adds latency to every assignment (not just racing ones), and does nothing to
prevent the row from ever being duplicated if a future code path creates an Assignment outside
this transaction — a DB constraint is a stronger, more future-proof guarantee.
- *Row-level lock (`SELECT ... FOR UPDATE`) on a per-ticket lock row*: works, but requires
inventing a new lock-row concept for a case Postgres's own partial unique index already solves
natively.
## 2. SLA pause/resume/sweep race
**Decision**: Add `version Int @default(0)` to `SLARun`. Replace `SlaRunRepository.update(id,
data)` with `updateWithVersion(id, expectedVersion, data)`, mirroring
`TicketsRepository.updateStatus`'s existing atomic `updateMany({where:{id, version:
expectedVersion}, data:{...data, version:{increment:1}}})` pattern exactly. `SlaService.pause`,
`resume`, `complete`, and `runBreachDetectionSweep` each move to a small
read-compute-write-retry loop (bounded to 3 attempts): re-read the run fresh on a version
conflict, recompute the operation's own delta (e.g. resume's `pausedMs` shift) against the fresh
state, and retry the versioned write.
**Rationale**: Every one of `pause`/`resume`/`complete`/the sweep does an unconditional
read-then-`update(run.id, {...})` with no guard — two of these racing (e.g. `resume` and the
sweep evaluating the same run at once) can silently clobber each other: the sweep's own
`update(run.id, {status:'breached', breachedAt: now})` could be overwritten moments later by a
`resume` that read the run *before* the sweep's write and still thinks it's `paused`, un-breaching
a run that was legitimately breached and permanently losing that breach from SLA-compliance
figures — a real, silent correctness bug, not a hypothetical one. `Ticket` already has exactly
this problem solved for its own status field with a `version` counter and an atomic
conditional-update; reusing that identical mechanism (rather than inventing a new one) keeps the
codebase's concurrency idiom singular and matches Principle III's spirit even though it isn't
a cross-module boundary concern.
**Alternatives considered**:
- *Wrap each operation in a Postgres advisory lock keyed by run ID*: works but adds a new
locking primitive to the codebase for a problem the existing version-counter idiom already
solves; rejected for consistency, not because it wouldn't work.
- *A single DB transaction spanning the sweep's read+write for all runs at once*: would only
protect the sweep against itself, not against `pause`/`resume` racing it from an unrelated
request path — doesn't close the actual gap.
## 3. Escalation idempotency
**Decision**: Add a PostgreSQL partial unique index — `CREATE UNIQUE INDEX
escalation_events_ticket_rule_unique ON escalation_events (ticket_id, rule_id) WHERE rule_id IS
NOT NULL;` — and change `EscalationEventRepository.create` (called from
`EscalationService.fire`) to catch the resulting `P2002` and return the already-existing event
for that `(ticketId, ruleId)` pair instead of creating a duplicate or throwing.
**Rationale**: `SLARun.ticketId` is `@unique` and "no reopen-cycle support" (existing schema
comment) means a given rule can only ever legitimately fire once per ticket's lifetime for a
rule-triggered breach (`handleBreach`'s `ruleId` is always a real rule ID scoped to one specific
`triggerType`; `resolution_breach` and `first_response_breach` runs are naturally different
rules, so this constraint doesn't conflate the two). Manual escalation
(`escalateManually`/`fire(ticketId, ruleId: null, ...)`) is deliberately excluded from the
constraint (`WHERE rule_id IS NOT NULL`) because an admin legitimately re-escalating the same
ticket manually more than once must keep working exactly as it does today. This directly closes
the gap the audit found: `runBreachDetectionSweep`'s `findRunningPastResolutionDueAt` can return
the same still-`running` row to two overlapping sweep passes (e.g. a slow sweep still finishing
when the next scheduled tick fires, or a duplicate BullMQ job delivery calling `handleBreach`
directly) before either pass's own `update(run.id, {status:'breached', ...})` commits — without
this constraint, both passes independently call `fire` and each successfully creates its own
`EscalationEvent` plus its own `assignToSpecificNode`.
**Alternatives considered**:
- *A dedicated idempotency-key column populated by the caller (e.g. a sweep-run ID)*: more
general, but overkill here — the natural, already-unique business key for a rule-triggered
escalation genuinely is `(ticketId, ruleId)` given the "no reopen-cycle" constraint already in
place; inventing a separate key would duplicate information the schema already expresses.
- *Making the sweep single-flight via a Redis lock around the whole sweep function*: would
prevent two sweep passes from overlapping, but does not protect against a duplicate BullMQ job
calling `handleBreach` directly for the same trigger outside the sweep's own loop — the DB
constraint protects the actual invariant regardless of caller, which is the correct place per
Principle VII ("job handlers MUST be idempotent").
## 4. Ticket optimistic-concurrency proof
**Decision**: No implementation change. Add `tests/concurrency/ticket-status-race.test.ts`
firing a batch of genuinely concurrent `TicketsRepository.updateStatus` calls at the same
ticket, all from the same starting version, against the real throwaway Postgres, and asserting
exactly one succeeds (returns the updated ticket) while every other call returns `null` (the
existing stale-version-mismatch signal) — proving FR-004/SC-004 against the mechanism that
already exists (see `tests/concurrency/round-robin.test.ts:12`'s own reference to "003-ticketing's
optimistic ticket-status concurrency" as prior art that was never itself concurrency-tested).
**Rationale**: The existing `updateMany({where:{id, version: expectedVersion}, ...})` is a
single atomic SQL statement — Postgres itself guarantees only one concurrent `UPDATE` matching
that `WHERE` clause can succeed before the row's `version` changes underneath the others. This
is sound by construction; the gap is purely "never proven under real concurrency," which this
research assumes will simply confirm the existing guarantee (per spec.md's own Assumptions) —
but the test is still written to fail loudly if that assumption turns out to be wrong.
## 5. Load-test tooling choice
**Decision**: `autocannon` (npm devDependency), invoked via small TypeScript runner scripts
under `tests/load/`, one per named endpoint group (ticket creation, AI support flow, admin
reporting), each producing a JSON report (`autocannon`'s own `Result` shape: requests/sec,
latency `p50`/`p90`/`p99`, non-2xx count) written to `tests/load/reports/` (gitignored — these
are run artifacts, not fixtures) plus a printed console summary.
**Rationale**: `autocannon` is a pure Node.js package (no separate binary to install, unlike
k6), is TypeScript-friendly, and its programmatic API (`autocannon({url, connections, duration,
requests: [...]}, callback)`) fits scripting multi-step flows (e.g. sign-in once, then hammer an
authenticated endpoint) far more naturally than k6's separate-runtime JS dialect — keeping this
feature's new tooling inside the same Node/TS toolchain as the rest of the project (Technical
Context), consistent with this project's existing minimal-new-tooling bias.
**Alternatives considered**:
- *k6*: the industry-standard load-testing tool with richer scripting and threshold
assertions, but ships as a separate Go binary requiring its own install/Docker image outside
npm — heavier footprint for a project whose stack is otherwise 100% npm-managed.
Reconsider if this project later needs distributed/cloud load generation, which `autocannon`
does not support and k6 does.
- *artillery*: also npm-native and closer to k6 in scripting richness, but pulls in a much
larger dependency tree for YAML-driven scenario files this feature doesn't need — `autocannon`
is a lighter fit for three hand-written TS scripts.
## 6. Load-test pass/fail thresholds
**Decision**: Per FR-009, no numeric throughput/latency/error-rate threshold is hardcoded as
pass/fail. Each load-test report prints its own measured numbers and the tooling exits `0`
regardless of the numbers observed (this is a measurement tool, not a gate) — a comment in each
script marks the threshold question as `OPEN BUSINESS DECISION` and links back to spec.md
Assumptions, so a future feature can wire an explicit pass/fail gate into CI once the business
sets a real target.
**Rationale**: Inventing an arbitrary "must handle 500 req/s at p99 < 200ms" number would
violate the roadmap's own explicit rule ("Never hardcode a placeholder value for any of the
[open business decisions] and ship it as if it were final") — throughput/latency targets are
exactly this kind of business-owned number, not an engineering default.
+240
View File
@@ -0,0 +1,240 @@
# Feature Specification: Load and Concurrency Testing
**Feature Branch**: `016-load-concurrency-testing`
**Created**: 2026-09-09
**Status**: Draft
**Input**: User description: "Load and concurrency testing (Phase 11): exercise the concurrency-safety guarantees docs/09-testing-observability-cicd.md's own testing strategy already calls for — assignment race conditions, SLA pause/resume durability, escalation idempotency, ticket optimistic concurrency — under genuinely concurrent requests against real infrastructure, fixing any real race a test reveals; and add real HTTP load/throughput testing against the API's own critical endpoints."
## User Scenarios & Testing *(mandatory)*
### User Story 1 - A ticket is never assigned to two agents at once under concurrent escalation (Priority: P1)
An operator needs confidence that when a ticket is escalated to a human (or reassigned) from
more than one trigger at nearly the same moment — for example, a manual reassignment landing at
the same instant as an automatic escalation-rule firing — the ticket ends up with exactly one
current assignment, never two agents both believing they own the same case.
**Why this priority**: A double-assignment is a customer- and agent-facing correctness failure
(two agents work the same ticket, or the SLA/workload dashboards silently double-count it) and
undermines every dashboard and workload figure already shipped in this system. This is the most
severe class of bug this feature can find.
**Independent Test**: Can be fully tested by firing many genuinely concurrent assignment
requests at the same ticket against a real running instance of the API and a real Postgres
database, then confirming exactly one `Assignment` row is marked current for that ticket
afterward — no reliance on timing assumptions or sequential calls.
**Acceptance Scenarios**:
1. **Given** a ticket eligible for assignment, **When** many concurrent assignment attempts are
made against it at once, **Then** exactly one assignment ends up marked as the ticket's
current assignment, and the database itself (not just the last response received) confirms
this.
2. **Given** the race in Scenario 1 is exercised repeatedly, **When** the test is run multiple
times, **Then** the result is consistent every time — the protection does not depend on
lucky timing.
---
### User Story 2 - An SLA clock is never corrupted by overlapping pause/resume activity (Priority: P1)
An operator needs confidence that when a ticket's SLA clock is paused and resumed by more than
one concurrent trigger — for example, a customer-reply webhook resuming the clock at the same
moment the scheduled breach-detection sweep is evaluating that same ticket — the SLA run ends up
in one coherent, correct state, never a state where the clock is simultaneously "paused" and
"counting toward breach," and never a state that silently drops a pause/resume event.
**Why this priority**: SLA correctness is a contractual promise to customers and already backs
the Management and Support dashboards shipped in 015-reporting-dashboards; a corrupted SLA clock
produces wrong compliance figures and wrong breach alerts without any visible error.
**Independent Test**: Can be fully tested by firing concurrent pause and resume operations at
the same SLA run against a real running instance of the API and a real Postgres database, then
confirming the run's final stored state (paused/active, due-at timestamps) is internally
consistent and matches one coherent ordering of the operations — not a mix of both.
**Acceptance Scenarios**:
1. **Given** an active SLA run, **When** a pause and a resume are triggered concurrently,
**Then** the run's final state is exactly one of "paused" or "active" — never a state with
contradictory fields (e.g., marked paused with no pause timestamp recorded, or marked active
with a stale due-at that never accounted for the pause).
2. **Given** the breach-detection sweep is evaluating a run at the same moment a resume is
requested for it, **When** both complete, **Then** the run is not double-processed (no
duplicate breach event, no lost resume).
---
### User Story 3 - An escalation rule firing twice never creates two escalation events (Priority: P1)
An operator needs confidence that if the same escalation trigger is delivered more than once —
for example, a retried background job or a re-processed event — the ticket is escalated exactly
once, not reassigned and re-notified redundantly.
**Why this priority**: Duplicate escalations would double-notify agents, double-count in the
Support and Management dashboards, and could re-trigger reassignment away from an agent who has
already started work — a direct regression of work already done in this session.
**Independent Test**: Can be fully tested by firing the same escalation trigger concurrently
more than once for the same ticket against a real running instance of the API and a real
Postgres database, then confirming only one `EscalationEvent` row exists for that trigger
afterward.
**Acceptance Scenarios**:
1. **Given** a ticket eligible for escalation, **When** the same escalation trigger is delivered
twice at nearly the same moment, **Then** exactly one escalation event is recorded for it.
2. **Given** Scenario 1's duplicate delivery, **When** the escalation event is created,
**Then** the ticket is reassigned exactly once, not twice.
---
### User Story 4 - A ticket's status can never be corrupted by two simultaneous updates (Priority: P2)
An operator needs confidence that the ticket status-transition safeguard already built for this
system actually holds under real concurrent load, not just in isolated sequential tests — this
is existing protection, but has never been proven under genuine concurrency.
**Why this priority**: Lower priority than User Stories 1-3 because a real defensive mechanism
already exists here (see Assumptions); this story exists to convert an untested assumption into
a proven guarantee, and is valuable but lower-risk than the three unguarded races above.
**Independent Test**: Can be fully tested by firing multiple concurrent status-update attempts
at the same ticket, each based on the same starting version, against a real running API and
database, then confirming exactly one update succeeds and every other attempt receives a clear
conflict response rather than silently corrupting or skipping the ticket's state.
**Acceptance Scenarios**:
1. **Given** a ticket at a known status and version, **When** multiple concurrent status-update
requests are made from that same version, **Then** exactly one succeeds and the rest are
rejected with a conflict response, and the ticket's final status matches the one update that
succeeded.
---
### User Story 5 - The API's critical endpoints hold up under realistic concurrent traffic (Priority: P2)
An operator needs a documented, repeatable measurement of how the system's most important
endpoints — new support requests coming in, the AI support flow, and the admin reporting
dashboards — behave under sustained concurrent load, so that a future capacity or performance
regression can be caught by comparing against this baseline rather than guessed at.
**Why this priority**: This is about establishing a measurable baseline and repeatable tooling
rather than proving or fixing a specific correctness bug (unlike User Stories 1-4), so it is
valuable but not blocking for the correctness guarantees above.
**Independent Test**: Can be fully tested by running a load-test tool against a real running
instance of the API for each of the three named endpoint groups and producing a report of
throughput, latency percentiles, and error rate, independent of whether any other user story in
this feature has been completed.
**Acceptance Scenarios**:
1. **Given** the API is running against real infrastructure, **When** a defined concurrent load
is sent to the ticket-creation endpoint for a sustained period, **Then** a report is produced
showing throughput, latency percentiles, and error rate for that run.
2. **Given** the same setup, **When** the same load profile is sent to the AI support flow and
to the admin reporting endpoints, **Then** an equivalent report is produced for each,
allowing the three to be compared against each other and against future runs.
### Edge Cases
- What happens when a concurrent assignment race includes a ticket that is simultaneously being
closed or reopened? The assignment/reassignment safeguard must not be bypassable by a
status change racing the same window.
- What happens when a pause and a breach both become due at the exact same instant? The final
state must reflect one coherent, auditable outcome, not an unresolvable both-happened state.
- What happens when the load test itself pushes an endpoint into its own rate limiter (e.g. the
013-auth-hardening login rate limit, or the SaaS integration per-minute rate limits)? The
report must distinguish "rejected by design (rate limit)" from "failed under load" rather than
counting both as the same kind of failure.
- What happens when two of these races are exercised back-to-back against the same throwaway
database without cleanup? Each test must use its own uniquely-identified fixtures so repeated
runs (and CI re-runs) don't produce false positives or false negatives from leftover state.
## Requirements *(mandatory)*
### Functional Requirements
- **FR-001**: The system MUST guarantee that a ticket never has more than one assignment marked
as current, even when multiple assignment operations are attempted concurrently against it.
- **FR-002**: The system MUST guarantee that an SLA run's paused/active state and its associated
timestamps remain internally consistent when pause, resume, and the breach-detection sweep are
triggered concurrently against the same run.
- **FR-003**: The system MUST guarantee that the same escalation trigger delivered more than
once for the same ticket produces exactly one escalation event and exactly one resulting
reassignment.
- **FR-004**: The system MUST reject a ticket status update whose expected starting version no
longer matches the ticket's actual current version, even when the conflicting updates are
concurrent, and MUST leave the ticket in the state produced by whichever single update
actually succeeded.
- **FR-005**: The system's automated test suite MUST include a dedicated concurrency test for
each of FR-001 through FR-004, each exercising genuinely concurrent requests against real,
live infrastructure (not mocked timers or sequential calls standing in for concurrency).
- **FR-006**: Where a concurrency test written for this feature reveals that a guarantee in
FR-001, FR-002, or FR-003 does not currently hold, the underlying race MUST be fixed as part
of this feature, not merely documented.
- **FR-007**: The system MUST provide repeatable load-test tooling covering, at minimum: new
support request submission, the AI support flow, and the admin reporting dashboard endpoints.
- **FR-008**: Each load test run MUST produce a report including throughput, latency
percentiles, and error rate, with rate-limited responses reported separately from failures.
- **FR-009**: Pass/fail thresholds for the load tests (target throughput, acceptable latency,
acceptable error rate) MUST be explicitly marked as `OPEN BUSINESS DECISION` wherever the
business has not already specified a number, per this project's own roadmap convention —
never hardcoded as if final.
### Key Entities
- **Assignment race scenario**: A reusable test setup representing "many concurrent attempts to
assign or reassign the same ticket," used to exercise FR-001.
- **SLA race scenario**: A reusable test setup representing "concurrent pause, resume, and sweep
activity against the same SLA run," used to exercise FR-002.
- **Escalation race scenario**: A reusable test setup representing "the same escalation trigger
delivered more than once for the same ticket," used to exercise FR-003.
- **Load test report**: The recorded output of a load-test run against one endpoint group —
throughput, latency percentiles, error rate, and rate-limited-response count — kept so a
future run can be compared against it.
## Success Criteria *(mandatory)*
### Measurable Outcomes
- **SC-001**: A test run that fires at least 20 genuinely concurrent assignment attempts at the
same ticket always results in exactly one current assignment, with zero exceptions across at
least 10 repeated runs.
- **SC-002**: A test run that fires concurrent pause/resume/sweep activity against the same SLA
run always leaves that run in one internally-consistent, auditable state, with zero
contradictory-state outcomes across at least 10 repeated runs.
- **SC-003**: A test run that delivers the same escalation trigger twice for the same ticket
always results in exactly one escalation event and exactly one reassignment, with zero
duplicate outcomes across at least 10 repeated runs.
- **SC-004**: A test run that fires at least 20 genuinely concurrent status-update attempts from
the same starting version against the same ticket always results in exactly one success and
the ticket left in that one succeeding state.
- **SC-005**: A load-test report exists for each of the three named endpoint groups (ticket
creation, AI support flow, admin reporting), each independently re-runnable on demand and
producing consistent-shape output for comparison across runs.
## Assumptions
- Ticket status optimistic concurrency (User Story 4) already has a real defensive mechanism in
the codebase (a version-checked atomic update) — this feature's job for that story is to prove
it under genuine concurrency with a new test, not to build new protection, unless that test
surprises this assumption and reveals a real gap.
- Assignment double-assignment, SLA pause/resume races, and escalation duplicate-event risk (User
Stories 1-3) are NOT currently guarded against — this feature's job for those stories is both
to prove the gap with a real concurrency test and to implement the fix, per FR-006.
- "Genuinely concurrent" means real parallel requests issued against a real running instance of
the API backed by real Postgres/Redis (this project's standing verification discipline
throughout every prior feature), not fake-timer or mocked-clock simulations.
- Load testing (User Story 5) targets the existing dev/throwaway infrastructure already used for
this project's own manual verification, not a separate staging or production environment —
provisioning a dedicated load-test environment is out of scope.
- Specific throughput/latency/error-rate thresholds for "pass" are an `OPEN BUSINESS DECISION`
per FR-009; this feature delivers the tooling and a baseline report, not a final SLA number.
- Round-robin assignment-selection counter safety is already covered by an existing genuine
concurrency test and is explicitly out of scope for this feature.
+221
View File
@@ -0,0 +1,221 @@
---
description: "Task list for 016-load-concurrency-testing"
---
# Tasks: Load and Concurrency Testing
**Input**: Design documents from `specs/016-load-concurrency-testing/`
**Organization**: Tasks are grouped by user story (US1 = assignment race, US2 = SLA race, US3 =
escalation idempotency, US4 = ticket-status race proof, US5 = load-test tooling). US1-US4 share
one Foundational phase (the schema migration all four rely on); US5 has no schema dependency and
can proceed independently of it.
## Format: `[ID] [P?] [Story] Description`
All file paths are relative to `supporthub-api/` (repo root).
---
## Phase 1: Setup
- [x] T001 [P] Add `autocannon` as a devDependency (`package.json`) and add
`tests/load/reports/` to `.gitignore` (run artifacts, not fixtures)
---
## Phase 2: Foundational (Blocking Prerequisites for US1-US4)
**Purpose**: The one shared schema migration US1, US2, and US3's fixes each depend on. US4 (no
schema change, see research.md §4) and US5 (no schema dependency) do not need this phase and can
proceed in parallel with it.
- [x] T002 In `prisma/schema.prisma`, add `version Int @default(0)` to `SLARun`; generate one
migration (`npx prisma migrate dev --name concurrency_guards`) that also includes, as raw
SQL, `CREATE UNIQUE INDEX assignments_one_current_per_ticket ON assignments (ticket_id)
WHERE is_current = true;` and `CREATE UNIQUE INDEX escalation_events_ticket_rule_unique ON
escalation_events (ticket_id, rule_id) WHERE rule_id IS NOT NULL;` (data-model.md); apply
to the throwaway test Postgres (`supporthub-test-pg`, port 5433) and the real dev Postgres
(`postgres-development`, port 5434, via `prisma migrate diff` + direct `psql` per this
project's own established non-destructive dev-sync approach); regenerate the Prisma client
**Checkpoint**: Schema ready — US1, US2, US3 implementation can now begin.
---
## Phase 3: User Story 1 - Assignment never double-assigned under concurrency (Priority: P1)
**Goal**: Two concurrent assignment attempts on the same ticket always leave exactly one current
assignment.
**Independent Test**: Run `tests/concurrency/assignment-race.test.ts` alone against the
throwaway Postgres — it creates its own ticket and needs nothing from US2-US5.
- [x] T003 [US1] Write `tests/concurrency/assignment-race.test.ts`: create one ticket, fire
>=20 genuinely concurrent assignment attempts at it (via the real assignment
engine/service entry point, not the repository directly), then query `assignments`
directly and assert exactly one row has `is_current = true` for that ticket (depends on
T002)
- [x] T004 [US1] Fix `AssignmentRepository.createAssignment` in
`src/modules/orchestration/assignments/repository/assignment.repository.ts` to catch the
`assignments_one_current_per_ticket` unique-violation (Prisma `P2002`) and retry the whole
supersede-then-create transaction, bounded to 3 attempts, per research.md §1 (depends on
T002)
- [x] T005 [US1] Re-run `assignment-race.test.ts` at least 10 times in a row (or extend the test
with its own internal repeat loop) confirming zero failures — SC-001 (depends on T003, T004)
**Checkpoint**: Quickstart Scenario 1 passes against real infrastructure, consistently.
---
## Phase 4: User Story 2 - SLA clock never corrupted by overlapping pause/resume/sweep (Priority: P1)
**Goal**: Concurrent pause/resume/breach-sweep activity against the same SLA run always leaves
it in one internally-consistent state.
**Independent Test**: Run `tests/concurrency/sla-race.test.ts` alone against the throwaway
Postgres — it creates its own ticket + SLA run and needs nothing from US1/US3/US4/US5.
- [x] T006 [US2] Replace `SlaRunRepository.update` with `updateWithVersion(id, expectedVersion,
data)` in `src/modules/orchestration/sla/repository/sla-run.repository.ts`, mirroring
`TicketsRepository.updateStatus`'s atomic `updateMany({where:{id, version:
expectedVersion}, data:{...data, version:{increment:1}}})` pattern exactly (depends on T002)
- [x] T007 [US2] Update `pause`, `resume`, `complete`, and `runBreachDetectionSweep` in
`src/modules/orchestration/sla/service/sla.service.ts` to call `updateWithVersion` with
each run's last-read version, and to re-read + recompute + retry (bounded to 3 attempts)
on a version-conflict `null` result, per research.md §2 (depends on T006)
- [x] T008 [US2] Write `tests/concurrency/sla-race.test.ts`: create a ticket with an active SLA
run, fire concurrent `pause`/`resume` calls and a `runBreachDetectionSweep()` pass against
it, then query the run directly and assert its final state is internally consistent (never
`paused` with `pausedAt: null`, never a legitimately `breached` run silently reverted to
`running`) (depends on T007)
- [x] T009 [US2] Re-run `sla-race.test.ts` at least 10 times confirming zero
contradictory-state outcomes — SC-002 (depends on T008)
**Checkpoint**: Quickstart Scenario 2 passes against real infrastructure, consistently.
---
## Phase 5: User Story 3 - An escalation trigger fired twice never duplicates (Priority: P1)
**Goal**: The same escalation trigger delivered twice for the same ticket always results in
exactly one escalation event and one reassignment.
**Independent Test**: Run `tests/concurrency/escalation-idempotency.test.ts` alone against the
throwaway Postgres — it creates its own ticket + escalation rule and needs nothing from
US1/US2/US4/US5 (though it exercises the same `Assignment` table US1 protects, as a
cross-check).
- [x] T010 [US3] Fix `EscalationEventRepository.create` in
`src/modules/orchestration/escalation/repository/escalation-event.repository.ts` to catch
the `escalation_events_ticket_rule_unique` unique-violation (Prisma `P2002`) and return
the pre-existing row for that `(ticketId, ruleId)` pair via a `findFirst` fallback instead
of throwing, per research.md §3 (depends on T002)
- [x] T011 [US3] Confirm `EscalationService.fire` in
`src/modules/orchestration/escalation/service/escalation.service.ts` behaves correctly
when `create` returns a pre-existing event (it must not also re-run
`assignToSpecificNode` for a duplicate trigger) — adjust `fire` if needed so a
duplicate-conflict short-circuits before reassignment (depends on T010)
- [x] T012 [US3] Write `tests/concurrency/escalation-idempotency.test.ts`: create a ticket
eligible for a specific escalation rule, call the real trigger path (e.g.
`escalationService.handleBreach`) twice concurrently for the identical trigger, then query
`escalation_events` and `assignments` directly and assert exactly one of each resulted
(depends on T011)
- [x] T013 [US3] Re-run `escalation-idempotency.test.ts` at least 10 times confirming zero
duplicate outcomes — SC-003 (depends on T012)
**Checkpoint**: Quickstart Scenario 3 passes against real infrastructure, consistently.
---
## Phase 6: User Story 4 - Ticket status optimistic concurrency, proven (Priority: P2)
**Goal**: Prove the existing version-checked ticket-status update holds under genuine
concurrency.
**Independent Test**: Run `tests/concurrency/ticket-status-race.test.ts` alone against the
throwaway Postgres — no dependency on T002 or any other user story (research.md §4: no
implementation change expected).
- [x] T014 [US4] Write `tests/concurrency/ticket-status-race.test.ts`: create a ticket at a
known status/version, fire >=20 genuinely concurrent `ticketsRepository.updateStatus`
calls all starting from that same version, and assert exactly one returns the updated
ticket while every other call returns `null` — SC-004
**Checkpoint**: Quickstart Scenario 4 passes, confirming the existing mechanism (no fix
expected; a failure here would mean research.md's assumption was wrong and needs revisiting).
---
## Phase 7: User Story 5 - Repeatable load/throughput baseline (Priority: P2)
**Goal**: Repeatable `autocannon`-based load-test tooling and a baseline report for the three
named critical endpoint groups.
**Independent Test**: Run each `tests/load/*.load.ts` script alone against a real running dev
server — no dependency on T002 or any other user story.
- [x] T015 [P] [US5] Create `tests/load/autocannon.config.ts`: a shared runner helper wrapping
`autocannon`'s programmatic API, producing the report shape from data-model.md
(`requestsPerSec`, `latencyP50Ms`/`P90Ms`/`P99Ms`, `non2xxCount`, `rateLimitedCount`),
printing a console summary and writing JSON to `tests/load/reports/` (depends on T001)
- [x] T016 [P] [US5] Create `tests/load/ticket-creation.load.ts` using the T015 helper against
`POST /v1/support/requests` (depends on T015)
- [x] T017 [P] [US5] Create `tests/load/ai-support-flow.load.ts` using the T015 helper against
the AI support flow's own endpoints (depends on T015)
- [x] T018 [P] [US5] Create `tests/load/admin-reporting.load.ts` using the T015 helper, signing
in as the seeded admin first, against the 015-reporting-dashboards endpoints (depends on
T015)
- [x] T019 [US5] Run all three scripts against a real running dev server, confirm each produces
a report, and run each twice to confirm consistent-shape output for comparison — SC-005
(depends on T016, T017, T018)
**Checkpoint**: Quickstart Scenario 5 passes; a baseline report exists for each endpoint group.
---
## Phase 8: Polish & Cross-Cutting Concerns
- [x] T020 Update `specs/016-load-concurrency-testing/checklists/requirements.md` Notes with any
implementation-time findings
- [x] T021 `npx tsc --noEmit` / `npm run lint` / `npx tsx scripts/check-architecture.ts` clean
- [x] T022 Full existing unit + integration + concurrency suite re-run (throwaway DB), confirming
no regression in 007-orchestration-assignment's, 008-sla-escalation's,
012-admin-list-views's, and 015-reporting-dashboards's own existing coverage of
`Assignment`/`SLARun`/`EscalationEvent`
- [x] T023 Mark all of this file's checkboxes complete once verified
---
## Dependencies & Execution Order
- **Setup (Phase 1)**: No dependencies — can start immediately
- **Foundational (Phase 2)**: No dependencies — BLOCKS User Stories 1, 2, 3 only
- **User Story 4**: No dependency on Phase 2 or any other story — can start immediately
- **User Story 5**: No dependency on Phase 2 or any other story — can start immediately (only
needs Phase 1's `autocannon` devDependency)
- **User Stories 1, 2, 3**: Each depends only on Phase 2 — independent of each other and of
User Stories 4/5
- **Polish (Phase 8)**: Depends on all five user stories
## Parallel Example: Foundational-independent stories
```text
# Once Phase 1 completes, these can start immediately in parallel, without waiting on Phase 2:
Task: "Write tests/concurrency/ticket-status-race.test.ts" (US4, T014)
Task: "Create tests/load/autocannon.config.ts" (US5, T015)
```
## Implementation Strategy
### Suggested order
1. Phase 1 (Setup) and Phase 2 (Foundational) — Phase 2 unblocks the three highest-severity
real-bug fixes (US1, US2, US3)
2. User Stories 1, 2, 3 (all P1) — each is a real, currently-unguarded race; fix and prove each
in turn, or in parallel across files since they touch different modules
3. User Story 4 (P2) — quick to add, proves existing protection, can be done any time after
Phase 1
4. User Story 5 (P2) — independent tooling work, can be done any time after Phase 1, in parallel
with 1-4
5. Phase 8 (Polish) once all five stories are verified
@@ -8,6 +8,24 @@ export interface CreateAssignmentData {
reason?: string | undefined;
}
// Bounded, but generous: under N genuinely concurrent attempts on the same ticket, a given
// attempt can collide with a different still-in-flight one on each of several retries before
// the field of contenders drains — 3 was observed to be too few under a 20-way race in this
// project's own concurrency test (tests/concurrency/assignment-race.test.ts).
const MAX_CREATE_ASSIGNMENT_ATTEMPTS = 20;
function isCurrentAssignmentConflict(error: unknown): boolean {
return (
error instanceof Prisma.PrismaClientKnownRequestError &&
error.code === 'P2002' &&
(error.meta?.target as string[] | undefined)?.includes('ticketId') === true
);
}
function jitterDelayMs(): number {
return Math.floor(Math.random() * 15);
}
export class AssignmentRepository {
constructor(private readonly prisma = prismaClient) {}
@@ -16,23 +34,42 @@ export class AssignmentRepository {
* supersedes any existing current row for this ticket (isCurrent: false, unassignedAt: now())
* and inserts the new current row — the same "never overwrite, always a new row" guarantee
* 004's KnowledgeEntry versioning already established for a different entity.
*
* 016-load-concurrency-testing research.md §1: at Postgres's default READ COMMITTED
* isolation, two concurrent calls can each see "nothing current to supersede" and both
* attempt to `create` their own `isCurrent: true` row. The `assignments_one_current_per_ticket`
* partial unique index (migration 20260909120000) makes the second one fail fast with `P2002`
* instead of silently succeeding — caught here and retried (bounded) so the loser's own
* request still applies, correctly superseding the winner's row on the next attempt, rather
* than surfacing a raw conflict to a caller that did nothing wrong.
*/
async createAssignment(data: CreateAssignmentData): Promise<Assignment> {
return this.prisma.$transaction(async (tx) => {
await tx.assignment.updateMany({
where: { ticketId: data.ticketId, isCurrent: true },
data: { isCurrent: false, unassignedAt: new Date() },
});
return tx.assignment.create({
data: {
ticketId: data.ticketId,
agentId: data.agentId,
strategy: data.strategy,
reason: data.reason,
isCurrent: true,
} as Prisma.AssignmentUncheckedCreateInput,
});
});
for (let attempt = 1; attempt <= MAX_CREATE_ASSIGNMENT_ATTEMPTS; attempt++) {
try {
return await this.prisma.$transaction(async (tx) => {
await tx.assignment.updateMany({
where: { ticketId: data.ticketId, isCurrent: true },
data: { isCurrent: false, unassignedAt: new Date() },
});
return tx.assignment.create({
data: {
ticketId: data.ticketId,
agentId: data.agentId,
strategy: data.strategy,
reason: data.reason,
isCurrent: true,
} as Prisma.AssignmentUncheckedCreateInput,
});
});
} catch (error) {
if (!isCurrentAssignmentConflict(error) || attempt === MAX_CREATE_ASSIGNMENT_ATTEMPTS) {
throw error;
}
await new Promise((resolve) => setTimeout(resolve, jitterDelayMs()));
}
}
/* istanbul ignore next -- unreachable: the loop above always returns or throws */
throw new Error('createAssignment: exhausted retry attempts unexpectedly.');
}
async findCurrent(ticketId: string): Promise<Assignment | null> {
@@ -10,13 +10,54 @@ export interface CreateEscalationEventData {
triggeredBy: string;
}
export interface CreateEscalationEventResult {
event: EscalationEvent;
/** False when `create` returned a pre-existing event instead of inserting a new one — the
* caller (EscalationService.fire) uses this to skip re-running side effects (reassignment,
* the audit event-bus publish) for a duplicate trigger. */
wasNewlyCreated: boolean;
}
function isDuplicateRuleEscalationConflict(error: unknown): boolean {
return (
error instanceof Prisma.PrismaClientKnownRequestError &&
error.code === 'P2002' &&
(error.meta?.target as string[] | undefined)?.includes('ruleId') === true
);
}
export class EscalationEventRepository {
constructor(private readonly prisma = prismaClient) {}
async create(data: CreateEscalationEventData): Promise<EscalationEvent> {
return this.prisma.escalationEvent.create({
data: data as Prisma.EscalationEventUncheckedCreateInput,
});
/**
* 016-load-concurrency-testing research.md §3: a rule-triggered escalation
* (`data.ruleId` set) can only ever legitimately fire once per ticket over that ticket's
* lifetime (SLARun.ticketId is @unique — no reopen-cycle support). The
* `escalation_events_ticket_rule_unique` partial unique index (migration 20260909120000)
* enforces this at the database level; a duplicate trigger (e.g. two overlapping breach-sweep
* passes, or a re-delivered job) fails with `P2002` here, and this method returns the
* pre-existing event instead of throwing — the duplicate is silently absorbed, never
* surfaced as an error to a caller that did nothing wrong. Manual escalations (`ruleId: null`)
* are unaffected and always insert a new row.
*/
async create(data: CreateEscalationEventData): Promise<CreateEscalationEventResult> {
try {
const event = await this.prisma.escalationEvent.create({
data: data as Prisma.EscalationEventUncheckedCreateInput,
});
return { event, wasNewlyCreated: true };
} catch (error) {
if (!isDuplicateRuleEscalationConflict(error)) throw error;
const existing = await this.prisma.escalationEvent.findFirst({
where: {
ticketId: data.ticketId,
...(data.ruleId !== undefined ? { ruleId: data.ruleId } : {}),
},
});
if (!existing) throw error; // conflict raced with a delete — surface the original error.
return { event: existing, wasNewlyCreated: false };
}
}
async findAllForTicket(ticketId: string): Promise<EscalationEvent[]> {
@@ -76,7 +76,7 @@ export class EscalationService {
actor: string,
reason: string,
): Promise<EscalationEvent> {
const event = await this.events.create({
const { event, wasNewlyCreated } = await this.events.create({
ticketId,
ruleId,
// No existing model persists "which hierarchy node is this ticket currently in" — Assignment
@@ -88,6 +88,12 @@ export class EscalationService {
triggeredBy: actor,
});
// 016-load-concurrency-testing research.md §3/FR-003: a duplicate rule-triggered trigger
// (the repository already detected and absorbed it) must not also reassign or re-publish —
// both already happened for the winning attempt; doing them again would be the exact
// duplicate-side-effect bug this feature exists to close.
if (!wasNewlyCreated) return event;
await this.assignments.assignToSpecificNode(ticketId, targetNodeId, actor, reason);
// research.md "SLA_BREACHED/ESCALATION_TRIGGERED are also published, for audit, not for
@@ -21,8 +21,26 @@ export class SlaRunRepository {
return this.prisma.sLARun.findUnique({ where: { ticketId } });
}
async update(id: string, data: Prisma.SLARunUpdateInput): Promise<SLARun> {
return this.prisma.sLARun.update({ where: { id }, data });
/**
* 016-load-concurrency-testing research.md §2: optimistic concurrency, identical shape to
* TicketsRepository.updateStatus (003-ticketing) — an atomic single-statement `updateMany`
* conditioned on the row's `version` still matching `expectedVersion`. Replaces the old plain
* `update(id, data)`, which let two racing callers (e.g. `resume` and the breach sweep
* evaluating the same run at once) silently clobber each other's writes. Returns `null` on a
* stale-version mismatch — the caller re-reads and retries, exactly like `TicketsService`
* already does for ticket-status conflicts.
*/
async updateWithVersion(
id: string,
expectedVersion: number,
data: Prisma.SLARunUpdateInput,
): Promise<SLARun | null> {
const result = await this.prisma.sLARun.updateMany({
where: { id, version: expectedVersion },
data: { ...data, version: { increment: 1 } } as Prisma.SLARunUncheckedUpdateManyInput,
});
if (result.count === 0) return null;
return this.prisma.sLARun.findUnique({ where: { id } });
}
/** research.md "Breach detection — one repeatable BullMQ job": every running run whose
@@ -98,13 +98,28 @@ export class SlaService {
});
}
// 016-load-concurrency-testing research.md §2: pause/resume/complete each read-then-write a
// run with no guard, so two of them racing (or one racing the sweep below) could silently
// clobber each other — e.g. a resume reading a run just before the sweep marks it breached,
// then overwriting that breach back to 'running' moments later. Bounded to a handful of
// attempts, re-reading fresh state each time (mirroring TicketsService's own version-conflict
// handling), so a losing attempt still applies correctly against the winner's result instead
// of being silently dropped or corrupting state.
private static readonly MAX_UPDATE_ATTEMPTS = 5;
/** FR-007: pausing on WAITING_FOR_CUSTOMER records pausedAt and flips status — no-ops if
* there's no run or it isn't currently running. */
async pause(ticketId: string): Promise<void> {
const run = await this.runs.findByTicketId(ticketId);
if (!run || run.status !== 'running') return;
for (let attempt = 1; attempt <= SlaService.MAX_UPDATE_ATTEMPTS; attempt++) {
const run = await this.runs.findByTicketId(ticketId);
if (!run || run.status !== 'running') return;
await this.runs.update(run.id, { status: 'paused', pausedAt: new Date() });
const updated = await this.runs.updateWithVersion(run.id, run.version, {
status: 'paused',
pausedAt: new Date(),
});
if (updated) return;
}
}
/**
@@ -113,38 +128,49 @@ export class SlaService {
* durability mechanism (no separate remaining-minutes bookkeeping, no in-memory state).
*/
async resume(ticketId: string): Promise<void> {
const run = await this.runs.findByTicketId(ticketId);
if (!run || run.status !== 'paused' || !run.pausedAt) return;
for (let attempt = 1; attempt <= SlaService.MAX_UPDATE_ATTEMPTS; attempt++) {
const run = await this.runs.findByTicketId(ticketId);
if (!run || run.status !== 'paused' || !run.pausedAt) return;
const pausedMs = Date.now() - run.pausedAt.getTime();
await this.runs.update(run.id, {
status: 'running',
pausedAt: null,
resumedAt: new Date(),
firstResponseDueAt: run.firstResponseDueAt
? new Date(run.firstResponseDueAt.getTime() + pausedMs)
: null,
resolutionDueAt: run.resolutionDueAt
? new Date(run.resolutionDueAt.getTime() + pausedMs)
: null,
});
const pausedMs = Date.now() - run.pausedAt.getTime();
const updated = await this.runs.updateWithVersion(run.id, run.version, {
status: 'running',
pausedAt: null,
resumedAt: new Date(),
firstResponseDueAt: run.firstResponseDueAt
? new Date(run.firstResponseDueAt.getTime() + pausedMs)
: null,
resolutionDueAt: run.resolutionDueAt
? new Date(run.resolutionDueAt.getTime() + pausedMs)
: null,
});
if (updated) return;
}
}
/** FR-010: a run that resolves before its due date is marked completed and is never later
* flagged breached (the breach sweep only ever looks at status: 'running' runs). */
async complete(ticketId: string): Promise<void> {
const run = await this.runs.findByTicketId(ticketId);
if (!run || run.status === 'completed') return;
for (let attempt = 1; attempt <= SlaService.MAX_UPDATE_ATTEMPTS; attempt++) {
const run = await this.runs.findByTicketId(ticketId);
if (!run || run.status === 'completed') return;
// 014-full-observability data-model.md #6: read BEFORE the update below — a run already
// 'breached' by the time it resolves was already counted breached by the sweep and must
// never also be counted 'met' here, even though this update still (pre-existing behavior,
// unrelated to this feature — see research.md §5) overwrites its status to 'completed'.
if (run.status !== 'breached') {
slaRunOutcomesCounter.inc({ outcome: 'met' });
const updated = await this.runs.updateWithVersion(run.id, run.version, {
status: 'completed',
completedAt: new Date(),
});
if (updated) {
// 014-full-observability data-model.md #6: gated on this same successful transition's
// pre-update status (not re-read afterward) — a run already 'breached' by the time it
// resolves was already counted breached by the sweep and must never also be counted
// 'met' here, even though this update still (pre-existing behavior, unrelated to this
// feature — see research.md §5) overwrites its status to 'completed'.
if (run.status !== 'breached') {
slaRunOutcomesCounter.inc({ outcome: 'met' });
}
return;
}
}
await this.runs.update(run.id, { status: 'completed', completedAt: new Date() });
}
/**
@@ -153,13 +179,23 @@ export class SlaService {
* worker process needed to invoke it, tests call this directly). Marks resolution breaches
* (status -> breached) and first-response breaches (firstResponseBreachedAt, status
* unchanged), then fires escalation for each newly-detected breach.
*
* 016-load-concurrency-testing research.md §2: each run's update is now version-guarded. A
* lost race here (a concurrent pause/resume/complete changed the run first) means this run's
* status is no longer what the sweep's own query assumed — skipped for this pass rather than
* retried, since the next scheduled sweep re-evaluates every run fresh against real
* conditions anyway; this only ever defers, never drops, a genuine breach.
*/
async runBreachDetectionSweep(): Promise<void> {
const now = new Date();
const resolutionBreaches = await this.runs.findRunningPastResolutionDueAt(now);
for (const run of resolutionBreaches) {
await this.runs.update(run.id, { status: 'breached', breachedAt: now });
const updated = await this.runs.updateWithVersion(run.id, run.version, {
status: 'breached',
breachedAt: now,
});
if (!updated) continue;
slaRunOutcomesCounter.inc({ outcome: 'breached' });
await this.escalation.handleBreach(run.ticketId, 'resolution_breach');
}
@@ -170,7 +206,10 @@ export class SlaService {
const hasAgentResponse = messages.some((m) => m.type === 'AGENT_MESSAGE');
if (hasAgentResponse) continue;
await this.runs.update(run.id, { firstResponseBreachedAt: now });
const updated = await this.runs.updateWithVersion(run.id, run.version, {
firstResponseBreachedAt: now,
});
if (!updated) continue;
await this.escalation.handleBreach(run.ticketId, 'first_response_breach');
}
}
+178
View File
@@ -0,0 +1,178 @@
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
import { buildApp } from '@/app';
import { prismaClient } from '@/infrastructure/database';
import { FastifyInstance } from 'fastify';
import { loginAs, authHeader } from '../helpers/auth';
import {
encryptCredential,
generateCredentialSecret,
issueIntegrationToken,
} from '@/modules/catalog/products';
import { assignmentEngine } from '@/modules/orchestration/assignments';
/**
* specs/016-load-concurrency-testing User Story 1 / FR-001 / SC-001: two or more concurrent
* assignment attempts on the same ticket must never leave more than one Assignment row marked
* `isCurrent`. Before research.md §1's fix (a Postgres partial unique index on
* `assignments(ticketId) WHERE isCurrent = true`, plus a bounded retry in
* AssignmentRepository.createAssignment), Postgres's default READ COMMITTED isolation let two
* concurrent `assignmentEngine.assignToSpecificNode` calls each see "nothing current to
* supersede" and both successfully create their own `isCurrent: true` row.
*/
describe('Assignment double-assignment race (User Story 1)', () => {
let app: FastifyInstance;
let authToken: string;
const externalProductId = `TEST_ASSIGN_RACE_PROD_${Date.now()}`;
const skillTag = `assign_race_skill_${Date.now()}`;
let productId: string;
let teamId: string;
const agentIds: string[] = [];
let nodeId: string;
let secret: string;
const createdTicketIds: string[] = [];
async function createTicket(): Promise<string> {
const token = issueIntegrationToken(secret, {
externalProductId,
tenantId: 'tenant-1',
userId: 'user-1',
});
const created = await app.inject({
method: 'POST',
url: '/v1/support/requests',
headers: { authorization: `Bearer ${token}` },
payload: {
productId: externalProductId,
tenantId: 'tenant-1',
userId: 'user-1',
source: 'test',
problem: `Assignment race test ${Date.now()}-${Math.random()}`,
},
});
expect(created.statusCode).toBe(202);
const ticketId = created.json().data.ticketId as string;
createdTicketIds.push(ticketId);
return ticketId;
}
beforeAll(async () => {
app = await buildApp();
authToken = await loginAs(app, 'ADMIN');
const product = await prismaClient.product.create({
data: { externalProductId, name: 'Assignment Race Test Product', status: 'active' },
});
productId = product.id;
secret = generateCredentialSecret();
await prismaClient.productIntegration.create({
data: {
productId,
credentialRef: encryptCredential(secret),
authMechanism: 'signed_token',
allowedScope: { tenantIds: ['tenant-1'] },
status: 'active',
rateLimitPerMinute: 1000,
rateLimitPerUserPerMinute: 1000,
},
});
const team = await app.inject({
method: 'POST',
url: '/admin/teams',
headers: authHeader(authToken),
payload: { name: `Assign Race Team ${Date.now()}` },
});
teamId = team.json().data.id;
// Several eligible agents so a race has real agents to (incorrectly) double-assign across,
// not just one candidate every attempt would trivially agree on.
for (let i = 0; i < 5; i++) {
const agent = await app.inject({
method: 'POST',
url: `/admin/teams/${teamId}/agents`,
headers: authHeader(authToken),
payload: { name: `Assign Race Agent ${i}` },
});
const agentId = agent.json().data.id;
agentIds.push(agentId);
await app.inject({
method: 'PUT',
url: `/admin/agents/${agentId}/skills/${skillTag}`,
headers: authHeader(authToken),
payload: { level: 3 },
});
}
const node = await app.inject({
method: 'POST',
url: '/admin/hierarchy-nodes',
headers: authHeader(authToken),
payload: {
name: 'Assign Race Node',
order: 0,
productScope: [externalProductId],
skills: [skillTag],
assignmentStrategy: 'ROUND_ROBIN',
},
});
nodeId = node.json().data.id;
});
afterAll(async () => {
const ticketFilter = { ticketId: { in: createdTicketIds } };
await prismaClient.assignmentHistory.deleteMany({ where: ticketFilter });
await prismaClient.assignment.deleteMany({ where: ticketFilter });
await prismaClient.hierarchyNode.deleteMany({ where: { id: nodeId } });
await prismaClient.agentSkill.deleteMany({ where: { agentId: { in: agentIds } } });
await prismaClient.agent.deleteMany({ where: { teamId } });
await prismaClient.team.deleteMany({ where: { id: teamId } });
await prismaClient.ticketMessage.deleteMany({ where: ticketFilter });
await prismaClient.ticket.deleteMany({ where: { id: { in: createdTicketIds } } });
await prismaClient.problem.deleteMany({ where: { productId } });
await prismaClient.productIntegration.deleteMany({ where: { productId } });
await prismaClient.product.deleteMany({ where: { id: productId } });
await app.close();
});
it('leaves exactly one current assignment after 20 genuinely concurrent assignment attempts', async () => {
const ticketId = await createTicket();
const attempts = 20;
await Promise.all(
Array.from({ length: attempts }, () =>
assignmentEngine.assignToSpecificNode(ticketId, nodeId, 'system', 'concurrency test'),
),
);
const currentAssignments = await prismaClient.assignment.findMany({
where: { ticketId, isCurrent: true },
});
expect(currentAssignments.length).toBe(1);
// Every attempt was still recorded (superseded or current) — the race must not have
// silently dropped attempts, only converged them onto a single current row.
const allAssignments = await prismaClient.assignment.findMany({ where: { ticketId } });
expect(allAssignments.length).toBe(attempts);
});
it(
'holds consistently across 10 repeated runs (SC-001: zero exceptions)',
async () => {
for (let run = 0; run < 10; run++) {
const ticketId = await createTicket();
await Promise.all(
Array.from({ length: 20 }, () =>
assignmentEngine.assignToSpecificNode(ticketId, nodeId, 'system', `run ${run}`),
),
);
const currentAssignments = await prismaClient.assignment.findMany({
where: { ticketId, isCurrent: true },
});
expect(currentAssignments.length).toBe(1);
}
},
60000,
);
});
@@ -0,0 +1,228 @@
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
import { buildApp } from '@/app';
import { prismaClient } from '@/infrastructure/database';
import { FastifyInstance } from 'fastify';
import { loginAs, authHeader } from '../helpers/auth';
import {
encryptCredential,
generateCredentialSecret,
issueIntegrationToken,
} from '@/modules/catalog/products';
import { escalationService } from '@/modules/orchestration/escalation';
/**
* specs/016-load-concurrency-testing User Story 3 / FR-003 / SC-003: the same escalation
* trigger delivered more than once for the same ticket (e.g. two overlapping breach-sweep
* passes, or a re-delivered job) must result in exactly one EscalationEvent and exactly one
* resulting reassignment — never two. Before research.md §3's fix (the
* `escalation_events_ticket_rule_unique` partial unique index plus
* EscalationEventRepository.create's catch-and-absorb), `EscalationService.fire` unconditionally
* created a new event and reassigned on every call, with no dedup mechanism at all.
*/
describe('Escalation idempotency (User Story 3)', () => {
let app: FastifyInstance;
let authToken: string;
const externalProductId = `TEST_ESCALATION_IDEMPOTENCY_PROD_${Date.now()}`;
const skillTag = `escalation_idempotency_skill_${Date.now()}`;
let productId: string;
let teamId: string;
let agentId: string;
let nodeId: string;
let policyId: string;
let escalationPolicyId: string;
let secret: string;
const createdTicketIds: string[] = [];
async function createTicketAndAssign(): Promise<string> {
const token = issueIntegrationToken(secret, {
externalProductId,
tenantId: 'tenant-1',
userId: 'user-1',
});
const created = await app.inject({
method: 'POST',
url: '/v1/support/requests',
headers: { authorization: `Bearer ${token}` },
payload: {
productId: externalProductId,
tenantId: 'tenant-1',
userId: 'user-1',
source: 'test',
problem: `Escalation idempotency test ${Date.now()}-${Math.random()}`,
},
});
expect(created.statusCode).toBe(202);
const ticketId = created.json().data.ticketId as string;
createdTicketIds.push(ticketId);
const ticket = await prismaClient.ticket.findUniqueOrThrow({ where: { id: ticketId } });
await app.inject({
method: 'PATCH',
url: `/tickets/${ticketId}/status`,
headers: authHeader(authToken),
payload: { status: 'HUMAN_ESCALATION', expectedVersion: ticket.version },
});
// Force the run overdue — the concrete condition handleBreach fires for in production, via
// the breach sweep.
await prismaClient.sLARun.update({
where: { ticketId },
data: { resolutionDueAt: new Date(Date.now() - 60_000) },
});
return ticketId;
}
beforeAll(async () => {
app = await buildApp();
authToken = await loginAs(app, 'ADMIN');
const product = await prismaClient.product.create({
data: { externalProductId, name: 'Escalation Idempotency Test Product', status: 'active' },
});
productId = product.id;
secret = generateCredentialSecret();
await prismaClient.productIntegration.create({
data: {
productId,
credentialRef: encryptCredential(secret),
authMechanism: 'signed_token',
allowedScope: { tenantIds: ['tenant-1'] },
status: 'active',
rateLimitPerMinute: 1000,
rateLimitPerUserPerMinute: 1000,
},
});
const team = await app.inject({
method: 'POST',
url: '/admin/teams',
headers: authHeader(authToken),
payload: { name: `Escalation Idempotency Team ${Date.now()}` },
});
teamId = team.json().data.id;
const agent = await app.inject({
method: 'POST',
url: `/admin/teams/${teamId}/agents`,
headers: authHeader(authToken),
payload: { name: 'Escalation Idempotency Agent' },
});
agentId = agent.json().data.id;
await app.inject({
method: 'PUT',
url: `/admin/agents/${agentId}/skills/${skillTag}`,
headers: authHeader(authToken),
payload: { level: 3 },
});
const node = await app.inject({
method: 'POST',
url: '/admin/hierarchy-nodes',
headers: authHeader(authToken),
payload: {
name: 'Escalation Idempotency Node',
order: 0,
productScope: [externalProductId],
skills: [skillTag],
assignmentStrategy: 'ROUND_ROBIN',
},
});
nodeId = node.json().data.id;
const policy = await app.inject({
method: 'POST',
url: '/admin/sla-policies',
headers: authHeader(authToken),
payload: {
name: 'Escalation Idempotency SLA Policy',
productId,
firstResponseMinutes: 30,
resolutionMinutes: 240,
},
});
policyId = policy.json().data.id;
const escPolicy = await app.inject({
method: 'POST',
url: '/admin/escalation-policies',
headers: authHeader(authToken),
payload: { name: 'Escalation Idempotency Policy', productId },
});
escalationPolicyId = escPolicy.json().data.id;
await app.inject({
method: 'POST',
url: `/admin/escalation-policies/${escalationPolicyId}/rules`,
headers: authHeader(authToken),
payload: {
triggerType: 'resolution_breach',
condition: {},
targetNodeId: nodeId,
notify: {},
},
});
});
afterAll(async () => {
const ticketFilter = { ticketId: { in: createdTicketIds } };
await prismaClient.escalationEvent.deleteMany({ where: ticketFilter });
await prismaClient.escalationRule.deleteMany({ where: { targetNodeId: nodeId } });
await prismaClient.escalationPolicy.deleteMany({ where: { id: escalationPolicyId } });
await prismaClient.sLARun.deleteMany({ where: ticketFilter });
await prismaClient.sLAPolicy.deleteMany({ where: { id: policyId } });
await prismaClient.assignmentHistory.deleteMany({ where: ticketFilter });
await prismaClient.assignment.deleteMany({ where: ticketFilter });
await prismaClient.hierarchyNode.deleteMany({ where: { id: nodeId } });
await prismaClient.agentSkill.deleteMany({ where: { agentId } });
await prismaClient.agent.deleteMany({ where: { teamId } });
await prismaClient.team.deleteMany({ where: { id: teamId } });
await prismaClient.ticketMessage.deleteMany({ where: ticketFilter });
await prismaClient.ticket.deleteMany({ where: { id: { in: createdTicketIds } } });
await prismaClient.problem.deleteMany({ where: { productId } });
await prismaClient.productIntegration.deleteMany({ where: { productId } });
await prismaClient.product.deleteMany({ where: { id: productId } });
await app.close();
});
it('results in exactly one escalation event and one assignment when the same trigger fires twice concurrently', async () => {
const ticketId = await createTicketAndAssign();
await Promise.all([
escalationService.handleBreach(ticketId, 'resolution_breach'),
escalationService.handleBreach(ticketId, 'resolution_breach'),
]);
const events = await prismaClient.escalationEvent.findMany({ where: { ticketId } });
expect(events.length).toBe(1);
const currentAssignments = await prismaClient.assignment.findMany({
where: { ticketId, isCurrent: true },
});
expect(currentAssignments.length).toBe(1);
});
it(
'holds consistently across 10 repeated runs (SC-003: zero duplicate outcomes)',
async () => {
for (let run = 0; run < 10; run++) {
const ticketId = await createTicketAndAssign();
await Promise.all([
escalationService.handleBreach(ticketId, 'resolution_breach'),
escalationService.handleBreach(ticketId, 'resolution_breach'),
escalationService.handleBreach(ticketId, 'resolution_breach'),
]);
const events = await prismaClient.escalationEvent.findMany({ where: { ticketId } });
expect(events.length).toBe(1);
const currentAssignments = await prismaClient.assignment.findMany({
where: { ticketId, isCurrent: true },
});
expect(currentAssignments.length).toBe(1);
}
},
60000,
);
});
+218
View File
@@ -0,0 +1,218 @@
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
import { buildApp } from '@/app';
import { prismaClient } from '@/infrastructure/database';
import { FastifyInstance } from 'fastify';
import { loginAs, authHeader } from '../helpers/auth';
import {
encryptCredential,
generateCredentialSecret,
issueIntegrationToken,
} from '@/modules/catalog/products';
import { slaService } from '@/modules/orchestration/sla';
/**
* specs/016-load-concurrency-testing User Story 2 / FR-002 / SC-002: concurrent pause, resume,
* and breach-sweep activity against the same SLA run must always leave it in one
* internally-consistent, auditable state — never a state with contradictory fields (paused with
* no pause timestamp, or a legitimately breached run silently reverted to running by a racing
* resume). Before research.md §2's fix (SLARun.version + SlaRunRepository.updateWithVersion),
* SlaService.pause/resume/complete/runBreachDetectionSweep each did a plain read-then-write with
* no guard, so two racing calls could clobber each other's writes.
*/
describe('SLA pause/resume/sweep race (User Story 2)', () => {
let app: FastifyInstance;
let authToken: string;
const externalProductId = `TEST_SLA_RACE_PROD_${Date.now()}`;
const skillTag = `sla_race_skill_${Date.now()}`;
let productId: string;
let teamId: string;
let agentId: string;
let nodeId: string;
let policyId: string;
let secret: string;
const createdTicketIds: string[] = [];
async function createTicketAndAssign(): Promise<string> {
const token = issueIntegrationToken(secret, {
externalProductId,
tenantId: 'tenant-1',
userId: 'user-1',
});
const created = await app.inject({
method: 'POST',
url: '/v1/support/requests',
headers: { authorization: `Bearer ${token}` },
payload: {
productId: externalProductId,
tenantId: 'tenant-1',
userId: 'user-1',
source: 'test',
problem: `SLA race test ${Date.now()}-${Math.random()}`,
},
});
expect(created.statusCode).toBe(202);
const ticketId = created.json().data.ticketId as string;
createdTicketIds.push(ticketId);
const ticket = await prismaClient.ticket.findUniqueOrThrow({ where: { id: ticketId } });
await app.inject({
method: 'PATCH',
url: `/tickets/${ticketId}/status`,
headers: authHeader(authToken),
payload: { status: 'HUMAN_ESCALATION', expectedVersion: ticket.version },
});
return ticketId;
}
beforeAll(async () => {
app = await buildApp();
authToken = await loginAs(app, 'ADMIN');
const product = await prismaClient.product.create({
data: { externalProductId, name: 'SLA Race Test Product', status: 'active' },
});
productId = product.id;
secret = generateCredentialSecret();
await prismaClient.productIntegration.create({
data: {
productId,
credentialRef: encryptCredential(secret),
authMechanism: 'signed_token',
allowedScope: { tenantIds: ['tenant-1'] },
status: 'active',
rateLimitPerMinute: 1000,
rateLimitPerUserPerMinute: 1000,
},
});
const team = await app.inject({
method: 'POST',
url: '/admin/teams',
headers: authHeader(authToken),
payload: { name: `SLA Race Team ${Date.now()}` },
});
teamId = team.json().data.id;
const agent = await app.inject({
method: 'POST',
url: `/admin/teams/${teamId}/agents`,
headers: authHeader(authToken),
payload: { name: 'SLA Race Agent' },
});
agentId = agent.json().data.id;
await app.inject({
method: 'PUT',
url: `/admin/agents/${agentId}/skills/${skillTag}`,
headers: authHeader(authToken),
payload: { level: 3 },
});
const node = await app.inject({
method: 'POST',
url: '/admin/hierarchy-nodes',
headers: authHeader(authToken),
payload: {
name: 'SLA Race Node',
order: 0,
productScope: [externalProductId],
skills: [skillTag],
assignmentStrategy: 'ROUND_ROBIN',
},
});
nodeId = node.json().data.id;
const policy = await app.inject({
method: 'POST',
url: '/admin/sla-policies',
headers: authHeader(authToken),
payload: { name: 'SLA Race Policy', productId, firstResponseMinutes: 30, resolutionMinutes: 240 },
});
policyId = policy.json().data.id;
});
afterAll(async () => {
const ticketFilter = { ticketId: { in: createdTicketIds } };
await prismaClient.escalationEvent.deleteMany({ where: ticketFilter });
await prismaClient.sLARun.deleteMany({ where: ticketFilter });
await prismaClient.sLAPolicy.deleteMany({ where: { id: policyId } });
await prismaClient.assignmentHistory.deleteMany({ where: ticketFilter });
await prismaClient.assignment.deleteMany({ where: ticketFilter });
await prismaClient.hierarchyNode.deleteMany({ where: { id: nodeId } });
await prismaClient.agentSkill.deleteMany({ where: { agentId } });
await prismaClient.agent.deleteMany({ where: { teamId } });
await prismaClient.team.deleteMany({ where: { id: teamId } });
await prismaClient.ticketMessage.deleteMany({ where: ticketFilter });
await prismaClient.ticket.deleteMany({ where: { id: { in: createdTicketIds } } });
await prismaClient.problem.deleteMany({ where: { productId } });
await prismaClient.productIntegration.deleteMany({ where: { productId } });
await prismaClient.product.deleteMany({ where: { id: productId } });
await app.close();
});
function assertInternallyConsistent(run: {
status: string;
pausedAt: Date | null;
resumedAt: Date | null;
breachedAt: Date | null;
}) {
if (run.status === 'paused') {
expect(run.pausedAt).not.toBeNull();
} else {
expect(run.pausedAt).toBeNull();
}
// A run the sweep has genuinely marked breached must never be silently reverted to running
// by a racing resume — status and breachedAt must agree with each other.
if (run.status === 'breached') {
expect(run.breachedAt).not.toBeNull();
}
}
it(
'leaves an internally-consistent final state under concurrent pause/resume/sweep',
async () => {
const ticketId = await createTicketAndAssign();
// Force the run's resolution due date into the past so the breach sweep genuinely has
// something real to detect concurrently with pause/resume, not a no-op query.
await prismaClient.sLARun.update({
where: { ticketId },
data: { resolutionDueAt: new Date(Date.now() - 60_000) },
});
await Promise.all([
slaService.pause(ticketId),
slaService.resume(ticketId),
slaService.runBreachDetectionSweep(),
slaService.pause(ticketId),
slaService.resume(ticketId),
]);
const run = await prismaClient.sLARun.findUniqueOrThrow({ where: { ticketId } });
assertInternallyConsistent(run);
},
30000,
);
it(
'holds consistently across 10 repeated runs (SC-002: zero contradictory-state outcomes)',
async () => {
for (let run = 0; run < 10; run++) {
const ticketId = await createTicketAndAssign();
await prismaClient.sLARun.update({
where: { ticketId },
data: { resolutionDueAt: new Date(Date.now() - 60_000) },
});
await Promise.all([
slaService.pause(ticketId),
slaService.resume(ticketId),
slaService.runBreachDetectionSweep(),
]);
const finalRun = await prismaClient.sLARun.findUniqueOrThrow({ where: { ticketId } });
assertInternallyConsistent(finalRun);
}
},
60000,
);
});
@@ -0,0 +1,121 @@
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
import { buildApp } from '@/app';
import { prismaClient } from '@/infrastructure/database';
import { FastifyInstance } from 'fastify';
import {
encryptCredential,
generateCredentialSecret,
issueIntegrationToken,
} from '@/modules/catalog/products';
import { ticketsRepository } from '@/modules/ticketing/tickets';
/**
* specs/016-load-concurrency-testing User Story 4 / FR-004 / SC-004: proves — rather than
* assumes — that 003-ticketing's own optimistic-concurrency guarantee
* (TicketsRepository.updateStatus's atomic `updateMany({where:{id, version: expectedVersion}})`)
* actually holds under genuinely concurrent requests, not just the sequential checks that
* existed before this feature. research.md §4: no implementation change is expected here — this
* is a real Postgres, real concurrency proof of an already-sound mechanism.
*/
describe('Ticket status optimistic concurrency (User Story 4)', () => {
let app: FastifyInstance;
const externalProductId = `TEST_TICKET_STATUS_RACE_PROD_${Date.now()}`;
let productId: string;
let secret: string;
const createdTicketIds: string[] = [];
async function createTicket(): Promise<string> {
const token = issueIntegrationToken(secret, {
externalProductId,
tenantId: 'tenant-1',
userId: 'user-1',
});
const created = await app.inject({
method: 'POST',
url: '/v1/support/requests',
headers: { authorization: `Bearer ${token}` },
payload: {
productId: externalProductId,
tenantId: 'tenant-1',
userId: 'user-1',
source: 'test',
problem: `Ticket status race test ${Date.now()}-${Math.random()}`,
},
});
expect(created.statusCode).toBe(202);
const ticketId = created.json().data.ticketId as string;
createdTicketIds.push(ticketId);
return ticketId;
}
beforeAll(async () => {
app = await buildApp();
const product = await prismaClient.product.create({
data: { externalProductId, name: 'Ticket Status Race Test Product', status: 'active' },
});
productId = product.id;
secret = generateCredentialSecret();
await prismaClient.productIntegration.create({
data: {
productId,
credentialRef: encryptCredential(secret),
authMechanism: 'signed_token',
allowedScope: { tenantIds: ['tenant-1'] },
status: 'active',
rateLimitPerMinute: 1000,
rateLimitPerUserPerMinute: 1000,
},
});
});
afterAll(async () => {
await prismaClient.ticketMessage.deleteMany({ where: { ticketId: { in: createdTicketIds } } });
await prismaClient.ticket.deleteMany({ where: { id: { in: createdTicketIds } } });
await prismaClient.problem.deleteMany({ where: { productId } });
await prismaClient.productIntegration.deleteMany({ where: { productId } });
await prismaClient.product.deleteMany({ where: { id: productId } });
await app.close();
});
it('exactly one of 20 concurrent status updates from the same version succeeds', async () => {
const ticketId = await createTicket();
const original = await prismaClient.ticket.findUniqueOrThrow({ where: { id: ticketId } });
expect(original.status).toBe('NEW');
const attempts = 20;
const results = await Promise.all(
Array.from({ length: attempts }, () =>
ticketsRepository.updateStatus(ticketId, 'AI_ANALYZING', original.version),
),
);
const successes = results.filter((r) => r !== null);
const failures = results.filter((r) => r === null);
expect(successes.length).toBe(1);
expect(failures.length).toBe(attempts - 1);
const finalTicket = await prismaClient.ticket.findUniqueOrThrow({ where: { id: ticketId } });
expect(finalTicket.status).toBe('AI_ANALYZING');
expect(finalTicket.version).toBe(original.version + 1);
});
it(
'holds consistently across 10 repeated runs (SC-004)',
async () => {
for (let run = 0; run < 10; run++) {
const ticketId = await createTicket();
const original = await prismaClient.ticket.findUniqueOrThrow({ where: { id: ticketId } });
const results = await Promise.all(
Array.from({ length: 20 }, () =>
ticketsRepository.updateStatus(ticketId, 'AI_ANALYZING', original.version),
),
);
expect(results.filter((r) => r !== null).length).toBe(1);
}
},
60000,
);
});
+69
View File
@@ -0,0 +1,69 @@
/**
* specs/016-load-concurrency-testing User Story 5 / FR-007: load/throughput baseline for the
* 015-reporting-dashboards admin endpoints — pure database aggregation reads, no third-party
* cost (unlike ai-support-flow.load.ts). Run against a real running dev server:
* `npx tsx tests/load/admin-reporting.load.ts`.
*
* Signs in as the project's own seeded admin account once (a session JWT is reusable across
* requests, unlike 002-saas-integration's single-use integration tokens), then cycles across
* all four dashboards so the report reflects a realistic mix of the endpoint group, not just one
* route.
*/
import { runLoadTest } from './autocannon.config';
const API_URL = process.env.LOAD_TEST_API_URL ?? 'http://localhost:4501';
const CONNECTIONS = Number(process.env.LOAD_TEST_CONNECTIONS ?? 10);
const DURATION_SEC = Number(process.env.LOAD_TEST_DURATION_SEC ?? 10);
const ADMIN_EMAIL = process.env.LOAD_TEST_ADMIN_EMAIL ?? 'admin@supporthub.internal';
const ADMIN_PASSWORD = process.env.LOAD_TEST_ADMIN_PASSWORD ?? 'ChangeMe123!';
const DASHBOARD_PATHS = [
'/admin/reports/management',
'/admin/reports/support',
'/admin/reports/ai',
];
async function signIn(): Promise<string> {
const response = await fetch(`${API_URL}/auth/login`, {
method: 'POST',
headers: { 'content-type': 'application/json' },
body: JSON.stringify({ email: ADMIN_EMAIL, password: ADMIN_PASSWORD }),
});
if (!response.ok) {
throw new Error(
`Admin sign-in failed (${response.status}) — set LOAD_TEST_ADMIN_EMAIL/PASSWORD if the seeded admin credentials differ.`,
);
}
const body = (await response.json()) as { data: { token: string } };
return body.data.token;
}
async function main(): Promise<void> {
const token = await signIn();
let requestIndex = 0;
await runLoadTest('admin-reporting', {
url: API_URL,
connections: CONNECTIONS,
duration: DURATION_SEC,
requests: [
{
method: 'GET',
headers: { authorization: `Bearer ${token}` },
setupRequest: (request) => {
request.path = DASHBOARD_PATHS[requestIndex % DASHBOARD_PATHS.length];
requestIndex += 1;
return request;
},
},
],
});
}
main()
.then(() => process.exit(0))
.catch((error) => {
// eslint-disable-next-line no-console
console.error(error);
process.exit(1);
});
+166
View File
@@ -0,0 +1,166 @@
/**
* specs/016-load-concurrency-testing User Story 5 / FR-007: load/throughput baseline for the AI
* support flow's customer-reply turn (`POST /tickets/:ticketId/ai-session/messages`).
*
* IMPORTANT — real cost: 005-ai-support calls the real Anthropic API for every diagnosis turn
* (via @anthropic-ai/sdk), both when a ticket's first AI turn runs (asynchronously, right after
* ticket creation) AND for every customer-reply turn this script sends. Running this script
* fires genuine, billed Claude API calls — it is NOT a free, purely-internal load test like
* ticket-creation.load.ts's own database-only path. Confirm the intended request volume
* (LOAD_TEST_REQUESTS below) with whoever owns the Anthropic billing before running this against
* anything but a small smoke-sized amount.
*
* Observed in practice: a synthetic problem string with no matching entry in this throwaway
* product's (empty) knowledge base often escalates on the very first AI turn — there is nothing
* for the model to diagnose confidently. That is still a real, honestly-measured code path (a
* fast 404 from `handleCustomerReply`'s "no active session" guard), not a script bug — this
* script measures whatever the endpoint actually does, rather than forcing every session to
* stay open by only ever picking already-active ones.
*
* Run against a real running dev server: `npx tsx tests/load/ai-support-flow.load.ts`.
* `LOAD_TEST_REQUESTS` (default 5) hard-caps the total number of real API-consuming requests —
* this script uses autocannon's `amount` option, never an open-ended `duration`, specifically to
* keep the real-money cost bounded and predictable.
*/
import { prismaClient } from '@/infrastructure/database';
import {
encryptCredential,
generateCredentialSecret,
issueIntegrationToken,
} from '@/modules/catalog/products';
import { runLoadTest } from './autocannon.config';
const API_URL = process.env.LOAD_TEST_API_URL ?? 'http://localhost:4501';
const CONNECTIONS = Number(process.env.LOAD_TEST_CONNECTIONS ?? 1);
const REQUESTS = Number(process.env.LOAD_TEST_REQUESTS ?? 5);
const SESSION_POLL_TIMEOUT_MS = 30_000;
const SESSION_POLL_INTERVAL_MS = 500;
async function sleep(ms: number): Promise<void> {
return new Promise((resolve) => setTimeout(resolve, ms));
}
/** The AI_SESSION worker creates the session asynchronously after ticket creation (FR-001,
* session.service.ts's own runFirstTurn docstring) — poll until SOME session row exists (any
* status) rather than assuming it's ready the instant the create-ticket request returns.
* Deliberately not restricted to the "active" statuses: this throwaway product's problems have
* no matching knowledge, so the very first turn commonly escalates immediately, which is a
* legitimate terminal outcome to measure, not a wait condition. */
async function waitForAnySession(ticketId: string): Promise<void> {
const deadline = Date.now() + SESSION_POLL_TIMEOUT_MS;
while (Date.now() < deadline) {
const session = await prismaClient.aISupportSession.findFirst({ where: { ticketId } });
if (session) return;
await sleep(SESSION_POLL_INTERVAL_MS);
}
throw new Error(`Timed out waiting for an AI session to be created on ticket ${ticketId}.`);
}
async function cleanup(productId: string): Promise<void> {
// AI-support rows form a deeper chain than a plain ticket (session -> diagnosis/interaction/
// action/knowledge-reference), all RESTRICT-constrained back to the ticket — every level must
// be cleared before the ticket itself can be deleted.
const sessions = await prismaClient.aISupportSession.findMany({
where: { ticket: { productId } },
select: { id: true },
});
const sessionIds = sessions.map((s) => s.id);
await prismaClient.aIActionResult.deleteMany({
where: { action: { sessionId: { in: sessionIds } } },
});
await prismaClient.aIAction.deleteMany({ where: { sessionId: { in: sessionIds } } });
await prismaClient.aIDiagnosis.deleteMany({ where: { sessionId: { in: sessionIds } } });
await prismaClient.aIInteraction.deleteMany({ where: { sessionId: { in: sessionIds } } });
await prismaClient.aIKnowledgeReference.deleteMany({ where: { sessionId: { in: sessionIds } } });
await prismaClient.aISupportSession.deleteMany({ where: { id: { in: sessionIds } } });
await prismaClient.ticketMessage.deleteMany({ where: { ticket: { productId } } });
await prismaClient.assignmentHistory.deleteMany({ where: { ticket: { productId } } });
await prismaClient.assignment.deleteMany({ where: { ticket: { productId } } });
await prismaClient.sLARun.deleteMany({ where: { ticket: { productId } } });
await prismaClient.escalationEvent.deleteMany({ where: { ticket: { productId } } });
await prismaClient.ticket.deleteMany({ where: { productId } });
await prismaClient.problem.deleteMany({ where: { productId } });
await prismaClient.productIntegration.deleteMany({ where: { productId } });
await prismaClient.product.deleteMany({ where: { id: productId } });
}
async function main(): Promise<void> {
const externalProductId = `LOAD_TEST_AI_FLOW_${Date.now()}`;
const product = await prismaClient.product.create({
data: { externalProductId, name: 'Load Test — AI Support Flow', status: 'active' },
});
try {
const secret = generateCredentialSecret();
await prismaClient.productIntegration.create({
data: {
productId: product.id,
credentialRef: encryptCredential(secret),
authMechanism: 'signed_token',
allowedScope: { tenantIds: ['load-test-tenant'] },
status: 'active',
rateLimitPerMinute: 100000,
rateLimitPerUserPerMinute: 100000,
},
});
// eslint-disable-next-line no-console
console.log(`Pre-creating ${REQUESTS} tickets and waiting for each one's AI session...`);
const ticketIds: string[] = [];
for (let i = 0; i < REQUESTS; i++) {
const token = issueIntegrationToken(secret, {
externalProductId,
tenantId: 'load-test-tenant',
userId: `load-test-user-${i}`,
});
const response = await fetch(`${API_URL}/v1/support/requests`, {
method: 'POST',
headers: { 'content-type': 'application/json', authorization: `Bearer ${token}` },
body: JSON.stringify({
productId: externalProductId,
tenantId: 'load-test-tenant',
userId: `load-test-user-${i}`,
source: 'load-test',
problem: `AI flow load test problem ${i} ${Date.now()}`,
}),
});
const body = (await response.json()) as { data: { ticketId: string } };
ticketIds.push(body.data.ticketId);
await waitForAnySession(body.data.ticketId);
}
// eslint-disable-next-line no-console
console.log('Every ticket has an AI session (active or already resolved/escalated) — starting the load test.');
let requestIndex = 0;
await runLoadTest('ai-support-flow', {
url: `${API_URL}/tickets/placeholder/ai-session/messages`,
connections: CONNECTIONS,
amount: REQUESTS,
requests: [
{
method: 'POST',
headers: { 'content-type': 'application/json' },
setupRequest: (request) => {
const ticketId = ticketIds[requestIndex % ticketIds.length];
requestIndex += 1;
request.path = `/tickets/${ticketId}/ai-session/messages`;
request.body = JSON.stringify({
message: 'I already tried restarting, still broken.',
});
return request;
},
},
],
});
} finally {
await cleanup(product.id);
}
}
main()
.then(() => process.exit(0))
.catch((error) => {
// eslint-disable-next-line no-console
console.error(error);
process.exit(1);
});
+79
View File
@@ -0,0 +1,79 @@
import autocannon from 'autocannon';
import { mkdirSync, writeFileSync } from 'fs';
import path from 'path';
/**
* specs/016-load-concurrency-testing data-model.md "Load test report". Printed to the console
* and written as JSON under tests/load/reports/ (gitignored — a run artifact, not a fixture)
* for every run, so two runs of the same script can be compared.
*/
export interface LoadTestReport {
endpoint: string;
connections: number;
durationSec: number;
requestsPerSec: number;
latencyP50Ms: number;
latencyP90Ms: number;
latencyP99Ms: number;
non2xxCount: number;
rateLimitedCount: number;
}
/**
* FR-007/FR-008: a thin wrapper over autocannon's programmatic API producing this feature's own
* report shape. FR-009: deliberately has NO pass/fail assertion on the numbers — throughput and
* latency targets are an `OPEN BUSINESS DECISION` (spec.md Assumptions) this project's roadmap
* says must never be invented and shipped as if final. This tool measures and records; wiring an
* explicit CI gate is future work once the business sets a real target.
*/
export async function runLoadTest(
endpoint: string,
options: autocannon.Options,
): Promise<LoadTestReport> {
const result = await autocannon(options);
const report: LoadTestReport = {
endpoint,
connections: result.connections,
durationSec: result.duration,
requestsPerSec: Number(result.requests.average.toFixed(2)),
latencyP50Ms: result.latency.p50,
latencyP90Ms: result.latency.p90,
latencyP99Ms: result.latency.p99,
non2xxCount: result.non2xx,
rateLimitedCount: result.statusCodeStats?.['429']?.count ?? 0,
};
printReport(report);
writeReport(report);
return report;
}
function printReport(report: LoadTestReport): void {
// eslint-disable-next-line no-console
console.log(`\n=== Load test report: ${report.endpoint} ===`);
// eslint-disable-next-line no-console
console.log(`connections: ${report.connections} duration: ${report.durationSec}s`);
// eslint-disable-next-line no-console
console.log(`requests/sec (avg): ${report.requestsPerSec}`);
// eslint-disable-next-line no-console
console.log(
`latency p50/p90/p99 (ms): ${report.latencyP50Ms}/${report.latencyP90Ms}/${report.latencyP99Ms}`,
);
// eslint-disable-next-line no-console
console.log(`non-2xx: ${report.non2xxCount} rate-limited (429): ${report.rateLimitedCount}`);
// eslint-disable-next-line no-console
console.log(
'No pass/fail threshold applied — throughput/latency targets are an OPEN BUSINESS DECISION.',
);
}
function writeReport(report: LoadTestReport): void {
const dir = path.join(__dirname, 'reports');
mkdirSync(dir, { recursive: true });
const safeName = report.endpoint.replace(/[^a-z0-9-]/gi, '_');
const filePath = path.join(dir, `${safeName}-${Date.now()}.json`);
writeFileSync(filePath, JSON.stringify(report, null, 2));
// eslint-disable-next-line no-console
console.log(`Report written to ${filePath}`);
}
+94
View File
@@ -0,0 +1,94 @@
/**
* specs/016-load-concurrency-testing User Story 5 / FR-007: load/throughput baseline for new
* support-request submission — the entry point of the entire support flow. Run against a real
* running dev server: `npx tsx tests/load/ticket-creation.load.ts`.
*
* IMPORTANT — real cost: a successful ticket creation asynchronously triggers 005-ai-support's
* AI_SESSION worker, which makes a real, billed Anthropic API call for that ticket's first
* diagnosis turn. This endpoint's own HTTP response is fast and free, but the request still has
* a real downstream cost — set `LOAD_TEST_REQUESTS` to bound the total ticket count explicitly
* rather than relying on an open-ended `LOAD_TEST_DURATION_SEC` run whose total is
* latency-dependent and less predictable.
*
* Creates its own throwaway product + integration credential, generates a fresh single-use
* integration token per request (002-saas-integration's own jti replay protection means a
* single static token can't be reused across requests), and cleans up everything it created
* once the run finishes.
*/
import { prismaClient } from '@/infrastructure/database';
import {
encryptCredential,
generateCredentialSecret,
issueIntegrationToken,
} from '@/modules/catalog/products';
import { runLoadTest } from './autocannon.config';
const API_URL = process.env.LOAD_TEST_API_URL ?? 'http://localhost:4501';
const CONNECTIONS = Number(process.env.LOAD_TEST_CONNECTIONS ?? 10);
const DURATION_SEC = Number(process.env.LOAD_TEST_DURATION_SEC ?? 10);
// When set, caps the total number of tickets created (and thus the total downstream AI cost)
// instead of running for an open-ended duration.
const REQUESTS = process.env.LOAD_TEST_REQUESTS ? Number(process.env.LOAD_TEST_REQUESTS) : undefined;
async function main(): Promise<void> {
const externalProductId = `LOAD_TEST_TICKET_CREATION_${Date.now()}`;
const product = await prismaClient.product.create({
data: { externalProductId, name: 'Load Test — Ticket Creation', status: 'active' },
});
const secret = generateCredentialSecret();
await prismaClient.productIntegration.create({
data: {
productId: product.id,
credentialRef: encryptCredential(secret),
authMechanism: 'signed_token',
allowedScope: { tenantIds: ['load-test-tenant'] },
status: 'active',
rateLimitPerMinute: 100000,
rateLimitPerUserPerMinute: 100000,
},
});
try {
await runLoadTest('ticket-creation', {
url: `${API_URL}/v1/support/requests`,
connections: CONNECTIONS,
...(REQUESTS !== undefined ? { amount: REQUESTS } : { duration: DURATION_SEC }),
requests: [
{
method: 'POST',
headers: { 'content-type': 'application/json' },
setupRequest: (request) => {
const token = issueIntegrationToken(secret, {
externalProductId,
tenantId: 'load-test-tenant',
userId: 'load-test-user',
});
request.headers = { ...request.headers, authorization: `Bearer ${token}` };
request.body = JSON.stringify({
productId: externalProductId,
tenantId: 'load-test-tenant',
userId: 'load-test-user',
source: 'load-test',
problem: `Load test problem ${Date.now()}-${Math.random()}`,
});
return request;
},
},
],
});
} finally {
await prismaClient.ticketMessage.deleteMany({ where: { ticket: { productId: product.id } } });
await prismaClient.ticket.deleteMany({ where: { productId: product.id } });
await prismaClient.problem.deleteMany({ where: { productId: product.id } });
await prismaClient.productIntegration.deleteMany({ where: { productId: product.id } });
await prismaClient.product.deleteMany({ where: { id: product.id } });
}
}
main()
.then(() => process.exit(0))
.catch((error) => {
// eslint-disable-next-line no-console
console.error(error);
process.exit(1);
});
@@ -7,6 +7,7 @@ function fakeRun(overrides: Partial<Record<string, unknown>> = {}) {
id: 'run-1',
ticketId: 'ticket-1',
status: 'running',
version: 1,
...overrides,
};
}
@@ -20,7 +21,7 @@ describe('SLA compliance metric (014-full-observability data-model.md #6)', () =
const incSpy = vi.spyOn(observability.slaRunOutcomesCounter, 'inc');
const runs = {
findByTicketId: vi.fn().mockResolvedValue(fakeRun({ status: 'running' })),
update: vi.fn().mockResolvedValue(undefined),
updateWithVersion: vi.fn().mockResolvedValue(fakeRun({ status: 'completed' })),
} as never;
const service = new SlaService(undefined, runs);
@@ -33,7 +34,7 @@ describe('SLA compliance metric (014-full-observability data-model.md #6)', () =
const incSpy = vi.spyOn(observability.slaRunOutcomesCounter, 'inc');
const runs = {
findByTicketId: vi.fn().mockResolvedValue(fakeRun({ status: 'breached' })),
update: vi.fn().mockResolvedValue(undefined),
updateWithVersion: vi.fn().mockResolvedValue(fakeRun({ status: 'completed' })),
} as never;
const service = new SlaService(undefined, runs);
@@ -46,13 +47,13 @@ describe('SLA compliance metric (014-full-observability data-model.md #6)', () =
const incSpy = vi.spyOn(observability.slaRunOutcomesCounter, 'inc');
const runs = {
findByTicketId: vi.fn().mockResolvedValue(fakeRun({ status: 'completed' })),
update: vi.fn().mockResolvedValue(undefined),
updateWithVersion: vi.fn().mockResolvedValue(undefined),
} as never;
const service = new SlaService(undefined, runs);
await service.complete('ticket-1');
expect(incSpy).not.toHaveBeenCalled();
expect((runs as unknown as { update: ReturnType<typeof vi.fn> }).update).not.toHaveBeenCalled();
expect((runs as unknown as { updateWithVersion: ReturnType<typeof vi.fn> }).updateWithVersion).not.toHaveBeenCalled();
});
});
@@ -19,7 +19,9 @@ describe('EscalationService.handleBreach', () => {
findApplicable: vi.fn().mockResolvedValue({ id: 'policy-1', productId: 'prod-1' }),
} as never;
const rules = { findActiveRules: vi.fn().mockResolvedValue([rule]) } as never;
const events = { create: vi.fn().mockResolvedValue({ id: 'event-1' }) } as never;
const events = {
create: vi.fn().mockResolvedValue({ event: { id: 'event-1' }, wasNewlyCreated: true }),
} as never;
const assignmentEngine = {
assignToSpecificNode: vi.fn().mockResolvedValue(undefined),
} as never;
@@ -10,6 +10,7 @@ function run(overrides: Partial<SLARun>): SLARun {
firstResponseDueAt: null,
resolutionDueAt: null,
status: 'running',
version: 1,
pausedAt: null,
resumedAt: null,
breachedAt: null,
@@ -22,11 +23,11 @@ function run(overrides: Partial<SLARun>): SLARun {
describe('SlaService.runBreachDetectionSweep', () => {
it('marks every running run past its resolution due date as breached and fires escalation', async () => {
const overdue = run({ id: 'r1', ticketId: 't1' });
const update = vi.fn().mockResolvedValue(overdue);
const updateWithVersion = vi.fn().mockResolvedValue(overdue);
const runsRepo = {
findRunningPastResolutionDueAt: vi.fn().mockResolvedValue([overdue]),
findRunningPastFirstResponseDueAt: vi.fn().mockResolvedValue([]),
update,
updateWithVersion,
} as never;
const handleBreach = vi.fn().mockResolvedValue(undefined);
const escalation = { handleBreach } as never;
@@ -34,7 +35,7 @@ describe('SlaService.runBreachDetectionSweep', () => {
const service = new SlaService(undefined, runsRepo, undefined, undefined, escalation);
await service.runBreachDetectionSweep();
expect(update).toHaveBeenCalledWith('r1', expect.objectContaining({ status: 'breached' }));
expect(updateWithVersion).toHaveBeenCalledWith('r1', 1, expect.objectContaining({ status: 'breached' }));
expect(handleBreach).toHaveBeenCalledWith('t1', 'resolution_breach');
});
@@ -42,7 +43,7 @@ describe('SlaService.runBreachDetectionSweep', () => {
const runsRepo = {
findRunningPastResolutionDueAt: vi.fn().mockResolvedValue([]),
findRunningPastFirstResponseDueAt: vi.fn().mockResolvedValue([]),
update: vi.fn(),
updateWithVersion: vi.fn(),
} as never;
const handleBreach = vi.fn();
const service = new SlaService(undefined, runsRepo, undefined, undefined, {
@@ -10,6 +10,7 @@ function run(overrides: Partial<SLARun>): SLARun {
firstResponseDueAt: new Date('2026-01-05T12:00:00.000Z'),
resolutionDueAt: new Date('2026-01-05T17:00:00.000Z'),
status: 'running',
version: 1,
pausedAt: null,
resumedAt: null,
breachedAt: null,
@@ -22,13 +23,13 @@ function run(overrides: Partial<SLARun>): SLARun {
describe('SlaService pause/resume', () => {
it('pause records pausedAt and flips status to paused', async () => {
const found = run({});
const update = vi.fn().mockResolvedValue(found);
const runsRepo = { findByTicketId: vi.fn().mockResolvedValue(found), update } as never;
const updateWithVersion = vi.fn().mockResolvedValue(found);
const runsRepo = { findByTicketId: vi.fn().mockResolvedValue(found), updateWithVersion } as never;
const service = new SlaService(undefined, runsRepo);
await service.pause('ticket-1');
expect(update).toHaveBeenCalledWith('run-1', expect.objectContaining({ status: 'paused' }));
expect(updateWithVersion).toHaveBeenCalledWith('run-1', 1, expect.objectContaining({ status: 'paused' }));
});
it('resume shifts both due dates forward by exactly the paused wall-clock duration', async () => {
@@ -39,16 +40,16 @@ describe('SlaService pause/resume', () => {
firstResponseDueAt: new Date('2026-01-05T12:00:00.000Z'),
resolutionDueAt: new Date('2026-01-05T17:00:00.000Z'),
});
const update = vi.fn().mockResolvedValue(paused);
const runsRepo = { findByTicketId: vi.fn().mockResolvedValue(paused), update } as never;
const updateWithVersion = vi.fn().mockResolvedValue(paused);
const runsRepo = { findByTicketId: vi.fn().mockResolvedValue(paused), updateWithVersion } as never;
const service = new SlaService(undefined, runsRepo);
const before = Date.now();
await service.resume('ticket-1');
const after = Date.now();
expect(update).toHaveBeenCalledTimes(1);
const [, patch] = update.mock.calls[0] as [string, Record<string, unknown>];
expect(updateWithVersion).toHaveBeenCalledTimes(1);
const [, , patch] = updateWithVersion.mock.calls[0] as [string, number, Record<string, unknown>];
expect(patch.status).toBe('running');
expect(patch.pausedAt).toBeNull();
@@ -74,13 +75,13 @@ describe('SlaService pause/resume', () => {
status: 'paused',
pausedAt: secondPausedAt,
} as SLARun;
const update = vi.fn().mockResolvedValue(pausedAgain);
const runsRepo = { findByTicketId: vi.fn().mockResolvedValue(pausedAgain), update } as never;
const updateWithVersion = vi.fn().mockResolvedValue(pausedAgain);
const runsRepo = { findByTicketId: vi.fn().mockResolvedValue(pausedAgain), updateWithVersion } as never;
const service = new SlaService(undefined, runsRepo);
await service.resume('ticket-1');
const [, patch] = update.mock.calls[0] as [string, Record<string, unknown>];
const [, , patch] = updateWithVersion.mock.calls[0] as [string, number, Record<string, unknown>];
const shifted = (patch.resolutionDueAt as Date).getTime();
// Must be shifted from the ALREADY-shifted 18:00 baseline, not the original 17:00 baseline.
expect(shifted).toBeGreaterThan(new Date('2026-01-05T18:00:00.000Z').getTime());
@@ -88,11 +89,11 @@ describe('SlaService pause/resume', () => {
it('never resumes a run that is not currently paused', async () => {
const runningRun = run({ status: 'running', pausedAt: null });
const update = vi.fn();
const runsRepo = { findByTicketId: vi.fn().mockResolvedValue(runningRun), update } as never;
const updateWithVersion = vi.fn();
const runsRepo = { findByTicketId: vi.fn().mockResolvedValue(runningRun), updateWithVersion } as never;
const service = new SlaService(undefined, runsRepo);
await service.resume('ticket-1');
expect(update).not.toHaveBeenCalled();
expect(updateWithVersion).not.toHaveBeenCalled();
});
});