14 KiB
Backend handover
For the next developer. What this service was, what it is now, what changed, and what is still waiting for somebody.
Two companion documents sit one directory up and go deeper:
../REVIEW.md— the assessment. What was wrong, what was built, what I would not claim. Read §1 and §10 if you read nothing else.../SAAS_HARDENING.md— the chronological log, including the mistakes and the reasoning behind each decision.
This file is the orientation. Those two are the detail.
1. In one paragraph
This is a multi-tenant SaaS control plane: workspaces (tenants), the people in them, what those people are allowed to do, what their workspace has paid for, and which downstream modules they can reach. It was a working application with a serious isolation problem — any signed-in user of any customer could read every other customer's data through several endpoints, because tenancy was a convention in the query layer rather than a rule in the database. That is now enforced by PostgreSQL row-level security, and roughly twenty capabilities a business customer expects have been built on top.
2. What was here before
Fifteen migrations, and these concerns:
| Area | What existed |
|---|---|
| Auth | Email/password sign-in, JWT access tokens, refresh tokens |
| Tenancy | A tenant_id column, filtered by hand in each query |
| Roles | Roles and access codes per workspace |
| Subscriptions | Plans, and an assignment of plan to workspace |
| Modules | A registry of downstream modules and their environments |
| SSO outbound | Signing users into modules — the platform as identity provider |
| Audit | An audit_logs table and a service writing to it |
| Theme | Colour palettes |
Not present: row-level security, MFA, inbound SSO, SCIM, invitations, API keys, webhooks, notifications, org units, documents, reference data, sessions anybody could see, seat enforcement, rate limiting, an outbox, or a test suite.
The state of the code is recoverable exactly: everything in this repository is
uncommitted on branch furqan. git diff shows every change to a pre-existing
file, and git ls-files --others --exclude-standard shows every new one. Nothing
was committed by design — see §7.
3. The single most important change
Workspace isolation is now a database rule, not a coding convention.
Migration c3d5e7f9a801 enables and FORCEs row-level security on every
workspace-owned table. The application connects as a role created by
scripts/create_app_role.py, which is NOBYPASSRLS — so a query that forgets
its WHERE tenant_id = ... returns nothing rather than everything.
Three pieces make it work, and you need all three:
app/core/tenant_context.py— acontextvarholding the current workspace.scoped_to(tenant_id)sets it;unscoped()lifts it for genuine platform-wide work.app/middleware/tenant_scope_middleware.py— sets it per request from the access token.- A session listener that pushes the value into PostgreSQL via
set_configbefore each statement.
The trap: unscoped() only affects sessions wired to that listener. A raw
create_engine session has no listener, so the flag never reaches PostgreSQL and
every policy hides every row. Such a session must issue
SELECT set_config('app.bypass_rls', 'on', false) itself.
app/services/system/audit_retention.py is the worked example.
4. What was built, grouped
Twenty-six migrations were added. Rather than list them, here is what they
bought, roughly in dependency order. Each has tests; ../REVIEW.md §9 argues why
each was built the way it was.
Security and correctness
| Capability | Why it mattered |
|---|---|
| Row-level security | Above. The reason for everything else. |
| Encrypted module trust credentials | They were stored in plaintext. |
| Superadmin as an explicit flag | It was an absent tenant_id, so a signup that omitted the workspace produced a platform superadmin. |
| MFA (TOTP) + lockout | No second factor existed, for anyone, including superadmins. |
| Sessions people can see and end | Refresh tokens rotated and revoked correctly; nobody could ever see the result. |
| Rate limiting | Credential endpoints had none. |
| One account per address | Alice@ and alice@ were two accounts. Matched on lower(email) now. |
| Append-only audit log | UPDATE/DELETE revoked from the application role. |
| Soft delete | On tenants and org_units, where a hard delete destroyed history. |
Customer-facing capability
| Capability | What it is |
|---|---|
| Inbound SSO | Sign in with the customer's own Azure AD / Okta / Google. The opposite direction to the outbound SSO that already existed. |
| SCIM 2.0 | Provisioning and de-provisioning from the customer's directory. Without it, leavers kept their accounts. |
| Invitations | Inviting somebody instead of choosing their password for them. |
| API keys | Keys a customer can automate against, scoped. |
| Webhooks | Telling a customer's systems when something happens. HMAC-signed. |
| Notifications + preferences | In-product notices, and the ability to turn each kind off. |
| Org units | Departments and branches, with user administration scoped to a sub-tree. |
| Seat allocation | Seats divided between branches, not only counted per workspace. |
| Documents | Files attached to things, with content sniffed by magic bytes. |
| Reference data | Generic lookup lists — the things dropdowns are made of. |
| Per-workspace outgoing email | Sending from the customer's own address. |
| Subscription lifecycle | Grace periods, cancellation, expiry notices, history. |
Operational
| Piece | Where |
|---|---|
| Background worker, six jobs | app/services/system/worker.py — outbox, alerts, sessions, webhooks, notices, audit-retention |
| Event outbox | Events survive a crash between the write and the send |
| Idempotency | A retried request is safe to retry |
| Alert state | Remembers which alerts are already open, so they do not re-fire hourly |
| Operations endpoints | /api/admin/operations/* — what the background work has been doing |
5. Where to look
app/
core/
tenant_context.py Read this first. Workspace scoping.
rls.py Policy helpers and the enforcement check.
crypto.py Encryption at rest for module credentials.
ssrf.py Resolve-then-pin. Used by webhooks and SSO discovery.
document_storage.py Four functions. Swap for S3 here and nowhere else.
file_types.py Magic-byte sniffing. Never trusts Content-Type.
middleware/
tenant_scope_middleware.py Sets the workspace per request.
rate_limit.py Credential endpoints.
idempotency_middleware.py Safe retries.
services/
auth/ Workspace-owned concerns.
system/ Platform concerns, plus worker.py.
routes/ Thin. Validation and permission checks only.
scripts/ Operational. See §6.
tests/ 52 files, 878 tests.
docs/
MODULE_CONTRACT.md Hand to the authors of downstream modules.
SCIM.md, WEBHOOKS.md Customer-facing integration docs.
56 pre-existing files were modified. The largest changes are in
app/__init__.py (router mounting, middleware order), app/config/settings.py,
and the tenant/user/role services, which had their hand-written workspace filters
replaced by policy-backed queries.
6. Running it
python -m venv venv && venv/Scripts/pip install -r requirements.txt
# Tests. The role matters — see the note below.
TEST_DATABASE_URL=postgresql://saas_app:...@localhost/saas_test \
python -m pytest tests/ -q # 878 pass, 3 skip
python scripts/verify_security_fixes.py # 23 checks
python scripts/report_drift.py # read-only; what the live data looks like
python scripts/measure_query_plans.py # query plans at volume
Run the suite as saas_app, not as postgres. A superuser bypasses RLS, so
the policy tests cannot prove anything and skip themselves with a message saying
so. Three audit-backfill tests do the reverse — they need a role that can
UPDATE audit_logs, so they skip on the app role. Between the two
configurations everything is covered; in either one alone, something is not.
7. Pending — needs you, not me
These cannot be done from this machine. ../REVIEW.md §5 has the full text.
7.1 Apply the migrations, in this order
RLS FORCEs its policies, which applies them to the owner too. Without the context-setting code already deployed, every query returns nothing.
1. Deploy the code first
2. alembic upgrade head
3. python scripts/create_app_role.py # prints a password once
4. Put that role in DATABASE_URL, restart
5. python scripts/create_audit_retention_role.py # prints a second password once
6. Put that one in AUDIT_RETENTION_DATABASE_URL, restart
- Until step 4 the policies exist but do not apply. The application is still
connecting as a role that bypasses them, and nothing in the schema shows the
difference.
check_rls_enforced()reports it. - Step 3 must be re-run even if it was run before, because it now also
revokes
UPDATE, DELETEonaudit_logs. A role created by the earlier version still holds those rights. - Steps 5–6 are not optional. The nightly retention sweep is a DELETE and the application role can no longer do it. Unconfigured, the job raises on every run — deliberately: a retention job that no-ops for a year while its last-run timestamp keeps updating is the failure nobody notices until the table is why a query times out.
- Rolling back is
alembic downgrade b2e1d4f5a602.
7.2 Rotate the secrets
.env.local and .env.production are in the working tree with live values and
have been for the whole history. Treat every value in them as known. Rotating
them and purging the files from git history is outstanding and is not something I
should do to your repository.
7.3 Run the drift report against real data
python scripts/report_drift.py
Read-only. It has only ever run against an empty local database, so it currently proves the plumbing works and nothing about your data. Nobody yet knows how many workspaces are over their seat limit, how many roles grant permissions their plan never allowed, or how many accounts are stranded with no workspace. Expect the seat number to be non-zero — the limit was never enforced, so the data can exceed it.
7.4 Ship signature version 2 in the modules
The receiving half is built, tested and off by default.
MODULE_TRUST_REQUIRE_REPLAY_CONTROLS cannot be turned on until the modules send
the new headers — turning it on first refuses every legitimate call. Hand
docs/MODULE_CONTRACT.md to their authors.
7.5 Point the alerts somewhere
ALERT_WEBHOOK_URL and ALERT_EMAIL are unset, so alerting is built and silent.
Where they go, and who answers at three in the morning, is not a code decision.
7.6 Decide about /internal/sso/exchange
It works, and nothing issues a grant code, so no module can be using it. I finished it rather than deleting it, because removing a mounted public endpoint is your call. I would lean towards deleting it — the signed-payload handoff supersedes it.
7.7 Backfill audit attribution (optional)
Entries written before audit_logs had a workspace column are superadmin-only.
scripts/backfill_audit_tenants.py recovers most of them from entity_id and
has a --dry-run. It needs the schema owner, not the application role,
passed as --database-url or AUDIT_BACKFILL_DATABASE_URL — audit_logs is
append-only for the application now. It says so and stops if you forget.
8. Deliberately not built
Named as decisions so the next person finds an answer rather than a gap.
locations,cost_codes,currenciesexist in the base application. They are the customer's domain, not the platform's — a construction business needs cost codes; a platform that hosts one does not. The reference-list tables hold them if you want them, with no migration.- Virus scanning on uploads is absent, and named as absent. A function that pretends to scan is worse than an honest gap: it converts "we do not check" into "we checked", which is the belief that gets a file opened.
user_sessionsis outside RLS on purpose. It is written during sign-in before workspace context exists and read during refresh after the access token has expired. A policy there isolates nothing and breaks signing in. Every query on it is keyed onuser_idor an unguessablejti.- No reference-list content is seeded. Which currencies and document types
the platform publishes is a decision;
lookup_service.seed_platform_listis where it goes.
9. What I would not claim
Being explicit, because you are inheriting this.
- Nothing has been verified against production data. Every test runs against a local scratch database.
- CI has never been run by GitHub Actions. The repository has no remote. Every step has been run locally in order against a database created from scratch, which is most of the value, but the YAML has only been parsed.
- Coverage is ~78%, not 100%. The thinnest areas are the email service, parts
of
role_service, and route modules generally. - The migration chain is verified up from empty and back down to base on a throwaway database — never against anything shared.
- Defects were introduced and caught in the same pass. Several. They are
listed in
../REVIEW.md§7 rather than quietly omitted, because the pattern is the useful part: each was caught by a test written for a different reason.