maskantech/productcatalogue_backend
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: add comprehensive product engine architecture and database dictionary

Browse Source
This commit is contained in:
Inamul-hasan-tec
2026-08-19 16:13:11 +05:30
parent 4d419dcd36
commit 891b8157bc
7 changed files with 700 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/product_engine_deep_dive/01_EXECUTIVE_ARCHITECTURE_AND_ECOSYSTEM_MAP.md
+103
View File
@@ -0,0 +1,103 @@
# 🏛️ Enterprise PIM Architecture & Product Creation Ecosystem Manual
## 1. Executive Summary & Purpose
This document provides an exhaustive, authoritative blueprint of the **Product Information Management (PIM)** engine for both human software engineers, QA architects, and AI autonomous agents. It establishes the definitive standard for how products, their taxonomy, attributes, assets, channels, and metadata are modeled, validated, persisted, audited, and syndicated.
---
## 2. Global Entity-Relationship Architecture
```mermaid
erDiagram
TENANTS ||--o{ USERS : "owns"
TENANTS ||--o{ CATALOGS : "owns (Product Families)"
TENANTS ||--o{ CATEGORIES : "owns"
TENANTS ||--o{ BRANDS : "owns"
TENANTS ||--o{ UNITS : "owns"
TENANTS ||--o{ ATTRIBUTES : "owns"
TENANTS ||--o{ ATTRIBUTE_SETS : "owns"
TENANTS ||--o{ ASSETS : "owns"
TENANTS ||--o{ PRODUCTS : "owns"
TENANTS ||--o{ CHANNELS : "owns"
CATEGORIES ||--o{ CATEGORIES : "parent_id (Hierarchy Tree)"
CATEGORIES ||--o{ PRODUCTS : "classifies"
CATEGORIES ||--o{ CATALOGS : "binds default"
BRANDS ||--o{ PRODUCTS : "labels"
UNITS ||--o{ PRODUCTS : "measures"
ATTRIBUTE_SETS ||--o{ ATTRIBUTE_SET_GROUPS : "contains"
ATTRIBUTE_GROUPS ||--o{ ATTRIBUTE_SET_GROUPS : "assigned to"
ATTRIBUTE_GROUPS ||--o{ ATTRIBUTES : "groups"
ATTRIBUTES ||--o{ ATTRIBUTE_OPTIONS : "defines choices"
CATALOGS ||--o{ ATTRIBUTE_SETS : "binds attribute_set_id"
CATALOGS ||--o{ FAMILY_ATTRIBUTES : "binds direct attributes"
CATALOGS ||--o{ FAMILY_VARIANT_AXES : "defines variant dimensions"
CATALOGS ||--o{ FAMILY_ASSET_REQUIREMENTS : "enforces media rules"
CATALOGS ||--o{ FAMILY_CHANNELS : "subscribes channels"
PRODUCTS ||--o| CATALOGS : "instantiates (family_id)"
PRODUCTS ||--o{ PRODUCT_ATTRIBUTE_VALUES : "stores custom data"
PRODUCTS ||--o{ PRODUCT_ASSETS : "maps media files"
PRODUCTS ||--o{ VARIANTS : "has SKU children"
PRODUCTS ||--o{ PRODUCT_COMPLETENESS : "scores data readiness"
ATTRIBUTES ||--o{ PRODUCT_ATTRIBUTE_VALUES : "defines data point"
ASSETS ||--o{ PRODUCT_ASSETS : "maps physical file"
ASSET_TYPES ||--o{ ASSETS : "classifies media"
ASSET_FAMILIES ||--o{ ASSET_TYPES : "bundles requirements"
```
---
## 3. High-Level Core Subsystems & Dependency Hierarchy
To construct a valid Product, the system depends on an immutable, layered dependency hierarchy:
```
[Layer 0: Multi-Tenant Foundation]
├── Tenant Isolation (tenant_id scoping)
└── RBAC & Audit Trails (User Sessions, Permissions)
│
[Layer 1: Fundamental Catalog Primitives]
├── Units of Measure (kg, pcs, cm, l, etc.)
├── Brands (Manufacturers, Trademarks, Logos)
└── Categories (Nested Hierarchy, Slugs, Breadcrumbs)
│
[Layer 2: Attribute & Classification Engine]
├── Attributes (Types, Validations, Regex, Options)
├── Attribute Groups (UI Organizers & Logical Groupings)
└── Attribute Sets (Templates combining multiple Groups)
│
[Layer 3: Media & Digital Asset Management (DAM)]
├── Asset Types (Hero, Gallery, Manuals, Spec Sheets)
├── Asset Families (Required Media Rules & Validations)
└── Central File Registry (S3 / Local Storage, URLs, MIME)
│
[Layer 4: Blueprint Orchestration (Product Family / Catalog)]
├── Family Blueprint Definition (Inherits Category + Set)
├── Variant Axis Rules (Size, Color, Storage, RAM)
├── Media Requirements (Minimum Asset Dimensions & Formats)
└── Channel Syndication Subscriptions (Shopify, Amazon, CSV)
│
[Layer 5: Product Core Entity & Lifecycle Engine]
├── Draft Creation (POST /api/v1/products)
├── Dynamic Attribute Persistence (EAV / JSONB Hybrid)
├── Variant Matrix Generator (Cartesian Product of Axes)
├── Asset Role Assignment (Hero Image, Gallery, Video)
├── Channel Scoping & Overrides
└── Completeness Engine (Automated 0-100% Scoring)
```
---
## 4. Architectural Rules & Invariants
1. **Tenant Isolation**: Every database query in multi-tenant mode MUST enforce `tenant_id` scoping to prevent data leakage between organizations.
2. **Soft Deletes (`paranoid: true`)**: Deleting any primitive (Category, Attribute, Product, Unit, Brand) sets `deleted_at = NOW()`. Hard deletion is strictly disallowed to preserve audit integrity.
3. **SKU Invariant**:
- A Product in `draft` status may have `sku = null`.
- A Product transitioning to `pending` or `active` MUST have a globally unique SKU (either manually supplied or auto-generated by the Sequence Engine).
4. **Code Normalization**: `code` across all entities (Categories, Attributes, Units, Families, Products) must be lowercase, alphanumeric, and underscore-delimited (e.g., `tech_electronics_01`).
5. **Data Completeness Independence**: A product's completeness score is computed per channel and locale, evaluating mandatory general fields, required attributes, DAM media assets, and syndication channels.
docs/product_engine_deep_dive/02_DATABASE_SCHEMA_AND_TABLE_DICTIONARY.md
+188
View File
@@ -0,0 +1,188 @@
# 🗄️ PIM Database Schema & Table Dictionary
## 1. Overview
The PIM database is structured on PostgreSQL with Sequelize ORM, leveraging UUID primary keys, JSONB for flexible extensible metadata, and strict foreign-key integrity constraints with tenant isolation.
---
## 2. Table-by-Table Data Dictionary
### 2.1. `products` (Core Master Catalog Record)
| Column Name | PostgreSQL Type | Nullable | Default | Description & Foreign Key References |
| :--- | :--- | :--- | :--- | :--- |
| `id` | `UUID` | NO | `gen_random_uuid()` | Primary Key (Global Product UUID) |
| `tenant_id` | `INTEGER` | YES | `NULL` | Tenant isolation scope (`tenants.id`) |
| `code` | `VARCHAR(100)` | NO | — | Unique URL-safe identifier (e.g. `technova_audio_pro_x1`) |
| `sku` | `VARCHAR(100)` | YES | `NULL` | Stock Keeping Unit (Unique when status != draft) |
| `name` | `VARCHAR(255)` | NO | — | Display Name of the product |
| `description` | `TEXT` | YES | `NULL` | Full rich-text or plain-text product description |
| `price` | `NUMERIC(15,2)` | YES | `0.00` | Base retail catalog price |
| `stock` | `INTEGER` | YES | `0` | Base physical warehouse inventory on hand |
| `status` | `VARCHAR(20)` | NO | `'draft'` | Lifecycle Status: `draft`, `pending`, `active`, `archived` |
| `type` | `VARCHAR(20)` | NO | `'simple'` | Product Type: `simple`, `variant`, `bundle`, `virtual` |
| `family_id` | `UUID` | YES | `NULL` | Foreign Key -> `catalogs.id` (Product Family Blueprint) |
| `category_id` | `UUID` | YES | `NULL` | Foreign Key -> `categories.id` (Primary Taxonomy Category) |
| `brand_id` | `UUID` | YES | `NULL` | Foreign Key -> `brands.id` (Brand Manufacturer) |
| `unit_id` | `UUID` | YES | `NULL` | Foreign Key -> `units.id` (Unit of Measure) |
| `barcode` | `VARCHAR(100)` | YES | `NULL` | Universal barcode value |
| `gtin` | `VARCHAR(100)` | YES | `NULL` | Global Trade Item Number |
| `upc` | `VARCHAR(100)` | YES | `NULL` | Universal Product Code (12-digit) |
| `ean` | `VARCHAR(100)` | YES | `NULL` | European Article Number (13-digit) |
| `country` | `VARCHAR(100)` | YES | `NULL` | Country of Origin (ISO code or string) |
| `hsn` | `VARCHAR(50)` | YES | `NULL` | Harmonized System of Nomenclature code |
| `metadata` | `JSONB` | YES | `{}` | Extensible attributes, syndication channels, staging info |
| `version` | `INTEGER` | NO | `1` | Optimistic locking revision counter |
| `created_at` | `TIMESTAMP WITH TZ` | NO | `NOW()` | Audit record creation timestamp |
| `updated_at` | `TIMESTAMP WITH TZ` | NO | `NOW()` | Audit record last modification timestamp |
| `deleted_at` | `TIMESTAMP WITH TZ` | YES | `NULL` | Soft delete marker (Paranoid mode) |
---
### 2.2. `catalogs` (Product Families Blueprint)
| Column Name | PostgreSQL Type | Nullable | Default | Description & Foreign Key References |
| :--- | :--- | :--- | :--- | :--- |
| `id` | `UUID` | NO | `gen_random_uuid()` | Primary Key |
| `tenant_id` | `INTEGER` | YES | `NULL` | Tenant isolation scope (`tenants.id`) |
| `code` | `VARCHAR(50)` | NO | — | Unique family code (e.g. `electronics_family`) |
| `name` | `VARCHAR(100)` | NO | — | Family display name (e.g. `Electronics Family`) |
| `description`| `TEXT` | YES | `NULL` | Family blueprint description |
| `status` | `VARCHAR(20)` | NO | `'draft'` | `draft`, `active`, `inactive` |
| `category_id` | `UUID` | YES | `NULL` | Default inherited category -> `categories.id` |
| `attribute_set_id` | `UUID` | YES | `NULL` | Default bound Attribute Set -> `attribute_sets.id` |
| `workflow_code` | `VARCHAR(50)`| NO | `'standard'` | Workflow state machine configuration code |
| `completeness_rules` | `JSONB` | YES | `{}` | Allowed brands, allowed units, required thresholds |
| `created_at` | `TIMESTAMP WITH TZ` | NO | `NOW()` | Timestamp |
| `updated_at` | `TIMESTAMP WITH TZ` | NO | `NOW()` | Timestamp |
| `deleted_at` | `TIMESTAMP WITH TZ` | YES | `NULL` | Soft delete marker |
---
### 2.3. `categories` (Hierarchical Taxonomy Tree)
| Column Name | PostgreSQL Type | Nullable | Default | Description & Foreign Key References |
| :--- | :--- | :--- | :--- | :--- |
| `id` | `UUID` | NO | `gen_random_uuid()` | Primary Key |
| `tenant_id` | `INTEGER` | YES | `NULL` | Tenant isolation scope (`tenants.id`) |
| `parent_id` | `UUID` | YES | `NULL` | Self-referencing FK -> `categories.id` (Parent Category) |
| `name` | `VARCHAR(100)` | NO | — | Category Name (e.g., `Audio & Headphones`) |
| `code` | `VARCHAR(50)` | NO | — | Unique taxonomy code |
| `slug` | `VARCHAR(100)` | NO | — | SEO slug (e.g., `audio-headphones`) |
| `description`| `TEXT` | YES | `NULL` | Category description |
| `status` | `VARCHAR(20)` | NO | `'active'` | `active`, `inactive` |
| `display_order` | `INTEGER` | NO | `0` | UI sort order |
| `created_at` | `TIMESTAMP WITH TZ` | NO | `NOW()` | Timestamp |
| `updated_at` | `TIMESTAMP WITH TZ` | NO | `NOW()` | Timestamp |
| `deleted_at` | `TIMESTAMP WITH TZ` | YES | `NULL` | Soft delete marker |
---
### 2.4. `brands` (Brand Registry)
| Column Name | PostgreSQL Type | Nullable | Default | Description & Foreign Key References |
| :--- | :--- | :--- | :--- | :--- |
| `id` | `UUID` | NO | `gen_random_uuid()` | Primary Key |
| `tenant_id` | `INTEGER` | YES | `NULL` | Tenant isolation scope (`tenants.id`) |
| `name` | `VARCHAR(100)` | NO | — | Brand Name (e.g. `TechNova`) |
| `code` | `VARCHAR(50)` | NO | — | Unique code (e.g. `technova`) |
| `website` | `VARCHAR(255)` | YES | `NULL` | Brand official website URL |
| `description`| `TEXT` | YES | `NULL` | Brand profile text |
| `logo_url` | `TEXT` | YES | `NULL` | Media URL to brand logo |
| `status` | `VARCHAR(20)` | NO | `'active'` | `active`, `inactive` |
| `created_at` | `TIMESTAMP WITH TZ` | NO | `NOW()` | Timestamp |
| `updated_at` | `TIMESTAMP WITH TZ` | NO | `NOW()` | Timestamp |
| `deleted_at` | `TIMESTAMP WITH TZ` | YES | `NULL` | Soft delete marker |
---
### 2.5. `units` (Units of Measure Registry)
| Column Name | PostgreSQL Type | Nullable | Default | Description & Foreign Key References |
| :--- | :--- | :--- | :--- | :--- |
| `id` | `UUID` | NO | `gen_random_uuid()` | Primary Key |
| `tenant_id` | `INTEGER` | YES | `NULL` | Tenant isolation scope (`tenants.id`) |
| `name` | `VARCHAR(100)` | NO | — | Unit Name (e.g. `Piece`, `Set`, `Kilogram`) |
| `code` | `VARCHAR(50)` | NO | — | Unique code (e.g. `pcs`, `set`, `kg`) |
| `symbol` | `VARCHAR(20)` | NO | — | Display symbol (e.g. `pc`, `set`, `kg`) |
| `unit_type` | `VARCHAR(50)` | NO | `'Other'` | `Weight`, `Length`, `Volume`, `Count`, `Other` |
| `conversion_factor` | `NUMERIC(15,6)` | YES | `1.000000` | Multiplier relative to standard base unit |
| `base_unit_id` | `UUID` | YES | `NULL` | Self-referencing FK -> `units.id` (Standard base unit) |
| `status` | `VARCHAR(20)` | NO | `'active'` | `active`, `inactive` |
| `created_at` | `TIMESTAMP WITH TZ` | NO | `NOW()` | Timestamp |
| `updated_at` | `TIMESTAMP WITH TZ` | NO | `NOW()` | Timestamp |
| `deleted_at` | `TIMESTAMP WITH TZ` | YES | `NULL` | Soft delete marker |
---
### 2.6. `attributes` (Dynamic Field Definitions)
| Column Name | PostgreSQL Type | Nullable | Default | Description & Foreign Key References |
| :--- | :--- | :--- | :--- | :--- |
| `id` | `UUID` | NO | `gen_random_uuid()` | Primary Key |
| `tenant_id` | `INTEGER` | YES | `NULL` | Tenant isolation scope (`tenants.id`) |
| `code` | `VARCHAR(50)` | NO | — | Attribute Code (e.g. `color_spec`, `weight_grams`) |
| `name` | `VARCHAR(100)` | NO | — | Display Name (e.g. `Color Spec`, `Weight (Grams)`) |
| `type` | `VARCHAR(30)` | NO | `'text'` | `text`, `textarea`, `number`, `decimal`, `date`, `boolean`, `select`, `multiselect` |
| `is_required` | `BOOLEAN` | NO | `FALSE` | Mandate flag for completeness evaluation |
| `is_unique` | `BOOLEAN` | NO | `FALSE` | Requires unique value across catalog |
| `is_variant_eligible` | `BOOLEAN` | NO | `FALSE` | Allowed as matrix generator axis |
| `min_length` / `max_length` | `INTEGER` | YES | `NULL` | String length bounds |
| `options` | `JSONB` | YES | `[]` | Array of strings for select/multiselect fallback |
| `status` | `VARCHAR(20)` | NO | `'active'` | `active`, `inactive` |
| `created_at` | `TIMESTAMP WITH TZ` | NO | `NOW()` | Timestamp |
| `updated_at` | `TIMESTAMP WITH TZ` | NO | `NOW()` | Timestamp |
| `deleted_at` | `TIMESTAMP WITH TZ` | YES | `NULL` | Soft delete marker |
---
### 2.7. `attribute_options` (Select / Multiselect Option Choices)
| Column Name | PostgreSQL Type | Nullable | Default | Description & Foreign Key References |
| :--- | :--- | :--- | :--- | :--- |
| `id` | `UUID` | NO | `gen_random_uuid()` | Primary Key |
| `attribute_id` | `UUID` | NO | — | Foreign Key -> `attributes.id` (Parent Attribute) |
| `code` | `VARCHAR(50)` | NO | — | Value Code (e.g. `black`, `red`, `wireless_bt`) |
| `value` | `VARCHAR(255)` | NO | — | Human Display Label (e.g. `Black`, `Red`) |
| `display_order` | `INTEGER` | NO | `0` | Dropdown presentation position |
| `created_at` | `TIMESTAMP WITH TZ` | NO | `NOW()` | Timestamp |
| `updated_at` | `TIMESTAMP WITH TZ` | NO | `NOW()` | Timestamp |
---
### 2.8. `product_attribute_values` (EAV Product Data Storage)
| Column Name | PostgreSQL Type | Nullable | Default | Description & Foreign Key References |
| :--- | :--- | :--- | :--- | :--- |
| `id` | `UUID` | NO | `gen_random_uuid()` | Primary Key |
| `product_id` | `UUID` | NO | — | Foreign Key -> `products.id` (Cascade on delete) |
| `attribute_id` | `UUID` | NO | — | Foreign Key -> `attributes.id` |
| `value` | `TEXT` | YES | `NULL` | Serialized value (String, number, date, JSON array) |
| `locale` | `VARCHAR(10)` | NO | `'en'` | Internationalization locale code |
| `channel_code` | `VARCHAR(50)` | YES | `NULL` | Channel specific value override |
| `created_at` | `TIMESTAMP WITH TZ` | NO | `NOW()` | Timestamp |
| `updated_at` | `TIMESTAMP WITH TZ` | NO | `NOW()` | Timestamp |
---
### 2.9. `product_assets` (Product Media Junction)
| Column Name | PostgreSQL Type | Nullable | Default | Description & Foreign Key References |
| :--- | :--- | :--- | :--- | :--- |
| `id` | `UUID` | NO | `gen_random_uuid()` | Primary Key |
| `product_id` | `UUID` | NO | — | Foreign Key -> `products.id` |
| `asset_id` | `UUID` | NO | — | Foreign Key -> `assets.id` |
| `role` | `VARCHAR(50)` | NO | `'gallery_image'`| `hero_image`, `gallery_image`, `thumbnail`, `video`, `document` |
| `is_primary` | `BOOLEAN` | NO | `FALSE` | Primary hero display flag |
| `display_order` | `INTEGER` | NO | `0` | Image gallery sort order |
| `created_at` | `TIMESTAMP WITH TZ` | NO | `NOW()` | Timestamp |
| `updated_at` | `TIMESTAMP WITH TZ` | NO | `NOW()` | Timestamp |
---
### 2.10. `product_completeness` (Data Readiness Engine Scoring)
| Column Name | PostgreSQL Type | Nullable | Default | Description & Foreign Key References |
| :--- | :--- | :--- | :--- | :--- |
| `id` | `UUID` | NO | `gen_random_uuid()` | Primary Key |
| `product_id` | `UUID` | NO | — | Foreign Key -> `products.id` |
| `channel` | `VARCHAR(50)` | NO | `'default'` | Distribution channel scope (`default`, `shopify`, `amazon`) |
| `locale` | `VARCHAR(10)` | NO | `'en'` | Locale scope |
| `percentage` | `INTEGER` | NO | `0` | Calculated score (0 to 100%) |
| `is_complete` | `BOOLEAN` | NO | `FALSE` | `TRUE` if `percentage === 100` |
| `missing_attributes` | `JSONB` | YES | `[]` | Array of missing attribute codes & labels |
| `missing_assets` | `JSONB` | YES | `[]` | Array of missing mandatory asset type roles |
| `missing_channels` | `JSONB` | YES | `[]` | Array of missing required syndication channels |
| `missing_general` | `JSONB` | YES | `[]` | Missing basic fields (`name`, `category`, `brand`, `unit`) |
| `created_at` | `TIMESTAMP WITH TZ` | NO | `NOW()` | Timestamp |
| `updated_at` | `TIMESTAMP WITH TZ` | NO | `NOW()` | Timestamp |
docs/product_engine_deep_dive/03_PRODUCT_CREATION_POST_API_LIFECYCLE.md
+131
View File
@@ -0,0 +1,131 @@
# ⚡ Product Creation POST API & Lifecycle Engine
## 1. Overview
The `POST /api/v1/products` endpoint is the gateway for catalog authoring. It coordinates JSON Schema validation, taxonomy binding, dynamic EAV attribute parsing, code generation, SKU reservation, transaction execution, socket broadcasting, and automated completeness calculation.
---
## 2. API Contract & Payload Schema
### Request Definition
- **Endpoint**: `POST /api/v1/products`
- **Headers**:
- `Content-Type: application/json`
- `Authorization: Bearer <JWT_TOKEN>`
- `x-tenant-id: <TENANT_ID>`
### Canonical JSON Payload
```json
{
"name": "TechNova Audio Pro X1 Wireless Headphones",
"code": "technova_audio_pro_x1",
"sku": "TECHNOVA-AUDIO-PRO-X1-BLK",
"status": "draft",
"type": "simple",
"price": 249.99,
"stock": 100,
"family_id": "b0343591-1bfe-4b02-8556-626616be518a",
"category": "d13554e2-763b-4886-9a3d-4c312781dc41",
"brand": "0e527d71-5582-4fec-beea-682442cf8947",
"unit": "3dbf77c3-3765-4f46-9538-4e8971f1e695",
"description": "Premium noise-cancelling over-ear headphones with 40-hour battery life.",
"barcode": "8901234567890",
"gtin": "00890123456789",
"upc": "890123456789",
"ean": "8901234567890",
"country": "Germany",
"hsn": "85183000",
"metadata": {
"attributeSetId": "eb6706cf-71c3-47c4-8182-a36ee06ce564",
"channels": ["shopify", "amazon", "custom_csv"],
"currentStage": "draft"
},
"attributes": {
"color_spec": "black",
"weight_grams": 250,
"connectivity": "Bluetooth 5.3"
}
}
```
---
## 3. End-to-End Execution Sequence Diagram
```mermaid
sequenceDiagram
autonumber
actor Client as Frontend Wizard / API Client
participant Auth as Auth & Context Middleware
participant Ctrl as ProductController
participant Svc as ProductService
participant CodeGen as Code/SKU Engine
participant DB as PostgreSQL Database
participant Comp as CompletenessService
participant Audit as AuditService
participant WS as Socket.IO Broadcaster
Client->>Auth: POST /api/v1/products
Auth->>Auth: Validate JWT, Tenant ID & Permission (products.create)
Auth->>Ctrl: create(req, res)
Ctrl->>Svc: create(productData, context)
rect rgb(240, 248, 255)
Note over Svc,DB: BEGIN Database Transaction
Svc->>CodeGen: generateUniqueCode(Product, baseCode)
CodeGen-->>Svc: Normalized unique code (e.g. technova_audio_pro_x1_1)
alt Manual SKU provided
Svc->>Svc: Preserve user-entered SKU
else Status is active/pending & SKU is empty
Svc->>CodeGen: generateSku(Product, prefix)
CodeGen-->>Svc: Generated SKU (e.g. ELEC-00042)
else Status is draft & SKU empty
Svc->>Svc: Set SKU = null (Valid in draft)
end
Svc->>DB: INSERT INTO products (...) VALUES (...)
DB-->>Svc: Created Product record (UUID)
opt Dynamic Attributes Supplied
Svc->>DB: Bulk INSERT INTO product_attribute_values (...)
end
Svc->>DB: COMMIT Transaction
end
Svc->>Comp: CompletenessService.calculate(productId)
Comp->>DB: Compute & Upsert ProductCompleteness
Svc->>Audit: log({ action: 'CREATE', entity: 'Product', id })
Svc->>WS: emit('product:created', productPayload)
Svc-->>Ctrl: Hydrated Product with Associations
Ctrl-->>Client: HTTP 201 Created (Product JSON)
```
---
## 4. Lifecycle Status Transitions & Validation Rules
```mermaid
stateDiagram-v2
[*] --> Draft: Initial Save (POST)
Draft --> Pending: Submit for QA / Approval
Pending --> Active: Publish & Approve
Active --> Archived: Catalog Retirement
Archived --> Active: Reactivate
Pending --> Draft: QA Reject / Revisions Needed
note right of Draft
- Missing attributes allowed
- Missing assets allowed
- SKU may be null
- Completeness: 0% - 100%
end note
note right of Active
- Requires Valid SKU
- Name, Category, Brand, Unit required
- Completeness verified
- Ready for Syndication Push
end note
```
docs/product_engine_deep_dive/04_DEPENDENT_ENTITIES_AND_CASCADE_ENGINE.md
+107
View File
@@ -0,0 +1,107 @@
# 🔗 Dependent Entities & Cascading Architecture
## 1. Overview
A Product in modern PIM is not an isolated table row; it is an aggregated composite entity. This document details each sub-primitive, its database schema, cascade behaviors, and lifecycle rules.
---
## 2. Taxonomy & Hierarchy (Categories)
### Data Architecture
Categories are modeled as an Adjacency List hierarchy using self-referencing `parent_id`.
```mermaid
graph TD
Root[Root: Electronics] --> Audio[Category: Audio & Sound]
Audio --> Headphones[Subcategory: Wireless Headphones]
Audio --> Speakers[Subcategory: Bluetooth Speakers]
Root --> Computers[Category: Computers & Laptops]
```
### Inheritance & Rules
- **Category Inheritance**: When selecting a `Product Family`, the product automatically inherits the family's default category if none is set.
- **Slug Normalization**: Slugs are generated recursively (`electronics/audio-sound/wireless-headphones`).
- **Cascade Rule**: Deleting a category does NOT delete products; it sets product `category_id = NULL` to prevent orphaned cascades.
---
## 3. Brand & Manufacturer Registry
### Data Architecture
Brands store trademark data, manufacturer info, and logo DAM assets.
### Blueprint Filtering
- Product Families define `allowedBrands` in their blueprint.
- When creating a product from a family, the UI automatically filters the Brand dropdown to only include allowed brands.
- If allowed brands is empty, all active workspace brands are selectable.
---
## 4. Units of Measure (UOM) Engine
### Data Architecture & Conversions
- `unit_type`: Categorizes units (`Weight`, `Length`, `Volume`, `Count`, `Other`).
- `conversion_factor`: Standard multiplier against a base unit (e.g. `g` has factor `0.001` relative to base unit `kg`).
- **Conflict Handling**: The inline unit creation endpoint enforces unique code constraints. On `HTTP 409 Conflict`, the frontend auto-selects the existing unit matching the code/name to prevent workflow interruption.
---
## 5. Attributes, Sets & Groups (The EAV Model)
### Hierarchy Model
```mermaid
graph TD
Set[Attribute Set: Electronics Set] --> G1[Group: Technical Specifications]
Set --> G2[Group: Physical Dimensions]
Set --> G3[Group: Marketing & Media]
G1 --> A1[Attribute: Connectivity]
G1 --> A2[Attribute: Battery Life]
G2 --> A3[Attribute: Weight Grams]
G2 --> A4[Attribute: Color Spec]
G3 --> A5[Attribute: SEO Title]
```
### Supported Attribute Types & Validations
1. `text`: String values with `min_length` and `max_length`.
2. `textarea`: Multi-line text for descriptions and spec tables.
3. `number` / `decimal`: Integer or floating-point numbers with `min_value` and `max_value` limits (e.g., non-negative `min: 0` for physical weights).
4. `date`: ISO date format (`YYYY-MM-DD`).
5. `boolean`: Binary `true` / `false` flags.
6. `select`: Single-choice enumeration reading from `attribute_options` or `options` array.
7. `multiselect`: Comma-delimited or JSON array of allowed choices.
---
## 6. Digital Asset Management (DAM) & Media Junction
### Role Matrix
| Role Code | Display Label | Purpose | Completeness Evaluation |
| :--- | :--- | :--- | :--- |
| `hero_image` | **HERO IMAGE** | Primary high-res catalog cover image | Evaluated for completeness (is_primary) |
| `gallery_image`| **GALLERY** | Additional multi-angle product views | Optional supporting media |
| `thumbnail` | **THUMBNAIL** | Low-res compressed preview icon | Optional UI thumbnail |
| `video` | **VIDEO** | Product showcase video (MP4/WebM) | Optional media |
| `document` | **DOCUMENT** | PDF user manuals, safety sheets | Optional compliance assets |
---
## 7. Completeness Calculation Mathematical Engine
The PIM Data Completeness score is calculated as a real-time ratio (0% to 100%):
$$\text{Completeness } \% = \text{round}\left( \frac{\text{Fulfilled Fields}}{\text{Total Expected Fields}} \times 100 \right)$$
### Weighted Breakdown:
1. **General Prerequisites (4 fields)**:
- `name` (Product Name present)
- `category_id` (Category assigned)
- `brand_id` (Brand assigned)
- `unit_id` (Unit of Measure assigned)
2. **Evaluated Attributes**:
- Evaluates all configured attributes in the active Attribute Set.
- Each configured attribute value contributes proportionally to the attribute score.
3. **Required Media Assets**:
- Evaluates mandatory asset families (e.g. at least 1 primary Hero Image).
4. **Subscribed Syndication Channels**:
- Evaluates whether mandatory syndication channels are linked.
docs/product_engine_deep_dive/05_SYSTEM_GAPS_RISK_AUDIT_AND_OPTIMIZATION_BLUEPRINT.md
+48
View File
@@ -0,0 +1,48 @@
# 🔍 System Gaps, Risk Audit & Optimization Blueprint
## 1. Executive Summary
This document captures architectural vulnerabilities, potential race conditions, database indexing requirements, and optimization blueprints identified during the comprehensive Product Creation deep-dive.
---
## 2. Identified Vulnerabilities & Audit Findings
| Category | Finding & Risk | Impact | Resolution & Architecture Recommendation |
| :--- | :--- | :--- | :--- |
| **Concurrency** | Non-atomic SKU generation on concurrent product creations | Duplicate SKU generation if two products are created in the exact same millisecond | Use PostgreSQL sequence or database-level lock on `sku_counters` table rather than `COUNT(*)` lookups. |
| **Data Integrity** | Soft-deleted attribute options lingering in product values | A product displays a deleted option code with no human-readable label | In `product_attribute_values`, store both `option_id` (FK) and string fallback, or enforce cascade warning on attribute option deletion. |
| **Performance** | Missing composite index on `product_attribute_values` | Sluggish search queries when filtering products by multiple attribute values | Add compound index: `CREATE INDEX idx_pav_attr_val ON product_attribute_values(attribute_id, value);` |
| **Tenant Scope** | In-memory filtering instead of database query scoping | Unnecessary memory overhead fetching cross-tenant records | Always apply `where: { tenant_id }` in repository layer before executing Sequelize `findAll()`. |
---
## 3. High-Priority Database Indexing Strategy
To guarantee sub-50ms response times for a catalog containing 500,000+ products:
```sql
-- 1. Product Registry Primary Lookups
CREATE INDEX IF NOT EXISTS idx_products_tenant_status ON products(tenant_id, status) WHERE deleted_at IS NULL;
CREATE INDEX IF NOT EXISTS idx_products_code_tenant ON products(code, tenant_id);
CREATE INDEX IF NOT EXISTS idx_products_sku_tenant ON products(sku, tenant_id);
CREATE INDEX IF NOT EXISTS idx_products_family ON products(family_id);
CREATE INDEX IF NOT EXISTS idx_products_category ON products(category_id);
CREATE INDEX IF NOT EXISTS idx_products_brand ON products(brand_id);
-- 2. Dynamic Attribute EAV Indexing
CREATE INDEX IF NOT EXISTS idx_pav_product_attr ON product_attribute_values(product_id, attribute_id);
CREATE INDEX IF NOT EXISTS idx_pav_attr_value ON product_attribute_values(attribute_id, value);
-- 3. Completeness Indexing
CREATE INDEX IF NOT EXISTS idx_completeness_product_channel ON product_completeness(product_id, channel, locale);
-- 4. Media Asset Junction Indexing
CREATE INDEX IF NOT EXISTS idx_product_assets_product_primary ON product_assets(product_id, is_primary);
```
---
## 4. Architectural Next Steps & Roadmap
1. **Bulk Product Import & Syndication Queue**: Implement Redis / BullMQ worker pipeline for processing 10,000+ CSV / XML product imports asynchronously with streaming batch inserts.
2. **Channel Transformation Engine (Phase 5)**: Build dynamic attribute mapping formulas (e.g. mapping `weight_grams / 1000` to Shopify's `weight_kg`).
3. **Audit Log Timeline UI**: Render chronological diff timeline in product editor showing exact field changes and who approved them.
docs/product_engine_deep_dive/06_PRODUCT_LISTING_AND_QUERY_ARCHITECTURE.md
+95
View File
@@ -0,0 +1,95 @@
# 📋 Product Listing & Query Engine Architecture
## 1. Overview
The Product Listing system (`GET /api/v1/products`) powers the core catalog data grid. It supports multi-facet filtering (by category, brand, family, completeness, channel, status, tag, and custom attributes), full-text search across product name and SKU, server-side pagination, eager-loading of primary hero assets, and real-time completeness score aggregation.
---
## 2. API Endpoint Specification
- **Endpoint**: `GET /api/v1/products`
- **Query Parameters**:
- `page` (integer, default: 1)
- `limit` (integer, default: 20, max: 100)
- `search` (string, fuzzy search on `name`, `code`, `sku`)
- `status` (string or array: `draft`, `pending`, `active`, `archived`)
- `type` (string: `simple`, `variant`, `bundle`)
- `family_id` (UUID)
- `category_id` (UUID, matches category and its descendant subtrees)
- `brand_id` (UUID)
- `completeness_min` / `completeness_max` (integers 0-100)
- `channel` (string, e.g. `shopify`, `amazon`)
- `sortBy` (`name`, `sku`, `created_at`, `updated_at`, `completeness`, `price`)
- `sortOrder` (`ASC` or `DESC`)
---
## 3. Query Optimization & Eager-Loading Structure
To avoid the $N+1$ query problem, the product listing query uses targeted eager-loading with selected columns:
```javascript
const queryOptions = {
where: baseWhereClause,
attributes: [
'id', 'code', 'sku', 'name', 'price', 'stock', 'status', 'type',
'family_id', 'category_id', 'brand_id', 'unit_id', 'created_at', 'updated_at'
],
include: [
{
model: models.Catalog,
as: 'family',
attributes: ['id', 'code', 'name']
},
{
model: models.Category,
as: 'category',
attributes: ['id', 'code', 'name', 'slug']
},
{
model: models.Brand,
as: 'brand',
attributes: ['id', 'code', 'name', 'logo_url']
},
{
model: models.Unit,
as: 'unit',
attributes: ['id', 'code', 'name', 'symbol']
},
{
model: models.ProductAsset,
as: 'productAssets',
where: { is_primary: true },
required: false,
include: [
{
model: models.Asset,
as: 'asset',
attributes: ['id', 'url', 'thumbnail_url', 'name', 'mime_type']
}
]
},
{
model: models.ProductCompleteness,
as: 'completenessEntries',
where: { channel: 'default' },
required: false,
attributes: ['percentage', 'is_complete', 'missing_attributes', 'missing_assets']
}
],
order: [[sortField, sortOrder]],
limit,
offset: (page - 1) * limit
};
```
---
## 4. Frontend Data Grid Hydration & Performance
- **Primary Hero Image**: Displayed as a responsive 40x40 thumbnail from `productAssets[0].asset.thumbnail_url || productAssets[0].asset.url`.
- **Completeness Indicator**: Circular or pill progress badge colored dynamically:
- `0 - 49%`: Red (Incomplete)
- `50 - 84%`: Amber (Partially Configured)
- `85 - 99%`: Blue (Ready for Review)
- `100%`: Emerald Green (Fully Complete & Ready for Publish)
- **Fast Filter Sync**: Search query debounced at 300ms, updating URL search params to preserve filter state on browser reload.
docs/product_engine_deep_dive/README.md
+28
View File
@@ -0,0 +1,28 @@
# 📚 PIM Product Creation & Management Master Documentation
Welcome to the central architectural and operational knowledge base for the **Product Information Management (PIM)** engine. This directory contains end-to-end specifications, database table dictionaries, sequence diagrams, and optimization blueprints designed for software engineers, QA architects, and AI autonomous agents.
---
## 📑 Manuals & Reference Guides
| File | Scope & Contents |
| :--- | :--- |
| **[`01_EXECUTIVE_ARCHITECTURE_AND_ECOSYSTEM_MAP.md`](file:///Users/maskantech/Desktop/PIM/docs/product_engine_deep_dive/01_EXECUTIVE_ARCHITECTURE_AND_ECOSYSTEM_MAP.md)** | Global Entity-Relationship (ER) diagram, 5-layer dependency hierarchy, multi-tenant isolation principles, and core architectural invariants. |
| **[`02_DATABASE_SCHEMA_AND_TABLE_DICTIONARY.md`](file:///Users/maskantech/Desktop/PIM/docs/product_engine_deep_dive/02_DATABASE_SCHEMA_AND_TABLE_DICTIONARY.md)** | Complete table-by-table dictionary covering 10+ core tables (`products`, `catalogs`, `categories`, `brands`, `units`, `attributes`, `attribute_options`, `product_attribute_values`, `product_assets`, `product_completeness`), exact column types, constraints, and defaults. |
| **[`03_PRODUCT_CREATION_POST_API_LIFECYCLE.md`](file:///Users/maskantech/Desktop/PIM/docs/product_engine_deep_dive/03_PRODUCT_CREATION_POST_API_LIFECYCLE.md)** | Micro-detailed specification of `POST /api/v1/products`, request payload schema, transaction lifecycle sequence diagram, code/SKU generation rules, and status state machine. |
| **[`04_DEPENDENT_ENTITIES_AND_CASCADE_ENGINE.md`](file:///Users/maskantech/Desktop/PIM/docs/product_engine_deep_dive/04_DEPENDENT_ENTITIES_AND_CASCADE_ENGINE.md)** | In-depth breakdown of Category trees, Brand filters, Units of Measure conversions, Dynamic Attribute EAV models, DAM Media Asset roles, and the mathematical Completeness Calculation Engine. |
| **[`05_SYSTEM_GAPS_RISK_AUDIT_AND_OPTIMIZATION_BLUEPRINT.md`](file:///Users/maskantech/Desktop/PIM/docs/product_engine_deep_dive/05_SYSTEM_GAPS_RISK_AUDIT_AND_OPTIMIZATION_BLUEPRINT.md)** | Identified vulnerabilities, concurrency risks, recommended high-performance PostgreSQL composite indexes, and future syndication queue roadmap. |
| **[`06_PRODUCT_LISTING_AND_QUERY_ARCHITECTURE.md`](file:///Users/maskantech/Desktop/PIM/docs/product_engine_deep_dive/06_PRODUCT_LISTING_AND_QUERY_ARCHITECTURE.md)** | Complete blueprint for `GET /api/v1/products`, multi-facet filtering, eager-loading relations, sorting, pagination, and frontend grid hydration. |
---
## 🎯 Quick Verification Commands
```bash
# Verify backend server health
curl -sI http://localhost:5002/api/v1/categories
# Verify frontend build & dev server
curl -sI http://localhost:5173
cd productcatalogue_frontend && npx tsc --noEmit
```