support_backend/docs/03-ai-support-architecture.md

03 — AI Support Architecture

The AI Support Agent is the first support layer, not a generic chatbot. It must be product-aware, evidence-driven, and tightly bounded by deterministic policy.

1. High-level flow

Customer Problem
   → Problem Classification
   → Knowledge Retrieval (RAG)
   → Relevant Context
   → AI Reasoning
   → Next Action (direct solution / guided runbook / tool call / escalate)

2. Product knowledge system

Knowledge is retrieved, not stuffed into one giant system prompt. Use a RAG architecture scoped per product.

Conceptual structure per product:

Product
├── Overview
├── Features
├── Troubleshooting
├── Known Issues
├── Error Catalog
├── Runbooks
├── FAQs
├── Resolution Procedures
└── Product Operations

Example knowledge entry:

Knowledge ID:   KB-DQ-102
Product:        DocuQube
Feature:        PDF → HTML
Type:           Known Issue
Problem:        Conversion fails for complex layouts.
Symptoms:       Layout parser failure.
Error:          LAYOUT_PARSE_042
Cause:          Specific PDF layouts fail through the primary parser.
Recommended:    Use fallback parser.
Verification:   Retry conversion and confirm output.
Escalation:     Escalate if fallback parser fails.

Knowledge must be:

• Versioned, searchable, auditable
• Product-scoped, and category-scoped where necessary
• Permission-aware (never expose raw internal knowledge to customers — the AI converts retrieved knowledge into simple, product-specific guidance)
• Maintainable by administrators, with quality metadata:

Field	Purpose
version	Change tracking
status	draft / published / unpublished
effectiveDate	When it becomes eligible for retrieval
productScope, featureScope, categoryScope	Retrieval filters
validationStatus	Has this been verified to actually work?
owner, lastReview	Accountability / staleness detection
source	Where it originated (runbook author, resolved ticket, docs)

The AI should prefer validated knowledge over unvalidated entries when both are retrieved.

3. Knowledge retrieval example

Customer says: "My PDF is not converting to HTML."

The system resolves:

Product:      DocuQube
Feature:      PDF → HTML
Problem type: Conversion Failure

Then searches: feature documentation → error catalog → known issues → troubleshooting → runbooks → previously validated solutions. Only relevant, filtered knowledge is returned to the AI's context — never a bulk dump.

4. AI diagnosis

The AI produces a structured diagnosis, not prose alone:

{
  "product": "docuqube",
  "feature": "pdf_to_html",
  "problemType": "conversion_failure",
  "severity": "medium",
  "confidence": 0.91,
  "possibleCauses": ["unsupported_layout", "parser_failure", "processing_timeout"]
}

Confidence policy (configurable)

Confidence band	Behavior
High	AI may proceed automatically
Medium	AI may ask additional diagnostic questions
Low	Escalate to human support

Thresholds are stored in configuration, never hardcoded, and must be tunable per product/category without a deploy.

5. AI tool system

The AI requests tools; the application decides whether execution is allowed. The LLM never gets unrestricted database/application access.

Example tools (DocuQube):

Tool	Purpose
getDocumentStatus()	Read current processing state
getDocumentMetadata()	Read document metadata
getProcessingStatus()	Read pipeline stage status
getErrorDetails()	Read structured error info
retryConversion()	Re-trigger conversion job
retryOCR()	Re-trigger OCR job
enableFallbackParser()	Toggle fallback parser for this document
checkServiceStatus()	Read upstream service health
getSupportedFileTypes()	Read static capability info
escalateToSupport()	Trigger human escalation

Each tool definition must include:

interface AITool {
  name: string;
  description: string;
  inputSchema: ZodSchema;
  outputSchema: ZodSchema;
  permission: string;         // permission required to invoke
  riskLevel: "low" | "medium" | "high";
  supportedProducts: string[];
  auditRequired: boolean;
}

High-risk operations require deterministic policy checks and/or human approval before execution — the AI's request is a proposal, not an authorization.

6. Troubleshooting runbook engine

Runbooks are configurable step sequences, not something the LLM improvises. The AI communicates the runbook naturally in conversation; the workflow engine controls which steps are actually permitted next.

Example — PDF_HTML_CONVERSION_FAILURE:

Step 1: Check file size          → if over limit, guide customer to reduce size
Step 2: Check file type
Step 3: Check document structure
Step 4: Try fallback parser
Step 5: Verify conversion        → if still failing, escalate via configured path

Do not allow the LLM to invent arbitrary troubleshooting steps. The runbook engine is the source of the permitted sequence; the AI's job is presentation and interpretation of results, not authorship of new steps.

7. Guided customer experience

When the AI identifies a known solution, it presents:

Problem → Likely cause → Recommended action → Customer instruction → Verification

Example:

"Your document failed during layout processing." → "Enable fallback parser."

1. Open Conversion Settings
2. Open Advanced
3. Enable Use Fallback Parser
4. Save
5. Retry conversion

The system then waits for the customer/product action rather than assuming completion.

8. Verification (first-class, evidence-based)

Never assume "customer says done" equals "problem resolved." Prefer actual system evidence.

Conversion started → processing completed → HTML generated → output validated
   → Verification = SUCCESS → only then can AI mark "Resolved by AI"

Supported verification modes:

• Product signal verification (webhook/event from the product confirming success)
• Automated verification (poll a status endpoint via an approved tool)
• Customer confirmation (used as a secondary/confirming signal, not primary evidence)
• Agent verification (for human-handled cases)

9. AI safety and control

AI must never:

• Invent product behavior, configuration settings, or troubleshooting steps
• Execute unauthorized actions
• Access arbitrary customer data
• Expose internal notes or private knowledge
• Modify support configuration, SLA policies, or hierarchy
• Escalate arbitrarily without going through policy evaluation

AI can recommend. Deterministic application policies decide. Tool execution is always permission-aware, and high-risk actions require stronger controls (policy check and/or human approval) regardless of AI confidence.

10. Escalation trigger conditions (AI → Human)

Escalate when any of the following hold:

• AI cannot identify the issue
• Confidence is below the configured threshold
• No matching knowledge exists
• The runbook is exhausted without success
• A required tool execution fails
• The problem requires human judgment/intervention
• The problem looks like a product defect
• It's flagged as a critical incident
• The customer explicitly asks for a human
• Policy requires a human for this case type
• The AI cannot safely execute a required action

When this happens, the AI session hands off to a human support ticket with full context — see 04 and 05.

Example AI hand-off summary:

Problem:      PDF → HTML conversion failure.
Diagnosis:    Layout parser issue.
Steps attempted:
  ✓ File validation
  ✓ Processing status check
  ✓ Fallback parser
  ✗ Still failed
AI confidence: 62%
Recommendation: Investigate conversion service.