pdf/docs/phase3-dev1-plan.md

Phase 3 — Dev-1 Plan (Form Filling · Hit-Testing · Test Harness) — v2

Status: Review before build (reworked with your decisions). No code changed. Grounded in the current codebase (verified). Scope: the three Phase-3 items not blocked by the content-stream analyser. Build them in parallel with (separate) content-stream work.

Locked decisions (from review)

1. Form fill UI → Acrobat-style on-page overlay inputs (edit fields directly on the page), built now — not the lighter in-panel version.
2. Form rendering → production-grade (FPDF_FFLDraw + proper appearance regeneration; robust per-type).
3. Hit-testing → build BOTH the engine hitGlyph/spatial index (the roadmap Dev-1 deliverable — not deferred, since it's not scheduled in any other phase) and the frontend real-time selection model. They share one algorithm; the engine is authoritative/tested/SDK-exposed, the frontend is the responsive web layer.
4. Regression gate → freeze PDFium baseline now, becomes meaningful when the custom renderer lands.
5. Fuzzing → Windows AND Linux (clang-cl on Windows + clang on Linux).
6. Corpus → download script into a gitignored dir (500+ public PDFs).

None of these need the content-stream analyser. They build on Phase-1 glyphs, the form-field reader, and the existing gtest/ASan infra.

---

Workstream A — Production-Grade Form Filling + Acrobat-Style On-Page Editing

Current state (verified)

• ✅ update_field handler exists (pdfium_document.cpp:1809) — sets /V, bool→checkbox, saves. ~50%.
• ✅ Form-fill env initialized; field reader returns name/value/type/flags/options; Forms tab lists them (read-only).
• ❌ render() uses flags=0 (line 717) → form widgets not drawn (filled values invisible).
• ❌ No /AS for checkboxes, no FORM_SetIndexSelected for choices, no appearance regen, no on-page edit UI.

Build — Engine (C++)

1. Render forms (production): retain the form-fill env on the document; in render() call FPDF_FFLDraw(formHandle, bitmap, page, x, y, w, h, rotate, FPDF_ANNOT) after FPDF_RenderPageBitmap so field appearances (incl. filled values) composite into the page image.
2. Complete update_field per field type:
   • Text (Tx): set /V; regenerate appearance (via form env FORM_ReplaceSelection/FORM_ForceToKillFocus, or set AcroForm /NeedAppearances) so the value renders + exports.
   • Checkbox/Radio (Btn): set /V and /AS to the field's on-state name (read the AP states); radios update the sibling group.
   • Choice (Ch): combo → /V; listbox/multi → FORM_SetIndexSelected(formHandle, page, index, selected) (+ /V).
3. Robust field identity: match by the same id convention as delete_annotation (NM else anno_<page>_<idx>), not raw index parsing.
4. Save: incremental (form edits are dict changes); save_full on export.

Build — Gateway

• Extend UpdateFieldData if needed: value: str | bool, optional selectedIndices: int[] (listbox multi-select).
• (Optional) a GET /…/form-fields convenience endpoint — or keep reading them from /annotations (already includes field props).

Build — Frontend (Acrobat-style on-page editing)

1. FormFieldLayer (new viewer layer, sibling to AnnotationLayer): for each widget annotation on the page, render an HTML control positioned over the field rect (×zoom), type-mapped:
   • Tx → <input> / <textarea> (multiline per fieldFlags)
   • Btn checkbox → <input type=checkbox>; radio → grouped radios
   • Ch combo → <select> (or input+datalist if editable); listbox → <select multiple>
   • Sig → click-to-sign placeholder (reuses Signature flow)
2. Positioning: field rect comes from extractAnnotations (top-left device space) → place + scale with zoom, like the other layers; transparent styling so the PDF field box shows through.
3. Interaction: edit on the page → on blur/change emit update_field → optimistic value + save → page re-renders with baked appearance (so it's consistent on export).
4. Modes: a "Fill & Sign" affordance to toggle the form layer on (so inputs don't interfere with other tools).

Acceptance

On a form PDF you can type into text fields, tick checkboxes, pick dropdowns directly on the page; values persist after save, render in the page image, and survive export. Verified on text_form / combobox_form / listbox_form.

Risk / effort

PDFium form rendering + appearance regen is the classic gotcha (/AS state names vary; multiline/comb fields). On-page overlay positioning across zoom/rotation needs care. ~1.5–2 weeks.

---

Workstream B — Adobe-Level Hit-Testing & Text Selection (engine + frontend)

Current state (verified)

• ✅ Glyph bounds (extractTextWithBounds).
• 🟡 SelectionLayer = rectangular marquee (rectsIntersect) — not reading-order selection.

Build — Engine (the roadmap Dev-1 deliverable; authoritative + SDK)

1. Reading-order model per page: line detection (cluster by baseline/y), order glyphs L→R within line, top→bottom across lines; assign sequential index. (v1 = LTR single/simple multi-column; note RTL/complex as limitation.)
2. Spatial index: sorted-vector + binary search (or uniform grid) over glyph rects.
3. APIs (bound to Python/SDK):
   • hit_glyph(x, y) → {glyphIndex, caretSide} (nearest glyph + before/after).
   • select_range(p1, p2) → {glyphIndices[], text} (ordered range between two points).
   • word_at(x,y), line_at(x,y).
4. Tests: corpus-based accuracy (hit/range correctness; reading order sanity).

Build — Frontend (Adobe-grade interactive selection)

1. TextSelectionModel (client-side): mirrors the engine's reading-order + spatial index for real-time interaction (no HTTP per mouse-move). Shares the algorithm; an automated test compares frontend vs engine selection on the corpus to prevent drift.
2. Interactions (Adobe parity):
   • Drag = anchor→focus reading-order range (flows across lines, partial first/last line).
   • Double-click = word, triple-click = line/paragraph.
   • Shift-click / Shift-drag = extend; caret position; Ctrl/⌘+A = select page.
   • Copy = ordered text with correct spaces/newlines.
   • (Stretch) cross-page selection (Adobe does this; v1 may be per-page — flagged).
3. Visual: selection rendered as highlighted glyph runs (smooth at any zoom).

Acceptance

Selection behaves like Acrobat/a browser: drag selects a flowing range across lines, double/triple-click select word/line, shift extends, copy yields correctly-ordered text; smooth on a dense/100-page doc. Engine hit_glyph/select_range exposed + tested. Also the foundation for "click a glyph to edit it" once content-stream editing lands.

Risk / effort

Reading order for columns/RTL is hard — v1 targets LTR; complex layouts flagged. Engine + frontend + parity test. ~1.5 weeks.

---

Workstream C — Fuzz + Regression Harness (Windows + Linux)

Current state (verified)

• ✅ 41 corpus PDFs; gtest suite (engine/tests/); CTest; ASan/UBSan presets (windows-asan/linux-asan).

Build — Regression suite

1. tests/regression/ — Python harness: render each corpus PDF (engine bindings) at 144 DPI → PNG → SSIM vs baseline/; fail < 0.95; HTML/JSON report.
2. Freeze the PDFium baseline now (golden images). When the custom renderer lands, switch engine-render to the custom path → gate becomes the real safety net.
3. scripts/fetch_corpus.ps1/.sh — pull 500+ permissive public PDFs (pdf.js / pdfium test resources) into a gitignored corpus dir.
4. CI: regression job wired as a gate.

Build — Fuzzing (Windows + Linux)

1. tests/fuzz/fuzz_load_render.cpp — LLVMFuzzerTestOneInput(data,size) → loadFromMemory → page/render/extractText/applyEdits; seed = the 41 PDFs.
2. Toolchain: Linux = clang -fsanitize=fuzzer,address,undefined; Windows = clang-cl -fsanitize=fuzzer,address (UBSan partial on Windows — run full UBSan on Linux). New CMake fuzz preset(s) (fuzz-linux, fuzz-windows).
3. engine/include/pdfengine/hardened_limits.h — MAX_OBJECTS / MAX_STREAM_SIZE / MAX_PAGE_COUNT checked in load/parse paths.
4. Runs: nightly fuzz (both OSes), ASan/UBSan over the existing gtest suite via the *-asan presets.

Acceptance

CI runs SSIM regression (green baseline now); fuzz targets build and run on Windows + Linux, N hours zero-crash, ASan clean; UBSan clean on Linux. hardened_limits guards enforced.

Risk / effort

clang-cl + libFuzzer on Windows needs LLVM installed (the MSVC windows-asan preset stays for unit-test ASan). Corpus licensing → permissive sets only. ~1–1.5 weeks, mostly background/parallel.

---

Build order & parallelization

1. A — Form filling first (real Adobe feature; one engine rebuild; production-grade on-page editing).
2. B — Hit-testing/selection next (engine spatial index + frontend Adobe selection; sets up future text editing).
3. C — Harness stood up from day 1 in the background (protects later Dev-2/Dev-3 content-stream work).

All independent of each other and of the content-stream analyser. Each ends with: engine rebuild (A, B-engine) + frontend typecheck/build + end-to-end verification, and the gateway test instance stopped afterward.

Revised effort

Workstream	Effort	Engine rebuild?
A — Form filling (on-page, production)	~1.5–2 wk	yes
B — Hit-testing (engine + frontend Adobe selection)	~1.5 wk	yes
C — Fuzz + regression (Win + Linux)	~1–1.5 wk	new fuzz preset

What this delivers

Form filling (Acrobat-style, on-page), Adobe-grade text selection/copy (engine-authoritative + responsive UI), and a production quality safety net (regression + fuzz on Windows & Linux). Still not editing existing page text/images — that's the content-stream chain (separate critical path).