docs: add experiment designer redesign spec and update work in progress status

2025-12-11 14:44:44 -05:00 · 2025-08-08 00:42:01 -04:00
parent 1ac8296ab7
commit 524eff89fd
2 changed files with 450 additions and 5 deletions
--- a/docs/experiment-designer-redesign.md
+++ b/docs/experiment-designer-redesign.md
@@ -0,0 +1,434 @@
+# Experiment Designer Redesign (Production Baseline)
+
+This document defines the production-ready redesign of the Experiment Designer. It supersedes prior "modular refactor" notes and consolidates architecture, hashing, drift detection, UI/UX, state management, validation, plugin integration, saving/versioning, and export strategy. All implementation must adhere to this specification to ensure reproducibility, extensibility, and maintainability.
+
+---
+
+## 1. Goals
+
+- Provide a clear, fast, and intuitive hierarchical design workflow.
+- Guarantee reproducibility via deterministic hashing and provenance retention.
+- Surface plugin dependency health and schema drift early.
+- Support scalable experiments (many steps/actions) without UI degradation.
+- Enable structured validation (structural, parameter-level, semantic) before compilation.
+- Provide robust export/import and future multi-user collaboration readiness.
+- Maintain 100% type safety and consistent design patterns across the platform.
+
+---
+
+## 2. Conceptual Model
+
+Hierarchy:
+```
+Experiment
+  └─ Step (ordered, typed)
+        └─ Action (ordered, typed, provenance-bound)
+```
+
+Key invariants:
+- Step `order` is zero-based, contiguous after any mutation.
+- Action `orderIndex` is zero-based within its parent step.
+- Each action retains provenance: `{ source.kind, pluginId?, pluginVersion?, baseActionId? }`
+- Execution descriptors are retained for reproducibility (`transport`, `ros2`, `rest`, `retryable`, `timeoutMs`).
+
+---
+
+## 3. Hashing & Integrity Guarantees
+
+### 3.1 Purposes of Hashing
+- Detect structural drift between persisted design and working edits.
+- Bind design to plugin and action schemas for reproducibility.
+- Support execution graph provenance (hash at compile time).
+- Enable export integrity and offline verification.
+
+### 3.2 Design Hash Components
+Included:
+- Steps (ordered): `id`, `name`, `type`, `order`, `trigger.type`, sorted trigger condition keys.
+- Actions (per step, ordered): `id`, `type`, `source.kind`, `pluginId`, `pluginVersion`, `baseActionId`, `execution.transport`, parameter key set (NOT values by default—configurable).
+- Optional: parameter values (toggle design mode if needed—default: exclude values to reduce false-positive drift).
+
+Excluded:
+- Ephemeral UI state (`expanded`, selection state).
+- Human-friendly timestamps or transient meta.
+
+### 3.3 Canonicalization Steps
+1. Deep clone design subset.
+2. Remove `undefined` keys.
+3. Sort object keys recursively.
+4. Sort arrays where semantic ordering is not meaningful (e.g. condition key sets).
+5. JSON stringify (no whitespace).
+6. SHA-256 digest → hex.
+
+### 3.4 Incremental Hash Optimization
+Per-action hash → per-step hash → design hash:
+```
+H_action = hash(canonical(actionMeta))
+H_step = hash(stepMeta + concat(H_action_i))
+H_design = hash(globalMeta + concat(H_step_i))
+```
+Recompute only modified branches for responsiveness.
+
+### 3.5 Drift States
+| State        | Condition                                                                                          |
+|--------------|----------------------------------------------------------------------------------------------------|
+| Unvalidated  | No validation performed since load                                                                 |
+| Validated    | Last validated hash matches stored experiment integrity hash and working snapshot unchanged        |
+| Drift        | (A) Working snapshot changed since last validation OR (B) validated hash ≠ stored integrity hash   |
+| Plugin Drift | Design validated, but one or more action definitions (schema/provenance) no longer match registry  |
+
+### 3.6 Plugin Signature Drift
+Signature hash per action definition: hash of `(type + category + parameterSchema + transport + baseActionId + pluginVersion)`.
+- If signature mismatch for existing action instance: mark as “schema drift”.
+- Provide reconciliation CTA to refetch schema and optionally run migration.
+
+---
+
+## 4. UI Architecture
+
+High-level layout:
+
+```
+┌───────────────────────────────────────────────────────────────────────────┐
+│ Header: breadcrumbs • experiment name • hash badge • plugin deps summary  │
+├──────────────┬───────────────────────────────┬────────────────────────────┤
+│ Action       │ Step Flow (sortable linear)   │ Properties / Inspector      │
+│ Library      │ - Step cards (collapsible)     │ - Step editor               │
+│ - Search     │ - Inline action list           │ - Action parameters         │
+│ - Categories │ - DnD (steps/actions/library)  │ - Validation & drift panel  │
+│ - Plugins    │ - Structural markers           │ - Dependencies & provenance │
+├──────────────┴───────────────────────────────┴────────────────────────────┤
+│ Bottom Save Bar: dirty state • versioning • export • last saved • conflicts│
+└───────────────────────────────────────────────────────────────────────────┘
+```
+
+Component responsibilities:
+| Component                     | Responsibility |
+|------------------------------|----------------|
+| `DesignerShell`              | Data loading, permission guard, store boot |
+| `ActionLibraryPanel`         | Search/filter, categorized draggable items |
+| `StepFlow`                   | Rendering + reordering steps & actions |
+| `StepCard`                   | Step context container |
+| `ActionItem`                 | Visual + selectable action row |
+| `PropertiesPanel`            | Context editing (step/action) |
+| `ParameterFieldFactory`      | Schema → control mapping |
+| `ValidationPanel`            | Issue listing + filtering |
+| `DependencyInspector`        | Plugin + action provenance health |
+| `SaveBar`                    | Hash/drift/dirtiness/export/version controls |
+| `hashing.ts`                 | Canonicalization + incremental hashing |
+| `validators.ts`              | Rule execution (structural, parameter) |
+| `exporters.ts`               | Export bundle builder |
+| `store/useDesignerStore.ts`  | Central state store |
+
+---
+
+## 5. State Management
+
+Use a lightweight, framework-agnostic, fully typed store (Zustand). Core fields:
+
+```
+{
+  steps: ExperimentStep[]
+  dirty: Set<string>
+  selectedStepId?: string
+  selectedActionId?: string
+  lastPersistedHash?: string
+  lastValidatedHash?: string
+  lastValidatedSnapshot?: string
+  pluginSignatureIndex: Record<actionTypeOrId, string>
+  validationIssues: Record<entityId, string[]>
+  pendingSave?: boolean
+  conflict?: { serverHash: string; localHash: string }
+}
+```
+
+### Actions
+- `setSteps`, `upsertStep`, `removeStep`, `reorderStep`
+- `upsertAction`, `removeAction`, `reorderAction`
+- `markDirty(entityId)`
+- `computeDesignHash()`
+- `setValidationResult({ hash, issues, snapshot })`
+- `applyServerSync({ steps, hash })`
+- `recordConflict(serverHash, localHash)`
+
+---
+
+## 6. Drag & Drop
+
+Library → Step
+- Drag action definition placeholder onto step drop zone triggers action instantiation.
+Step reorder
+- Sortable vertical list with keyboard accessibility.
+Action reorder
+- Sortable within a step (no cross-step move initially; future extension).
+Framework: `@dnd-kit/core` + `@dnd-kit/sortable`.
+
+Accessibility:
+- Custom keyboard handlers: Up/Down to move selection; Alt+Up/Down to reorder.
+
+---
+
+## 7. Parameters & Control Mapping
+
+Mapping:
+| Schema Type | Control |
+|-------------|---------|
+| boolean     | Switch |
+| number (bounded) | Slider + numeric display |
+| number (unbounded) | Numeric input |
+| text short  | Text input |
+| text long   | Textarea (expandable) |
+| select/enumeration | Select / Combobox |
+| multi-select | Tag list (future) |
+| json/object | Lazy code editor (future) |
+
+Enhancements:
+- Show required indicator
+- Inline validation message
+- Revert-to-default button if default provided
+- Modified badge (dot) for overridden parameters
+
+---
+
+## 8. Validation System
+
+Levels:
+1. Structural
+   - No empty step names
+   - Steps must have valid `type`
+   - Conditional/loop must define required condition semantics
+2. Parameter
+   - Required values present
+   - Number bounds enforced
+   - Enum membership
+3. Semantic
+   - No unresolved plugin dependency
+   - Unique action IDs and step IDs
+   - Loop guard presence (future)
+4. Execution Preflight (optional)
+   - Detect obviously irrecoverable execution (e.g. empty parallel block)
+
+Severity classification:
+- Error: blocks save/compile
+- Warning: surfaced but non-blocking
+- Info: advisory (e.g. unused parameter)
+
+Store format:
+```
+validationIssues: {
+  [entityId: string]: { severity: "error" | "warning" | "info"; message: string }[]
+}
+```
+
+Rendering:
+- Badge on step/actions with count + highest severity color.
+- Panel filter toggles (Errors / Warnings / All).
+- Inline icon markers.
+
+---
+
+## 9. Plugin Integration & Drift
+
+Definitions loaded into `ActionRegistry`:
+```
+{
+  id,
+  type,
+  name,
+  category,
+  parameters[],
+  source: { kind, pluginId?, pluginVersion?, baseActionId? },
+  execution?,
+  parameterSchemaRaw
+}
+```
+
+Per-definition signatureHash computed:
+`hash(type + category + JSON(parameterSchemaRaw) + execution.transport + baseActionId + pluginVersion)`
+
+Per-action drift logic:
+- If action.source.pluginVersion missing in registry → Blocked (hard drift).
+- If signatureHash mismatch → Soft drift (allow edit, prompt reconcile).
+- Provide “Reconcile” overlay listing changed parameters (added / removed / type changed).
+
+---
+
+## 10. Saving & Versioning
+
+Save triggers:
+- Manual (primary save button or Cmd+S)
+- Debounced auto-save (idle > 5s, no pending errors)
+- Forced save before export
+
+Payload:
+```
+{
+  id,
+  visualDesign: { steps, version, lastSaved },
+  createSteps: true,
+  compileExecution: true
+}
+```
+
+Version management:
+- If structural change (add/remove/reorder step/action or change type) → version bump (auto unless user disables).
+- Parameter-only changes may optionally not bump version (configurable toggle; default: no bump).
+- UI: dropdown / toggle “Increment version on save”.
+
+Conflict detection:
+- Server returns persisted integrityHash.
+- If local lastPersistedHash differs and serverHash differs from our pre-save computed hash → conflict modal.
+- Provide:
+  - View diff summary (step added/removed, action count deltas).
+  - Override server (if authorized) or Reload.
+
+---
+
+## 11. Export / Import
+
+Export bundle structure:
+```json
+{
+  "format": "hristudio.design.v1",
+  "exportedAt": "...ISO8601...",
+  "experiment": {
+    "id": "...",
+    "name": "...",
+    "version": 4,
+    "integrityHash": "...",
+    "steps": [... canonical steps ...],
+    "pluginDependencies": ["pluginA@1.2.1", "robot.voice@0.9.3"]
+  },
+  "compiled": {
+    "graphHash": "...",
+    "steps": 12,
+    "actions": 47,
+    "plan": { /* normalized execution nodes */ }
+  }
+}
+```
+
+Import validation:
+- Check `format` signature & version.
+- Recompute design hash vs embedded.
+- Check plugin availability (list missing).
+- Offer “Install required plugins” if privileges allow.
+
+---
+
+## 12. Accessibility & UX Standards
+
+- Keyboard traversal (Tab / Arrow) across steps & actions.
+- Focus ring on selected entities.
+- ARIA labels on all interactive controls (drag handles, add buttons).
+- Color usage passes contrast (WCAG 2.1 AA).
+- Non-color indicators for drift (icons / labels).
+- All icons: Lucide only.
+
+---
+
+## 13. Performance Considerations
+
+- Virtualize StepFlow when step count > threshold (e.g. 50).
+- Memoize parameter forms (avoid re-render on unrelated step changes).
+- Incremental hashing—avoid full recompute on each keystroke.
+- Lazy-load advanced components (JSON editor, large schema viewers).
+
+---
+
+## 14. Testing Strategy
+
+Unit Tests:
+- Hash determinism (same structure -> same hash).
+- Hash sensitivity (reorder step / change action type -> different hash).
+- Signature drift detection logic.
+- Validation rule coverage (structural & parameter).
+
+Integration Tests:
+- Save cycle (design → save → fetch → hash equality).
+- Plugin dependency missing scenario.
+- Conflict resolution workflow.
+
+Edge Cases:
+- Empty experiment (0 steps) → validation error.
+- Conditional step with no conditions → error.
+- Loop step missing guard → warning (future escalation).
+- Plugin removed post-design → plugin drift surfaced.
+
+---
+
+## 15. Migration Plan (Internal)
+
+1. Introduce new store + hashing modules.
+2. Replace current `BlockDesigner` usage with `DesignerShell`.
+3. Port ActionLibrary / StepFlow / PropertiesPanel to new contract.
+4. Add SaveBar + drift/UI overlays.
+5. Remove deprecated legacy design references (no “enhanced” terminology).
+6. Update docs cross-links (`project-overview.md`, `implementation-details.md`).
+7. Add export/import UI.
+8. Stabilize, then enforce hash validation before trial creation.
+
+---
+
+## 16. Security & Integrity
+
+- Server must recompute its own structural hash during compile to trust design.
+- Client-submitted integrityHash considered advisory; never trusted alone.
+- Plugin versions pinned explicitly inside actions (no implicit latest resolution).
+- Attempting to execute an experiment with unresolved drift prompts required validation.
+
+---
+
+## 17. Future Extensions
+
+| Feature                  | Description |
+|--------------------------|-------------|
+| Real-time co-editing     | WebSocket presence + granular patch sync |
+| Branching workflows      | Conditional branches forming DAG (graph mode) |
+| Step templates library   | Shareable reproducible step/action sets |
+| Parameter presets        | Save named parameter bundles per action type |
+| Timeline estimation view | Aggregate predicted duration |
+| Replay / provenance diff | Compare two design versions side-by-side |
+| Plugin action migration  | Scripted param transforms on schema changes |
+| Execution simulation     | Dry-run graph traversal with timing estimates |
+
+---
+
+## 18. Implementation Checklist (Actionable)
+
+- [ ] hashing.ts (canonical + incremental)
+- [ ] validators.ts (structural + param rules)
+- [ ] store/useDesignerStore.ts
+- [ ] ActionRegistry rewrite with signature hashing
+- [ ] ActionLibraryPanel (search, categories, drift indicators)
+- [ ] StepFlow + StepCard + ActionItem (DnD with @dnd-kit)
+- [ ] PropertiesPanel + ParameterFieldFactory
+- [ ] ValidationPanel + badges
+- [ ] DependencyInspector + plugin drift mapping
+- [ ] SaveBar (dirty, versioning, export)
+- [ ] Exporter (JSON bundle) + import hook
+- [ ] Conflict modal
+- [ ] Drift reconciliation UI
+- [ ] Unit & integration tests
+- [ ] Docs cross-link updates
+- [ ] Remove obsolete legacy code paths
+
+(Track progress in `docs/work_in_progress.md` under “Experiment Designer Redesign Implementation”.)
+
+---
+
+## 19. Cross-References
+
+- See `docs/implementation-details.md` (update with hashing + drift model)
+- See `docs/api-routes.md` (`experiments.update`, `experiments.validateDesign`)
+- See `docs/database-schema.md` (`steps`, `actions`, provenance fields)
+- See `docs/project-overview.md` (designer feature summary)
+- See `docs/work_in_progress.md` (status tracking)
+
+---
+
+## 20. Summary
+
+This redesign formalizes a production-grade, reproducible, and extensible experiment design environment with deterministic hashing, plugin-aware action provenance, structured validation, export integrity, and a modular, performance-conscious UI framework. Implementation should now proceed directly against this spec; deviations require documentation updates and justification.
+
+--- 
+End of specification.
--- a/docs/work_in_progress.md
+++ b/docs/work_in_progress.md
@@ -4,7 +4,7 @@

 ### Experiment Designer Iteration (February 2025)

-#### **Current Focus: Modularized Designer (ActionLibrary / StepFlow / PropertiesPanel / Registry) & Validation Drift Handling**
+#### **Current Focus: Experiment Designer Redesign (Hashing / Drift / Action Library / Properties / DnD / Save & Export)**
 **Status**: In active iteration (not stable)

 The experiment designer has been refactored into modular components (registry + library + flow + properties panel). Active iteration now targets validation drift visibility, richer parameter controls, and continued provenance/compilation integrity.
@@ -54,9 +54,9 @@ The experiment designer has been refactored into modular components (registry +
 **Current State Summary:**
 Functional baseline with modular extraction complete. Parameter UI upgraded (boolean → Switch, ranged number → Slider). Hash drift indicator implemented (Validated / Drift / Unvalidated). Still pending: enum grouping polish, keyboard DnD accessibility, version pin drift resolution, structured plugin load error surfaces.

-### Seed Script Simplification & Repository Integration (stable)
+### Experiment Designer Redesign Implementation (in progress)

-#### **Unified Repository-Based Plugin Loading**
+#### **Status Snapshot**
 Simplified and unified the seed scripts to load all plugins (core and robot) through the same repository sync mechanism.

 **Seed Script Consolidation:**
@@ -90,9 +90,20 @@ bun db:seed  # Single command loads everything

 ---

-### Plugin Store Repository Integration
+- Spec Document: `docs/experiment-designer-redesign.md` (completed)
+- Hashing Model: Implementing (canonical + incremental planned)
+- State Store: Planned (Zustand-based)
+- Action Library: Pending rebuild (categorization + search + drift markers)
+- Step Flow: Existing structure to be replaced with new DnD + keyboard support
+- Properties Panel: To adopt dynamic ParameterFieldFactory (Switch / Slider / etc.)
+- Validation Layer: Rule set drafting (structural + param + plugin)
+- Drift Detection: Planned (design vs last validated hash + plugin signature drift)
+- Save / Versioning: Pending (auto-save + manual + conflict detection)
+- Export / Import: Export bundle utility planned
+- Removal of legacy naming (“enhanced” / transitional) in progress
+- Docs Cross-linking: Partially updated

-#### **Complete Repository Synchronization System**
+#### **Next Milestones**
 Fixed and implemented full repository synchronization for dynamic plugin loading from remote repositories.

 **Core Fixes:**