docs: consolidate and archive documentation

- Move 30+ outdated docs to docs/_archive/ - Move obsolete root files to _archive/ - Update README.md (Better Auth, current features) - Update docs/README.md (new architecture diagram) - Update docs/quick-reference.md (consolidated) - Update docs/project-status.md (March 2026 state) - Update docs/nao6-quick-reference.md (14 actions, Docker services) - Update docs/implementation-guide.md (Better Auth, git submodule) - Update docs/proposal.tex (timeline updates) - Archive errors.txt, plugin_dump.json, test HTML files
2026-03-23 19:27:51 -04:00 · 2026-03-22 16:38:28 -04:00
parent add3380307
commit cf3597881b
46 changed files with 439 additions and 1193 deletions
--- a/docs/_archive/block-designer-implementation.md
+++ b/docs/_archive/block-designer-implementation.md
@@ -0,0 +1,481 @@
+# Block Designer Implementation Tracking
+
+## Project Status: COMPLETED ✅
+
+**Implementation Date**: December 2024  
+**Total Development Time**: ~8 hours  
+**Final Status**: Production ready with database integration
+
+## ✨ Key Improvements Implemented
+
+### 1. **Fixed Save Functionality** ✅
+- **API Integration**: Added `visualDesign` field to experiments.update API route
+- **Database Storage**: Visual designs are saved as JSONB to PostgreSQL with GIN indexes
+- **Real-time Feedback**: Loading states, success/error toasts, unsaved changes indicators
+- **Auto-recovery**: Loads existing designs from database on page load
+
+### 2. **Proper Drag & Drop from Palette** ✅
+- **From Palette**: Drag blocks directly from the palette to the canvas
+- **To Canvas**: Drop blocks on main canvas or into control structures
+- **Visual Feedback**: Clear drop zones, hover states, and drag overlays
+- **Touch Support**: Works on tablets and touch devices
+
+### 3. **Clean, Professional UI** ✅
+- **No Double Borders**: Fixed border conflicts between panels and containers
+- **Consistent Spacing**: Proper padding, margins, and visual hierarchy
+- **Modern Design**: Clean color scheme, proper shadows, and hover effects
+- **Responsive Layout**: Three-panel resizable interface with proper constraints
+
+### 4. **Enhanced User Experience** ✅
+- **Better Block Shapes**: Distinct visual shapes (hat, action, control) for different block types
+- **Parameter Previews**: Live preview of block parameters in both palette and canvas
+- **Intuitive Selection**: Click to select, visual selection indicators
+- **Smart Nesting**: Easy drag-and-drop into control structures with clear drop zones
+
+## What Was Built
+
+### Core Interface
+- **Three-panel layout**: Block Library | Experiment Flow | Properties
+- **Dense, structured design** replacing freeform canvas approach
+- **Resizable panels** with proper responsive behavior
+- **Dashboard integration** with existing breadcrumb system
+
+### Block System
+- **Six block categories** with distinct visual design:
+  - Events (Green/Play) - Trial triggers
+  - Wizard (Purple/Users) - Human actions  
+  - Robot (Blue/Bot) - Automated actions
+  - Control (Orange/GitBranch) - Flow control
+  - Sensors (Green/Activity) - Data collection
+- **Shape-based functionality**:
+  - Action blocks: Standard rounded rectangles
+  - Control blocks: C-shaped with nesting areas
+  - Hat blocks: Event triggers with distinctive tops
+- **Parameter system** with type-safe inputs and live preview
+
+### Advanced Features
+- **dnd-kit integration** for reliable cross-platform drag and drop
+- **Block nesting** for control structures (repeat, if statements)
+- **Visual hierarchy** with indentation and connecting lines
+- **Real-time parameter editing** in dedicated properties panel
+- **Block removal** from nested structures
+- **Parameter preview** in block library drawer
+
+### Database Integration
+- **Enhanced schema** with new JSONB columns:
+  - `visual_design`: Complete block layout and parameters
+  - `execution_graph`: Compiled execution sequence  
+  - `plugin_dependencies`: Required robot platform plugins
+- **GIN indexes** on JSONB for fast query performance
+- **Plugin registry** tables for extensible block types
+
+## 🏗️ Technical Architecture
+
+### Core Components
+
+1. **EnhancedBlockDesigner** - Main container component
+2. **BlockPalette** - Left panel with draggable block categories
+3. **SortableBlock** - Individual block component with drag/sort capabilities
+4. **DroppableContainer** - Drop zones for control structures
+5. **DraggablePaletteBlock** - Draggable blocks in the palette
+
+### Block Registry System
+
+```typescript
+class BlockRegistry {
+  private blocks = new Map<string, PluginBlockDefinition>();
+  
+  // Core blocks: Events, Wizard Actions, Robot Actions, Control Flow, Sensors
+  // Extensible plugin architecture for additional robot platforms
+}
+```
+
+### Data Flow
+
+```
+1. Palette Block Drag → 2. Canvas Drop → 3. Block Creation → 4. State Update → 5. Database Save
+     ↓                      ↓                ↓                  ↓                 ↓
+ DraggablePaletteBlock → DroppableContainer → BlockRegistry → React State → tRPC API
+```
+
+## 🎨 Block Categories & Types
+
+### Events (Green - Play Icon)
+- **when trial starts** - Hat-shaped trigger block
+
+### Wizard Actions (Purple - Users Icon)  
+- **say** - Wizard speaks to participant
+- **gesture** - Wizard performs physical gesture
+
+### Robot Actions (Blue - Bot Icon)
+- **say** - Robot speaks using TTS
+- **move** - Robot moves in direction/distance
+- **look at** - Robot orients gaze to target
+
+### Control Flow (Orange - GitBranch Icon)
+- **wait** - Pause execution for time
+- **repeat** - Loop container with nesting
+- **if** - Conditional container with nesting
+
+### Sensors (Green - Activity Icon)
+- **observe** - Record behavioral observations
+
+## Technical Implementation
+
+### Drag & Drop System
+- **Library**: @dnd-kit/core with sortable and utilities
+- **Collision Detection**: closestCenter for optimal drop targeting
+- **Sensors**: Pointer (mouse/touch) + Keyboard for accessibility
+- **Drop Zones**: Main canvas, control block interiors, reordering
+
+### State Management
+```typescript
+interface BlockDesign {
+  id: string;
+  name: string;
+  description: string;
+  blocks: ExperimentBlock[];
+  version: number;
+  lastSaved: Date;
+}
+```
+
+### Database Schema
+```sql
+-- experiments table
+visualDesign JSONB,           -- Complete block design
+executionGraph JSONB,         -- Compiled execution plan  
+pluginDependencies TEXT[],    -- Required robot plugins
+
+-- GIN index for fast JSONB queries
+CREATE INDEX experiment_visual_design_idx ON experiments USING GIN (visual_design);
+```
+
+### API Integration
+```typescript
+// tRPC route: experiments.update
+updateExperiment.mutate({
+  id: experimentId,
+  visualDesign: {
+    blocks: design.blocks,
+    version: design.version,
+    lastSaved: new Date().toISOString(),
+  }
+});
+```
+
+### Architecture Decisions
+1. **Abandoned freeform canvas** in favor of structured vertical list
+2. **Used dnd-kit instead of native drag/drop** for reliability
+3. **Integrated with existing dashboard patterns** rather than custom UI
+4. **JSONB storage** for flexible schema evolution
+5. **Plugin-based block registry** for robot platform extensibility
+
+### Key Components
+- `EnhancedBlockDesigner.tsx` - Main interface (1,200+ lines)
+- `BlockRegistry` class - Manages available block types
+- Database schema extensions for visual design storage
+- Breadcrumb integration with existing dashboard system
+
+### Performance Optimizations
+- **Efficient rendering** with minimal re-renders
+- **Direct DOM manipulation** during drag operations
+- **Lazy loading** of block libraries
+- **Optimized state management** with React hooks
+
+## Challenges Solved
+
+### 1. Layout Conflicts
+- **Problem**: Full-screen designer conflicting with dashboard layout
+- **Solution**: Integrated within dashboard container with proper height management
+
+### 2. Drag and Drop Reliability
+- **Problem**: Native HTML drag/drop was buggy and inconsistent
+- **Solution**: Switched to dnd-kit for cross-platform reliability
+
+### 3. Control Flow Nesting
+- **Problem**: Complex logic for nested block structures
+- **Solution**: Droppable containers with visual feedback and proper data management
+
+### 4. Breadcrumb Integration
+- **Problem**: Custom breadcrumb conflicting with dashboard system
+- **Solution**: Used existing `useBreadcrumbsEffect` hook for proper integration
+
+### 5. Parameter Management
+- **Problem**: Complex parameter editing workflows
+- **Solution**: Dedicated properties panel with type-safe form controls
+
+## Code Quality Improvements
+
+### Removed Deprecated Files
+- `BlockDesigner.tsx` - Old implementation
+- `ExperimentDesigner.tsx` - Card-based approach
+- `ExperimentDesignerClient.tsx` - Wrapper component
+- `FlowDesigner.tsx` - React Flow attempt
+- `FreeFormDesigner.tsx` - Canvas approach
+- `flow-theme.css` - React Flow styling
+
+### Documentation Cleanup
+- Removed outdated step-by-step documentation
+- Removed planning documents that are now implemented
+- Consolidated into single comprehensive guide
+- Added implementation tracking (this document)
+
+### Code Standards
+- **100% TypeScript** with strict type checking
+- **Emoji-free interface** using only lucide icons
+- **Consistent naming** following project conventions
+- **Proper error handling** with user-friendly messages
+- **Accessibility support** with keyboard navigation
+
+## Database Schema Changes
+
+### New Tables
+```sql
+-- Robot plugin registry
+CREATE TABLE hs_robot_plugin (
+  id UUID PRIMARY KEY,
+  name VARCHAR(255) NOT NULL,
+  version VARCHAR(50) NOT NULL,
+  -- ... plugin metadata
+);
+
+-- Block type registry  
+CREATE TABLE hs_block_registry (
+  id UUID PRIMARY KEY,
+  block_type VARCHAR(100) NOT NULL,
+  plugin_id UUID REFERENCES hs_robot_plugin(id),
+  shape block_shape_enum NOT NULL,
+  category block_category_enum NOT NULL,
+  -- ... block definition
+);
+```
+
+### Enhanced Experiments Table
+```sql
+ALTER TABLE hs_experiment ADD COLUMN visual_design JSONB;
+ALTER TABLE hs_experiment ADD COLUMN execution_graph JSONB;
+ALTER TABLE hs_experiment ADD COLUMN plugin_dependencies TEXT[];
+
+CREATE INDEX experiment_visual_design_idx ON hs_experiment 
+USING gin (visual_design);
+```
+
+### New Enums
+```sql
+CREATE TYPE block_shape_enum AS ENUM (
+  'action', 'control', 'value', 'boolean', 'hat', 'cap'
+);
+
+CREATE TYPE block_category_enum AS ENUM (
+  'wizard', 'robot', 'control', 'sensor', 'logic', 'event'
+);
+```
+
+## User Experience Achievements
+
+### Workflow Improvements
+- **Reduced complexity**: No more confusing freeform canvas
+- **Clear hierarchy**: Linear top-to-bottom execution order
+- **Intuitive nesting**: Visual drop zones for control structures
+- **Fast iteration**: Quick block addition and configuration
+- **Professional feel**: Clean, dense interface design
+
+### Accessibility Features
+- **Keyboard navigation** through all interface elements
+- **Screen reader support** with proper ARIA labels
+- **Touch-friendly** sizing for tablet interfaces
+- **High contrast** color schemes for visibility
+- **Clear visual hierarchy** with consistent typography
+
+## Future Enhancement Opportunities
+
+### Short Term
+- **Inline parameter editing** in block drawer
+- **Block templates** with pre-configured parameters
+- **Export/import** of block designs
+- **Undo/redo** functionality
+
+### Medium Term
+- **Real-time collaboration** for multi-researcher editing
+- **Execution visualization** showing current block during trials
+- **Error handling blocks** for robust trial management
+- **Variable blocks** for data manipulation
+
+### Long Term
+- **Machine learning integration** for adaptive experiments
+- **Multi-robot coordination** blocks
+- **Advanced sensor integration** 
+- **Template library** with community sharing
+
+## Lessons Learned
+
+### Design Principles
+1. **Structure over flexibility**: Linear flow is better than freeform for most users
+2. **Integration over isolation**: Work with existing patterns, not against them
+3. **Progressive enhancement**: Start simple, add complexity gradually
+4. **User feedback**: Visual feedback is crucial for drag operations
+5. **Performance matters**: Smooth interactions are essential for user adoption
+
+### Technical Insights
+1. **dnd-kit is superior** to native HTML drag and drop for complex interfaces
+2. **JSONB storage** provides excellent flexibility for evolving schemas
+3. **Type safety** prevents many runtime errors in complex interfaces
+4. **Proper state management** is critical for responsive UI updates
+5. **Database indexing** is essential for JSONB query performance
+
+## Success Metrics
+
+### Quantitative
+- **0 known bugs** in current implementation
+- **<100ms response time** for most user interactions
+- **50+ blocks** supported efficiently in single experiment
+- **3 panel layout** with smooth resizing performance
+- **6 block categories** with 12+ block types implemented
+
+### Qualitative
+- **Intuitive workflow** - Users can create experiments without training
+- **Professional appearance** - Interface feels polished and complete
+- **Reliable interactions** - Drag and drop works consistently
+- **Clear hierarchy** - Experiment flow is easy to understand
+- **Extensible architecture** - Ready for robot platform plugins
+
+## Deployment Status
+
+### Production Ready
+- ✅ **Database migrations** applied successfully
+- ✅ **Code integration** complete with no conflicts
+- ✅ **Documentation** comprehensive and current
+- ✅ **Error handling** robust with user-friendly messages
+- ✅ **Performance** optimized for production workloads
+
+### Access
+- **Route**: `/experiments/[id]/designer`
+- **Permissions**: Requires experiment edit access
+- **Dependencies**: PostgreSQL with JSONB support
+- **Browser support**: Modern browsers with drag/drop APIs
+
+## 🚀 Usage Instructions
+
+### Basic Workflow
+1. **Open Designer**: Navigate to Experiments → [Experiment Name] → Designer
+2. **Add Blocks**: Drag blocks from left palette to main canvas
+3. **Configure**: Click blocks to edit parameters in right panel
+4. **Nest Blocks**: Drag blocks into control structures (repeat, if)
+5. **Save**: Click Save button or Cmd/Ctrl+S
+
+### Advanced Features
+- **Reorder Blocks**: Drag blocks up/down in the sequence
+- **Remove from Control**: Delete nested blocks or drag them out
+- **Parameter Types**: Text inputs, number inputs, select dropdowns
+- **Visual Feedback**: Hover states, selection rings, drag overlays
+
+### Keyboard Shortcuts
+- `Delete` - Remove selected block
+- `Escape` - Deselect all blocks  
+- `↑/↓` - Navigate block selection
+- `Enter` - Edit selected block parameters
+- `Cmd/Ctrl+S` - Save design
+
+## 🎯 Testing the Implementation
+
+### Manual Testing Checklist
+- [ ] Drag blocks from palette to canvas
+- [ ] Drag blocks into repeat/if control structures
+- [ ] Reorder blocks by dragging
+- [ ] Select blocks and edit parameters
+- [ ] Save design (check for success toast)
+- [ ] Reload page (design should persist)
+- [ ] Test touch/tablet interactions
+
+### Browser Compatibility
+- ✅ Chrome/Chromium 90+
+- ✅ Firefox 88+  
+- ✅ Safari 14+
+- ✅ Edge 90+
+- ✅ Mobile Safari (iOS 14+)
+- ✅ Chrome Mobile (Android 10+)
+
+## 🐛 Troubleshooting
+
+### Common Issues
+
+**Blocks won't drag from palette:**
+- Ensure you're dragging from the block area (not just the icon)
+- Check browser drag/drop API support
+- Try refreshing the page
+
+**Save not working:**
+- Check network connection
+- Verify user has edit permissions for experiment
+- Check browser console for API errors
+
+**Drag state gets stuck:**
+- Press Escape to reset drag state
+- Refresh page if issues persist
+- Check for JavaScript errors in console
+
+**Parameters not updating:**
+- Ensure block is selected (blue ring around block)
+- Click outside input fields to trigger save
+- Check for validation errors
+
+### Performance Tips
+- Keep experiments under 50 blocks for optimal performance
+- Use control blocks to organize complex sequences
+- Close unused browser tabs to free memory
+- Clear browser cache if experiencing issues
+
+## 🔮 Future Enhancements
+
+### Planned Features
+- **Inline Parameter Editing**: Edit parameters directly on blocks
+- **Block Templates**: Save and reuse common block sequences  
+- **Visual Branching**: Better visualization of conditional logic
+- **Collaboration**: Real-time collaborative editing
+- **Version History**: Track and restore design versions
+
+### Plugin Extensibility
+```typescript
+// Robot platform plugins can register new blocks
+registry.registerBlock({
+  type: "ur5_move_joint",
+  category: "robot",
+  displayName: "move joint",
+  description: "Move UR5 robot joint to position",
+  icon: "Bot",
+  color: "#3b82f6",
+  parameters: [
+    { id: "joint", name: "Joint", type: "select", options: ["shoulder", "elbow", "wrist"] },
+    { id: "angle", name: "Angle (deg)", type: "number", min: -180, max: 180 }
+  ]
+});
+```
+
+## 📊 Performance Metrics
+
+### Rendering Performance
+- **Initial Load**: <100ms for 20 blocks
+- **Drag Operations**: 60fps smooth animations
+- **Save Operations**: <500ms for typical designs
+- **Memory Usage**: <50MB for complex experiments
+
+### Bundle Size Impact
+- **@dnd-kit/core**: +45KB (gzipped: +12KB)
+- **Component Code**: +25KB (gzipped: +8KB)
+- **Total Addition**: +70KB (gzipped: +20KB)
+
+## 🏆 Success Criteria - All Met ✅
+
+- ✅ **Drag & Drop Works**: Palette to canvas, reordering, nesting
+- ✅ **Save Functionality**: Persistent storage with API integration
+- ✅ **Clean UI**: No double borders, professional appearance
+- ✅ **Parameter Editing**: Full configuration support
+- ✅ **Performance**: Smooth for typical experiment sizes
+- ✅ **Accessibility**: Keyboard navigation and screen reader support
+- ✅ **Mobile Support**: Touch-friendly interactions
+- ✅ **Type Safety**: TypeScript with strict mode
+
+---
+
+**Implementation completed**: Production-ready block designer successfully replacing all previous experimental interfaces. Ready for researcher adoption and robot platform plugin development.