From 4bed5379434a452c66b3f76b6abd10d97aa86b75 Mon Sep 17 00:00:00 2001 From: Sean O'Connor Date: Sat, 21 Mar 2026 20:51:08 -0400 Subject: [PATCH] Update docs: add March 2026 session summary, NAO6 Docker integration docs, and quick reference updates - Add MARCH-2026-SESSION.md with complete summary of work done - Update nao6-quick-reference.md for Docker-based deployment - Update quick-reference.md with NAO6 Docker integration section --- docs/MARCH-2026-SESSION.md | 159 +++++++++++++++++++++++++ docs/nao6-quick-reference.md | 223 ++++++++++++++++++----------------- docs/quick-reference.md | 33 ++++++ 3 files changed, 304 insertions(+), 111 deletions(-) create mode 100644 docs/MARCH-2026-SESSION.md diff --git a/docs/MARCH-2026-SESSION.md b/docs/MARCH-2026-SESSION.md new file mode 100644 index 0000000..c881ddb --- /dev/null +++ b/docs/MARCH-2026-SESSION.md @@ -0,0 +1,159 @@ +# HRIStudio - March 2026 Development Summary + +## What We Did This Session + +### 1. Docker Integration for NAO6 Robot +**Files**: `nao6-hristudio-integration/` + +- Created `Dockerfile` with ROS2 Humble + naoqi packages +- Created `docker-compose.yaml` with 3 services: `nao_driver`, `ros_bridge`, `ros_api` +- Created `scripts/init_robot.sh` - Bash script to wake up robot via SSH when Docker starts +- Fixed autonomous life disable issue (previously used Python `naoqi` package which isn't on PyPI) + +**Key insight**: Robot init via SSH + `qicli` calls instead of Python SDK + +### 2. Plugin System Fixes +**Files**: `robot-plugins/plugins/nao6-ros2.json`, `src/lib/ros/wizard-ros-service.ts` + +- **Topic fixes**: Removed `/naoqi_driver/` prefix from topics (driver already provides unprefixed topics) +- **say_with_emotion**: Fixed with proper NAOqi markup (`\rspd=120\^start(animations/...)`) +- **wave_goodbye**: Added animated speech with waving gesture +- **play_animation**: Added for predefined NAO animations +- **Sensor topics**: Fixed camera, IMU, bumper, sonar, touch topics (removed prefix) + +### 3. Database Schema - Plugin Identifier +**Files**: `src/server/db/schema.ts`, `src/server/services/trial-execution.ts` + +- Added `identifier` column to `plugins` table (unique, machine-readable ID like `nao6-ros2`) +- `name` now for display only ("NAO6 Robot (ROS2 Integration)") +- Updated trial-execution to look up by `identifier` first, then `name` (backwards compat) +- Created migration script: `scripts/migrate-add-identifier.ts` + +### 4. Seed Script Improvements +**Files**: `scripts/seed-dev.ts` + +- Fixed to use local plugin file (not remote `repo.hristudio.com`) +- Added `identifier` field for all plugins (nao6, hristudio-core, hristudio-woz) +- Experiment structure: + - Step 1: The Hook + - Step 2: The Narrative + - Step 3: Comprehension Check (conditional with wizard choices) + - Step 4a/4b: Branch A/B (with `nextStepId` conditions to converge) + - Step 5: Story Continues (convergence point) + - Step 6: Conclusion + +### 5. Robot Action Timing Fix +**Files**: `src/lib/ros/wizard-ros-service.ts` + +- Speech actions now estimate duration: `1500ms emotion overhead + word_count * 300ms` +- Added `say_with_emotion` and `wave_goodbye` as explicit built-in actions +- Fixed 100ms timeout that was completing actions before robot finished + +### 6. Branching Logic Fixes (Critical!) +**Files**: `src/components/trials/wizard/` + +**Bug 1**: `onClick={onNextStep}` passed event object instead of calling function +- Fixed: `onClick={() => onNextStep()}` + +**Bug 2**: `onCompleted()` called after branch choice incremented action count +- Fixed: Removed `onCompleted()` call after branch selection + +**Bug 3**: Branch A/B had no `nextStepId` condition, fell through to linear progression +- Fixed: Added `conditions.nextStepId: step5.id` to Branch A and B + +**Bug 4**: Robot actions from previous step executed on new step (branching jumped but actions from prior step still triggered) +- Root cause: `completedActionsCount` not being reset properly +- Fixed: `handleNextStep()` now resets `completedActionsCount(0)` on explicit jump + +### 7. Auth.js to Better Auth Migration (Attempted, Reverted) +**Status**: Incomplete - 41+ type errors remain + +The migration requires significant changes to how `session.user.roles` is accessed since Better Auth doesn't include roles in session by default. Would need to fetch roles from database on each request. + +**Recommendation**: Defer until more development time available. + +--- + +## Current Architecture + +### Plugin Identifier System +``` +plugins table: + - id: UUID (primary key) + - identifier: varchar (unique, e.g. "nao6-ros2") + - name: varchar (display, e.g. "NAO6 Robot (ROS2 Integration)") + - robotId: UUID (optional FK to robots) + - actionDefinitions: JSONB + +actions table: + - type: "plugin.action" (e.g., "nao6-ros2.say_with_emotion") + - pluginId: varchar (references plugins.identifier) +``` + +### Branching Flow +``` +Step 3 (Comprehension Check) + └── wizard_wait_for_response action + ├── Click "Correct" → setLastResponse("Correct") → nextStepId=step4a.id + └── Click "Incorrect" → setLastResponse("Incorrect") → nextStepId=step4b.id + +Step 4a/4b (Branches) + └── conditions.nextStepId: step5.id → jump to Story Continues + +Step 5 (Story Continues) + └── Linear progression to Step 6 + +Step 6 (Conclusion) + └── Trial complete +``` + +### ROS Topics (NAO6) +``` +/speech - Text-to-speech +/cmd_vel - Velocity commands +/joint_angles - Joint position commands +/camera/front/image_raw +/camera/bottom/image_raw +/imu/torso +/bumper +/{hand,head}_touch +/sonar/{left,right} +/info +``` + +--- + +## Known Issues / Remaining Work + +1. **Auth.js to Better Auth Migration** - Deferred, requires significant refactoring +2. **robots.executeSystemAction** - Procedure not found error (fallback works but should investigate) +3. **say_with_emotion via WebSocket** - May need proper plugin config to avoid fallback + +--- + +## Docker Deployment + +```bash +cd nao6-hristudio-integration +docker compose up -d +``` + +Robot init runs automatically on startup (via `init_robot.sh`). + +--- + +## Testing Checklist + +- [x] Docker builds and starts +- [x] Robot wakes up (autonomous life disabled) +- [x] Seed script runs successfully +- [x] Trial executes with proper branching +- [x] Branch A → Story Continues (not Branch B) +- [x] Robot speaks with emotion (say_with_emotion) +- [x] Wave gesture works +- [ ] Robot movement (walk, turn) tested +- [ ] All NAO6 actions verified + +--- + +*Last Updated: March 21, 2026* diff --git a/docs/nao6-quick-reference.md b/docs/nao6-quick-reference.md index 1240c62..6d283df 100755 --- a/docs/nao6-quick-reference.md +++ b/docs/nao6-quick-reference.md @@ -2,88 +2,92 @@ Essential commands for using NAO6 robots with HRIStudio. -## Quick Start +## Quick Start (Docker) -### 1. Start NAO Integration +### 1. Start Docker Integration ```bash -cd ~/naoqi_ros2_ws -source install/setup.bash -ros2 launch nao_launch nao6_hristudio.launch.py nao_ip:=nao.local password:=robolab +cd ~/Documents/Projects/nao6-hristudio-integration +docker compose up -d ``` -### 2. Wake Robot -Press chest button for 3 seconds, or use: -```bash -# Via SSH (institution-specific password) -ssh nao@nao.local -# Then run wake-up command (see integration repo docs) -``` +The robot will automatically wake up and autonomous life will be disabled on startup. -### 3. Start HRIStudio +### 2. Start HRIStudio ```bash cd ~/Documents/Projects/hristudio bun dev ``` -### 4. Test Connection -- Open: `http://localhost:3000/nao-test` -- Click "Connect" -- Test robot commands +### 3. Verify Connection +- Open: `http://localhost:3000` +- Navigate to trial wizard +- WebSocket should connect automatically -## Essential Commands +## Docker Services -### Test Connectivity -```bash -ping nao.local # Test network -ros2 topic list | grep naoqi # Check ROS topics -``` - -### Manual Control -```bash -# Speech -ros2 topic pub --once /speech std_msgs/String "data: 'Hello world'" - -# Movement (robot must be awake!) -ros2 topic pub --once /cmd_vel geometry_msgs/msg/Twist '{linear: {x: 0.1}}' - -# Stop -ros2 topic pub --once /cmd_vel geometry_msgs/msg/Twist '{linear: {x: 0.0}}' -``` - -### Monitor Status -```bash -ros2 topic echo /naoqi_driver/battery # Battery level -ros2 topic echo /naoqi_driver/joint_states # Joint positions -``` - -## Troubleshooting - -**Robot not moving:** Press chest button for 3 seconds to wake up - -**WebSocket fails:** Check rosbridge is running on port 9090 -```bash -ss -an | grep 9090 -``` - -**Connection lost:** Restart rosbridge -```bash -pkill -f rosbridge -ros2 run rosbridge_server rosbridge_websocket -``` +| Service | Port | Description | +|---------|------|-------------| +| nao_driver | - | NAOqi driver node | +| ros_bridge | 9090 | WebSocket bridge | +| ros_api | - | ROS API services | ## ROS Topics -**Commands (Input):** -- `/speech` - Text-to-speech -- `/cmd_vel` - Movement -- `/joint_angles` - Joint control +**Commands (Publish to these):** +``` +/speech - Text-to-speech +/cmd_vel - Velocity commands (movement) +/joint_angles - Joint position commands +``` -**Sensors (Output):** -- `/naoqi_driver/joint_states` - Joint data -- `/naoqi_driver/battery` - Battery level -- `/naoqi_driver/bumper` - Foot sensors -- `/naoqi_driver/sonar/*` - Distance sensors -- `/naoqi_driver/camera/*` - Camera feeds +**Sensors (Subscribe to these):** +``` +/camera/front/image_raw - Front camera +/camera/bottom/image_raw - Bottom camera +/joint_states - Joint positions +/imu/torso - IMU data +/bumper - Foot bumpers +/{hand,head}_touch - Touch sensors +/sonar/{left,right} - Ultrasonic sensors +/info - Robot info +``` + +## Manual Control + +### Test Connectivity +```bash +# Network +ping 10.0.0.42 + +# ROS topics (inside Docker) +docker exec -it nao6-hristudio-integration-nao_driver-1 ros2 topic list +``` + +### Direct Commands (inside Docker) +```bash +# Speech +docker exec -it nao6-hristudio-integration-nao_driver-1 \ + ros2 topic pub --once /speech std_msgs/String "{data: 'Hello'}" + +# Movement (robot must be awake!) +docker exec -it nao6-hristudio-integration-nao_driver-1 \ + ros2 topic pub --once /cmd_vel geometry_msgs/Twist "{linear: {x: 0.1, y: 0.0, z: 0.0}}" +``` + +### Robot Control via SSH +```bash +# SSH to robot +sshpass -p "nao" ssh nao@10.0.0.42 + +# Wake up +qicli call ALMotion.wakeUp + +# Disable autonomous life +qicli call ALAutonomousLife.setState disabled + +# Go to stand +qicli call ALRobotPosture.goToPosture Stand 0.5 +``` ## WebSocket @@ -99,79 +103,76 @@ ros2 run rosbridge_server rosbridge_websocket } ``` -## More Information +## Troubleshooting -See **[nao6-hristudio-integration](../../nao6-hristudio-integration/)** repository for: -- Complete installation guide -- Detailed usage instructions -- Full troubleshooting guide -- Plugin definitions -- Launch file configurations +**Robot not moving:** +- Check robot is awake: `qicli call ALMotion.isWakeUp` → returns `true` +- If not: `qicli call ALMotion.wakeUp` -## Common Use Cases - -### Make Robot Speak +**WebSocket fails:** ```bash -ros2 topic pub --once /speech std_msgs/String "data: 'Welcome to the experiment'" +# Check rosbridge is running +docker compose ps + +# View logs +docker compose logs ros_bridge ``` -### Walk Forward 3 Steps +**Connection issues:** ```bash -ros2 topic pub --times 3 /cmd_vel geometry_msgs/msg/Twist '{linear: {x: 0.1, y: 0.0, z: 0.0}, angular: {x: 0.0, y: 0.0, z: 0.0}}' +# Restart Docker +docker compose down && docker compose up -d + +# Check robot IP in .env +cat nao6-hristudio-integration/.env ``` -### Turn Head Left -```bash -ros2 topic pub --once /joint_angles naoqi_bridge_msgs/msg/JointAnglesWithSpeed '{joint_names: ["HeadYaw"], joint_angles: [0.8], speed: 0.2}' -``` +## Environment Variables -### Emergency Stop -```bash -ros2 topic pub --once /cmd_vel geometry_msgs/msg/Twist '{linear: {x: 0.0, y: 0.0, z: 0.0}, angular: {x: 0.0, y: 0.0, z: 0.0}}' +Create `nao6-hristudio-integration/.env`: +``` +NAO_IP=10.0.0.42 +NAO_USERNAME=nao +NAO_PASSWORD=nao +BRIDGE_PORT=9090 ``` ## 🚨 Safety Notes -- **Always wake up robot before movement commands** -- **Keep emergency stop accessible** +- **Always verify robot is awake before movement commands** +- **Keep emergency stop accessible** (`qicli call ALMotion.rest()`) - **Start with small movements (0.05 m/s)** -- **Monitor battery level during experiments** +- **Monitor battery level** - **Ensure clear space around robot** -## 📝 Credentials +## Credentials -**Default NAO Login:** +**NAO Robot:** +- IP: `10.0.0.42` (configurable) - Username: `nao` -- Password: `robolab` (institution-specific) +- Password: `nao` -**HRIStudio Login:** +**HRIStudio:** - Email: `sean@soconnor.dev` - Password: `password123` -## 🔄 Complete Restart Procedure +## Complete Restart ```bash -# 1. Kill all processes -sudo fuser -k 9090/tcp -pkill -f "rosbridge\|naoqi\|ros2" +# 1. Restart Docker integration +cd ~/Documents/Projects/nao6-hristudio-integration +docker compose down +docker compose up -d -# 2. Restart database -sudo docker compose down && sudo docker compose up -d +# 2. Verify robot is awake (check logs) +docker compose logs nao_driver | grep -i "wake\|autonomous" -# 3. Start ROS integration -cd ~/naoqi_ros2_ws && source install/setup.bash -ros2 launch install/nao_launch/share/nao_launch/launch/nao6_hristudio.launch.py nao_ip:=nao.local password:=robolab - -# 4. Wake up robot (in another terminal) -sshpass -p "robolab" ssh nao@nao.local "python2 -c \"import sys; sys.path.append('/opt/aldebaran/lib/python2.7/site-packages'); import naoqi; naoqi.ALProxy('ALMotion', '127.0.0.1', 9559).wakeUp()\"" - -# 5. Start HRIStudio (in another terminal) -cd /home/robolab/Documents/Projects/hristudio && bun dev +# 3. Start HRIStudio +cd ~/Documents/Projects/hristudio +bun dev ``` --- -**📖 For detailed setup instructions, see:** [NAO6 Complete Integration Guide](./nao6-integration-complete-guide.md) - **✅ Integration Status:** Production Ready -**🤖 Tested With:** NAO V6.0 / NAOqi 2.8.7.4 / ROS2 Humble \ No newline at end of file +**🤖 Tested With:** NAO V6 / ROS2 Humble / Docker diff --git a/docs/quick-reference.md b/docs/quick-reference.md index 7388912..1b4ff0d 100755 --- a/docs/quick-reference.md +++ b/docs/quick-reference.md @@ -111,6 +111,39 @@ http://localhost:3000/api/trpc/ - **`dashboard`**: Overview stats, recent activity, study progress - **`admin`**: Repository management, system settings +--- + +## 🤖 NAO6 Docker Integration + +### Quick Start +```bash +cd ~/Documents/Projects/nao6-hristudio-integration +docker compose up -d +``` + +Robot automatically wakes up and disables autonomous life on startup. + +### ROS Topics +``` +/speech - Text-to-speech +/cmd_vel - Movement commands +/joint_angles - Joint position control +/camera/front/image_raw +/camera/bottom/image_raw +/imu/torso +/bumper +/{hand,head}_touch +/sonar/{left,right} +/info +``` + +### Plugin System +- Plugin identifier: `nao6-ros2` +- Plugin name: `NAO6 Robot (ROS2 Integration)` +- Action types: `nao6-ros2.say_with_emotion`, `nao6-ros2.move_arm`, etc. + +See [nao6-quick-reference.md](./nao6-quick-reference.md) for full details. + ### Example Usage ```typescript // Get user's studies