mirror of
https://github.com/soconnor0919/hristudio.git
synced 2026-03-23 19:27:51 -04:00
Update docs: add March 2026 session summary, NAO6 Docker integration docs, and quick reference updates
- Add MARCH-2026-SESSION.md with complete summary of work done - Update nao6-quick-reference.md for Docker-based deployment - Update quick-reference.md with NAO6 Docker integration section
This commit is contained in:
159
docs/MARCH-2026-SESSION.md
Normal file
159
docs/MARCH-2026-SESSION.md
Normal file
@@ -0,0 +1,159 @@
|
||||
# HRIStudio - March 2026 Development Summary
|
||||
|
||||
## What We Did This Session
|
||||
|
||||
### 1. Docker Integration for NAO6 Robot
|
||||
**Files**: `nao6-hristudio-integration/`
|
||||
|
||||
- Created `Dockerfile` with ROS2 Humble + naoqi packages
|
||||
- Created `docker-compose.yaml` with 3 services: `nao_driver`, `ros_bridge`, `ros_api`
|
||||
- Created `scripts/init_robot.sh` - Bash script to wake up robot via SSH when Docker starts
|
||||
- Fixed autonomous life disable issue (previously used Python `naoqi` package which isn't on PyPI)
|
||||
|
||||
**Key insight**: Robot init via SSH + `qicli` calls instead of Python SDK
|
||||
|
||||
### 2. Plugin System Fixes
|
||||
**Files**: `robot-plugins/plugins/nao6-ros2.json`, `src/lib/ros/wizard-ros-service.ts`
|
||||
|
||||
- **Topic fixes**: Removed `/naoqi_driver/` prefix from topics (driver already provides unprefixed topics)
|
||||
- **say_with_emotion**: Fixed with proper NAOqi markup (`\rspd=120\^start(animations/...)`)
|
||||
- **wave_goodbye**: Added animated speech with waving gesture
|
||||
- **play_animation**: Added for predefined NAO animations
|
||||
- **Sensor topics**: Fixed camera, IMU, bumper, sonar, touch topics (removed prefix)
|
||||
|
||||
### 3. Database Schema - Plugin Identifier
|
||||
**Files**: `src/server/db/schema.ts`, `src/server/services/trial-execution.ts`
|
||||
|
||||
- Added `identifier` column to `plugins` table (unique, machine-readable ID like `nao6-ros2`)
|
||||
- `name` now for display only ("NAO6 Robot (ROS2 Integration)")
|
||||
- Updated trial-execution to look up by `identifier` first, then `name` (backwards compat)
|
||||
- Created migration script: `scripts/migrate-add-identifier.ts`
|
||||
|
||||
### 4. Seed Script Improvements
|
||||
**Files**: `scripts/seed-dev.ts`
|
||||
|
||||
- Fixed to use local plugin file (not remote `repo.hristudio.com`)
|
||||
- Added `identifier` field for all plugins (nao6, hristudio-core, hristudio-woz)
|
||||
- Experiment structure:
|
||||
- Step 1: The Hook
|
||||
- Step 2: The Narrative
|
||||
- Step 3: Comprehension Check (conditional with wizard choices)
|
||||
- Step 4a/4b: Branch A/B (with `nextStepId` conditions to converge)
|
||||
- Step 5: Story Continues (convergence point)
|
||||
- Step 6: Conclusion
|
||||
|
||||
### 5. Robot Action Timing Fix
|
||||
**Files**: `src/lib/ros/wizard-ros-service.ts`
|
||||
|
||||
- Speech actions now estimate duration: `1500ms emotion overhead + word_count * 300ms`
|
||||
- Added `say_with_emotion` and `wave_goodbye` as explicit built-in actions
|
||||
- Fixed 100ms timeout that was completing actions before robot finished
|
||||
|
||||
### 6. Branching Logic Fixes (Critical!)
|
||||
**Files**: `src/components/trials/wizard/`
|
||||
|
||||
**Bug 1**: `onClick={onNextStep}` passed event object instead of calling function
|
||||
- Fixed: `onClick={() => onNextStep()}`
|
||||
|
||||
**Bug 2**: `onCompleted()` called after branch choice incremented action count
|
||||
- Fixed: Removed `onCompleted()` call after branch selection
|
||||
|
||||
**Bug 3**: Branch A/B had no `nextStepId` condition, fell through to linear progression
|
||||
- Fixed: Added `conditions.nextStepId: step5.id` to Branch A and B
|
||||
|
||||
**Bug 4**: Robot actions from previous step executed on new step (branching jumped but actions from prior step still triggered)
|
||||
- Root cause: `completedActionsCount` not being reset properly
|
||||
- Fixed: `handleNextStep()` now resets `completedActionsCount(0)` on explicit jump
|
||||
|
||||
### 7. Auth.js to Better Auth Migration (Attempted, Reverted)
|
||||
**Status**: Incomplete - 41+ type errors remain
|
||||
|
||||
The migration requires significant changes to how `session.user.roles` is accessed since Better Auth doesn't include roles in session by default. Would need to fetch roles from database on each request.
|
||||
|
||||
**Recommendation**: Defer until more development time available.
|
||||
|
||||
---
|
||||
|
||||
## Current Architecture
|
||||
|
||||
### Plugin Identifier System
|
||||
```
|
||||
plugins table:
|
||||
- id: UUID (primary key)
|
||||
- identifier: varchar (unique, e.g. "nao6-ros2")
|
||||
- name: varchar (display, e.g. "NAO6 Robot (ROS2 Integration)")
|
||||
- robotId: UUID (optional FK to robots)
|
||||
- actionDefinitions: JSONB
|
||||
|
||||
actions table:
|
||||
- type: "plugin.action" (e.g., "nao6-ros2.say_with_emotion")
|
||||
- pluginId: varchar (references plugins.identifier)
|
||||
```
|
||||
|
||||
### Branching Flow
|
||||
```
|
||||
Step 3 (Comprehension Check)
|
||||
└── wizard_wait_for_response action
|
||||
├── Click "Correct" → setLastResponse("Correct") → nextStepId=step4a.id
|
||||
└── Click "Incorrect" → setLastResponse("Incorrect") → nextStepId=step4b.id
|
||||
|
||||
Step 4a/4b (Branches)
|
||||
└── conditions.nextStepId: step5.id → jump to Story Continues
|
||||
|
||||
Step 5 (Story Continues)
|
||||
└── Linear progression to Step 6
|
||||
|
||||
Step 6 (Conclusion)
|
||||
└── Trial complete
|
||||
```
|
||||
|
||||
### ROS Topics (NAO6)
|
||||
```
|
||||
/speech - Text-to-speech
|
||||
/cmd_vel - Velocity commands
|
||||
/joint_angles - Joint position commands
|
||||
/camera/front/image_raw
|
||||
/camera/bottom/image_raw
|
||||
/imu/torso
|
||||
/bumper
|
||||
/{hand,head}_touch
|
||||
/sonar/{left,right}
|
||||
/info
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Known Issues / Remaining Work
|
||||
|
||||
1. **Auth.js to Better Auth Migration** - Deferred, requires significant refactoring
|
||||
2. **robots.executeSystemAction** - Procedure not found error (fallback works but should investigate)
|
||||
3. **say_with_emotion via WebSocket** - May need proper plugin config to avoid fallback
|
||||
|
||||
---
|
||||
|
||||
## Docker Deployment
|
||||
|
||||
```bash
|
||||
cd nao6-hristudio-integration
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
Robot init runs automatically on startup (via `init_robot.sh`).
|
||||
|
||||
---
|
||||
|
||||
## Testing Checklist
|
||||
|
||||
- [x] Docker builds and starts
|
||||
- [x] Robot wakes up (autonomous life disabled)
|
||||
- [x] Seed script runs successfully
|
||||
- [x] Trial executes with proper branching
|
||||
- [x] Branch A → Story Continues (not Branch B)
|
||||
- [x] Robot speaks with emotion (say_with_emotion)
|
||||
- [x] Wave gesture works
|
||||
- [ ] Robot movement (walk, turn) tested
|
||||
- [ ] All NAO6 actions verified
|
||||
|
||||
---
|
||||
|
||||
*Last Updated: March 21, 2026*
|
||||
@@ -2,88 +2,92 @@
|
||||
|
||||
Essential commands for using NAO6 robots with HRIStudio.
|
||||
|
||||
## Quick Start
|
||||
## Quick Start (Docker)
|
||||
|
||||
### 1. Start NAO Integration
|
||||
### 1. Start Docker Integration
|
||||
```bash
|
||||
cd ~/naoqi_ros2_ws
|
||||
source install/setup.bash
|
||||
ros2 launch nao_launch nao6_hristudio.launch.py nao_ip:=nao.local password:=robolab
|
||||
cd ~/Documents/Projects/nao6-hristudio-integration
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
### 2. Wake Robot
|
||||
Press chest button for 3 seconds, or use:
|
||||
```bash
|
||||
# Via SSH (institution-specific password)
|
||||
ssh nao@nao.local
|
||||
# Then run wake-up command (see integration repo docs)
|
||||
```
|
||||
The robot will automatically wake up and autonomous life will be disabled on startup.
|
||||
|
||||
### 3. Start HRIStudio
|
||||
### 2. Start HRIStudio
|
||||
```bash
|
||||
cd ~/Documents/Projects/hristudio
|
||||
bun dev
|
||||
```
|
||||
|
||||
### 4. Test Connection
|
||||
- Open: `http://localhost:3000/nao-test`
|
||||
- Click "Connect"
|
||||
- Test robot commands
|
||||
### 3. Verify Connection
|
||||
- Open: `http://localhost:3000`
|
||||
- Navigate to trial wizard
|
||||
- WebSocket should connect automatically
|
||||
|
||||
## Essential Commands
|
||||
## Docker Services
|
||||
|
||||
### Test Connectivity
|
||||
```bash
|
||||
ping nao.local # Test network
|
||||
ros2 topic list | grep naoqi # Check ROS topics
|
||||
```
|
||||
|
||||
### Manual Control
|
||||
```bash
|
||||
# Speech
|
||||
ros2 topic pub --once /speech std_msgs/String "data: 'Hello world'"
|
||||
|
||||
# Movement (robot must be awake!)
|
||||
ros2 topic pub --once /cmd_vel geometry_msgs/msg/Twist '{linear: {x: 0.1}}'
|
||||
|
||||
# Stop
|
||||
ros2 topic pub --once /cmd_vel geometry_msgs/msg/Twist '{linear: {x: 0.0}}'
|
||||
```
|
||||
|
||||
### Monitor Status
|
||||
```bash
|
||||
ros2 topic echo /naoqi_driver/battery # Battery level
|
||||
ros2 topic echo /naoqi_driver/joint_states # Joint positions
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
**Robot not moving:** Press chest button for 3 seconds to wake up
|
||||
|
||||
**WebSocket fails:** Check rosbridge is running on port 9090
|
||||
```bash
|
||||
ss -an | grep 9090
|
||||
```
|
||||
|
||||
**Connection lost:** Restart rosbridge
|
||||
```bash
|
||||
pkill -f rosbridge
|
||||
ros2 run rosbridge_server rosbridge_websocket
|
||||
```
|
||||
| Service | Port | Description |
|
||||
|---------|------|-------------|
|
||||
| nao_driver | - | NAOqi driver node |
|
||||
| ros_bridge | 9090 | WebSocket bridge |
|
||||
| ros_api | - | ROS API services |
|
||||
|
||||
## ROS Topics
|
||||
|
||||
**Commands (Input):**
|
||||
- `/speech` - Text-to-speech
|
||||
- `/cmd_vel` - Movement
|
||||
- `/joint_angles` - Joint control
|
||||
**Commands (Publish to these):**
|
||||
```
|
||||
/speech - Text-to-speech
|
||||
/cmd_vel - Velocity commands (movement)
|
||||
/joint_angles - Joint position commands
|
||||
```
|
||||
|
||||
**Sensors (Output):**
|
||||
- `/naoqi_driver/joint_states` - Joint data
|
||||
- `/naoqi_driver/battery` - Battery level
|
||||
- `/naoqi_driver/bumper` - Foot sensors
|
||||
- `/naoqi_driver/sonar/*` - Distance sensors
|
||||
- `/naoqi_driver/camera/*` - Camera feeds
|
||||
**Sensors (Subscribe to these):**
|
||||
```
|
||||
/camera/front/image_raw - Front camera
|
||||
/camera/bottom/image_raw - Bottom camera
|
||||
/joint_states - Joint positions
|
||||
/imu/torso - IMU data
|
||||
/bumper - Foot bumpers
|
||||
/{hand,head}_touch - Touch sensors
|
||||
/sonar/{left,right} - Ultrasonic sensors
|
||||
/info - Robot info
|
||||
```
|
||||
|
||||
## Manual Control
|
||||
|
||||
### Test Connectivity
|
||||
```bash
|
||||
# Network
|
||||
ping 10.0.0.42
|
||||
|
||||
# ROS topics (inside Docker)
|
||||
docker exec -it nao6-hristudio-integration-nao_driver-1 ros2 topic list
|
||||
```
|
||||
|
||||
### Direct Commands (inside Docker)
|
||||
```bash
|
||||
# Speech
|
||||
docker exec -it nao6-hristudio-integration-nao_driver-1 \
|
||||
ros2 topic pub --once /speech std_msgs/String "{data: 'Hello'}"
|
||||
|
||||
# Movement (robot must be awake!)
|
||||
docker exec -it nao6-hristudio-integration-nao_driver-1 \
|
||||
ros2 topic pub --once /cmd_vel geometry_msgs/Twist "{linear: {x: 0.1, y: 0.0, z: 0.0}}"
|
||||
```
|
||||
|
||||
### Robot Control via SSH
|
||||
```bash
|
||||
# SSH to robot
|
||||
sshpass -p "nao" ssh nao@10.0.0.42
|
||||
|
||||
# Wake up
|
||||
qicli call ALMotion.wakeUp
|
||||
|
||||
# Disable autonomous life
|
||||
qicli call ALAutonomousLife.setState disabled
|
||||
|
||||
# Go to stand
|
||||
qicli call ALRobotPosture.goToPosture Stand 0.5
|
||||
```
|
||||
|
||||
## WebSocket
|
||||
|
||||
@@ -99,79 +103,76 @@ ros2 run rosbridge_server rosbridge_websocket
|
||||
}
|
||||
```
|
||||
|
||||
## More Information
|
||||
## Troubleshooting
|
||||
|
||||
See **[nao6-hristudio-integration](../../nao6-hristudio-integration/)** repository for:
|
||||
- Complete installation guide
|
||||
- Detailed usage instructions
|
||||
- Full troubleshooting guide
|
||||
- Plugin definitions
|
||||
- Launch file configurations
|
||||
**Robot not moving:**
|
||||
- Check robot is awake: `qicli call ALMotion.isWakeUp` → returns `true`
|
||||
- If not: `qicli call ALMotion.wakeUp`
|
||||
|
||||
## Common Use Cases
|
||||
|
||||
### Make Robot Speak
|
||||
**WebSocket fails:**
|
||||
```bash
|
||||
ros2 topic pub --once /speech std_msgs/String "data: 'Welcome to the experiment'"
|
||||
# Check rosbridge is running
|
||||
docker compose ps
|
||||
|
||||
# View logs
|
||||
docker compose logs ros_bridge
|
||||
```
|
||||
|
||||
### Walk Forward 3 Steps
|
||||
**Connection issues:**
|
||||
```bash
|
||||
ros2 topic pub --times 3 /cmd_vel geometry_msgs/msg/Twist '{linear: {x: 0.1, y: 0.0, z: 0.0}, angular: {x: 0.0, y: 0.0, z: 0.0}}'
|
||||
# Restart Docker
|
||||
docker compose down && docker compose up -d
|
||||
|
||||
# Check robot IP in .env
|
||||
cat nao6-hristudio-integration/.env
|
||||
```
|
||||
|
||||
### Turn Head Left
|
||||
```bash
|
||||
ros2 topic pub --once /joint_angles naoqi_bridge_msgs/msg/JointAnglesWithSpeed '{joint_names: ["HeadYaw"], joint_angles: [0.8], speed: 0.2}'
|
||||
```
|
||||
## Environment Variables
|
||||
|
||||
### Emergency Stop
|
||||
```bash
|
||||
ros2 topic pub --once /cmd_vel geometry_msgs/msg/Twist '{linear: {x: 0.0, y: 0.0, z: 0.0}, angular: {x: 0.0, y: 0.0, z: 0.0}}'
|
||||
Create `nao6-hristudio-integration/.env`:
|
||||
```
|
||||
NAO_IP=10.0.0.42
|
||||
NAO_USERNAME=nao
|
||||
NAO_PASSWORD=nao
|
||||
BRIDGE_PORT=9090
|
||||
```
|
||||
|
||||
## 🚨 Safety Notes
|
||||
|
||||
- **Always wake up robot before movement commands**
|
||||
- **Keep emergency stop accessible**
|
||||
- **Always verify robot is awake before movement commands**
|
||||
- **Keep emergency stop accessible** (`qicli call ALMotion.rest()`)
|
||||
- **Start with small movements (0.05 m/s)**
|
||||
- **Monitor battery level during experiments**
|
||||
- **Monitor battery level**
|
||||
- **Ensure clear space around robot**
|
||||
|
||||
## 📝 Credentials
|
||||
## Credentials
|
||||
|
||||
**Default NAO Login:**
|
||||
**NAO Robot:**
|
||||
- IP: `10.0.0.42` (configurable)
|
||||
- Username: `nao`
|
||||
- Password: `robolab` (institution-specific)
|
||||
- Password: `nao`
|
||||
|
||||
**HRIStudio Login:**
|
||||
**HRIStudio:**
|
||||
- Email: `sean@soconnor.dev`
|
||||
- Password: `password123`
|
||||
|
||||
## 🔄 Complete Restart Procedure
|
||||
## Complete Restart
|
||||
|
||||
```bash
|
||||
# 1. Kill all processes
|
||||
sudo fuser -k 9090/tcp
|
||||
pkill -f "rosbridge\|naoqi\|ros2"
|
||||
# 1. Restart Docker integration
|
||||
cd ~/Documents/Projects/nao6-hristudio-integration
|
||||
docker compose down
|
||||
docker compose up -d
|
||||
|
||||
# 2. Restart database
|
||||
sudo docker compose down && sudo docker compose up -d
|
||||
# 2. Verify robot is awake (check logs)
|
||||
docker compose logs nao_driver | grep -i "wake\|autonomous"
|
||||
|
||||
# 3. Start ROS integration
|
||||
cd ~/naoqi_ros2_ws && source install/setup.bash
|
||||
ros2 launch install/nao_launch/share/nao_launch/launch/nao6_hristudio.launch.py nao_ip:=nao.local password:=robolab
|
||||
|
||||
# 4. Wake up robot (in another terminal)
|
||||
sshpass -p "robolab" ssh nao@nao.local "python2 -c \"import sys; sys.path.append('/opt/aldebaran/lib/python2.7/site-packages'); import naoqi; naoqi.ALProxy('ALMotion', '127.0.0.1', 9559).wakeUp()\""
|
||||
|
||||
# 5. Start HRIStudio (in another terminal)
|
||||
cd /home/robolab/Documents/Projects/hristudio && bun dev
|
||||
# 3. Start HRIStudio
|
||||
cd ~/Documents/Projects/hristudio
|
||||
bun dev
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
**📖 For detailed setup instructions, see:** [NAO6 Complete Integration Guide](./nao6-integration-complete-guide.md)
|
||||
|
||||
**✅ Integration Status:** Production Ready
|
||||
**🤖 Tested With:** NAO V6.0 / NAOqi 2.8.7.4 / ROS2 Humble
|
||||
**🤖 Tested With:** NAO V6 / ROS2 Humble / Docker
|
||||
|
||||
@@ -111,6 +111,39 @@ http://localhost:3000/api/trpc/
|
||||
- **`dashboard`**: Overview stats, recent activity, study progress
|
||||
- **`admin`**: Repository management, system settings
|
||||
|
||||
---
|
||||
|
||||
## 🤖 NAO6 Docker Integration
|
||||
|
||||
### Quick Start
|
||||
```bash
|
||||
cd ~/Documents/Projects/nao6-hristudio-integration
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
Robot automatically wakes up and disables autonomous life on startup.
|
||||
|
||||
### ROS Topics
|
||||
```
|
||||
/speech - Text-to-speech
|
||||
/cmd_vel - Movement commands
|
||||
/joint_angles - Joint position control
|
||||
/camera/front/image_raw
|
||||
/camera/bottom/image_raw
|
||||
/imu/torso
|
||||
/bumper
|
||||
/{hand,head}_touch
|
||||
/sonar/{left,right}
|
||||
/info
|
||||
```
|
||||
|
||||
### Plugin System
|
||||
- Plugin identifier: `nao6-ros2`
|
||||
- Plugin name: `NAO6 Robot (ROS2 Integration)`
|
||||
- Action types: `nao6-ros2.say_with_emotion`, `nao6-ros2.move_arm`, etc.
|
||||
|
||||
See [nao6-quick-reference.md](./nao6-quick-reference.md) for full details.
|
||||
|
||||
### Example Usage
|
||||
```typescript
|
||||
// Get user's studies
|
||||
|
||||
Reference in New Issue
Block a user