soconnor/hristudio
1
0
Fork 0
mirror of https://github.com/soconnor0919/hristudio.git synced 2025-12-12 07:04:44 -05:00
Code Issues Packages Projects Releases Wiki Activity

nao6 ros2 integration updated

Browse Source
This commit is contained in:
Sean O'Connor
2025-11-13 10:58:45 -05:00
parent 70882b9dbb
commit 86b5ed80c4
276 changed files with 4288 additions and 1552 deletions
Download Patch File Download Diff File Expand all files Collapse all files

0
docs/README.md Normal file → Executable file
View File

0
docs/api-routes.md Normal file → Executable file
View File

0
docs/block-designer-implementation.md Normal file → Executable file
View File

0
docs/block-designer.md Normal file → Executable file
View File

0
docs/cleanup-summary.md Normal file → Executable file
View File

0
docs/core-blocks-system.md Normal file → Executable file
View File

0
docs/database-schema.md Normal file → Executable file
View File

0
docs/deployment-operations.md Normal file → Executable file
View File

0
docs/experiment-designer-redesign.md Normal file → Executable file
View File

0
docs/experiment-designer-step-integration.md Normal file → Executable file
View File

0
docs/feature-requirements.md Normal file → Executable file
View File

0
docs/flow-designer-connections.md Normal file → Executable file
View File

0
docs/implementation-details.md Normal file → Executable file
View File

0
docs/implementation-guide.md Normal file → Executable file
View File

630
docs/nao6-integration-complete-guide.md
View File

@@ -1,630 +0,0 @@
# NAO6 HRIStudio Integration: Complete Setup and Troubleshooting Guide
This comprehensive guide documents the complete process of integrating a NAO6 robot with HRIStudio, including all troubleshooting steps and solutions discovered during implementation.
## Overview
NAO6 integration with HRIStudio provides full robot control through a web-based interface, enabling researchers to conduct Human-Robot Interaction experiments with real-time robot control, sensor monitoring, and data collection.
**Integration Architecture:**
```
HRIStudio Web Interface → WebSocket → ROS Bridge → NAOqi Driver → NAO6 Robot
```
## Prerequisites
### Hardware Requirements
- NAO6 robot (NAOqi OS 2.8.7+)
- Ubuntu 22.04 LTS computer
- Network connectivity between computer and NAO6
- Administrative access to both systems
### Software Requirements
- ROS2 Humble
- NAOqi Driver2 for ROS2
- rosbridge-suite
- HRIStudio platform
- SSH access to NAO robot
## Part 1: ROS2 and NAO Driver Setup
### 1.1 Install ROS2 Humble
```bash
# Update system
sudo apt update && sudo apt upgrade -y
# Install ROS2 Humble
sudo apt install software-properties-common
sudo add-apt-repository universe
sudo apt update && sudo apt install curl -y
sudo curl -sSL https://raw.githubusercontent.com/ros/rosdistro/master/ros.asc | sudo apt-key add -
sudo sh -c 'echo "deb http://packages.ros.org/ros2/ubuntu $(lsb_release -cs) main" > /etc/apt/sources.list.d/ros2-latest.list'
sudo apt update
sudo apt install ros-humble-desktop
sudo apt install ros-dev-tools
# Source ROS2
echo "source /opt/ros/humble/setup.bash" >> ~/.bashrc
source ~/.bashrc
```
### 1.2 Install Required ROS2 Packages
```bash
# Install rosbridge for HRIStudio communication
sudo apt install ros-humble-rosbridge-suite
# Install additional useful packages
sudo apt install ros-humble-rqt
sudo apt install ros-humble-rqt-common-plugins
```
### 1.3 Set Up NAO Workspace
**Note:** We assume you already have a NAO workspace at `~/naoqi_ros2_ws` with the NAOqi driver installed.
```bash
# Verify workspace exists
ls ~/naoqi_ros2_ws/src/naoqi_driver2
```
If you need to set up the workspace from scratch, refer to the NAOqi ROS2 documentation.
### 1.4 Create Integrated Launch Package
Create a launch package that combines NAOqi driver with rosbridge:
```bash
cd ~/naoqi_ros2_ws
mkdir -p src/nao_launch/launch
```
**Create launch file** (`src/nao_launch/launch/nao6_hristudio.launch.py`):
```python
from launch import LaunchDescription
from launch.actions import DeclareLaunchArgument
from launch.substitutions import LaunchConfiguration
from launch_ros.actions import Node
def generate_launch_description():
return LaunchDescription([
# NAO IP configuration
DeclareLaunchArgument("nao_ip", default_value="nao.local"),
DeclareLaunchArgument("nao_port", default_value="9559"),
DeclareLaunchArgument("username", default_value="nao"),
DeclareLaunchArgument("password", default_value="nao"),
DeclareLaunchArgument("network_interface", default_value="eth0"),
DeclareLaunchArgument("qi_listen_url", default_value="tcp://0.0.0.0:0"),
DeclareLaunchArgument("namespace", default_value="naoqi_driver"),
DeclareLaunchArgument("bridge_port", default_value="9090"),
# NAOqi Driver
Node(
package="naoqi_driver",
executable="naoqi_driver_node",
name="naoqi_driver",
namespace=LaunchConfiguration("namespace"),
parameters=[{
"nao_ip": LaunchConfiguration("nao_ip"),
"nao_port": LaunchConfiguration("nao_port"),
"username": LaunchConfiguration("username"),
"password": LaunchConfiguration("password"),
"network_interface": LaunchConfiguration("network_interface"),
"qi_listen_url": LaunchConfiguration("qi_listen_url"),
"publish_joint_states": True,
"publish_odometry": True,
"publish_camera": True,
"publish_sensors": True,
"joint_states_frequency": 30.0,
"odom_frequency": 30.0,
"camera_frequency": 15.0,
"sensor_frequency": 10.0,
}],
output="screen",
),
# Rosbridge WebSocket Server for HRIStudio
Node(
package="rosbridge_server",
executable="rosbridge_websocket",
name="rosbridge_websocket",
parameters=[{
"port": LaunchConfiguration("bridge_port"),
"address": "0.0.0.0",
"authenticate": False,
"fragment_timeout": 600,
"delay_between_messages": 0,
"max_message_size": 10000000,
}],
output="screen",
),
# ROS API Server (required for rosbridge functionality)
Node(
package="rosapi",
executable="rosapi_node",
name="rosapi",
output="screen",
),
])
```
**Create package.xml**:
```xml
<?xml version="1.0"?>
<?xml-model href="http://download.ros.org/schema/package_format3.xsd" schematypeid="pf3"?>
<package format="3">
<name>nao_launch</name>
<version>1.0.0</version>
<description>Launch files for NAO6 HRIStudio integration</description>
<maintainer email="your@email.com">Your Name</maintainer>
<license>MIT</license>
<buildtool_depend>ament_cmake</buildtool_depend>
<exec_depend>launch</exec_depend>
<exec_depend>launch_ros</exec_depend>
<exec_depend>naoqi_driver</exec_depend>
<exec_depend>rosbridge_server</exec_depend>
<exec_depend>rosapi</exec_depend>
</package>
```
**Create CMakeLists.txt**:
```cmake
cmake_minimum_required(VERSION 3.8)
project(nao_launch)
if(CMAKE_COMPILER_IS_GNUCXX OR CMAKE_CXX_COMPILER_ID MATCHES "Clang")
add_compile_options(-Wall -Wextra -Wpedantic)
endif()
find_package(ament_cmake REQUIRED)
install(DIRECTORY launch/
DESTINATION share/${PROJECT_NAME}/launch/
)
if(BUILD_TESTING)
find_package(ament_lint_auto REQUIRED)
ament_lint_auto_find_test_dependencies()
endif()
ament_package()
```
### 1.5 Build the Workspace
```bash
cd ~/naoqi_ros2_ws
I_AGREE_TO_NAO_MESHES_LICENSE=1 I_AGREE_TO_PEPPER_MESHES_LICENSE=1 colcon build --symlink-install
source install/setup.bash
```
## Part 2: NAO Network Configuration and Connection
### 2.1 Verify NAO Network Connectivity
```bash
# Test basic connectivity
ping -c 4 nao.local
# Test NAOqi service port
timeout 5 bash -c 'echo "test" | nc nao.local 9559' && echo "NAOqi port is open!" || echo "NAOqi port might be closed"
# Alternative test
telnet nao.local 9559
# Press Ctrl+C to exit if connection succeeds
```
### 2.2 Find NAO Credentials
The default NAO credentials are typically:
- Username: `nao`
- Password: Usually `nao`, but can be custom
**Common passwords to try:**
- `nao` (default)
- Institution name (e.g., `bucknell`)
- Custom password set by administrator
## Part 3: HRIStudio Database Integration
### 3.1 Update Database Schema
The HRIStudio database needs to include NAO6 robot definitions and plugins.
**Update robots in seed script** (`scripts/seed-dev.ts`):
```typescript
const robots = [
{
name: "TurtleBot3 Burger",
manufacturer: "ROBOTIS",
model: "TurtleBot3 Burger",
description: "A compact, affordable, programmable, ROS2-based mobile robot for education and research",
capabilities: ["differential_drive", "lidar", "imu", "odometry"],
communicationProtocol: "ros2" as const,
},
{
name: "NAO Humanoid Robot",
manufacturer: "SoftBank Robotics",
model: "NAO V6",
description: "Humanoid robot designed for education, research, and social interaction with ROS2 integration",
capabilities: [
"speech",
"vision",
"walking",
"gestures",
"joint_control",
"touch_sensors",
"sonar_sensors",
"camera_feed",
"imu",
"odometry",
],
communicationProtocol: "ros2" as const,
},
];
```
### 3.2 Create NAO6 Plugin Repository
Create local plugin repository at `public/nao6-plugins/`:
**Repository metadata** (`public/nao6-plugins/repository.json`):
```json
{
"name": "NAO6 ROS2 Integration Repository",
"description": "Official NAO6 robot plugins for ROS2-based Human-Robot Interaction experiments",
"version": "1.0.0",
"author": {
"name": "HRIStudio Team",
"email": "support@hristudio.com"
},
"trust": "official",
"license": "MIT",
"robots": [
{
"name": "NAO6",
"manufacturer": "SoftBank Robotics",
"model": "NAO V6",
"communicationProtocol": "ros2"
}
],
"ros2": {
"distro": "humble",
"packages": ["naoqi_driver2", "naoqi_bridge_msgs", "rosbridge_suite"],
"bridge": {
"protocol": "websocket",
"defaultPort": 9090
}
}
}
```
### 3.3 Seed Database
```bash
# Start database
sudo docker compose up -d
# Push schema changes
bun db:push
# Seed with NAO6 data
bun db:seed
```
## Part 4: Web Interface Integration
### 4.1 Create NAO Test Page
Create `src/app/(dashboard)/nao-test/page.tsx` with the robot control interface.
**Key points:**
- Use `~` import alias (not `@`)
- Connect to WebSocket at `ws://YOUR_IP:9090`
- Use correct ROS topic names (without `/naoqi_driver` prefix for control topics)
**Important Topic Mapping:**
- Speech: `/speech` (not `/naoqi_driver/speech`)
- Movement: `/cmd_vel` (not `/naoqi_driver/cmd_vel`)
- Joint control: `/joint_angles` (not `/naoqi_driver/joint_angles`)
- Sensor data: `/naoqi_driver/joint_states`, `/naoqi_driver/bumper`, etc.
## Part 5: Critical Troubleshooting
### 5.1 Robot Not Responding to Commands
**Symptom:** ROS topics receive commands but robot doesn't move.
**Root Cause:** NAO robots start in "safe mode" with loose joints and need to be "awakened."
**Solution - SSH Wake-Up Method:**
```bash
# Install sshpass for automated SSH
sudo apt install sshpass -y
# Wake up robot via SSH
sshpass -p "YOUR_NAO_PASSWORD" ssh nao@nao.local "python2 -c \"
import sys
sys.path.append('/opt/aldebaran/lib/python2.7/site-packages')
import naoqi
try:
motion = naoqi.ALProxy('ALMotion', '127.0.0.1', 9559)
print 'Connected to ALMotion'
print 'Current stiffness:', motion.getStiffnesses('Body')[0] if motion.getStiffnesses('Body') else 'No stiffness data'
print 'Waking up robot...'
motion.wakeUp()
print 'Robot should now be awake!'
except Exception as e:
print 'Error:', str(e)
\""
```
**Alternative Physical Method:**
1. Press and hold the chest button for 3 seconds
2. Wait for the robot to stiffen and stand up
3. Robot should now respond to movement commands
### 5.2 Connection Issues
**Port Already in Use:**
```bash
# Kill existing processes
sudo fuser -k 9090/tcp
pkill -f "rosbridge\|naoqi\|ros2"
```
**Database Connection Issues:**
```bash
# Check Docker containers
sudo docker ps
# Restart database
sudo docker compose down
sudo docker compose up -d
```
### 5.3 Import Alias Issues
**Error:** Module import failures in React components.
**Solution:** Use `~` import alias consistently:
```typescript
import { Button } from "~/components/ui/button";
// NOT: import { Button } from "@/components/ui/button";
```
## Part 6: Verification and Testing
### 6.1 System Verification Script
Create verification script to test all components:
```bash
#!/bin/bash
echo "=== NAO6 HRIStudio Integration Verification ==="
# Test 1: ROS2 Setup
echo "✓ ROS2 Humble: $ROS_DISTRO"
# Test 2: NAO Connectivity
ping -c 1 nao.local && echo "✓ NAO reachable" || echo "✗ NAO not reachable"
# Test 3: Workspace Build
[ -f ~/naoqi_ros2_ws/install/setup.bash ] && echo "✓ Workspace built" || echo "✗ Workspace not built"
# Test 4: Database Running
sudo docker ps | grep -q postgres && echo "✓ Database running" || echo "✗ Database not running"
echo "=== Verification Complete ==="
```
### 6.2 End-to-End Test Procedure
**Terminal 1: Start ROS Integration**
```bash
cd ~/naoqi_ros2_ws
source install/setup.bash
ros2 launch install/nao_launch/share/nao_launch/launch/nao6_hristudio.launch.py nao_ip:=nao.local password:=YOUR_PASSWORD
```
**Terminal 2: Wake Up Robot**
```bash
# Use SSH method from Section 5.1
sshpass -p "YOUR_PASSWORD" ssh nao@nao.local "python2 -c \"...\""
```
**Terminal 3: Start HRIStudio**
```bash
cd /path/to/hristudio
bun dev
```
**Web Interface Test:**
1. Go to `http://localhost:3000/nao-test`
2. Click "Connect" - should show "Connected"
3. Test speech: Enter text and click "Say Text"
4. Test movement: Use arrow buttons to make robot walk
5. Test head control: Move sliders to control head position
6. Monitor sensor data in tabs
### 6.3 Command-Line Testing
**Test Speech:**
```bash
ros2 topic pub --once /speech std_msgs/String "data: 'Hello from ROS2'"
```
**Test Movement:**
```bash
ros2 topic pub --times 3 /cmd_vel geometry_msgs/msg/Twist '{linear: {x: 0.05, y: 0.0, z: 0.0}, angular: {x: 0.0, y: 0.0, z: 0.0}}'
```
**Test Head Movement:**
```bash
ros2 topic pub --once /joint_angles naoqi_bridge_msgs/msg/JointAnglesWithSpeed '{joint_names: ["HeadYaw"], joint_angles: [0.5], speed: 0.3}'
```
## Part 7: Production Deployment
### 7.1 Launch Script Creation
Create production-ready launch script (`scripts/launch_nao6.sh`):
```bash
#!/bin/bash
# NAO6 HRIStudio Integration Launch Script
set -e
# Configuration
NAO_IP="${NAO_IP:-nao.local}"
NAO_PASSWORD="${NAO_PASSWORD:-nao}"
BRIDGE_PORT="${BRIDGE_PORT:-9090}"
# Function to wake up robot
wake_up_robot() {
echo "Waking up NAO robot..."
sshpass -p "$NAO_PASSWORD" ssh nao@$NAO_IP "python2 -c \"
import sys
sys.path.append('/opt/aldebaran/lib/python2.7/site-packages')
import naoqi
motion = naoqi.ALProxy('ALMotion', '127.0.0.1', 9559)
motion.wakeUp()
print 'Robot awakened'
\""
}
# Main execution
echo "Starting NAO6 HRIStudio Integration"
echo "NAO IP: $NAO_IP"
echo "Bridge Port: $BRIDGE_PORT"
# Check connections
ping -c 1 $NAO_IP || { echo "Cannot reach NAO"; exit 1; }
# Start ROS integration
cd ~/naoqi_ros2_ws
source install/setup.bash
# Wake up robot in background
wake_up_robot &
# Launch ROS system
exec ros2 launch install/nao_launch/share/nao_launch/launch/nao6_hristudio.launch.py \
nao_ip:="$NAO_IP" \
password:="$NAO_PASSWORD" \
bridge_port:="$BRIDGE_PORT"
```
### 7.2 Service Integration (Optional)
Create systemd service for automatic startup:
```ini
[Unit]
Description=NAO6 HRIStudio Integration
After=network.target
[Service]
Type=simple
User=your_user
Environment=NAO_IP=nao.local
Environment=NAO_PASSWORD=your_password
ExecStart=/path/to/launch_nao6.sh
Restart=always
[Install]
WantedBy=multi-user.target
```
## Part 8: Safety and Best Practices
### 8.1 Safety Guidelines
- **Always keep emergency stop accessible** in the web interface
- **Start with small movements and low speeds** when testing
- **Monitor robot battery level** during long sessions
- **Ensure clear space around robot** before movement commands
- **Never leave robot unattended** during operation
### 8.2 Performance Optimization
**Network Optimization:**
```bash
# Increase network buffer sizes for camera data
sudo sysctl -w net.core.rmem_max=26214400
sudo sysctl -w net.core.rmem_default=26214400
```
**ROS2 Optimization:**
```bash
# Use optimized RMW implementation
export RMW_IMPLEMENTATION=rmw_cyclonedx_cpp
```
### 8.3 Troubleshooting Checklist
**Before Starting:**
- [ ] NAO robot powered on and connected to network
- [ ] ROS2 Humble installed and sourced
- [ ] NAO workspace built successfully
- [ ] Database running (Docker container)
- [ ] Correct NAO password known
**During Operation:**
- [ ] rosbridge WebSocket server running on port 9090
- [ ] NAO robot in standing position (not crouching)
- [ ] Robot joints stiffened (not loose)
- [ ] HRIStudio web interface connected to ROS bridge
**If Commands Not Working:**
1. Check robot is awake and standing
2. Verify topic names in web interface match ROS topics
3. Test commands from command line first
4. Check rosbridge logs for errors
## Part 9: Future Enhancements
### 9.1 Advanced Features
- **Multi-camera streaming** for experiment recording
- **Advanced gesture recognition** through touch sensors
- **Autonomous behavior integration** with navigation
- **Multi-robot coordination** for group interaction studies
### 9.2 Plugin Development
The NAO6 integration supports the HRIStudio plugin system for adding custom behaviors and extending robot capabilities.
## Conclusion
This guide provides a complete integration of NAO6 robots with HRIStudio, enabling researchers to conduct sophisticated Human-Robot Interaction experiments with full robot control, real-time data collection, and web-based interfaces.
The key insight discovered during implementation is that NAO robots require explicit "wake-up" commands to enable motor control, which must be performed before any movement commands will be executed.
**Support Resources:**
- NAO Documentation: https://developer.softbankrobotics.com/nao6
- naoqi_driver2: https://github.com/ros-naoqi/naoqi_driver2
- ROS2 Humble: https://docs.ros.org/en/humble/
- HRIStudio Documentation: See `docs/` folder
---
**Integration Status: Production Ready ✅**
*Last Updated: January 2025*
*Tested With: NAO V6.0 / NAOqi 2.8.7.4 / ROS2 Humble / HRIStudio v1.0*

233
docs/nao6-integration-summary.md
View File

@@ -1,233 +0,0 @@
# NAO6 ROS2 Integration Summary for HRIStudio
## Overview
This document summarizes the complete NAO6 ROS2 integration that has been implemented for HRIStudio, providing researchers with full access to NAO6 capabilities through the visual experiment designer and real-time wizard interface.
## What's Been Implemented
### 1. NAO6 ROS2 Plugin (`nao6-ros2.json`)
A comprehensive robot plugin that exposes all NAO6 capabilities through standard ROS2 topics:
**Location**: `robot-plugins/plugins/nao6-ros2.json`
**Key Features**:
- Full ROS2 integration via `naoqi_driver2`
- 10 robot actions across movement, interaction, and sensors
- Proper HRIStudio plugin schema compliance
- Safety limits and parameter validation
- Transform functions for message conversion
### 2. Robot Actions Available
#### Movement Actions
- **Walk with Velocity**: Control linear/angular walking velocities
- **Stop Walking**: Emergency stop for immediate movement cessation
- **Set Joint Angle**: Control individual joint positions (25 DOF)
- **Turn Head**: Dedicated head orientation control
#### Interaction Actions
- **Say Text**: Text-to-speech via ROS2 `/speech` topic
#### Sensor Actions
- **Get Camera Image**: Capture from front or bottom cameras
- **Get Joint States**: Read current joint positions and velocities
- **Get IMU Data**: Inertial measurement from torso sensor
- **Get Bumper Status**: Foot contact sensor readings
- **Get Touch Sensors**: Hand and head tactile sensor states
- **Get Sonar Range**: Ultrasonic distance measurements
- **Get Robot Info**: General robot status and information
### 3. ROS2 Topic Mapping
The plugin maps to these standard NAO6 ROS2 topics:
```
/cmd_vel → Robot velocity commands (Twist)
/odom → Odometry data (Odometry)
/joint_states → Joint positions/velocities (JointState)
/joint_angles → NAO-specific joint control (JointAnglesWithSpeed)
/camera/front/image_raw → Front camera stream (Image)
/camera/bottom/image_raw → Bottom camera stream (Image)
/imu/torso → Inertial measurement (Imu)
/speech → Text-to-speech commands (String)
/bumper → Foot bumper sensors (Bumper)
/hand_touch → Hand touch sensors (HandTouch)
/head_touch → Head touch sensors (HeadTouch)
/sonar/left → Left ultrasonic sensor (Range)
/sonar/right → Right ultrasonic sensor (Range)
/info → Robot information (RobotInfo)
```
### 4. Transform Functions (`nao6-transforms.ts`)
**Location**: `src/lib/nao6-transforms.ts`
Comprehensive message conversion functions:
- Parameter validation and safety limits
- ROS2 message format compliance
- Joint limit enforcement (25 DOF with proper ranges)
- Velocity clamping for safe operation
- Helper functions for UI integration
### 5. Setup Documentation (`nao6-ros2-setup.md`)
**Location**: `docs/nao6-ros2-setup.md`
Complete setup guide covering:
- ROS2 Humble installation on Ubuntu 22.04
- NAO6 network configuration
- naoqi_driver2 and rosbridge setup
- Custom launch file creation
- Testing and validation procedures
- HRIStudio plugin configuration
- Troubleshooting and safety guidelines
## Technical Architecture
### ROS2 Integration Stack
```
HRIStudio (Web Interface)
↓ WebSocket
rosbridge_server (Port 9090)
↓ ROS2 Topics/Services
naoqi_driver2
↓ NAOqi Protocol (Port 9559)
NAO6 Robot
```
### Message Flow
1. **Command Execution**:
- HRIStudio wizard interface → WebSocket → rosbridge → ROS2 topic → naoqi_driver2 → NAO6
2. **Sensor Data**:
- NAO6 → naoqi_driver2 → ROS2 topic → rosbridge → WebSocket → HRIStudio
3. **Real-time Feedback**:
- Continuous sensor streams for live monitoring
- Event logging for research data capture
## Safety Features
### Joint Limits Enforcement
- All 25 NAO6 joints have proper min/max limits defined
- Automatic clamping prevents damage from invalid commands
- Parameter validation before message transmission
### Velocity Limits
- Linear velocity: -0.55 to 0.55 m/s
- Angular velocity: -2.0 to 2.0 rad/s
- Automatic clamping for safe operation
### Emergency Stops
- Dedicated stop action for immediate movement cessation
- Timeout protection on all actions
- Connection monitoring and error handling
## Integration Status
### ✅ Completed Components
1. **Plugin Definition**: Full NAO6 plugin with proper schema
2. **Action Library**: 10 comprehensive robot actions
3. **Transform Functions**: Complete message conversion system
4. **Documentation**: Setup guide and integration instructions
5. **Safety Systems**: Joint limits, velocity clamping, emergency stops
6. **Repository Integration**: Plugin added to official repository
### 🔄 Usage Workflow
1. **Setup Phase**:
- Install ROS2 Humble on companion computer
- Configure NAO6 network connection
- Launch naoqi_driver2 and rosbridge
2. **HRIStudio Configuration**:
- Install NAO6 plugin in study
- Configure ROS bridge URL
- Design experiments using NAO6 actions
3. **Experiment Execution**:
- Real-time robot control through wizard interface
- Live sensor data monitoring
- Comprehensive event logging
## Research Capabilities
### Experiment Design
- Visual programming with NAO6-specific actions
- Parameter configuration with safety validation
- Multi-modal data collection coordination
### Data Capture
- Synchronized robot commands and sensor data
- Video streams from dual cameras
- Inertial, tactile, and proximity sensor logs
- Speech synthesis and timing records
### Reproducibility
- Standardized action definitions
- Consistent parameter schemas
- Version-controlled plugin specifications
- Complete experiment protocol documentation
## Next Steps for Researchers
### Immediate Use
1. Follow setup guide in `docs/nao6-ros2-setup.md`
2. Install NAO6 plugin in HRIStudio study
3. Create experiments using available actions
4. Run trials with real-time robot control
### Advanced Integration
1. **Custom Actions**: Extend plugin with study-specific behaviors
2. **Multi-Robot**: Scale to multiple NAO6 robots
3. **Navigation**: Add SLAM and path planning capabilities
4. **Manipulation**: Implement object interaction behaviors
### Research Applications
- Human-robot interaction studies
- Social robotics experiments
- Gesture and speech coordination research
- Sensor fusion and behavior analysis
- Wizard-of-Oz methodology validation
## Support and Resources
### Documentation
- **Setup Guide**: `docs/nao6-ros2-setup.md`
- **Plugin Schema**: `robot-plugins/docs/schema.md`
- **ROS2 Integration**: `docs/ros2-integration.md`
- **Transform Functions**: `src/lib/nao6-transforms.ts`
### External Resources
- **NAO6 Documentation**: https://developer.softbankrobotics.com/nao6
- **naoqi_driver2**: https://github.com/ros-naoqi/naoqi_driver2
- **ROS2 Humble**: https://docs.ros.org/en/humble/
- **rosbridge**: http://wiki.ros.org/rosbridge_suite
### Technical Support
- **HRIStudio Issues**: GitHub repository
- **ROS2 Community**: ROS Discourse forum
- **NAO6 Support**: SoftBank Robotics developer portal
## Conclusion
The NAO6 ROS2 integration provides researchers with a complete, production-ready system for conducting Human-Robot Interaction studies. The integration leverages:
- **Standard ROS2 protocols** for reliable communication
- **Comprehensive safety systems** for secure operation
- **Visual experiment design** for accessible research tools
- **Real-time control interfaces** for dynamic experiment execution
- **Complete data capture** for rigorous analysis
This implementation enables researchers to focus on their studies rather than technical integration, while maintaining the flexibility and control needed for cutting-edge HRI research.
---
**Status**: ✅ Production Ready
**Last Updated**: December 2024
**Compatibility**: HRIStudio v1.0+, ROS2 Humble, NAO6 with NAOqi 2.8.7+

157
docs/nao6-quick-reference.md Normal file → Executable file
View File

@@ -1,155 +1,114 @@
# NAO6 HRIStudio Quick Reference
# NAO6 Quick Reference
**Essential commands for NAO6 robot integration with HRIStudio**
Essential commands for using NAO6 robots with HRIStudio.
## 🚀 Quick Start (5 Steps)
## Quick Start
### 1. Start ROS Integration
### 1. Start NAO Integration
```bash
cd ~/naoqi_ros2_ws
source install/setup.bash
ros2 launch install/nao_launch/share/nao_launch/launch/nao6_hristudio.launch.py nao_ip:=nao.local password:=robolab
ros2 launch nao_launch nao6_hristudio.launch.py nao_ip:=nao.local password:=robolab
```
### 2. Wake Up Robot (CRITICAL!)
### 2. Wake Robot
Press chest button for 3 seconds, or use:
```bash
sshpass -p "robolab" ssh nao@nao.local "python2 -c \"
import sys
sys.path.append('/opt/aldebaran/lib/python2.7/site-packages')
import naoqi
motion = naoqi.ALProxy('ALMotion', '127.0.0.1', 9559)
motion.wakeUp()
print 'Robot awakened'
\""
# Via SSH (institution-specific password)
ssh nao@nao.local
# Then run wake-up command (see integration repo docs)
```
### 3. Start HRIStudio
```bash
cd /home/robolab/Documents/Projects/hristudio
cd ~/Documents/Projects/hristudio
bun dev
```
### 4. Access Test Interface
- URL: `http://localhost:3000/nao-test`
- Login: `sean@soconnor.dev` / `password123`
### 4. Test Connection
- Open: `http://localhost:3000/nao-test`
- Click "Connect"
- Test robot commands
### 5. Test Robot
- Click "Connect" to WebSocket
- Try speech: "Hello from HRIStudio!"
- Use movement buttons to control robot
## Essential Commands
## 🛠️ Essential Commands
### Connection Testing
### Test Connectivity
```bash
# Test NAO connectivity
ping nao.local
# Test NAOqi service
telnet nao.local 9559
# Check ROS topics
ros2 topic list | grep naoqi
ping nao.local # Test network
ros2 topic list | grep naoqi # Check ROS topics
```
### Manual Robot Control
### Manual Control
```bash
# Speech test
# Speech
ros2 topic pub --once /speech std_msgs/String "data: 'Hello world'"
# Movement test (robot must be awake!)
ros2 topic pub --times 3 /cmd_vel geometry_msgs/msg/Twist '{linear: {x: 0.05, y: 0.0, z: 0.0}, angular: {x: 0.0, y: 0.0, z: 0.0}}'
# Movement (robot must be awake!)
ros2 topic pub --once /cmd_vel geometry_msgs/msg/Twist '{linear: {x: 0.1}}'
# Head movement test
ros2 topic pub --once /joint_angles naoqi_bridge_msgs/msg/JointAnglesWithSpeed '{joint_names: ["HeadYaw"], joint_angles: [0.5], speed: 0.3}'
# Stop all movement
ros2 topic pub --once /cmd_vel geometry_msgs/msg/Twist '{linear: {x: 0.0, y: 0.0, z: 0.0}, angular: {x: 0.0, y: 0.0, z: 0.0}}'
# Stop
ros2 topic pub --once /cmd_vel geometry_msgs/msg/Twist '{linear: {x: 0.0}}'
```
### Status Checks
### Monitor Status
```bash
# Check robot info
ros2 service call /naoqi_driver/get_robot_config naoqi_bridge_msgs/srv/GetRobotInfo
ros2 topic echo /naoqi_driver/battery # Battery level
ros2 topic echo /naoqi_driver/joint_states # Joint positions
```
# Monitor joint states
ros2 topic echo /naoqi_driver/joint_states --once
## Troubleshooting
# Check ROS nodes
ros2 node list
**Robot not moving:** Press chest button for 3 seconds to wake up
# Check WebSocket connection
**WebSocket fails:** Check rosbridge is running on port 9090
```bash
ss -an | grep 9090
```
## 🔧 Troubleshooting
### Robot Not Moving
**Problem:** Commands sent but robot doesn't move
**Solution:** Robot needs to be awakened first
**Connection lost:** Restart rosbridge
```bash
# Wake up via SSH (see step 2 above)
# OR press chest button for 3 seconds
pkill -f rosbridge
ros2 run rosbridge_server rosbridge_websocket
```
### Connection Issues
```bash
# Kill existing processes
sudo fuser -k 9090/tcp
pkill -f "rosbridge\|naoqi\|ros2"
## ROS Topics
# Restart database
sudo docker compose down && sudo docker compose up -d
```
### Import Errors in Web Interface
**Problem:** React component import failures
**Solution:** Use `~` import alias consistently:
```typescript
import { Button } from "~/components/ui/button";
// NOT: import { Button } from "@/components/ui/button";
```
## 📊 Key Topics
### Input Topics (Robot Control)
**Commands (Input):**
- `/speech` - Text-to-speech
- `/cmd_vel` - Movement commands
- `/joint_angles` - Joint position control
- `/cmd_vel` - Movement
- `/joint_angles` - Joint control
### Output Topics (Sensor Data)
- `/naoqi_driver/joint_states` - Joint positions/velocities
**Sensors (Output):**
- `/naoqi_driver/joint_states` - Joint data
- `/naoqi_driver/battery` - Battery level
- `/naoqi_driver/bumper` - Foot sensors
- `/naoqi_driver/hand_touch` - Hand touch sensors
- `/naoqi_driver/head_touch` - Head touch sensors
- `/naoqi_driver/sonar/left` - Left ultrasonic sensor
- `/naoqi_driver/sonar/right` - Right ultrasonic sensor
- `/naoqi_driver/camera/front/image_raw` - Front camera
- `/naoqi_driver/camera/bottom/image_raw` - Bottom camera
- `/naoqi_driver/sonar/*` - Distance sensors
- `/naoqi_driver/camera/*` - Camera feeds
## 🔗 WebSocket Integration
## WebSocket
**ROS Bridge URL:** `ws://134.82.159.25:9090`
**URL:** `ws://localhost:9090`
**Message Format:**
**Example message:**
```javascript
// Publish command
{
"op": "publish",
"topic": "/speech",
"type": "std_msgs/String",
"msg": {"data": "Hello world"}
}
// Subscribe to topic
{
"op": "subscribe",
"topic": "/naoqi_driver/joint_states",
"type": "sensor_msgs/JointState"
}
```
## 🎯 Common Use Cases
## More Information
See **[nao6-hristudio-integration](../../nao6-hristudio-integration/)** repository for:
- Complete installation guide
- Detailed usage instructions
- Full troubleshooting guide
- Plugin definitions
- Launch file configurations
## Common Use Cases
### Make Robot Speak
```bash

376
docs/nao6-ros2-setup.md
View File

@@ -1,376 +0,0 @@
# NAO6 ROS2 Setup Guide for HRIStudio
This guide walks you through setting up your NAO6 robot with ROS2 integration for use with HRIStudio's experiment platform.
> **📋 For Complete Integration Guide:** See [NAO6 Complete Integration Guide](./nao6-integration-complete-guide.md) for comprehensive setup, troubleshooting, and production deployment instructions.
**⚠️ Critical Note:** NAO robots must be "awakened" (motors stiffened and standing) before movement commands will work. See the troubleshooting section below.
## Prerequisites
- NAO6 robot with NAOqi OS 2.8.7+
- Ubuntu 22.04.5 LTS computer (x86_64)
- Network connectivity between computer and NAO6
- Administrative access to both systems
## Overview
The integration uses the `naoqi_driver2` package to bridge NAOqi with ROS2, exposing all robot capabilities through standard ROS2 topics and services. HRIStudio connects via WebSocket using `rosbridge_server`.
## Step 1: NAO6 Network Configuration
1. **Power on your NAO6** and wait for boot completion
2. **Connect NAO6 to your network**:
- Press chest button to get IP address
- Or use Choregraphe to configure WiFi
3. **Verify connectivity**:
```bash
ping nao.local # or robot IP address
```
## Step 2: ROS2 Humble Installation
Install ROS2 Humble on your Ubuntu 22.04 system:
```bash
# Update system
sudo apt update && sudo apt upgrade -y
# Install ROS2 Humble
sudo apt install software-properties-common
sudo add-apt-repository universe
sudo apt update && sudo apt install curl -y
sudo curl -sSL https://raw.githubusercontent.com/ros/rosdistro/master/ros.asc | sudo apt-key add -
sudo sh -c 'echo "deb http://packages.ros.org/ros2/ubuntu $(lsb_release -cs) main" > /etc/apt/sources.list.d/ros2-latest.list'
sudo apt update
sudo apt install ros-humble-desktop
sudo apt install ros-dev-tools
# Source ROS2
echo "source /opt/ros/humble/setup.bash" >> ~/.bashrc
source ~/.bashrc
```
## Step 3: Install NAO ROS2 Packages
Install the required ROS2 packages for NAO6 integration:
```bash
# Install naoqi_driver2 and dependencies
sudo apt install ros-humble-naoqi-driver2
sudo apt install ros-humble-naoqi-bridge-msgs
sudo apt install ros-humble-geometry-msgs
sudo apt install ros-humble-sensor-msgs
sudo apt install ros-humble-nav-msgs
sudo apt install ros-humble-std-msgs
# Install rosbridge for HRIStudio communication
sudo apt install ros-humble-rosbridge-suite
# Install additional useful packages
sudo apt install ros-humble-rqt
sudo apt install ros-humble-rqt-common-plugins
```
## Step 4: Configure NAO Connection
Create a launch file for easy NAO6 connection:
```bash
# Create workspace
mkdir -p ~/nao_ws/src
cd ~/nao_ws
# Create launch file directory
mkdir -p src/nao_launch/launch
# Create the launch file
cat > src/nao_launch/launch/nao6_hristudio.launch.py << 'EOF'
from launch import LaunchDescription
from launch.actions import DeclareLaunchArgument, IncludeLaunchDescription
from launch.substitutions import LaunchConfiguration
from launch_ros.actions import Node
from launch.launch_description_sources import PythonLaunchDescriptionSource
from ament_index_python.packages import get_package_share_directory
import os
def generate_launch_description():
return LaunchDescription([
# NAO IP configuration
DeclareLaunchArgument('nao_ip', default_value='nao.local'),
DeclareLaunchArgument('nao_port', default_value='9559'),
DeclareLaunchArgument('bridge_port', default_value='9090'),
# NAOqi Driver
Node(
package='naoqi_driver2',
executable='naoqi_driver',
name='naoqi_driver',
parameters=[{
'nao_ip': LaunchConfiguration('nao_ip'),
'nao_port': LaunchConfiguration('nao_port'),
'publish_joint_states': True,
'publish_odometry': True,
'publish_camera': True,
'publish_sensors': True,
'joint_states_frequency': 30.0,
'odom_frequency': 30.0,
'camera_frequency': 15.0,
'sensor_frequency': 10.0
}],
output='screen'
),
# Rosbridge WebSocket Server
Node(
package='rosbridge_server',
executable='rosbridge_websocket',
name='rosbridge_websocket',
parameters=[{
'port': LaunchConfiguration('bridge_port'),
'address': '0.0.0.0',
'authenticate': False,
'fragment_timeout': 600,
'delay_between_messages': 0,
'max_message_size': 10000000
}],
output='screen'
)
])
EOF
# Create package.xml
cat > src/nao_launch/package.xml << 'EOF'
<?xml version="1.0"?>
<?xml-model href="http://download.ros.org/schema/package_format3.xsd" schematypeid="pf3"?>
<package format="3">
<name>nao_launch</name>
<version>1.0.0</version>
<description>Launch files for NAO6 HRIStudio integration</description>
<maintainer email="you@example.com">Your Name</maintainer>
<license>MIT</license>
<buildtool_depend>ament_cmake</buildtool_depend>
<exec_depend>launch</exec_depend>
<exec_depend>launch_ros</exec_depend>
<exec_depend>naoqi_driver2</exec_depend>
<exec_depend>rosbridge_server</exec_depend>
</package>
EOF
# Create CMakeLists.txt
cat > src/nao_launch/CMakeLists.txt << 'EOF'
cmake_minimum_required(VERSION 3.8)
project(nao_launch)
find_package(ament_cmake REQUIRED)
install(DIRECTORY launch/
DESTINATION share/${PROJECT_NAME}/launch/
)
ament_package()
EOF
# Build the workspace
colcon build
source install/setup.bash
```
## Step 5: Test NAO Connection
Start the NAO6 ROS2 integration:
```bash
cd ~/nao_ws
source install/setup.bash
# Launch with your NAO's IP address
ros2 launch nao_launch nao6_hristudio.launch.py nao_ip:=YOUR_NAO_IP
```
Replace `YOUR_NAO_IP` with your NAO's actual IP address (e.g., `192.168.1.100`).
## Step 6: Verify ROS2 Topics
In a new terminal, verify that NAO topics are publishing:
```bash
source /opt/ros/humble/setup.bash
# List all topics
ros2 topic list
# You should see these NAO6 topics:
# /cmd_vel - Robot velocity commands
# /odom - Odometry data
# /joint_states - Joint positions and velocities
# /joint_angles - NAO-specific joint control
# /camera/front/image_raw - Front camera
# /camera/bottom/image_raw - Bottom camera
# /imu/torso - Inertial measurement unit
# /bumper - Foot bumper sensors
# /hand_touch - Hand tactile sensors
# /head_touch - Head tactile sensors
# /sonar/left - Left ultrasonic sensor
# /sonar/right - Right ultrasonic sensor
# /info - Robot information
# Test specific topics
ros2 topic echo /joint_states --once
ros2 topic echo /odom --once
ros2 topic echo /info --once
```
## Step 7: Test Robot Control
Test basic robot control:
```bash
# Make NAO say something
ros2 topic pub /speech std_msgs/msg/String "data: 'Hello from ROS2!'" --once
# Move head (be careful with joint limits)
ros2 topic pub /joint_angles naoqi_bridge_msgs/msg/JointAnglesWithSpeed \
"joint_names: ['HeadYaw']
joint_angles: [0.5]
speed: 0.3" --once
# Basic walking command (very small movement)
ros2 topic pub /cmd_vel geometry_msgs/msg/Twist \
"linear: {x: 0.1, y: 0.0, z: 0.0}
angular: {x: 0.0, y: 0.0, z: 0.0}" --once
# Stop movement
ros2 topic pub /cmd_vel geometry_msgs/msg/Twist \
"linear: {x: 0.0, y: 0.0, z: 0.0}
angular: {x: 0.0, y: 0.0, z: 0.0}" --once
```
## Step 8: Configure HRIStudio
1. **Start HRIStudio** with your development setup
2. **Add NAO6 Plugin Repository**:
- Go to Admin → Plugin Repositories
- Add the HRIStudio official repository if not already present
- Sync to get the latest plugins including `nao6-ros2`
3. **Install NAO6 Plugin**:
- In your study, go to Plugins
- Install the "NAO6 Robot (ROS2 Integration)" plugin
- Configure the ROS bridge URL: `ws://YOUR_COMPUTER_IP:9090`
4. **Create Experiment**:
- Use the experiment designer
- Add NAO6 actions from the robot blocks section
- Configure parameters for each action
5. **Run Trial**:
- Ensure your NAO6 ROS2 system is running
- Start a trial in HRIStudio
- Control the robot through the wizard interface
## Available Robot Actions
Your NAO6 plugin provides these actions for experiments:
### Movement Actions
- **Walk with Velocity**: Control linear/angular velocity
- **Stop Walking**: Emergency stop
- **Set Joint Angle**: Control individual joints
- **Turn Head**: Head orientation control
### Interaction Actions
- **Say Text**: Text-to-speech via ROS2
### Sensor Actions
- **Get Camera Image**: Capture from front/bottom cameras
- **Get Joint States**: Read all joint positions
- **Get IMU Data**: Inertial measurement data
- **Get Bumper Status**: Foot contact sensors
- **Get Touch Sensors**: Hand/head touch detection
- **Get Sonar Range**: Ultrasonic distance sensors
- **Get Robot Info**: General robot status
## Troubleshooting
### NAO Connection Issues
```bash
# Check NAO network connectivity
ping nao.local
# Check NAOqi service
telnet nao.local 9559
# Restart NAOqi on NAO
# (Use robot's web interface or Choregraphe)
```
### ROS2 Issues
```bash
# Check if naoqi_driver2 is running
ros2 node list | grep naoqi
# Check topic publication rates
ros2 topic hz /joint_states
# Restart the launch file
ros2 launch nao_launch nao6_hristudio.launch.py nao_ip:=YOUR_NAO_IP
```
### HRIStudio Connection Issues
```bash
# Verify rosbridge is running
netstat -an | grep 9090
# Check WebSocket connection
curl -i -N -H "Connection: Upgrade" \
-H "Upgrade: websocket" \
-H "Sec-WebSocket-Key: test" \
-H "Sec-WebSocket-Version: 13" \
http://localhost:9090
```
### Robot Safety
- Always keep emergency stop accessible
- Start with small movements and low speeds
- Monitor robot battery level
- Ensure clear space around robot
- Never leave robot unattended during operation
## Performance Optimization
### Network Optimization
```bash
# Increase network buffer sizes for camera data
sudo sysctl -w net.core.rmem_max=26214400
sudo sysctl -w net.core.rmem_default=26214400
```
### ROS2 Optimization
```bash
# Adjust QoS settings for better performance
export RMW_IMPLEMENTATION=rmw_cyclonedx_cpp
export CYCLONEDX_URI=file:///path/to/cyclonedx.xml
```
## Next Steps
1. **Experiment Design**: Create experiments using NAO6 actions
2. **Data Collection**: Use sensor actions for research data
3. **Custom Actions**: Extend the plugin with custom behaviors
4. **Multi-Robot**: Scale to multiple NAO6 robots
5. **Advanced Features**: Implement navigation, manipulation, etc.
## Support Resources
- **NAO Documentation**: https://developer.softbankrobotics.com/nao6
- **naoqi_driver2**: https://github.com/ros-naoqi/naoqi_driver2
- **ROS2 Humble**: https://docs.ros.org/en/humble/
- **HRIStudio Docs**: See `docs/` folder
- **Community**: HRIStudio Discord/Forum
---
**Success!** Your NAO6 is now ready for use with HRIStudio experiments. The robot's capabilities are fully accessible through the visual experiment designer and real-time wizard interface.

0
docs/paper.md Normal file → Executable file
View File

0
docs/plugin-system-implementation-guide.md Normal file → Executable file
View File

0
docs/project-overview.md Normal file → Executable file
View File

0
docs/project-status.md Normal file → Executable file
View File

0
docs/proposal.tex Normal file → Executable file
View File

0
docs/quick-reference.md Normal file → Executable file
View File

0
docs/roman-2025-talk.md Normal file → Executable file
View File

0
docs/ros2-integration.md Normal file → Executable file
View File

0
docs/ros2_naoqi.md Normal file → Executable file
View File

0
docs/route-consolidation-summary.md Normal file → Executable file
View File

0
docs/thesis-project-priorities.md Normal file → Executable file
View File

0
docs/trial-system-overhaul.md Normal file → Executable file
View File

0
docs/wizard-interface-final.md Normal file → Executable file
View File

0
docs/wizard-interface-guide.md Normal file → Executable file
View File

0
docs/wizard-interface-redesign.md Normal file → Executable file
View File

0
docs/wizard-interface-summary.md Normal file → Executable file
View File

0
docs/work_in_progress.md Normal file → Executable file
View File