mirror of
https://github.com/soconnor0919/hristudio.git
synced 2025-12-11 06:34:44 -05:00
feat: Complete NAO6 ROS2 integration for HRIStudio
🤖 Full NAO6 Robot Integration with ROS2 and WebSocket Control ## New Features - **NAO6 Test Interface**: Real-time robot control via web browser at /nao-test - **ROS2 Integration**: Complete naoqi_driver2 + rosbridge setup with launch files - **WebSocket Control**: Direct robot control through HRIStudio web interface - **Plugin System**: NAO6 robot plugins for movement, speech, and sensors - **Database Integration**: Updated seed data with NAO6 robot and plugin definitions ## Key Components Added - **Web Interface**: src/app/(dashboard)/nao-test/page.tsx - Complete robot control dashboard - **Plugin Repository**: public/nao6-plugins/ - Local NAO6 plugin definitions - **Database Updates**: Updated robots table with ROS2 protocol and enhanced capabilities - **Comprehensive Documentation**: Complete setup, troubleshooting, and quick reference guides ## Documentation - **Complete Integration Guide**: docs/nao6-integration-complete-guide.md (630 lines) - **Quick Reference**: docs/nao6-quick-reference.md - Essential commands and troubleshooting - **Updated Setup Guide**: Enhanced docs/nao6-ros2-setup.md with critical notes - **Updated Main Docs**: docs/README.md with robot integration section ## Robot Capabilities - ✅ **Speech Control**: Text-to-speech with emotion and language support - ✅ **Movement Control**: Walking, turning, stopping with configurable speeds - ✅ **Head Control**: Precise yaw/pitch positioning with sliders - ✅ **Sensor Monitoring**: Joint states, touch sensors, sonar, cameras, IMU - ✅ **Safety Features**: Emergency stop, movement limits, real-time monitoring - ✅ **Real-time Data**: Live sensor data streaming through WebSocket ## Critical Discovery **Robot Wake-Up Requirement**: NAO robots start in safe mode with loose joints and must be explicitly awakened via SSH before movement commands work. This is now documented with automated solutions. ## Technical Implementation - **ROS2 Humble**: Complete naoqi_driver2 integration with rosbridge WebSocket server - **Topic Mapping**: Correct namespace handling for control vs. sensor topics - **Plugin Architecture**: Extensible NAO6 action definitions with parameter validation - **Database Schema**: Enhanced robots table with comprehensive NAO6 capabilities - **Import Consistency**: Fixed React import aliases to use ~ consistently ## Testing & Verification - ✅ Tested with NAO V6.0 / NAOqi 2.8.7.4 / ROS2 Humble - ✅ Complete end-to-end testing from web interface to robot movement - ✅ Comprehensive troubleshooting procedures documented - ✅ Production-ready launch scripts and deployment guides ## Production Ready This integration is fully tested and production-ready for Human-Robot Interaction research with complete documentation, safety guidelines, and troubleshooting procedures.
This commit is contained in:
@@ -112,10 +112,16 @@ This documentation suite provides everything needed to understand, build, deploy
|
||||
- Technical debt resolution
|
||||
- UI/UX enhancements
|
||||
|
||||
### **🤖 Robot Integration Guides**
|
||||
|
||||
14. **[NAO6 Complete Integration Guide](./nao6-integration-complete-guide.md)** - Comprehensive NAO6 setup, troubleshooting, and production deployment
|
||||
15. **[NAO6 Quick Reference](./nao6-quick-reference.md)** - Essential commands and troubleshooting for NAO6 integration
|
||||
16. **[NAO6 ROS2 Setup](./nao6-ros2-setup.md)** - Basic NAO6 ROS2 driver installation guide
|
||||
|
||||
### **📖 Academic References**
|
||||
|
||||
14. **[Research Paper](./root.tex)** - Academic LaTeX document
|
||||
15. **[Bibliography](./refs.bib)** - Research references
|
||||
17. **[Research Paper](./root.tex)** - Academic LaTeX document
|
||||
18. **[Bibliography](./refs.bib)** - Research references
|
||||
|
||||
---
|
||||
|
||||
@@ -152,8 +158,14 @@ This documentation suite provides everything needed to understand, build, deploy
|
||||
### **For Researchers**
|
||||
1. **[Project Overview](./project-overview.md)** - Research platform capabilities
|
||||
2. **[Feature Requirements](./feature-requirements.md)** - User workflows and features
|
||||
3. **[ROS2 Integration](./ros2-integration.md)** - Robot platform integration
|
||||
4. **[Research Paper](./root.tex)** - Academic context and methodology
|
||||
3. **[NAO6 Quick Reference](./nao6-quick-reference.md)** - Essential NAO6 robot control commands
|
||||
4. **[ROS2 Integration](./ros2-integration.md)** - Robot platform integration
|
||||
5. **[Research Paper](./root.tex)** - Academic context and methodology
|
||||
|
||||
### **For Robot Integration**
|
||||
1. **[NAO6 Complete Integration Guide](./nao6-integration-complete-guide.md)** - Full NAO6 setup and troubleshooting
|
||||
2. **[NAO6 Quick Reference](./nao6-quick-reference.md)** - Essential commands and quick fixes
|
||||
3. **[ROS2 Integration](./ros2-integration.md)** - General robot integration patterns
|
||||
|
||||
---
|
||||
|
||||
@@ -219,6 +231,13 @@ bun dev
|
||||
- **Comprehensive Testing**: Realistic seed data with complete scenarios
|
||||
- **Developer Friendly**: Clear patterns and extensive documentation
|
||||
|
||||
### **Robot Integration**
|
||||
- **NAO6 Full Support**: Complete ROS2 integration with movement, speech, and sensor control
|
||||
- **Real-time Control**: WebSocket-based robot control through web interface
|
||||
- **Safety Features**: Emergency stops, movement limits, and comprehensive monitoring
|
||||
- **Production Ready**: Tested with NAO V6.0 / NAOqi 2.8.7.4 / ROS2 Humble
|
||||
- **Troubleshooting Guides**: Complete documentation for setup and problem resolution
|
||||
|
||||
---
|
||||
|
||||
## 🎊 **Project Status: Production Ready**
|
||||
@@ -238,6 +257,7 @@ bun dev
|
||||
- ✅ **Core Blocks System** - 26 blocks across events, wizard, control, observation
|
||||
- ✅ **Plugin Architecture** - Unified system for core blocks and robot actions
|
||||
- ✅ **Development Environment** - Realistic test data and scenarios
|
||||
- ✅ **NAO6 Robot Integration** - Full ROS2 integration with comprehensive control and monitoring
|
||||
|
||||
---
|
||||
|
||||
@@ -271,7 +291,7 @@ The platform is considered production-ready when:
|
||||
- ✅ Performance targets are achieved
|
||||
- ✅ Type safety is complete throughout
|
||||
|
||||
**All success criteria have been met. HRIStudio is ready for production deployment.**
|
||||
**All success criteria have been met. HRIStudio is ready for production deployment with full NAO6 robot integration support.**
|
||||
|
||||
---
|
||||
|
||||
|
||||
630
docs/nao6-integration-complete-guide.md
Normal file
630
docs/nao6-integration-complete-guide.md
Normal file
@@ -0,0 +1,630 @@
|
||||
# NAO6 HRIStudio Integration: Complete Setup and Troubleshooting Guide
|
||||
|
||||
This comprehensive guide documents the complete process of integrating a NAO6 robot with HRIStudio, including all troubleshooting steps and solutions discovered during implementation.
|
||||
|
||||
## Overview
|
||||
|
||||
NAO6 integration with HRIStudio provides full robot control through a web-based interface, enabling researchers to conduct Human-Robot Interaction experiments with real-time robot control, sensor monitoring, and data collection.
|
||||
|
||||
**Integration Architecture:**
|
||||
```
|
||||
HRIStudio Web Interface → WebSocket → ROS Bridge → NAOqi Driver → NAO6 Robot
|
||||
```
|
||||
|
||||
## Prerequisites
|
||||
|
||||
### Hardware Requirements
|
||||
- NAO6 robot (NAOqi OS 2.8.7+)
|
||||
- Ubuntu 22.04 LTS computer
|
||||
- Network connectivity between computer and NAO6
|
||||
- Administrative access to both systems
|
||||
|
||||
### Software Requirements
|
||||
- ROS2 Humble
|
||||
- NAOqi Driver2 for ROS2
|
||||
- rosbridge-suite
|
||||
- HRIStudio platform
|
||||
- SSH access to NAO robot
|
||||
|
||||
## Part 1: ROS2 and NAO Driver Setup
|
||||
|
||||
### 1.1 Install ROS2 Humble
|
||||
|
||||
```bash
|
||||
# Update system
|
||||
sudo apt update && sudo apt upgrade -y
|
||||
|
||||
# Install ROS2 Humble
|
||||
sudo apt install software-properties-common
|
||||
sudo add-apt-repository universe
|
||||
sudo apt update && sudo apt install curl -y
|
||||
sudo curl -sSL https://raw.githubusercontent.com/ros/rosdistro/master/ros.asc | sudo apt-key add -
|
||||
sudo sh -c 'echo "deb http://packages.ros.org/ros2/ubuntu $(lsb_release -cs) main" > /etc/apt/sources.list.d/ros2-latest.list'
|
||||
|
||||
sudo apt update
|
||||
sudo apt install ros-humble-desktop
|
||||
sudo apt install ros-dev-tools
|
||||
|
||||
# Source ROS2
|
||||
echo "source /opt/ros/humble/setup.bash" >> ~/.bashrc
|
||||
source ~/.bashrc
|
||||
```
|
||||
|
||||
### 1.2 Install Required ROS2 Packages
|
||||
|
||||
```bash
|
||||
# Install rosbridge for HRIStudio communication
|
||||
sudo apt install ros-humble-rosbridge-suite
|
||||
|
||||
# Install additional useful packages
|
||||
sudo apt install ros-humble-rqt
|
||||
sudo apt install ros-humble-rqt-common-plugins
|
||||
```
|
||||
|
||||
### 1.3 Set Up NAO Workspace
|
||||
|
||||
**Note:** We assume you already have a NAO workspace at `~/naoqi_ros2_ws` with the NAOqi driver installed.
|
||||
|
||||
```bash
|
||||
# Verify workspace exists
|
||||
ls ~/naoqi_ros2_ws/src/naoqi_driver2
|
||||
```
|
||||
|
||||
If you need to set up the workspace from scratch, refer to the NAOqi ROS2 documentation.
|
||||
|
||||
### 1.4 Create Integrated Launch Package
|
||||
|
||||
Create a launch package that combines NAOqi driver with rosbridge:
|
||||
|
||||
```bash
|
||||
cd ~/naoqi_ros2_ws
|
||||
mkdir -p src/nao_launch/launch
|
||||
```
|
||||
|
||||
**Create launch file** (`src/nao_launch/launch/nao6_hristudio.launch.py`):
|
||||
|
||||
```python
|
||||
from launch import LaunchDescription
|
||||
from launch.actions import DeclareLaunchArgument
|
||||
from launch.substitutions import LaunchConfiguration
|
||||
from launch_ros.actions import Node
|
||||
|
||||
def generate_launch_description():
|
||||
return LaunchDescription([
|
||||
# NAO IP configuration
|
||||
DeclareLaunchArgument("nao_ip", default_value="nao.local"),
|
||||
DeclareLaunchArgument("nao_port", default_value="9559"),
|
||||
DeclareLaunchArgument("username", default_value="nao"),
|
||||
DeclareLaunchArgument("password", default_value="nao"),
|
||||
DeclareLaunchArgument("network_interface", default_value="eth0"),
|
||||
DeclareLaunchArgument("qi_listen_url", default_value="tcp://0.0.0.0:0"),
|
||||
DeclareLaunchArgument("namespace", default_value="naoqi_driver"),
|
||||
DeclareLaunchArgument("bridge_port", default_value="9090"),
|
||||
|
||||
# NAOqi Driver
|
||||
Node(
|
||||
package="naoqi_driver",
|
||||
executable="naoqi_driver_node",
|
||||
name="naoqi_driver",
|
||||
namespace=LaunchConfiguration("namespace"),
|
||||
parameters=[{
|
||||
"nao_ip": LaunchConfiguration("nao_ip"),
|
||||
"nao_port": LaunchConfiguration("nao_port"),
|
||||
"username": LaunchConfiguration("username"),
|
||||
"password": LaunchConfiguration("password"),
|
||||
"network_interface": LaunchConfiguration("network_interface"),
|
||||
"qi_listen_url": LaunchConfiguration("qi_listen_url"),
|
||||
"publish_joint_states": True,
|
||||
"publish_odometry": True,
|
||||
"publish_camera": True,
|
||||
"publish_sensors": True,
|
||||
"joint_states_frequency": 30.0,
|
||||
"odom_frequency": 30.0,
|
||||
"camera_frequency": 15.0,
|
||||
"sensor_frequency": 10.0,
|
||||
}],
|
||||
output="screen",
|
||||
),
|
||||
|
||||
# Rosbridge WebSocket Server for HRIStudio
|
||||
Node(
|
||||
package="rosbridge_server",
|
||||
executable="rosbridge_websocket",
|
||||
name="rosbridge_websocket",
|
||||
parameters=[{
|
||||
"port": LaunchConfiguration("bridge_port"),
|
||||
"address": "0.0.0.0",
|
||||
"authenticate": False,
|
||||
"fragment_timeout": 600,
|
||||
"delay_between_messages": 0,
|
||||
"max_message_size": 10000000,
|
||||
}],
|
||||
output="screen",
|
||||
),
|
||||
|
||||
# ROS API Server (required for rosbridge functionality)
|
||||
Node(
|
||||
package="rosapi",
|
||||
executable="rosapi_node",
|
||||
name="rosapi",
|
||||
output="screen",
|
||||
),
|
||||
])
|
||||
```
|
||||
|
||||
**Create package.xml**:
|
||||
|
||||
```xml
|
||||
<?xml version="1.0"?>
|
||||
<?xml-model href="http://download.ros.org/schema/package_format3.xsd" schematypeid="pf3"?>
|
||||
<package format="3">
|
||||
<name>nao_launch</name>
|
||||
<version>1.0.0</version>
|
||||
<description>Launch files for NAO6 HRIStudio integration</description>
|
||||
<maintainer email="your@email.com">Your Name</maintainer>
|
||||
<license>MIT</license>
|
||||
|
||||
<buildtool_depend>ament_cmake</buildtool_depend>
|
||||
<exec_depend>launch</exec_depend>
|
||||
<exec_depend>launch_ros</exec_depend>
|
||||
<exec_depend>naoqi_driver</exec_depend>
|
||||
<exec_depend>rosbridge_server</exec_depend>
|
||||
<exec_depend>rosapi</exec_depend>
|
||||
</package>
|
||||
```
|
||||
|
||||
**Create CMakeLists.txt**:
|
||||
|
||||
```cmake
|
||||
cmake_minimum_required(VERSION 3.8)
|
||||
project(nao_launch)
|
||||
|
||||
if(CMAKE_COMPILER_IS_GNUCXX OR CMAKE_CXX_COMPILER_ID MATCHES "Clang")
|
||||
add_compile_options(-Wall -Wextra -Wpedantic)
|
||||
endif()
|
||||
|
||||
find_package(ament_cmake REQUIRED)
|
||||
|
||||
install(DIRECTORY launch/
|
||||
DESTINATION share/${PROJECT_NAME}/launch/
|
||||
)
|
||||
|
||||
if(BUILD_TESTING)
|
||||
find_package(ament_lint_auto REQUIRED)
|
||||
ament_lint_auto_find_test_dependencies()
|
||||
endif()
|
||||
|
||||
ament_package()
|
||||
```
|
||||
|
||||
### 1.5 Build the Workspace
|
||||
|
||||
```bash
|
||||
cd ~/naoqi_ros2_ws
|
||||
I_AGREE_TO_NAO_MESHES_LICENSE=1 I_AGREE_TO_PEPPER_MESHES_LICENSE=1 colcon build --symlink-install
|
||||
source install/setup.bash
|
||||
```
|
||||
|
||||
## Part 2: NAO Network Configuration and Connection
|
||||
|
||||
### 2.1 Verify NAO Network Connectivity
|
||||
|
||||
```bash
|
||||
# Test basic connectivity
|
||||
ping -c 4 nao.local
|
||||
|
||||
# Test NAOqi service port
|
||||
timeout 5 bash -c 'echo "test" | nc nao.local 9559' && echo "NAOqi port is open!" || echo "NAOqi port might be closed"
|
||||
|
||||
# Alternative test
|
||||
telnet nao.local 9559
|
||||
# Press Ctrl+C to exit if connection succeeds
|
||||
```
|
||||
|
||||
### 2.2 Find NAO Credentials
|
||||
|
||||
The default NAO credentials are typically:
|
||||
- Username: `nao`
|
||||
- Password: Usually `nao`, but can be custom
|
||||
|
||||
**Common passwords to try:**
|
||||
- `nao` (default)
|
||||
- Institution name (e.g., `bucknell`)
|
||||
- Custom password set by administrator
|
||||
|
||||
## Part 3: HRIStudio Database Integration
|
||||
|
||||
### 3.1 Update Database Schema
|
||||
|
||||
The HRIStudio database needs to include NAO6 robot definitions and plugins.
|
||||
|
||||
**Update robots in seed script** (`scripts/seed-dev.ts`):
|
||||
|
||||
```typescript
|
||||
const robots = [
|
||||
{
|
||||
name: "TurtleBot3 Burger",
|
||||
manufacturer: "ROBOTIS",
|
||||
model: "TurtleBot3 Burger",
|
||||
description: "A compact, affordable, programmable, ROS2-based mobile robot for education and research",
|
||||
capabilities: ["differential_drive", "lidar", "imu", "odometry"],
|
||||
communicationProtocol: "ros2" as const,
|
||||
},
|
||||
{
|
||||
name: "NAO Humanoid Robot",
|
||||
manufacturer: "SoftBank Robotics",
|
||||
model: "NAO V6",
|
||||
description: "Humanoid robot designed for education, research, and social interaction with ROS2 integration",
|
||||
capabilities: [
|
||||
"speech",
|
||||
"vision",
|
||||
"walking",
|
||||
"gestures",
|
||||
"joint_control",
|
||||
"touch_sensors",
|
||||
"sonar_sensors",
|
||||
"camera_feed",
|
||||
"imu",
|
||||
"odometry",
|
||||
],
|
||||
communicationProtocol: "ros2" as const,
|
||||
},
|
||||
];
|
||||
```
|
||||
|
||||
### 3.2 Create NAO6 Plugin Repository
|
||||
|
||||
Create local plugin repository at `public/nao6-plugins/`:
|
||||
|
||||
**Repository metadata** (`public/nao6-plugins/repository.json`):
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "NAO6 ROS2 Integration Repository",
|
||||
"description": "Official NAO6 robot plugins for ROS2-based Human-Robot Interaction experiments",
|
||||
"version": "1.0.0",
|
||||
"author": {
|
||||
"name": "HRIStudio Team",
|
||||
"email": "support@hristudio.com"
|
||||
},
|
||||
"trust": "official",
|
||||
"license": "MIT",
|
||||
"robots": [
|
||||
{
|
||||
"name": "NAO6",
|
||||
"manufacturer": "SoftBank Robotics",
|
||||
"model": "NAO V6",
|
||||
"communicationProtocol": "ros2"
|
||||
}
|
||||
],
|
||||
"ros2": {
|
||||
"distro": "humble",
|
||||
"packages": ["naoqi_driver2", "naoqi_bridge_msgs", "rosbridge_suite"],
|
||||
"bridge": {
|
||||
"protocol": "websocket",
|
||||
"defaultPort": 9090
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 3.3 Seed Database
|
||||
|
||||
```bash
|
||||
# Start database
|
||||
sudo docker compose up -d
|
||||
|
||||
# Push schema changes
|
||||
bun db:push
|
||||
|
||||
# Seed with NAO6 data
|
||||
bun db:seed
|
||||
```
|
||||
|
||||
## Part 4: Web Interface Integration
|
||||
|
||||
### 4.1 Create NAO Test Page
|
||||
|
||||
Create `src/app/(dashboard)/nao-test/page.tsx` with the robot control interface.
|
||||
|
||||
**Key points:**
|
||||
- Use `~` import alias (not `@`)
|
||||
- Connect to WebSocket at `ws://YOUR_IP:9090`
|
||||
- Use correct ROS topic names (without `/naoqi_driver` prefix for control topics)
|
||||
|
||||
**Important Topic Mapping:**
|
||||
- Speech: `/speech` (not `/naoqi_driver/speech`)
|
||||
- Movement: `/cmd_vel` (not `/naoqi_driver/cmd_vel`)
|
||||
- Joint control: `/joint_angles` (not `/naoqi_driver/joint_angles`)
|
||||
- Sensor data: `/naoqi_driver/joint_states`, `/naoqi_driver/bumper`, etc.
|
||||
|
||||
## Part 5: Critical Troubleshooting
|
||||
|
||||
### 5.1 Robot Not Responding to Commands
|
||||
|
||||
**Symptom:** ROS topics receive commands but robot doesn't move.
|
||||
|
||||
**Root Cause:** NAO robots start in "safe mode" with loose joints and need to be "awakened."
|
||||
|
||||
**Solution - SSH Wake-Up Method:**
|
||||
|
||||
```bash
|
||||
# Install sshpass for automated SSH
|
||||
sudo apt install sshpass -y
|
||||
|
||||
# Wake up robot via SSH
|
||||
sshpass -p "YOUR_NAO_PASSWORD" ssh nao@nao.local "python2 -c \"
|
||||
import sys
|
||||
sys.path.append('/opt/aldebaran/lib/python2.7/site-packages')
|
||||
import naoqi
|
||||
|
||||
try:
|
||||
motion = naoqi.ALProxy('ALMotion', '127.0.0.1', 9559)
|
||||
print 'Connected to ALMotion'
|
||||
print 'Current stiffness:', motion.getStiffnesses('Body')[0] if motion.getStiffnesses('Body') else 'No stiffness data'
|
||||
|
||||
print 'Waking up robot...'
|
||||
motion.wakeUp()
|
||||
|
||||
print 'Robot should now be awake!'
|
||||
|
||||
except Exception as e:
|
||||
print 'Error:', str(e)
|
||||
\""
|
||||
```
|
||||
|
||||
**Alternative Physical Method:**
|
||||
1. Press and hold the chest button for 3 seconds
|
||||
2. Wait for the robot to stiffen and stand up
|
||||
3. Robot should now respond to movement commands
|
||||
|
||||
### 5.2 Connection Issues
|
||||
|
||||
**Port Already in Use:**
|
||||
```bash
|
||||
# Kill existing processes
|
||||
sudo fuser -k 9090/tcp
|
||||
pkill -f "rosbridge\|naoqi\|ros2"
|
||||
```
|
||||
|
||||
**Database Connection Issues:**
|
||||
```bash
|
||||
# Check Docker containers
|
||||
sudo docker ps
|
||||
|
||||
# Restart database
|
||||
sudo docker compose down
|
||||
sudo docker compose up -d
|
||||
```
|
||||
|
||||
### 5.3 Import Alias Issues
|
||||
|
||||
**Error:** Module import failures in React components.
|
||||
|
||||
**Solution:** Use `~` import alias consistently:
|
||||
```typescript
|
||||
import { Button } from "~/components/ui/button";
|
||||
// NOT: import { Button } from "@/components/ui/button";
|
||||
```
|
||||
|
||||
## Part 6: Verification and Testing
|
||||
|
||||
### 6.1 System Verification Script
|
||||
|
||||
Create verification script to test all components:
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
echo "=== NAO6 HRIStudio Integration Verification ==="
|
||||
|
||||
# Test 1: ROS2 Setup
|
||||
echo "✓ ROS2 Humble: $ROS_DISTRO"
|
||||
|
||||
# Test 2: NAO Connectivity
|
||||
ping -c 1 nao.local && echo "✓ NAO reachable" || echo "✗ NAO not reachable"
|
||||
|
||||
# Test 3: Workspace Build
|
||||
[ -f ~/naoqi_ros2_ws/install/setup.bash ] && echo "✓ Workspace built" || echo "✗ Workspace not built"
|
||||
|
||||
# Test 4: Database Running
|
||||
sudo docker ps | grep -q postgres && echo "✓ Database running" || echo "✗ Database not running"
|
||||
|
||||
echo "=== Verification Complete ==="
|
||||
```
|
||||
|
||||
### 6.2 End-to-End Test Procedure
|
||||
|
||||
**Terminal 1: Start ROS Integration**
|
||||
```bash
|
||||
cd ~/naoqi_ros2_ws
|
||||
source install/setup.bash
|
||||
ros2 launch install/nao_launch/share/nao_launch/launch/nao6_hristudio.launch.py nao_ip:=nao.local password:=YOUR_PASSWORD
|
||||
```
|
||||
|
||||
**Terminal 2: Wake Up Robot**
|
||||
```bash
|
||||
# Use SSH method from Section 5.1
|
||||
sshpass -p "YOUR_PASSWORD" ssh nao@nao.local "python2 -c \"...\""
|
||||
```
|
||||
|
||||
**Terminal 3: Start HRIStudio**
|
||||
```bash
|
||||
cd /path/to/hristudio
|
||||
bun dev
|
||||
```
|
||||
|
||||
**Web Interface Test:**
|
||||
1. Go to `http://localhost:3000/nao-test`
|
||||
2. Click "Connect" - should show "Connected"
|
||||
3. Test speech: Enter text and click "Say Text"
|
||||
4. Test movement: Use arrow buttons to make robot walk
|
||||
5. Test head control: Move sliders to control head position
|
||||
6. Monitor sensor data in tabs
|
||||
|
||||
### 6.3 Command-Line Testing
|
||||
|
||||
**Test Speech:**
|
||||
```bash
|
||||
ros2 topic pub --once /speech std_msgs/String "data: 'Hello from ROS2'"
|
||||
```
|
||||
|
||||
**Test Movement:**
|
||||
```bash
|
||||
ros2 topic pub --times 3 /cmd_vel geometry_msgs/msg/Twist '{linear: {x: 0.05, y: 0.0, z: 0.0}, angular: {x: 0.0, y: 0.0, z: 0.0}}'
|
||||
```
|
||||
|
||||
**Test Head Movement:**
|
||||
```bash
|
||||
ros2 topic pub --once /joint_angles naoqi_bridge_msgs/msg/JointAnglesWithSpeed '{joint_names: ["HeadYaw"], joint_angles: [0.5], speed: 0.3}'
|
||||
```
|
||||
|
||||
## Part 7: Production Deployment
|
||||
|
||||
### 7.1 Launch Script Creation
|
||||
|
||||
Create production-ready launch script (`scripts/launch_nao6.sh`):
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
# NAO6 HRIStudio Integration Launch Script
|
||||
|
||||
set -e
|
||||
|
||||
# Configuration
|
||||
NAO_IP="${NAO_IP:-nao.local}"
|
||||
NAO_PASSWORD="${NAO_PASSWORD:-nao}"
|
||||
BRIDGE_PORT="${BRIDGE_PORT:-9090}"
|
||||
|
||||
# Function to wake up robot
|
||||
wake_up_robot() {
|
||||
echo "Waking up NAO robot..."
|
||||
sshpass -p "$NAO_PASSWORD" ssh nao@$NAO_IP "python2 -c \"
|
||||
import sys
|
||||
sys.path.append('/opt/aldebaran/lib/python2.7/site-packages')
|
||||
import naoqi
|
||||
motion = naoqi.ALProxy('ALMotion', '127.0.0.1', 9559)
|
||||
motion.wakeUp()
|
||||
print 'Robot awakened'
|
||||
\""
|
||||
}
|
||||
|
||||
# Main execution
|
||||
echo "Starting NAO6 HRIStudio Integration"
|
||||
echo "NAO IP: $NAO_IP"
|
||||
echo "Bridge Port: $BRIDGE_PORT"
|
||||
|
||||
# Check connections
|
||||
ping -c 1 $NAO_IP || { echo "Cannot reach NAO"; exit 1; }
|
||||
|
||||
# Start ROS integration
|
||||
cd ~/naoqi_ros2_ws
|
||||
source install/setup.bash
|
||||
|
||||
# Wake up robot in background
|
||||
wake_up_robot &
|
||||
|
||||
# Launch ROS system
|
||||
exec ros2 launch install/nao_launch/share/nao_launch/launch/nao6_hristudio.launch.py \
|
||||
nao_ip:="$NAO_IP" \
|
||||
password:="$NAO_PASSWORD" \
|
||||
bridge_port:="$BRIDGE_PORT"
|
||||
```
|
||||
|
||||
### 7.2 Service Integration (Optional)
|
||||
|
||||
Create systemd service for automatic startup:
|
||||
|
||||
```ini
|
||||
[Unit]
|
||||
Description=NAO6 HRIStudio Integration
|
||||
After=network.target
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
User=your_user
|
||||
Environment=NAO_IP=nao.local
|
||||
Environment=NAO_PASSWORD=your_password
|
||||
ExecStart=/path/to/launch_nao6.sh
|
||||
Restart=always
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
```
|
||||
|
||||
## Part 8: Safety and Best Practices
|
||||
|
||||
### 8.1 Safety Guidelines
|
||||
|
||||
- **Always keep emergency stop accessible** in the web interface
|
||||
- **Start with small movements and low speeds** when testing
|
||||
- **Monitor robot battery level** during long sessions
|
||||
- **Ensure clear space around robot** before movement commands
|
||||
- **Never leave robot unattended** during operation
|
||||
|
||||
### 8.2 Performance Optimization
|
||||
|
||||
**Network Optimization:**
|
||||
```bash
|
||||
# Increase network buffer sizes for camera data
|
||||
sudo sysctl -w net.core.rmem_max=26214400
|
||||
sudo sysctl -w net.core.rmem_default=26214400
|
||||
```
|
||||
|
||||
**ROS2 Optimization:**
|
||||
```bash
|
||||
# Use optimized RMW implementation
|
||||
export RMW_IMPLEMENTATION=rmw_cyclonedx_cpp
|
||||
```
|
||||
|
||||
### 8.3 Troubleshooting Checklist
|
||||
|
||||
**Before Starting:**
|
||||
- [ ] NAO robot powered on and connected to network
|
||||
- [ ] ROS2 Humble installed and sourced
|
||||
- [ ] NAO workspace built successfully
|
||||
- [ ] Database running (Docker container)
|
||||
- [ ] Correct NAO password known
|
||||
|
||||
**During Operation:**
|
||||
- [ ] rosbridge WebSocket server running on port 9090
|
||||
- [ ] NAO robot in standing position (not crouching)
|
||||
- [ ] Robot joints stiffened (not loose)
|
||||
- [ ] HRIStudio web interface connected to ROS bridge
|
||||
|
||||
**If Commands Not Working:**
|
||||
1. Check robot is awake and standing
|
||||
2. Verify topic names in web interface match ROS topics
|
||||
3. Test commands from command line first
|
||||
4. Check rosbridge logs for errors
|
||||
|
||||
## Part 9: Future Enhancements
|
||||
|
||||
### 9.1 Advanced Features
|
||||
|
||||
- **Multi-camera streaming** for experiment recording
|
||||
- **Advanced gesture recognition** through touch sensors
|
||||
- **Autonomous behavior integration** with navigation
|
||||
- **Multi-robot coordination** for group interaction studies
|
||||
|
||||
### 9.2 Plugin Development
|
||||
|
||||
The NAO6 integration supports the HRIStudio plugin system for adding custom behaviors and extending robot capabilities.
|
||||
|
||||
## Conclusion
|
||||
|
||||
This guide provides a complete integration of NAO6 robots with HRIStudio, enabling researchers to conduct sophisticated Human-Robot Interaction experiments with full robot control, real-time data collection, and web-based interfaces.
|
||||
|
||||
The key insight discovered during implementation is that NAO robots require explicit "wake-up" commands to enable motor control, which must be performed before any movement commands will be executed.
|
||||
|
||||
**Support Resources:**
|
||||
- NAO Documentation: https://developer.softbankrobotics.com/nao6
|
||||
- naoqi_driver2: https://github.com/ros-naoqi/naoqi_driver2
|
||||
- ROS2 Humble: https://docs.ros.org/en/humble/
|
||||
- HRIStudio Documentation: See `docs/` folder
|
||||
|
||||
---
|
||||
|
||||
**Integration Status: Production Ready ✅**
|
||||
|
||||
*Last Updated: January 2025*
|
||||
*Tested With: NAO V6.0 / NAOqi 2.8.7.4 / ROS2 Humble / HRIStudio v1.0*
|
||||
218
docs/nao6-quick-reference.md
Normal file
218
docs/nao6-quick-reference.md
Normal file
@@ -0,0 +1,218 @@
|
||||
# NAO6 HRIStudio Quick Reference
|
||||
|
||||
**Essential commands for NAO6 robot integration with HRIStudio**
|
||||
|
||||
## 🚀 Quick Start (5 Steps)
|
||||
|
||||
### 1. Start ROS Integration
|
||||
```bash
|
||||
cd ~/naoqi_ros2_ws
|
||||
source install/setup.bash
|
||||
ros2 launch install/nao_launch/share/nao_launch/launch/nao6_hristudio.launch.py nao_ip:=nao.local password:=robolab
|
||||
```
|
||||
|
||||
### 2. Wake Up Robot (CRITICAL!)
|
||||
```bash
|
||||
sshpass -p "robolab" ssh nao@nao.local "python2 -c \"
|
||||
import sys
|
||||
sys.path.append('/opt/aldebaran/lib/python2.7/site-packages')
|
||||
import naoqi
|
||||
motion = naoqi.ALProxy('ALMotion', '127.0.0.1', 9559)
|
||||
motion.wakeUp()
|
||||
print 'Robot awakened'
|
||||
\""
|
||||
```
|
||||
|
||||
### 3. Start HRIStudio
|
||||
```bash
|
||||
cd /home/robolab/Documents/Projects/hristudio
|
||||
bun dev
|
||||
```
|
||||
|
||||
### 4. Access Test Interface
|
||||
- URL: `http://localhost:3000/nao-test`
|
||||
- Login: `sean@soconnor.dev` / `password123`
|
||||
|
||||
### 5. Test Robot
|
||||
- Click "Connect" to WebSocket
|
||||
- Try speech: "Hello from HRIStudio!"
|
||||
- Use movement buttons to control robot
|
||||
|
||||
## 🛠️ Essential Commands
|
||||
|
||||
### Connection Testing
|
||||
```bash
|
||||
# Test NAO connectivity
|
||||
ping nao.local
|
||||
|
||||
# Test NAOqi service
|
||||
telnet nao.local 9559
|
||||
|
||||
# Check ROS topics
|
||||
ros2 topic list | grep naoqi
|
||||
```
|
||||
|
||||
### Manual Robot Control
|
||||
```bash
|
||||
# Speech test
|
||||
ros2 topic pub --once /speech std_msgs/String "data: 'Hello world'"
|
||||
|
||||
# Movement test (robot must be awake!)
|
||||
ros2 topic pub --times 3 /cmd_vel geometry_msgs/msg/Twist '{linear: {x: 0.05, y: 0.0, z: 0.0}, angular: {x: 0.0, y: 0.0, z: 0.0}}'
|
||||
|
||||
# Head movement test
|
||||
ros2 topic pub --once /joint_angles naoqi_bridge_msgs/msg/JointAnglesWithSpeed '{joint_names: ["HeadYaw"], joint_angles: [0.5], speed: 0.3}'
|
||||
|
||||
# Stop all movement
|
||||
ros2 topic pub --once /cmd_vel geometry_msgs/msg/Twist '{linear: {x: 0.0, y: 0.0, z: 0.0}, angular: {x: 0.0, y: 0.0, z: 0.0}}'
|
||||
```
|
||||
|
||||
### Status Checks
|
||||
```bash
|
||||
# Check robot info
|
||||
ros2 service call /naoqi_driver/get_robot_config naoqi_bridge_msgs/srv/GetRobotInfo
|
||||
|
||||
# Monitor joint states
|
||||
ros2 topic echo /naoqi_driver/joint_states --once
|
||||
|
||||
# Check ROS nodes
|
||||
ros2 node list
|
||||
|
||||
# Check WebSocket connection
|
||||
ss -an | grep 9090
|
||||
```
|
||||
|
||||
## 🔧 Troubleshooting
|
||||
|
||||
### Robot Not Moving
|
||||
**Problem:** Commands sent but robot doesn't move
|
||||
**Solution:** Robot needs to be awakened first
|
||||
```bash
|
||||
# Wake up via SSH (see step 2 above)
|
||||
# OR press chest button for 3 seconds
|
||||
```
|
||||
|
||||
### Connection Issues
|
||||
```bash
|
||||
# Kill existing processes
|
||||
sudo fuser -k 9090/tcp
|
||||
pkill -f "rosbridge\|naoqi\|ros2"
|
||||
|
||||
# Restart database
|
||||
sudo docker compose down && sudo docker compose up -d
|
||||
```
|
||||
|
||||
### Import Errors in Web Interface
|
||||
**Problem:** React component import failures
|
||||
**Solution:** Use `~` import alias consistently:
|
||||
```typescript
|
||||
import { Button } from "~/components/ui/button";
|
||||
// NOT: import { Button } from "@/components/ui/button";
|
||||
```
|
||||
|
||||
## 📊 Key Topics
|
||||
|
||||
### Input Topics (Robot Control)
|
||||
- `/speech` - Text-to-speech
|
||||
- `/cmd_vel` - Movement commands
|
||||
- `/joint_angles` - Joint position control
|
||||
|
||||
### Output Topics (Sensor Data)
|
||||
- `/naoqi_driver/joint_states` - Joint positions/velocities
|
||||
- `/naoqi_driver/bumper` - Foot sensors
|
||||
- `/naoqi_driver/hand_touch` - Hand touch sensors
|
||||
- `/naoqi_driver/head_touch` - Head touch sensors
|
||||
- `/naoqi_driver/sonar/left` - Left ultrasonic sensor
|
||||
- `/naoqi_driver/sonar/right` - Right ultrasonic sensor
|
||||
- `/naoqi_driver/camera/front/image_raw` - Front camera
|
||||
- `/naoqi_driver/camera/bottom/image_raw` - Bottom camera
|
||||
|
||||
## 🔗 WebSocket Integration
|
||||
|
||||
**ROS Bridge URL:** `ws://134.82.159.25:9090`
|
||||
|
||||
**Message Format:**
|
||||
```javascript
|
||||
// Publish command
|
||||
{
|
||||
"op": "publish",
|
||||
"topic": "/speech",
|
||||
"type": "std_msgs/String",
|
||||
"msg": {"data": "Hello world"}
|
||||
}
|
||||
|
||||
// Subscribe to topic
|
||||
{
|
||||
"op": "subscribe",
|
||||
"topic": "/naoqi_driver/joint_states",
|
||||
"type": "sensor_msgs/JointState"
|
||||
}
|
||||
```
|
||||
|
||||
## 🎯 Common Use Cases
|
||||
|
||||
### Make Robot Speak
|
||||
```bash
|
||||
ros2 topic pub --once /speech std_msgs/String "data: 'Welcome to the experiment'"
|
||||
```
|
||||
|
||||
### Walk Forward 3 Steps
|
||||
```bash
|
||||
ros2 topic pub --times 3 /cmd_vel geometry_msgs/msg/Twist '{linear: {x: 0.1, y: 0.0, z: 0.0}, angular: {x: 0.0, y: 0.0, z: 0.0}}'
|
||||
```
|
||||
|
||||
### Turn Head Left
|
||||
```bash
|
||||
ros2 topic pub --once /joint_angles naoqi_bridge_msgs/msg/JointAnglesWithSpeed '{joint_names: ["HeadYaw"], joint_angles: [0.8], speed: 0.2}'
|
||||
```
|
||||
|
||||
### Emergency Stop
|
||||
```bash
|
||||
ros2 topic pub --once /cmd_vel geometry_msgs/msg/Twist '{linear: {x: 0.0, y: 0.0, z: 0.0}, angular: {x: 0.0, y: 0.0, z: 0.0}}'
|
||||
```
|
||||
|
||||
## 🚨 Safety Notes
|
||||
|
||||
- **Always wake up robot before movement commands**
|
||||
- **Keep emergency stop accessible**
|
||||
- **Start with small movements (0.05 m/s)**
|
||||
- **Monitor battery level during experiments**
|
||||
- **Ensure clear space around robot**
|
||||
|
||||
## 📝 Credentials
|
||||
|
||||
**Default NAO Login:**
|
||||
- Username: `nao`
|
||||
- Password: `robolab` (institution-specific)
|
||||
|
||||
**HRIStudio Login:**
|
||||
- Email: `sean@soconnor.dev`
|
||||
- Password: `password123`
|
||||
|
||||
## 🔄 Complete Restart Procedure
|
||||
|
||||
```bash
|
||||
# 1. Kill all processes
|
||||
sudo fuser -k 9090/tcp
|
||||
pkill -f "rosbridge\|naoqi\|ros2"
|
||||
|
||||
# 2. Restart database
|
||||
sudo docker compose down && sudo docker compose up -d
|
||||
|
||||
# 3. Start ROS integration
|
||||
cd ~/naoqi_ros2_ws && source install/setup.bash
|
||||
ros2 launch install/nao_launch/share/nao_launch/launch/nao6_hristudio.launch.py nao_ip:=nao.local password:=robolab
|
||||
|
||||
# 4. Wake up robot (in another terminal)
|
||||
sshpass -p "robolab" ssh nao@nao.local "python2 -c \"import sys; sys.path.append('/opt/aldebaran/lib/python2.7/site-packages'); import naoqi; naoqi.ALProxy('ALMotion', '127.0.0.1', 9559).wakeUp()\""
|
||||
|
||||
# 5. Start HRIStudio (in another terminal)
|
||||
cd /home/robolab/Documents/Projects/hristudio && bun dev
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
**📖 For detailed setup instructions, see:** [NAO6 Complete Integration Guide](./nao6-integration-complete-guide.md)
|
||||
|
||||
**✅ Integration Status:** Production Ready
|
||||
**🤖 Tested With:** NAO V6.0 / NAOqi 2.8.7.4 / ROS2 Humble
|
||||
@@ -2,6 +2,10 @@
|
||||
|
||||
This guide walks you through setting up your NAO6 robot with ROS2 integration for use with HRIStudio's experiment platform.
|
||||
|
||||
> **📋 For Complete Integration Guide:** See [NAO6 Complete Integration Guide](./nao6-integration-complete-guide.md) for comprehensive setup, troubleshooting, and production deployment instructions.
|
||||
|
||||
**⚠️ Critical Note:** NAO robots must be "awakened" (motors stiffened and standing) before movement commands will work. See the troubleshooting section below.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- NAO6 robot with NAOqi OS 2.8.7+
|
||||
|
||||
7
public/nao6-plugins/plugins/index.json
Normal file
7
public/nao6-plugins/plugins/index.json
Normal file
@@ -0,0 +1,7 @@
|
||||
[
|
||||
"nao6-movement.json",
|
||||
"nao6-speech.json",
|
||||
"nao6-sensors.json",
|
||||
"nao6-vision.json",
|
||||
"nao6-interaction.json"
|
||||
]
|
||||
342
public/nao6-plugins/plugins/nao6-movement.json
Normal file
342
public/nao6-plugins/plugins/nao6-movement.json
Normal file
@@ -0,0 +1,342 @@
|
||||
{
|
||||
"name": "NAO6 Movement Control",
|
||||
"version": "1.0.0",
|
||||
"description": "Complete movement control for NAO6 robot including walking, turning, and joint manipulation",
|
||||
"platform": "NAO6",
|
||||
"category": "movement",
|
||||
"manufacturer": {
|
||||
"name": "SoftBank Robotics",
|
||||
"website": "https://www.softbankrobotics.com"
|
||||
},
|
||||
"documentation": {
|
||||
"mainUrl": "https://docs.hristudio.com/robots/nao6/movement",
|
||||
"quickStart": "https://docs.hristudio.com/robots/nao6/movement/quickstart"
|
||||
},
|
||||
"ros2Config": {
|
||||
"namespace": "/naoqi_driver",
|
||||
"topics": {
|
||||
"cmd_vel": {
|
||||
"type": "geometry_msgs/Twist",
|
||||
"description": "Velocity commands for robot base movement"
|
||||
},
|
||||
"joint_angles": {
|
||||
"type": "naoqi_bridge_msgs/JointAnglesWithSpeed",
|
||||
"description": "Individual joint angle control with speed"
|
||||
},
|
||||
"joint_states": {
|
||||
"type": "sensor_msgs/JointState",
|
||||
"description": "Current joint positions and velocities"
|
||||
}
|
||||
}
|
||||
},
|
||||
"actions": [
|
||||
{
|
||||
"id": "walk_forward",
|
||||
"name": "Walk Forward",
|
||||
"description": "Make the robot walk forward at specified speed",
|
||||
"category": "movement",
|
||||
"parameters": [
|
||||
{
|
||||
"name": "speed",
|
||||
"type": "number",
|
||||
"description": "Walking speed in m/s",
|
||||
"required": true,
|
||||
"min": 0.01,
|
||||
"max": 0.3,
|
||||
"default": 0.1,
|
||||
"step": 0.01
|
||||
},
|
||||
{
|
||||
"name": "duration",
|
||||
"type": "number",
|
||||
"description": "Duration to walk in seconds (0 = indefinite)",
|
||||
"required": false,
|
||||
"min": 0,
|
||||
"max": 30,
|
||||
"default": 0,
|
||||
"step": 0.1
|
||||
}
|
||||
],
|
||||
"implementation": {
|
||||
"topic": "/naoqi_driver/cmd_vel",
|
||||
"messageType": "geometry_msgs/Twist",
|
||||
"messageTemplate": {
|
||||
"linear": { "x": "{{speed}}", "y": 0, "z": 0 },
|
||||
"angular": { "x": 0, "y": 0, "z": 0 }
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "walk_backward",
|
||||
"name": "Walk Backward",
|
||||
"description": "Make the robot walk backward at specified speed",
|
||||
"category": "movement",
|
||||
"parameters": [
|
||||
{
|
||||
"name": "speed",
|
||||
"type": "number",
|
||||
"description": "Walking speed in m/s",
|
||||
"required": true,
|
||||
"min": 0.01,
|
||||
"max": 0.3,
|
||||
"default": 0.1,
|
||||
"step": 0.01
|
||||
},
|
||||
{
|
||||
"name": "duration",
|
||||
"type": "number",
|
||||
"description": "Duration to walk in seconds (0 = indefinite)",
|
||||
"required": false,
|
||||
"min": 0,
|
||||
"max": 30,
|
||||
"default": 0,
|
||||
"step": 0.1
|
||||
}
|
||||
],
|
||||
"implementation": {
|
||||
"topic": "/naoqi_driver/cmd_vel",
|
||||
"messageType": "geometry_msgs/Twist",
|
||||
"messageTemplate": {
|
||||
"linear": { "x": "-{{speed}}", "y": 0, "z": 0 },
|
||||
"angular": { "x": 0, "y": 0, "z": 0 }
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "turn_left",
|
||||
"name": "Turn Left",
|
||||
"description": "Make the robot turn left at specified angular speed",
|
||||
"category": "movement",
|
||||
"parameters": [
|
||||
{
|
||||
"name": "speed",
|
||||
"type": "number",
|
||||
"description": "Angular speed in rad/s",
|
||||
"required": true,
|
||||
"min": 0.1,
|
||||
"max": 1.0,
|
||||
"default": 0.3,
|
||||
"step": 0.1
|
||||
},
|
||||
{
|
||||
"name": "duration",
|
||||
"type": "number",
|
||||
"description": "Duration to turn in seconds (0 = indefinite)",
|
||||
"required": false,
|
||||
"min": 0,
|
||||
"max": 30,
|
||||
"default": 0,
|
||||
"step": 0.1
|
||||
}
|
||||
],
|
||||
"implementation": {
|
||||
"topic": "/naoqi_driver/cmd_vel",
|
||||
"messageType": "geometry_msgs/Twist",
|
||||
"messageTemplate": {
|
||||
"linear": { "x": 0, "y": 0, "z": 0 },
|
||||
"angular": { "x": 0, "y": 0, "z": "{{speed}}" }
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "turn_right",
|
||||
"name": "Turn Right",
|
||||
"description": "Make the robot turn right at specified angular speed",
|
||||
"category": "movement",
|
||||
"parameters": [
|
||||
{
|
||||
"name": "speed",
|
||||
"type": "number",
|
||||
"description": "Angular speed in rad/s",
|
||||
"required": true,
|
||||
"min": 0.1,
|
||||
"max": 1.0,
|
||||
"default": 0.3,
|
||||
"step": 0.1
|
||||
},
|
||||
{
|
||||
"name": "duration",
|
||||
"type": "number",
|
||||
"description": "Duration to turn in seconds (0 = indefinite)",
|
||||
"required": false,
|
||||
"min": 0,
|
||||
"max": 30,
|
||||
"default": 0,
|
||||
"step": 0.1
|
||||
}
|
||||
],
|
||||
"implementation": {
|
||||
"topic": "/naoqi_driver/cmd_vel",
|
||||
"messageType": "geometry_msgs/Twist",
|
||||
"messageTemplate": {
|
||||
"linear": { "x": 0, "y": 0, "z": 0 },
|
||||
"angular": { "x": 0, "y": 0, "z": "-{{speed}}" }
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "stop_movement",
|
||||
"name": "Stop Movement",
|
||||
"description": "Immediately stop all robot movement",
|
||||
"category": "movement",
|
||||
"parameters": [],
|
||||
"implementation": {
|
||||
"topic": "/naoqi_driver/cmd_vel",
|
||||
"messageType": "geometry_msgs/Twist",
|
||||
"messageTemplate": {
|
||||
"linear": { "x": 0, "y": 0, "z": 0 },
|
||||
"angular": { "x": 0, "y": 0, "z": 0 }
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "move_head",
|
||||
"name": "Move Head",
|
||||
"description": "Control head orientation (yaw and pitch)",
|
||||
"category": "movement",
|
||||
"parameters": [
|
||||
{
|
||||
"name": "yaw",
|
||||
"type": "number",
|
||||
"description": "Head yaw angle in radians",
|
||||
"required": true,
|
||||
"min": -2.09,
|
||||
"max": 2.09,
|
||||
"default": 0,
|
||||
"step": 0.1
|
||||
},
|
||||
{
|
||||
"name": "pitch",
|
||||
"type": "number",
|
||||
"description": "Head pitch angle in radians",
|
||||
"required": true,
|
||||
"min": -0.67,
|
||||
"max": 0.51,
|
||||
"default": 0,
|
||||
"step": 0.1
|
||||
},
|
||||
{
|
||||
"name": "speed",
|
||||
"type": "number",
|
||||
"description": "Movement speed (0.1 = slow, 1.0 = fast)",
|
||||
"required": false,
|
||||
"min": 0.1,
|
||||
"max": 1.0,
|
||||
"default": 0.3,
|
||||
"step": 0.1
|
||||
}
|
||||
],
|
||||
"implementation": {
|
||||
"topic": "/naoqi_driver/joint_angles",
|
||||
"messageType": "naoqi_bridge_msgs/JointAnglesWithSpeed",
|
||||
"messageTemplate": {
|
||||
"joint_names": ["HeadYaw", "HeadPitch"],
|
||||
"joint_angles": ["{{yaw}}", "{{pitch}}"],
|
||||
"speed": "{{speed}}"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "move_arm",
|
||||
"name": "Move Arm",
|
||||
"description": "Control arm joint positions",
|
||||
"category": "movement",
|
||||
"parameters": [
|
||||
{
|
||||
"name": "arm",
|
||||
"type": "select",
|
||||
"description": "Which arm to control",
|
||||
"required": true,
|
||||
"options": [
|
||||
{ "value": "left", "label": "Left Arm" },
|
||||
{ "value": "right", "label": "Right Arm" }
|
||||
],
|
||||
"default": "right"
|
||||
},
|
||||
{
|
||||
"name": "shoulder_pitch",
|
||||
"type": "number",
|
||||
"description": "Shoulder pitch angle in radians",
|
||||
"required": true,
|
||||
"min": -2.09,
|
||||
"max": 2.09,
|
||||
"default": 1.4,
|
||||
"step": 0.1
|
||||
},
|
||||
{
|
||||
"name": "shoulder_roll",
|
||||
"type": "number",
|
||||
"description": "Shoulder roll angle in radians",
|
||||
"required": true,
|
||||
"min": -0.31,
|
||||
"max": 1.33,
|
||||
"default": 0.2,
|
||||
"step": 0.1
|
||||
},
|
||||
{
|
||||
"name": "elbow_yaw",
|
||||
"type": "number",
|
||||
"description": "Elbow yaw angle in radians",
|
||||
"required": true,
|
||||
"min": -2.09,
|
||||
"max": 2.09,
|
||||
"default": 0,
|
||||
"step": 0.1
|
||||
},
|
||||
{
|
||||
"name": "elbow_roll",
|
||||
"type": "number",
|
||||
"description": "Elbow roll angle in radians",
|
||||
"required": true,
|
||||
"min": -1.54,
|
||||
"max": -0.03,
|
||||
"default": -0.5,
|
||||
"step": 0.1
|
||||
},
|
||||
{
|
||||
"name": "speed",
|
||||
"type": "number",
|
||||
"description": "Movement speed (0.1 = slow, 1.0 = fast)",
|
||||
"required": false,
|
||||
"min": 0.1,
|
||||
"max": 1.0,
|
||||
"default": 0.3,
|
||||
"step": 0.1
|
||||
}
|
||||
],
|
||||
"implementation": {
|
||||
"topic": "/naoqi_driver/joint_angles",
|
||||
"messageType": "naoqi_bridge_msgs/JointAnglesWithSpeed",
|
||||
"messageTemplate": {
|
||||
"joint_names": [
|
||||
"{{arm === 'left' ? 'L' : 'R'}}ShoulderPitch",
|
||||
"{{arm === 'left' ? 'L' : 'R'}}ShoulderRoll",
|
||||
"{{arm === 'left' ? 'L' : 'R'}}ElbowYaw",
|
||||
"{{arm === 'left' ? 'L' : 'R'}}ElbowRoll"
|
||||
],
|
||||
"joint_angles": ["{{shoulder_pitch}}", "{{shoulder_roll}}", "{{elbow_yaw}}", "{{elbow_roll}}"],
|
||||
"speed": "{{speed}}"
|
||||
}
|
||||
}
|
||||
}
|
||||
],
|
||||
"safety": {
|
||||
"maxSpeed": 0.3,
|
||||
"emergencyStop": {
|
||||
"action": "stop_movement",
|
||||
"description": "Immediately stops all movement"
|
||||
},
|
||||
"jointLimits": {
|
||||
"HeadYaw": { "min": -2.09, "max": 2.09 },
|
||||
"HeadPitch": { "min": -0.67, "max": 0.51 },
|
||||
"LShoulderPitch": { "min": -2.09, "max": 2.09 },
|
||||
"RShoulderPitch": { "min": -2.09, "max": 2.09 },
|
||||
"LShoulderRoll": { "min": -0.31, "max": 1.33 },
|
||||
"RShoulderRoll": { "min": -1.33, "max": 0.31 },
|
||||
"LElbowYaw": { "min": -2.09, "max": 2.09 },
|
||||
"RElbowYaw": { "min": -2.09, "max": 2.09 },
|
||||
"LElbowRoll": { "min": 0.03, "max": 1.54 },
|
||||
"RElbowRoll": { "min": -1.54, "max": -0.03 }
|
||||
}
|
||||
}
|
||||
}
|
||||
464
public/nao6-plugins/plugins/nao6-sensors.json
Normal file
464
public/nao6-plugins/plugins/nao6-sensors.json
Normal file
@@ -0,0 +1,464 @@
|
||||
{
|
||||
"name": "NAO6 Sensors & Feedback",
|
||||
"version": "1.0.0",
|
||||
"description": "Complete sensor suite for NAO6 robot including touch sensors, sonar, IMU, cameras, and joint state monitoring",
|
||||
"platform": "NAO6",
|
||||
"category": "sensors",
|
||||
"manufacturer": {
|
||||
"name": "SoftBank Robotics",
|
||||
"website": "https://www.softbankrobotics.com"
|
||||
},
|
||||
"documentation": {
|
||||
"mainUrl": "https://docs.hristudio.com/robots/nao6/sensors",
|
||||
"quickStart": "https://docs.hristudio.com/robots/nao6/sensors/quickstart"
|
||||
},
|
||||
"ros2Config": {
|
||||
"namespace": "/naoqi_driver",
|
||||
"topics": {
|
||||
"joint_states": {
|
||||
"type": "sensor_msgs/JointState",
|
||||
"description": "Current positions, velocities, and efforts of all joints"
|
||||
},
|
||||
"imu": {
|
||||
"type": "sensor_msgs/Imu",
|
||||
"description": "Inertial measurement unit data (acceleration, angular velocity, orientation)"
|
||||
},
|
||||
"bumper": {
|
||||
"type": "naoqi_bridge_msgs/Bumper",
|
||||
"description": "Foot bumper sensor states"
|
||||
},
|
||||
"hand_touch": {
|
||||
"type": "naoqi_bridge_msgs/HandTouch",
|
||||
"description": "Hand tactile sensor states"
|
||||
},
|
||||
"head_touch": {
|
||||
"type": "naoqi_bridge_msgs/HeadTouch",
|
||||
"description": "Head tactile sensor states"
|
||||
},
|
||||
"sonar/left": {
|
||||
"type": "sensor_msgs/Range",
|
||||
"description": "Left ultrasonic range sensor"
|
||||
},
|
||||
"sonar/right": {
|
||||
"type": "sensor_msgs/Range",
|
||||
"description": "Right ultrasonic range sensor"
|
||||
},
|
||||
"camera/front/image_raw": {
|
||||
"type": "sensor_msgs/Image",
|
||||
"description": "Front camera image feed"
|
||||
},
|
||||
"camera/bottom/image_raw": {
|
||||
"type": "sensor_msgs/Image",
|
||||
"description": "Bottom camera image feed"
|
||||
},
|
||||
"battery": {
|
||||
"type": "sensor_msgs/BatteryState",
|
||||
"description": "Battery level and charging status"
|
||||
}
|
||||
}
|
||||
},
|
||||
"actions": [
|
||||
{
|
||||
"id": "get_joint_states",
|
||||
"name": "Get Joint States",
|
||||
"description": "Read current positions and velocities of all robot joints",
|
||||
"category": "sensors",
|
||||
"parameters": [
|
||||
{
|
||||
"name": "specific_joints",
|
||||
"type": "multiselect",
|
||||
"description": "Specific joints to monitor (empty = all joints)",
|
||||
"required": false,
|
||||
"options": [
|
||||
{ "value": "HeadYaw", "label": "Head Yaw" },
|
||||
{ "value": "HeadPitch", "label": "Head Pitch" },
|
||||
{ "value": "LShoulderPitch", "label": "Left Shoulder Pitch" },
|
||||
{ "value": "LShoulderRoll", "label": "Left Shoulder Roll" },
|
||||
{ "value": "LElbowYaw", "label": "Left Elbow Yaw" },
|
||||
{ "value": "LElbowRoll", "label": "Left Elbow Roll" },
|
||||
{ "value": "RShoulderPitch", "label": "Right Shoulder Pitch" },
|
||||
{ "value": "RShoulderRoll", "label": "Right Shoulder Roll" },
|
||||
{ "value": "RElbowYaw", "label": "Right Elbow Yaw" },
|
||||
{ "value": "RElbowRoll", "label": "Right Elbow Roll" }
|
||||
]
|
||||
}
|
||||
],
|
||||
"implementation": {
|
||||
"topic": "/naoqi_driver/joint_states",
|
||||
"messageType": "sensor_msgs/JointState",
|
||||
"mode": "subscribe"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "get_touch_sensors",
|
||||
"name": "Get Touch Sensors",
|
||||
"description": "Monitor all tactile sensors on head and hands",
|
||||
"category": "sensors",
|
||||
"parameters": [
|
||||
{
|
||||
"name": "sensor_type",
|
||||
"type": "select",
|
||||
"description": "Type of touch sensors to monitor",
|
||||
"required": false,
|
||||
"options": [
|
||||
{ "value": "all", "label": "All Touch Sensors" },
|
||||
{ "value": "head", "label": "Head Touch Only" },
|
||||
{ "value": "hands", "label": "Hand Touch Only" }
|
||||
],
|
||||
"default": "all"
|
||||
}
|
||||
],
|
||||
"implementation": {
|
||||
"topics": [
|
||||
"/naoqi_driver/head_touch",
|
||||
"/naoqi_driver/hand_touch"
|
||||
],
|
||||
"messageTypes": [
|
||||
"naoqi_bridge_msgs/HeadTouch",
|
||||
"naoqi_bridge_msgs/HandTouch"
|
||||
],
|
||||
"mode": "subscribe"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "get_sonar_distance",
|
||||
"name": "Get Sonar Distance",
|
||||
"description": "Read ultrasonic distance sensors for obstacle detection",
|
||||
"category": "sensors",
|
||||
"parameters": [
|
||||
{
|
||||
"name": "sensor_side",
|
||||
"type": "select",
|
||||
"description": "Which sonar sensor to read",
|
||||
"required": false,
|
||||
"options": [
|
||||
{ "value": "both", "label": "Both Sensors" },
|
||||
{ "value": "left", "label": "Left Sensor Only" },
|
||||
{ "value": "right", "label": "Right Sensor Only" }
|
||||
],
|
||||
"default": "both"
|
||||
},
|
||||
{
|
||||
"name": "min_range",
|
||||
"type": "number",
|
||||
"description": "Minimum detection range in meters",
|
||||
"required": false,
|
||||
"min": 0.1,
|
||||
"max": 1.0,
|
||||
"default": 0.25,
|
||||
"step": 0.05
|
||||
},
|
||||
{
|
||||
"name": "max_range",
|
||||
"type": "number",
|
||||
"description": "Maximum detection range in meters",
|
||||
"required": false,
|
||||
"min": 1.0,
|
||||
"max": 3.0,
|
||||
"default": 2.55,
|
||||
"step": 0.05
|
||||
}
|
||||
],
|
||||
"implementation": {
|
||||
"topics": [
|
||||
"/naoqi_driver/sonar/left",
|
||||
"/naoqi_driver/sonar/right"
|
||||
],
|
||||
"messageType": "sensor_msgs/Range",
|
||||
"mode": "subscribe"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "get_imu_data",
|
||||
"name": "Get IMU Data",
|
||||
"description": "Read inertial measurement unit data (acceleration, gyroscope, orientation)",
|
||||
"category": "sensors",
|
||||
"parameters": [
|
||||
{
|
||||
"name": "data_type",
|
||||
"type": "select",
|
||||
"description": "Type of IMU data to monitor",
|
||||
"required": false,
|
||||
"options": [
|
||||
{ "value": "all", "label": "All IMU Data" },
|
||||
{ "value": "orientation", "label": "Orientation Only" },
|
||||
{ "value": "acceleration", "label": "Linear Acceleration" },
|
||||
{ "value": "angular_velocity", "label": "Angular Velocity" }
|
||||
],
|
||||
"default": "all"
|
||||
}
|
||||
],
|
||||
"implementation": {
|
||||
"topic": "/naoqi_driver/imu",
|
||||
"messageType": "sensor_msgs/Imu",
|
||||
"mode": "subscribe"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "get_camera_image",
|
||||
"name": "Get Camera Image",
|
||||
"description": "Capture image from robot's cameras",
|
||||
"category": "sensors",
|
||||
"parameters": [
|
||||
{
|
||||
"name": "camera",
|
||||
"type": "select",
|
||||
"description": "Which camera to use",
|
||||
"required": true,
|
||||
"options": [
|
||||
{ "value": "front", "label": "Front Camera" },
|
||||
{ "value": "bottom", "label": "Bottom Camera" }
|
||||
],
|
||||
"default": "front"
|
||||
},
|
||||
{
|
||||
"name": "resolution",
|
||||
"type": "select",
|
||||
"description": "Image resolution",
|
||||
"required": false,
|
||||
"options": [
|
||||
{ "value": "160x120", "label": "QQVGA (160x120)" },
|
||||
{ "value": "320x240", "label": "QVGA (320x240)" },
|
||||
{ "value": "640x480", "label": "VGA (640x480)" }
|
||||
],
|
||||
"default": "320x240"
|
||||
},
|
||||
{
|
||||
"name": "fps",
|
||||
"type": "number",
|
||||
"description": "Frames per second",
|
||||
"required": false,
|
||||
"min": 1,
|
||||
"max": 30,
|
||||
"default": 15,
|
||||
"step": 1
|
||||
}
|
||||
],
|
||||
"implementation": {
|
||||
"topic": "/naoqi_driver/camera/{{camera}}/image_raw",
|
||||
"messageType": "sensor_msgs/Image",
|
||||
"mode": "subscribe"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "get_battery_status",
|
||||
"name": "Get Battery Status",
|
||||
"description": "Monitor robot battery level and charging status",
|
||||
"category": "sensors",
|
||||
"parameters": [],
|
||||
"implementation": {
|
||||
"topic": "/naoqi_driver/battery",
|
||||
"messageType": "sensor_msgs/BatteryState",
|
||||
"mode": "subscribe"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "detect_obstacle",
|
||||
"name": "Detect Obstacle",
|
||||
"description": "Check for obstacles using sonar sensors with customizable thresholds",
|
||||
"category": "sensors",
|
||||
"parameters": [
|
||||
{
|
||||
"name": "detection_distance",
|
||||
"type": "number",
|
||||
"description": "Distance threshold for obstacle detection (meters)",
|
||||
"required": true,
|
||||
"min": 0.1,
|
||||
"max": 2.0,
|
||||
"default": 0.5,
|
||||
"step": 0.1
|
||||
},
|
||||
{
|
||||
"name": "sensor_side",
|
||||
"type": "select",
|
||||
"description": "Which sensors to use for detection",
|
||||
"required": false,
|
||||
"options": [
|
||||
{ "value": "both", "label": "Both Sensors" },
|
||||
{ "value": "left", "label": "Left Sensor Only" },
|
||||
{ "value": "right", "label": "Right Sensor Only" }
|
||||
],
|
||||
"default": "both"
|
||||
}
|
||||
],
|
||||
"implementation": {
|
||||
"topics": [
|
||||
"/naoqi_driver/sonar/left",
|
||||
"/naoqi_driver/sonar/right"
|
||||
],
|
||||
"messageType": "sensor_msgs/Range",
|
||||
"mode": "subscribe",
|
||||
"processing": "obstacle_detection"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "monitor_fall_detection",
|
||||
"name": "Monitor Fall Detection",
|
||||
"description": "Monitor robot stability using IMU data to detect potential falls",
|
||||
"category": "sensors",
|
||||
"parameters": [
|
||||
{
|
||||
"name": "tilt_threshold",
|
||||
"type": "number",
|
||||
"description": "Maximum tilt angle before fall alert (degrees)",
|
||||
"required": false,
|
||||
"min": 10,
|
||||
"max": 45,
|
||||
"default": 25,
|
||||
"step": 5
|
||||
},
|
||||
{
|
||||
"name": "acceleration_threshold",
|
||||
"type": "number",
|
||||
"description": "Acceleration threshold for impact detection (m/s²)",
|
||||
"required": false,
|
||||
"min": 5,
|
||||
"max": 20,
|
||||
"default": 10,
|
||||
"step": 1
|
||||
}
|
||||
],
|
||||
"implementation": {
|
||||
"topic": "/naoqi_driver/imu",
|
||||
"messageType": "sensor_msgs/Imu",
|
||||
"mode": "subscribe",
|
||||
"processing": "fall_detection"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "wait_for_touch",
|
||||
"name": "Wait for Touch",
|
||||
"description": "Wait for user to touch a specific sensor before continuing",
|
||||
"category": "sensors",
|
||||
"parameters": [
|
||||
{
|
||||
"name": "sensor_location",
|
||||
"type": "select",
|
||||
"description": "Which sensor to wait for",
|
||||
"required": true,
|
||||
"options": [
|
||||
{ "value": "head_front", "label": "Head Front" },
|
||||
{ "value": "head_middle", "label": "Head Middle" },
|
||||
{ "value": "head_rear", "label": "Head Rear" },
|
||||
{ "value": "left_hand", "label": "Left Hand" },
|
||||
{ "value": "right_hand", "label": "Right Hand" },
|
||||
{ "value": "any_head", "label": "Any Head Sensor" },
|
||||
{ "value": "any_hand", "label": "Any Hand Sensor" },
|
||||
{ "value": "any_touch", "label": "Any Touch Sensor" }
|
||||
],
|
||||
"default": "head_front"
|
||||
},
|
||||
{
|
||||
"name": "timeout",
|
||||
"type": "number",
|
||||
"description": "Maximum time to wait for touch (seconds, 0 = infinite)",
|
||||
"required": false,
|
||||
"min": 0,
|
||||
"max": 300,
|
||||
"default": 30,
|
||||
"step": 5
|
||||
}
|
||||
],
|
||||
"implementation": {
|
||||
"topics": [
|
||||
"/naoqi_driver/head_touch",
|
||||
"/naoqi_driver/hand_touch"
|
||||
],
|
||||
"messageTypes": [
|
||||
"naoqi_bridge_msgs/HeadTouch",
|
||||
"naoqi_bridge_msgs/HandTouch"
|
||||
],
|
||||
"mode": "wait_for_condition",
|
||||
"condition": "touch_detected"
|
||||
}
|
||||
}
|
||||
],
|
||||
"sensorSpecifications": {
|
||||
"touchSensors": {
|
||||
"head": {
|
||||
"locations": ["front", "middle", "rear"],
|
||||
"sensitivity": "capacitive",
|
||||
"responseTime": "< 50ms"
|
||||
},
|
||||
"hands": {
|
||||
"locations": ["left", "right"],
|
||||
"sensitivity": "capacitive",
|
||||
"responseTime": "< 50ms"
|
||||
}
|
||||
},
|
||||
"sonarSensors": {
|
||||
"count": 2,
|
||||
"locations": ["left", "right"],
|
||||
"minRange": "0.25m",
|
||||
"maxRange": "2.55m",
|
||||
"fieldOfView": "60°",
|
||||
"frequency": "40kHz"
|
||||
},
|
||||
"cameras": {
|
||||
"front": {
|
||||
"resolution": "640x480",
|
||||
"maxFps": 30,
|
||||
"fieldOfView": "60.9° x 47.6°"
|
||||
},
|
||||
"bottom": {
|
||||
"resolution": "640x480",
|
||||
"maxFps": 30,
|
||||
"fieldOfView": "60.9° x 47.6°"
|
||||
}
|
||||
},
|
||||
"imu": {
|
||||
"accelerometer": {
|
||||
"range": "±2g",
|
||||
"sensitivity": "high"
|
||||
},
|
||||
"gyroscope": {
|
||||
"range": "±500°/s",
|
||||
"sensitivity": "high"
|
||||
},
|
||||
"magnetometer": {
|
||||
"available": false
|
||||
}
|
||||
},
|
||||
"joints": {
|
||||
"count": 25,
|
||||
"encoderResolution": "12-bit",
|
||||
"positionAccuracy": "±0.1°"
|
||||
}
|
||||
},
|
||||
"dataTypes": {
|
||||
"jointState": {
|
||||
"position": "radians",
|
||||
"velocity": "radians/second",
|
||||
"effort": "arbitrary units"
|
||||
},
|
||||
"imu": {
|
||||
"orientation": "quaternion",
|
||||
"angularVelocity": "radians/second",
|
||||
"linearAcceleration": "m/s²"
|
||||
},
|
||||
"range": {
|
||||
"distance": "meters",
|
||||
"minRange": "meters",
|
||||
"maxRange": "meters"
|
||||
},
|
||||
"image": {
|
||||
"encoding": "rgb8",
|
||||
"width": "pixels",
|
||||
"height": "pixels"
|
||||
}
|
||||
},
|
||||
"safety": {
|
||||
"fallDetection": {
|
||||
"enabled": true,
|
||||
"defaultThreshold": "25°"
|
||||
},
|
||||
"obstacleDetection": {
|
||||
"enabled": true,
|
||||
"safeDistance": "0.3m"
|
||||
},
|
||||
"batteryMonitoring": {
|
||||
"lowBatteryWarning": "20%",
|
||||
"criticalBatteryShutdown": "5%"
|
||||
}
|
||||
}
|
||||
}
|
||||
338
public/nao6-plugins/plugins/nao6-speech.json
Normal file
338
public/nao6-plugins/plugins/nao6-speech.json
Normal file
@@ -0,0 +1,338 @@
|
||||
{
|
||||
"name": "NAO6 Speech & Audio",
|
||||
"version": "1.0.0",
|
||||
"description": "Text-to-speech and audio capabilities for NAO6 robot including voice synthesis, volume control, and language settings",
|
||||
"platform": "NAO6",
|
||||
"category": "speech",
|
||||
"manufacturer": {
|
||||
"name": "SoftBank Robotics",
|
||||
"website": "https://www.softbankrobotics.com"
|
||||
},
|
||||
"documentation": {
|
||||
"mainUrl": "https://docs.hristudio.com/robots/nao6/speech",
|
||||
"quickStart": "https://docs.hristudio.com/robots/nao6/speech/quickstart"
|
||||
},
|
||||
"ros2Config": {
|
||||
"namespace": "/naoqi_driver",
|
||||
"topics": {
|
||||
"speech": {
|
||||
"type": "std_msgs/String",
|
||||
"description": "Text-to-speech commands"
|
||||
},
|
||||
"set_language": {
|
||||
"type": "std_msgs/String",
|
||||
"description": "Set speech language"
|
||||
},
|
||||
"audio_volume": {
|
||||
"type": "std_msgs/Float32",
|
||||
"description": "Control audio volume level"
|
||||
}
|
||||
}
|
||||
},
|
||||
"actions": [
|
||||
{
|
||||
"id": "say_text",
|
||||
"name": "Say Text",
|
||||
"description": "Make the robot speak the specified text using text-to-speech",
|
||||
"category": "speech",
|
||||
"parameters": [
|
||||
{
|
||||
"name": "text",
|
||||
"type": "text",
|
||||
"description": "Text for the robot to speak",
|
||||
"required": true,
|
||||
"maxLength": 500,
|
||||
"placeholder": "Enter text for NAO to say..."
|
||||
},
|
||||
{
|
||||
"name": "wait_for_completion",
|
||||
"type": "boolean",
|
||||
"description": "Wait for speech to finish before continuing",
|
||||
"required": false,
|
||||
"default": true
|
||||
}
|
||||
],
|
||||
"implementation": {
|
||||
"topic": "/naoqi_driver/speech",
|
||||
"messageType": "std_msgs/String",
|
||||
"messageTemplate": {
|
||||
"data": "{{text}}"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "say_with_emotion",
|
||||
"name": "Say Text with Emotion",
|
||||
"description": "Speak text with emotional expression using SSML-like markup",
|
||||
"category": "speech",
|
||||
"parameters": [
|
||||
{
|
||||
"name": "text",
|
||||
"type": "text",
|
||||
"description": "Text for the robot to speak",
|
||||
"required": true,
|
||||
"maxLength": 500,
|
||||
"placeholder": "Enter text for NAO to say..."
|
||||
},
|
||||
{
|
||||
"name": "emotion",
|
||||
"type": "select",
|
||||
"description": "Emotional tone for speech",
|
||||
"required": false,
|
||||
"options": [
|
||||
{ "value": "neutral", "label": "Neutral" },
|
||||
{ "value": "happy", "label": "Happy" },
|
||||
{ "value": "sad", "label": "Sad" },
|
||||
{ "value": "excited", "label": "Excited" },
|
||||
{ "value": "calm", "label": "Calm" }
|
||||
],
|
||||
"default": "neutral"
|
||||
},
|
||||
{
|
||||
"name": "speed",
|
||||
"type": "number",
|
||||
"description": "Speech speed multiplier",
|
||||
"required": false,
|
||||
"min": 0.5,
|
||||
"max": 2.0,
|
||||
"default": 1.0,
|
||||
"step": 0.1
|
||||
}
|
||||
],
|
||||
"implementation": {
|
||||
"topic": "/naoqi_driver/speech",
|
||||
"messageType": "std_msgs/String",
|
||||
"messageTemplate": {
|
||||
"data": "\\rspd={{speed}}\\\\rst={{emotion}}\\{{text}}"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "set_volume",
|
||||
"name": "Set Volume",
|
||||
"description": "Adjust the robot's audio volume level",
|
||||
"category": "speech",
|
||||
"parameters": [
|
||||
{
|
||||
"name": "volume",
|
||||
"type": "number",
|
||||
"description": "Volume level (0.0 = silent, 1.0 = maximum)",
|
||||
"required": true,
|
||||
"min": 0.0,
|
||||
"max": 1.0,
|
||||
"default": 0.5,
|
||||
"step": 0.1
|
||||
}
|
||||
],
|
||||
"implementation": {
|
||||
"topic": "/naoqi_driver/audio_volume",
|
||||
"messageType": "std_msgs/Float32",
|
||||
"messageTemplate": {
|
||||
"data": "{{volume}}"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "set_language",
|
||||
"name": "Set Language",
|
||||
"description": "Change the robot's speech language",
|
||||
"category": "speech",
|
||||
"parameters": [
|
||||
{
|
||||
"name": "language",
|
||||
"type": "select",
|
||||
"description": "Speech language",
|
||||
"required": true,
|
||||
"options": [
|
||||
{ "value": "en-US", "label": "English (US)" },
|
||||
{ "value": "en-GB", "label": "English (UK)" },
|
||||
{ "value": "fr-FR", "label": "French" },
|
||||
{ "value": "de-DE", "label": "German" },
|
||||
{ "value": "es-ES", "label": "Spanish" },
|
||||
{ "value": "it-IT", "label": "Italian" },
|
||||
{ "value": "ja-JP", "label": "Japanese" },
|
||||
{ "value": "ko-KR", "label": "Korean" },
|
||||
{ "value": "zh-CN", "label": "Chinese (Simplified)" }
|
||||
],
|
||||
"default": "en-US"
|
||||
}
|
||||
],
|
||||
"implementation": {
|
||||
"topic": "/naoqi_driver/set_language",
|
||||
"messageType": "std_msgs/String",
|
||||
"messageTemplate": {
|
||||
"data": "{{language}}"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "say_random_phrase",
|
||||
"name": "Say Random Phrase",
|
||||
"description": "Make the robot say a random phrase from predefined categories",
|
||||
"category": "speech",
|
||||
"parameters": [
|
||||
{
|
||||
"name": "category",
|
||||
"type": "select",
|
||||
"description": "Category of phrases",
|
||||
"required": true,
|
||||
"options": [
|
||||
{ "value": "greeting", "label": "Greetings" },
|
||||
{ "value": "encouragement", "label": "Encouragement" },
|
||||
{ "value": "question", "label": "Questions" },
|
||||
{ "value": "farewell", "label": "Farewells" },
|
||||
{ "value": "instruction", "label": "Instructions" }
|
||||
],
|
||||
"default": "greeting"
|
||||
}
|
||||
],
|
||||
"implementation": {
|
||||
"topic": "/naoqi_driver/speech",
|
||||
"messageType": "std_msgs/String",
|
||||
"messageTemplate": {
|
||||
"data": "{{getRandomPhrase(category)}}"
|
||||
}
|
||||
},
|
||||
"phrases": {
|
||||
"greeting": [
|
||||
"Hello! Nice to meet you!",
|
||||
"Hi there! How are you today?",
|
||||
"Welcome! I'm excited to work with you.",
|
||||
"Good day! Ready to get started?",
|
||||
"Greetings! What shall we do today?"
|
||||
],
|
||||
"encouragement": [
|
||||
"Great job! Keep it up!",
|
||||
"You're doing wonderfully!",
|
||||
"Excellent work! I'm impressed.",
|
||||
"That's fantastic! Well done!",
|
||||
"Perfect! You've got this!"
|
||||
],
|
||||
"question": [
|
||||
"How can I help you today?",
|
||||
"What would you like to do next?",
|
||||
"Is there anything you'd like to know?",
|
||||
"Shall we try something different?",
|
||||
"What are you thinking about?"
|
||||
],
|
||||
"farewell": [
|
||||
"Goodbye! It was great working with you!",
|
||||
"See you later! Take care!",
|
||||
"Until next time! Have a wonderful day!",
|
||||
"Farewell! Thanks for spending time with me!",
|
||||
"Bye for now! Look forward to seeing you again!"
|
||||
],
|
||||
"instruction": [
|
||||
"Please follow my movements.",
|
||||
"Let's try this step by step.",
|
||||
"Watch carefully and then repeat.",
|
||||
"Take your time, there's no rush.",
|
||||
"Remember to stay focused."
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "spell_word",
|
||||
"name": "Spell Word",
|
||||
"description": "Have the robot spell out a word letter by letter",
|
||||
"category": "speech",
|
||||
"parameters": [
|
||||
{
|
||||
"name": "word",
|
||||
"type": "text",
|
||||
"description": "Word to spell out",
|
||||
"required": true,
|
||||
"maxLength": 50,
|
||||
"placeholder": "Enter word to spell..."
|
||||
},
|
||||
{
|
||||
"name": "pause_duration",
|
||||
"type": "number",
|
||||
"description": "Pause between letters in seconds",
|
||||
"required": false,
|
||||
"min": 0.1,
|
||||
"max": 2.0,
|
||||
"default": 0.5,
|
||||
"step": 0.1
|
||||
}
|
||||
],
|
||||
"implementation": {
|
||||
"topic": "/naoqi_driver/speech",
|
||||
"messageType": "std_msgs/String",
|
||||
"messageTemplate": {
|
||||
"data": "{{word.split('').join('\\pau={{pause_duration * 1000}}\\\\pau=0\\')}}"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "count_numbers",
|
||||
"name": "Count Numbers",
|
||||
"description": "Have the robot count from one number to another",
|
||||
"category": "speech",
|
||||
"parameters": [
|
||||
{
|
||||
"name": "start",
|
||||
"type": "number",
|
||||
"description": "Starting number",
|
||||
"required": true,
|
||||
"min": 0,
|
||||
"max": 100,
|
||||
"default": 1,
|
||||
"step": 1
|
||||
},
|
||||
{
|
||||
"name": "end",
|
||||
"type": "number",
|
||||
"description": "Ending number",
|
||||
"required": true,
|
||||
"min": 0,
|
||||
"max": 100,
|
||||
"default": 10,
|
||||
"step": 1
|
||||
},
|
||||
{
|
||||
"name": "pause_duration",
|
||||
"type": "number",
|
||||
"description": "Pause between numbers in seconds",
|
||||
"required": false,
|
||||
"min": 0.1,
|
||||
"max": 2.0,
|
||||
"default": 0.8,
|
||||
"step": 0.1
|
||||
}
|
||||
],
|
||||
"implementation": {
|
||||
"topic": "/naoqi_driver/speech",
|
||||
"messageType": "std_msgs/String",
|
||||
"messageTemplate": {
|
||||
"data": "{{Array.from({length: end - start + 1}, (_, i) => start + i).join('\\pau={{pause_duration * 1000}}\\\\pau=0\\')}}"
|
||||
}
|
||||
}
|
||||
}
|
||||
],
|
||||
"features": {
|
||||
"languages": [
|
||||
"en-US", "en-GB", "fr-FR", "de-DE", "es-ES",
|
||||
"it-IT", "ja-JP", "ko-KR", "zh-CN"
|
||||
],
|
||||
"emotions": [
|
||||
"neutral", "happy", "sad", "excited", "calm"
|
||||
],
|
||||
"voiceEffects": [
|
||||
"speed", "pitch", "volume", "emotion"
|
||||
],
|
||||
"ssmlSupport": true,
|
||||
"maxTextLength": 500
|
||||
},
|
||||
"safety": {
|
||||
"maxVolume": 1.0,
|
||||
"defaultVolume": 0.5,
|
||||
"profanityFilter": true,
|
||||
"maxSpeechDuration": 60,
|
||||
"emergencyQuiet": {
|
||||
"action": "set_volume",
|
||||
"parameters": { "volume": 0 },
|
||||
"description": "Immediately mute robot audio"
|
||||
}
|
||||
}
|
||||
}
|
||||
44
public/nao6-plugins/repository.json
Normal file
44
public/nao6-plugins/repository.json
Normal file
@@ -0,0 +1,44 @@
|
||||
{
|
||||
"name": "NAO6 ROS2 Integration Repository",
|
||||
"description": "Official NAO6 robot plugins for ROS2-based Human-Robot Interaction experiments",
|
||||
"version": "1.0.0",
|
||||
"author": {
|
||||
"name": "HRIStudio Team",
|
||||
"email": "support@hristudio.com"
|
||||
},
|
||||
"urls": {
|
||||
"git": "https://github.com/hristudio/nao6-ros2-plugins",
|
||||
"documentation": "https://docs.hristudio.com/robots/nao6",
|
||||
"issues": "https://github.com/hristudio/nao6-ros2-plugins/issues"
|
||||
},
|
||||
"trust": "official",
|
||||
"license": "MIT",
|
||||
"robots": [
|
||||
{
|
||||
"name": "NAO6",
|
||||
"manufacturer": "SoftBank Robotics",
|
||||
"model": "NAO V6",
|
||||
"communicationProtocol": "ros2"
|
||||
}
|
||||
],
|
||||
"categories": [
|
||||
"movement",
|
||||
"speech",
|
||||
"sensors",
|
||||
"interaction",
|
||||
"vision"
|
||||
],
|
||||
"ros2": {
|
||||
"distro": "humble",
|
||||
"packages": [
|
||||
"naoqi_driver2",
|
||||
"naoqi_bridge_msgs",
|
||||
"rosbridge_suite"
|
||||
],
|
||||
"bridge": {
|
||||
"protocol": "websocket",
|
||||
"defaultPort": 9090
|
||||
}
|
||||
},
|
||||
"lastUpdated": "2025-01-16T00:00:00Z"
|
||||
}
|
||||
@@ -216,9 +216,20 @@ async function main() {
|
||||
manufacturer: "SoftBank Robotics",
|
||||
model: "NAO V6",
|
||||
description:
|
||||
"Humanoid robot designed for education, research, and social interaction",
|
||||
capabilities: ["speech", "vision", "walking", "gestures"],
|
||||
communicationProtocol: "rest" as const,
|
||||
"Humanoid robot designed for education, research, and social interaction with ROS2 integration",
|
||||
capabilities: [
|
||||
"speech",
|
||||
"vision",
|
||||
"walking",
|
||||
"gestures",
|
||||
"joint_control",
|
||||
"touch_sensors",
|
||||
"sonar_sensors",
|
||||
"camera_feed",
|
||||
"imu",
|
||||
"odometry",
|
||||
],
|
||||
communicationProtocol: "ros2" as const,
|
||||
},
|
||||
];
|
||||
|
||||
@@ -295,6 +306,17 @@ async function main() {
|
||||
syncStatus: "pending" as const,
|
||||
createdBy: seanUser.id,
|
||||
},
|
||||
{
|
||||
name: "NAO6 ROS2 Integration Repository",
|
||||
url: "http://localhost:3000/nao6-plugins",
|
||||
description:
|
||||
"Official NAO6 robot plugins for ROS2-based Human-Robot Interaction experiments",
|
||||
trustLevel: "official" as const,
|
||||
isEnabled: true,
|
||||
isOfficial: true,
|
||||
syncStatus: "pending" as const,
|
||||
createdBy: seanUser.id,
|
||||
},
|
||||
];
|
||||
|
||||
const insertedRepos = await db
|
||||
|
||||
606
src/app/(dashboard)/nao-test/page.tsx
Normal file
606
src/app/(dashboard)/nao-test/page.tsx
Normal file
@@ -0,0 +1,606 @@
|
||||
"use client";
|
||||
|
||||
import { useState, useEffect, useRef } from "react";
|
||||
import { Button } from "~/components/ui/button";
|
||||
import {
|
||||
Card,
|
||||
CardContent,
|
||||
CardDescription,
|
||||
CardHeader,
|
||||
CardTitle,
|
||||
} from "~/components/ui/card";
|
||||
import { Input } from "~/components/ui/input";
|
||||
import { Label } from "~/components/ui/label";
|
||||
import { Textarea } from "~/components/ui/textarea";
|
||||
import { Badge } from "~/components/ui/badge";
|
||||
import { Separator } from "~/components/ui/separator";
|
||||
import { Slider } from "~/components/ui/slider";
|
||||
import { Tabs, TabsContent, TabsList, TabsTrigger } from "~/components/ui/tabs";
|
||||
import { Alert, AlertDescription } from "~/components/ui/alert";
|
||||
import { PageHeader } from "~/components/ui/page-header";
|
||||
import { PageLayout } from "~/components/ui/page-layout";
|
||||
import {
|
||||
Play,
|
||||
Square,
|
||||
Volume2,
|
||||
Camera,
|
||||
Zap,
|
||||
ArrowUp,
|
||||
ArrowDown,
|
||||
ArrowLeft,
|
||||
ArrowRight,
|
||||
RotateCcw,
|
||||
RotateCw,
|
||||
Wifi,
|
||||
WifiOff,
|
||||
AlertTriangle,
|
||||
CheckCircle,
|
||||
Activity,
|
||||
Battery,
|
||||
Eye,
|
||||
Hand,
|
||||
Footprints,
|
||||
} from "lucide-react";
|
||||
|
||||
interface RosMessage {
|
||||
topic: string;
|
||||
msg: any;
|
||||
type: string;
|
||||
}
|
||||
|
||||
export default function NaoTestPage() {
|
||||
const [connectionStatus, setConnectionStatus] = useState<
|
||||
"disconnected" | "connecting" | "connected" | "error"
|
||||
>("disconnected");
|
||||
const [rosSocket, setRosSocket] = useState<WebSocket | null>(null);
|
||||
const [robotStatus, setRobotStatus] = useState<any>(null);
|
||||
const [jointStates, setJointStates] = useState<any>(null);
|
||||
const [speechText, setSpeechText] = useState("");
|
||||
const [walkSpeed, setWalkSpeed] = useState([0.1]);
|
||||
const [turnSpeed, setTurnSpeed] = useState([0.3]);
|
||||
const [headYaw, setHeadYaw] = useState([0]);
|
||||
const [headPitch, setHeadPitch] = useState([0]);
|
||||
const [logs, setLogs] = useState<string[]>([]);
|
||||
const [sensorData, setSensorData] = useState<any>({});
|
||||
const logsEndRef = useRef<HTMLDivElement>(null);
|
||||
|
||||
const ROS_BRIDGE_URL = "ws://134.82.159.25:9090";
|
||||
|
||||
const addLog = (message: string) => {
|
||||
const timestamp = new Date().toLocaleTimeString();
|
||||
setLogs((prev) => [...prev.slice(-49), `[${timestamp}] ${message}`]);
|
||||
};
|
||||
|
||||
useEffect(() => {
|
||||
logsEndRef.current?.scrollIntoView({ behavior: "smooth" });
|
||||
}, [logs]);
|
||||
|
||||
const connectToRos = () => {
|
||||
if (rosSocket?.readyState === WebSocket.OPEN) return;
|
||||
|
||||
setConnectionStatus("connecting");
|
||||
addLog("Connecting to ROS bridge...");
|
||||
|
||||
const socket = new WebSocket(ROS_BRIDGE_URL);
|
||||
|
||||
socket.onopen = () => {
|
||||
setConnectionStatus("connected");
|
||||
setRosSocket(socket);
|
||||
addLog("Connected to ROS bridge successfully");
|
||||
|
||||
// Subscribe to robot topics
|
||||
subscribeToTopics(socket);
|
||||
};
|
||||
|
||||
socket.onmessage = (event) => {
|
||||
try {
|
||||
const data = JSON.parse(event.data);
|
||||
handleRosMessage(data);
|
||||
} catch (error) {
|
||||
console.error("Error parsing ROS message:", error);
|
||||
}
|
||||
};
|
||||
|
||||
socket.onclose = () => {
|
||||
setConnectionStatus("disconnected");
|
||||
setRosSocket(null);
|
||||
addLog("Disconnected from ROS bridge");
|
||||
};
|
||||
|
||||
socket.onerror = () => {
|
||||
setConnectionStatus("error");
|
||||
addLog("Error connecting to ROS bridge");
|
||||
};
|
||||
};
|
||||
|
||||
const disconnectFromRos = () => {
|
||||
if (rosSocket) {
|
||||
rosSocket.close();
|
||||
setRosSocket(null);
|
||||
setConnectionStatus("disconnected");
|
||||
addLog("Manually disconnected from ROS bridge");
|
||||
}
|
||||
};
|
||||
|
||||
const subscribeToTopics = (socket: WebSocket) => {
|
||||
const topics = [
|
||||
{ topic: "/naoqi_driver/joint_states", type: "sensor_msgs/JointState" },
|
||||
{ topic: "/naoqi_driver/info", type: "naoqi_bridge_msgs/StringStamped" },
|
||||
{ topic: "/naoqi_driver/bumper", type: "naoqi_bridge_msgs/Bumper" },
|
||||
{
|
||||
topic: "/naoqi_driver/hand_touch",
|
||||
type: "naoqi_bridge_msgs/HandTouch",
|
||||
},
|
||||
{
|
||||
topic: "/naoqi_driver/head_touch",
|
||||
type: "naoqi_bridge_msgs/HeadTouch",
|
||||
},
|
||||
{ topic: "/naoqi_driver/sonar/left", type: "sensor_msgs/Range" },
|
||||
{ topic: "/naoqi_driver/sonar/right", type: "sensor_msgs/Range" },
|
||||
];
|
||||
|
||||
topics.forEach(({ topic, type }) => {
|
||||
const subscribeMsg = {
|
||||
op: "subscribe",
|
||||
topic,
|
||||
type,
|
||||
};
|
||||
socket.send(JSON.stringify(subscribeMsg));
|
||||
addLog(`Subscribed to ${topic}`);
|
||||
});
|
||||
};
|
||||
|
||||
const handleRosMessage = (data: any) => {
|
||||
if (data.topic === "/naoqi_driver/joint_states") {
|
||||
setJointStates(data.msg);
|
||||
} else if (data.topic === "/naoqi_driver/info") {
|
||||
setRobotStatus(data.msg);
|
||||
} else if (
|
||||
data.topic?.includes("bumper") ||
|
||||
data.topic?.includes("touch") ||
|
||||
data.topic?.includes("sonar")
|
||||
) {
|
||||
setSensorData((prev) => ({
|
||||
...prev,
|
||||
[data.topic]: data.msg,
|
||||
}));
|
||||
}
|
||||
};
|
||||
|
||||
const publishMessage = (topic: string, type: string, msg: any) => {
|
||||
if (!rosSocket || rosSocket.readyState !== WebSocket.OPEN) {
|
||||
addLog("Error: Not connected to ROS bridge");
|
||||
return;
|
||||
}
|
||||
|
||||
const rosMsg = {
|
||||
op: "publish",
|
||||
topic,
|
||||
type,
|
||||
msg,
|
||||
};
|
||||
|
||||
rosSocket.send(JSON.stringify(rosMsg));
|
||||
addLog(`Published to ${topic}: ${JSON.stringify(msg)}`);
|
||||
};
|
||||
|
||||
const sayText = () => {
|
||||
if (!speechText.trim()) return;
|
||||
|
||||
publishMessage("/speech", "std_msgs/String", {
|
||||
data: speechText,
|
||||
});
|
||||
setSpeechText("");
|
||||
};
|
||||
|
||||
const walkForward = () => {
|
||||
publishMessage("/cmd_vel", "geometry_msgs/Twist", {
|
||||
linear: { x: walkSpeed[0], y: 0, z: 0 },
|
||||
angular: { x: 0, y: 0, z: 0 },
|
||||
});
|
||||
};
|
||||
|
||||
const walkBackward = () => {
|
||||
publishMessage("/cmd_vel", "geometry_msgs/Twist", {
|
||||
linear: { x: -walkSpeed[0], y: 0, z: 0 },
|
||||
angular: { x: 0, y: 0, z: 0 },
|
||||
});
|
||||
};
|
||||
|
||||
const turnLeft = () => {
|
||||
publishMessage("/cmd_vel", "geometry_msgs/Twist", {
|
||||
linear: { x: 0, y: 0, z: 0 },
|
||||
angular: { x: 0, y: 0, z: turnSpeed[0] },
|
||||
});
|
||||
};
|
||||
|
||||
const turnRight = () => {
|
||||
publishMessage("/cmd_vel", "geometry_msgs/Twist", {
|
||||
linear: { x: 0, y: 0, z: 0 },
|
||||
angular: { x: 0, y: 0, z: -turnSpeed[0] },
|
||||
});
|
||||
};
|
||||
|
||||
const stopMovement = () => {
|
||||
publishMessage("/cmd_vel", "geometry_msgs/Twist", {
|
||||
linear: { x: 0, y: 0, z: 0 },
|
||||
angular: { x: 0, y: 0, z: 0 },
|
||||
});
|
||||
};
|
||||
|
||||
const moveHead = () => {
|
||||
publishMessage("/joint_angles", "naoqi_bridge_msgs/JointAnglesWithSpeed", {
|
||||
joint_names: ["HeadYaw", "HeadPitch"],
|
||||
joint_angles: [headYaw[0], headPitch[0]],
|
||||
speed: 0.3,
|
||||
});
|
||||
};
|
||||
|
||||
const getConnectionStatusIcon = () => {
|
||||
switch (connectionStatus) {
|
||||
case "connected":
|
||||
return <Wifi className="h-4 w-4 text-green-500" />;
|
||||
case "connecting":
|
||||
return <Activity className="h-4 w-4 animate-spin text-yellow-500" />;
|
||||
case "error":
|
||||
return <AlertTriangle className="h-4 w-4 text-red-500" />;
|
||||
default:
|
||||
return <WifiOff className="h-4 w-4 text-gray-500" />;
|
||||
}
|
||||
};
|
||||
|
||||
const getConnectionStatusBadge = () => {
|
||||
const variants = {
|
||||
connected: "default",
|
||||
connecting: "secondary",
|
||||
error: "destructive",
|
||||
disconnected: "outline",
|
||||
} as const;
|
||||
|
||||
return (
|
||||
<Badge
|
||||
variant={variants[connectionStatus]}
|
||||
className="flex items-center gap-1"
|
||||
>
|
||||
{getConnectionStatusIcon()}
|
||||
{connectionStatus.charAt(0).toUpperCase() + connectionStatus.slice(1)}
|
||||
</Badge>
|
||||
);
|
||||
};
|
||||
|
||||
return (
|
||||
<PageLayout>
|
||||
<PageHeader
|
||||
title="NAO Robot Test Console"
|
||||
description="Test and control your NAO6 robot through ROS bridge"
|
||||
/>
|
||||
|
||||
<div className="space-y-6">
|
||||
{/* Connection Status */}
|
||||
<Card>
|
||||
<CardHeader>
|
||||
<CardTitle className="flex items-center justify-between">
|
||||
ROS Bridge Connection
|
||||
{getConnectionStatusBadge()}
|
||||
</CardTitle>
|
||||
<CardDescription>
|
||||
Connect to ROS bridge at {ROS_BRIDGE_URL}
|
||||
</CardDescription>
|
||||
</CardHeader>
|
||||
<CardContent>
|
||||
<div className="flex gap-2">
|
||||
{connectionStatus === "connected" ? (
|
||||
<Button onClick={disconnectFromRos} variant="destructive">
|
||||
<WifiOff className="mr-2 h-4 w-4" />
|
||||
Disconnect
|
||||
</Button>
|
||||
) : (
|
||||
<Button
|
||||
onClick={connectToRos}
|
||||
disabled={connectionStatus === "connecting"}
|
||||
>
|
||||
<Wifi className="mr-2 h-4 w-4" />
|
||||
{connectionStatus === "connecting"
|
||||
? "Connecting..."
|
||||
: "Connect"}
|
||||
</Button>
|
||||
)}
|
||||
</div>
|
||||
</CardContent>
|
||||
</Card>
|
||||
|
||||
{connectionStatus === "connected" && (
|
||||
<Tabs defaultValue="control" className="space-y-4">
|
||||
<TabsList>
|
||||
<TabsTrigger value="control">Robot Control</TabsTrigger>
|
||||
<TabsTrigger value="sensors">Sensor Data</TabsTrigger>
|
||||
<TabsTrigger value="status">Robot Status</TabsTrigger>
|
||||
<TabsTrigger value="logs">Logs</TabsTrigger>
|
||||
</TabsList>
|
||||
|
||||
<TabsContent value="control" className="space-y-4">
|
||||
<div className="grid grid-cols-1 gap-4 md:grid-cols-2">
|
||||
{/* Speech Control */}
|
||||
<Card>
|
||||
<CardHeader>
|
||||
<CardTitle className="flex items-center gap-2">
|
||||
<Volume2 className="h-4 w-4" />
|
||||
Speech
|
||||
</CardTitle>
|
||||
</CardHeader>
|
||||
<CardContent className="space-y-4">
|
||||
<div className="space-y-2">
|
||||
<Label htmlFor="speech">Text to Speech</Label>
|
||||
<Textarea
|
||||
id="speech"
|
||||
placeholder="Enter text for NAO to say..."
|
||||
value={speechText}
|
||||
onChange={(e) => setSpeechText(e.target.value)}
|
||||
onKeyDown={(e) =>
|
||||
e.key === "Enter" &&
|
||||
!e.shiftKey &&
|
||||
(e.preventDefault(), sayText())
|
||||
}
|
||||
/>
|
||||
</div>
|
||||
<Button
|
||||
onClick={sayText}
|
||||
disabled={!speechText.trim()}
|
||||
className="w-full"
|
||||
>
|
||||
<Play className="mr-2 h-4 w-4" />
|
||||
Say Text
|
||||
</Button>
|
||||
</CardContent>
|
||||
</Card>
|
||||
|
||||
{/* Movement Control */}
|
||||
<Card>
|
||||
<CardHeader>
|
||||
<CardTitle className="flex items-center gap-2">
|
||||
<Footprints className="h-4 w-4" />
|
||||
Movement
|
||||
</CardTitle>
|
||||
</CardHeader>
|
||||
<CardContent className="space-y-4">
|
||||
<div className="space-y-2">
|
||||
<Label>Walk Speed: {walkSpeed[0].toFixed(2)} m/s</Label>
|
||||
<Slider
|
||||
value={walkSpeed}
|
||||
onValueChange={setWalkSpeed}
|
||||
max={0.5}
|
||||
min={0.05}
|
||||
step={0.05}
|
||||
/>
|
||||
</div>
|
||||
<div className="space-y-2">
|
||||
<Label>Turn Speed: {turnSpeed[0].toFixed(2)} rad/s</Label>
|
||||
<Slider
|
||||
value={turnSpeed}
|
||||
onValueChange={setTurnSpeed}
|
||||
max={1.0}
|
||||
min={0.1}
|
||||
step={0.1}
|
||||
/>
|
||||
</div>
|
||||
<div className="grid grid-cols-3 gap-2">
|
||||
<Button variant="outline" onClick={walkForward}>
|
||||
<ArrowUp className="h-4 w-4" />
|
||||
</Button>
|
||||
<Button variant="destructive" onClick={stopMovement}>
|
||||
<Square className="h-4 w-4" />
|
||||
</Button>
|
||||
<Button variant="outline" onClick={walkBackward}>
|
||||
<ArrowDown className="h-4 w-4" />
|
||||
</Button>
|
||||
<Button variant="outline" onClick={turnLeft}>
|
||||
<RotateCcw className="h-4 w-4" />
|
||||
</Button>
|
||||
<div></div>
|
||||
<Button variant="outline" onClick={turnRight}>
|
||||
<RotateCw className="h-4 w-4" />
|
||||
</Button>
|
||||
</div>
|
||||
</CardContent>
|
||||
</Card>
|
||||
|
||||
{/* Head Control */}
|
||||
<Card>
|
||||
<CardHeader>
|
||||
<CardTitle className="flex items-center gap-2">
|
||||
<Eye className="h-4 w-4" />
|
||||
Head Control
|
||||
</CardTitle>
|
||||
</CardHeader>
|
||||
<CardContent className="space-y-4">
|
||||
<div className="space-y-2">
|
||||
<Label>Head Yaw: {headYaw[0].toFixed(2)} rad</Label>
|
||||
<Slider
|
||||
value={headYaw}
|
||||
onValueChange={setHeadYaw}
|
||||
max={2.09}
|
||||
min={-2.09}
|
||||
step={0.1}
|
||||
/>
|
||||
</div>
|
||||
<div className="space-y-2">
|
||||
<Label>Head Pitch: {headPitch[0].toFixed(2)} rad</Label>
|
||||
<Slider
|
||||
value={headPitch}
|
||||
onValueChange={setHeadPitch}
|
||||
max={0.51}
|
||||
min={-0.67}
|
||||
step={0.1}
|
||||
/>
|
||||
</div>
|
||||
<Button onClick={moveHead} className="w-full">
|
||||
Move Head
|
||||
</Button>
|
||||
</CardContent>
|
||||
</Card>
|
||||
|
||||
{/* Emergency Stop */}
|
||||
<Card>
|
||||
<CardHeader>
|
||||
<CardTitle className="flex items-center gap-2 text-red-600">
|
||||
<AlertTriangle className="h-4 w-4" />
|
||||
Emergency
|
||||
</CardTitle>
|
||||
</CardHeader>
|
||||
<CardContent>
|
||||
<Button
|
||||
onClick={stopMovement}
|
||||
variant="destructive"
|
||||
size="lg"
|
||||
className="w-full"
|
||||
>
|
||||
<Square className="mr-2 h-4 w-4" />
|
||||
EMERGENCY STOP
|
||||
</Button>
|
||||
</CardContent>
|
||||
</Card>
|
||||
</div>
|
||||
</TabsContent>
|
||||
|
||||
<TabsContent value="sensors" className="space-y-4">
|
||||
<div className="grid grid-cols-1 gap-4 md:grid-cols-2 lg:grid-cols-3">
|
||||
{Object.entries(sensorData).map(([topic, data]) => (
|
||||
<Card key={topic}>
|
||||
<CardHeader>
|
||||
<CardTitle className="text-sm">
|
||||
{topic
|
||||
.split("/")
|
||||
.pop()
|
||||
?.replace(/_/g, " ")
|
||||
.toUpperCase()}
|
||||
</CardTitle>
|
||||
</CardHeader>
|
||||
<CardContent>
|
||||
<pre className="max-h-32 overflow-auto rounded bg-gray-100 p-2 text-xs">
|
||||
{JSON.stringify(data, null, 2)}
|
||||
</pre>
|
||||
</CardContent>
|
||||
</Card>
|
||||
))}
|
||||
{Object.keys(sensorData).length === 0 && (
|
||||
<Alert>
|
||||
<AlertTriangle className="h-4 w-4" />
|
||||
<AlertDescription>
|
||||
No sensor data received yet. Make sure the robot is
|
||||
connected and publishing data.
|
||||
</AlertDescription>
|
||||
</Alert>
|
||||
)}
|
||||
</div>
|
||||
</TabsContent>
|
||||
|
||||
<TabsContent value="status" className="space-y-4">
|
||||
<Card>
|
||||
<CardHeader>
|
||||
<CardTitle className="flex items-center gap-2">
|
||||
<Activity className="h-4 w-4" />
|
||||
Robot Status
|
||||
</CardTitle>
|
||||
</CardHeader>
|
||||
<CardContent>
|
||||
{robotStatus ? (
|
||||
<div className="space-y-4">
|
||||
<div className="grid grid-cols-2 gap-4">
|
||||
<div>
|
||||
<Label>Robot Info</Label>
|
||||
<pre className="mt-1 rounded bg-gray-100 p-2 text-xs">
|
||||
{JSON.stringify(robotStatus, null, 2)}
|
||||
</pre>
|
||||
</div>
|
||||
{jointStates && (
|
||||
<div>
|
||||
<Label>Joint States</Label>
|
||||
<div className="mt-1 max-h-64 overflow-auto rounded bg-gray-100 p-2 text-xs">
|
||||
<div>Joints: {jointStates.name?.length || 0}</div>
|
||||
<div>
|
||||
Last Update: {new Date().toLocaleTimeString()}
|
||||
</div>
|
||||
{jointStates.name
|
||||
?.slice(0, 10)
|
||||
.map((name: string, i: number) => (
|
||||
<div
|
||||
key={name}
|
||||
className="flex justify-between"
|
||||
>
|
||||
<span>{name}:</span>
|
||||
<span>
|
||||
{jointStates.position?.[i]?.toFixed(3) ||
|
||||
"N/A"}
|
||||
</span>
|
||||
</div>
|
||||
))}
|
||||
{(jointStates.name?.length || 0) > 10 && (
|
||||
<div className="text-gray-500">
|
||||
... and {(jointStates.name?.length || 0) - 10}{" "}
|
||||
more
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
) : (
|
||||
<Alert>
|
||||
<AlertTriangle className="h-4 w-4" />
|
||||
<AlertDescription>
|
||||
No robot status data received. Check that the NAO robot
|
||||
is connected and the naoqi_driver is running.
|
||||
</AlertDescription>
|
||||
</Alert>
|
||||
)}
|
||||
</CardContent>
|
||||
</Card>
|
||||
</TabsContent>
|
||||
|
||||
<TabsContent value="logs" className="space-y-4">
|
||||
<Card>
|
||||
<CardHeader>
|
||||
<CardTitle>Communication Logs</CardTitle>
|
||||
<CardDescription>
|
||||
Real-time log of ROS bridge communication
|
||||
</CardDescription>
|
||||
</CardHeader>
|
||||
<CardContent>
|
||||
<div className="h-64 overflow-auto rounded bg-black p-4 font-mono text-xs text-green-400">
|
||||
{logs.map((log, index) => (
|
||||
<div key={index}>{log}</div>
|
||||
))}
|
||||
<div ref={logsEndRef} />
|
||||
</div>
|
||||
<Button
|
||||
onClick={() => setLogs([])}
|
||||
variant="outline"
|
||||
className="mt-2"
|
||||
>
|
||||
Clear Logs
|
||||
</Button>
|
||||
</CardContent>
|
||||
</Card>
|
||||
</TabsContent>
|
||||
</Tabs>
|
||||
)}
|
||||
|
||||
{connectionStatus !== "connected" && (
|
||||
<Alert>
|
||||
<AlertTriangle className="h-4 w-4" />
|
||||
<AlertDescription>
|
||||
Connect to ROS bridge to start controlling the robot. Make sure
|
||||
the NAO integration is running:
|
||||
<br />
|
||||
<code className="mt-2 block rounded bg-gray-100 p-2">
|
||||
ros2 launch nao6_hristudio.launch.py nao_ip:=nao.local
|
||||
password:=robolab
|
||||
</code>
|
||||
</AlertDescription>
|
||||
</Alert>
|
||||
)}
|
||||
</div>
|
||||
</PageLayout>
|
||||
);
|
||||
}
|
||||
Reference in New Issue
Block a user