feat: Complete NAO6 robot integration with HRIStudio platform

MAJOR INTEGRATION COMPLETE:

🤖 Robot Communication System:
- RobotCommunicationService for WebSocket ROS bridge integration
- Template-based message generation from plugin definitions
- Real-time action execution with error handling and reconnection

🔧 Trial Execution Engine:
- Updated TrialExecutionEngine to execute real robot actions
- Plugin-based action discovery and parameter validation
- Complete event logging for robot actions during trials

🎮 Wizard Interface Integration:
- RobotActionsPanel component for live robot control
- Plugin-based action discovery with categorized interface
- Real-time parameter forms auto-generated from schemas
- Emergency controls and safety features

📊 Database Integration:
- Enhanced plugin system with NAO6 definitions
- Robot action logging to trial events
- Study-scoped plugin installations

🔌 API Enhancement:
- executeRobotAction endpoint in trials router
- Parameter validation against plugin schemas
- Complete error handling and success tracking

 Production Ready Features:
- Parameter validation prevents invalid commands
- Emergency stop controls in wizard interface
- Connection management with auto-reconnect
- Complete audit trail of robot actions

TESTING READY:
- Seed script creates NAO6 experiment with robot actions
- Complete wizard interface for manual robot control
- Works with or without physical robot hardware

Ready for HRI research with live NAO6 robots!
This commit is contained in:
2025-10-17 11:35:36 -04:00
parent c206f86047
commit 7072ee487b
7 changed files with 1531 additions and 35 deletions

View File

@@ -25,7 +25,10 @@ import {
mediaCaptures,
users,
} from "~/server/db/schema";
import { TrialExecutionEngine } from "~/server/services/trial-execution";
import {
TrialExecutionEngine,
type ActionDefinition,
} from "~/server/services/trial-execution";
// Helper function to check if user has access to trial
async function checkTrialAccess(
@@ -894,4 +897,74 @@ export const trialsRouter = createTRPCRouter({
return { success: true };
}),
executeRobotAction: protectedProcedure
.input(
z.object({
trialId: z.string(),
pluginName: z.string(),
actionId: z.string(),
parameters: z.record(z.string(), z.unknown()).optional().default({}),
}),
)
.mutation(async ({ ctx, input }) => {
const { db } = ctx;
const userId = ctx.session.user.id;
await checkTrialAccess(db, userId, input.trialId, [
"owner",
"researcher",
"wizard",
]);
// Use execution engine to execute robot action
const executionEngine = getExecutionEngine();
// Create action definition for execution
const actionDefinition: ActionDefinition = {
id: `${input.pluginName}.${input.actionId}`,
stepId: "manual", // Manual execution
name: input.actionId,
type: `${input.pluginName}.${input.actionId}`,
orderIndex: 0,
parameters: input.parameters,
timeout: 30000,
required: false,
};
const result = await executionEngine.executeAction(
input.trialId,
actionDefinition,
);
if (!result.success) {
throw new TRPCError({
code: "INTERNAL_SERVER_ERROR",
message: result.error ?? "Robot action execution failed",
});
}
// Log the manual robot action execution
await db.insert(trialEvents).values({
trialId: input.trialId,
eventType: "manual_robot_action",
actionId: actionDefinition.id,
data: {
userId,
pluginName: input.pluginName,
actionId: input.actionId,
parameters: input.parameters,
result: result.data,
duration: result.duration,
},
timestamp: new Date(),
createdBy: userId,
});
return {
success: true,
data: result.data,
duration: result.duration,
};
}),
});