From 2fd89e9799013eab4b5589891af487a787b029a0 Mon Sep 17 00:00:00 2001 From: Sean O'Connor Date: Sun, 25 Jan 2026 01:28:16 -0500 Subject: [PATCH] update irb-application for new study --- irbapplication.tex | 84 +++++++++++++++++++++++++++++++++++++--------- 1 file changed, 68 insertions(+), 16 deletions(-) diff --git a/irbapplication.tex b/irbapplication.tex index 28954c7..946868a 100644 --- a/irbapplication.tex +++ b/irbapplication.tex @@ -57,77 +57,129 @@ Please describe in some detail the purpose of the proposed study (including, as appropriate, information about the research question and relevant hypothesis or, if the research is exploratory, what the researchers hope to learn). \vspace{0.3cm} -For my honors thesis project, I am evaluating HRIStudio, a web-based platform I developed for designing and executing Wizard-of-Oz experiments in Human-Robot Interaction research. My research question asks whether HRIStudio improves methodological consistency and user experience for non-technical researchers compared to traditional robot programming environments that come with common robots, like Choregraphe for the NAO robot. I hypothesize that participants using HRIStudio will achieve higher accuracy when recreating published HRI experiments, report better usability experiences, and complete tasks more efficiently than those using manufacturer-provided tools. This study will provide empirical evidence about whether visual programming interfaces can lower technical barriers in HRI research while maintaining experimental rigor. +For my honors thesis project, I am evaluating HRIStudio, a web-based platform I developed for designing and executing Wizard-of-Oz experiments in Human-Robot Interaction (HRI). +My research question compares HRIStudio against the industry standard (Choregraphe) to determine their impact on: +\begin{enumerate} + \item \textbf{Disciplinary Accessibility:} Can non-technical domain experts successfully design a robot interaction? + \item \textbf{Scientific Reproducibility:} Does the tool minimize human error and data loss during experiment execution? +\end{enumerate} +I hypothesize that participants using HRIStudio will achieve higher Design Fidelity Scores and Execution Reliability Scores than those using the control software. \item[\textbf{2.}] \textbf{Subject Sample Description} Describe the proposed subject sample. If subjects under the age of 18 will participate in your research, indicate the expected age range of the samples. If your research involves a category of subjects that is vulnerable to coercion or undue influence, such as children, prisoners, individuals with impaired decision-making capacity, or economically or educationally disadvantaged persons, you must indicate clearly why the use of these subjects is scientifically necessary. \vspace{0.3cm} -I will recruit 10-12 students and faculty from non-engineering departments at Bucknell University, specifically Psychology, Education, and Sociology. All participants will be adults aged 18-65 with experience designing behavioral experiments but little to no programming background. I will exclude anyone with engineering or computer science background, prior experience with Choregraphe or similar robot programming tools, or undergraduate students to avoid any perceived academic coercion. This population is scientifically necessary because my research specifically evaluates tools designed to make HRI research accessible to non-technical domain experts, so using participants with programming backgrounds would not test the platform's core value proposition of lowering technical barriers for interdisciplinary researchers. +I will recruit a total of \textbf{N=20 participants}, divided into two distinct groups: + +\textbf{Group A: The "Wizards" (N=10)} +\begin{itemize} + \item \textbf{Population:} Faculty from the Computer Science Department at Bucknell University. + \item \textbf{Exclusion Criteria:} Must have NO prior experience with the NAO robot or Choregraphe software. +\end{itemize} + +\textbf{Group B: The "Test Subjects" (N=10)} +\begin{itemize} + \item \textbf{Population:} Undergraduate students at Bucknell University. + \item \textbf{Role:} They will serve as the "student" taking the geography quiz administered by the Wizard. + \item \textbf{Exclusion Criteria:} Must not be enrolled in a course taught by the "Wizard" participant they are paired with (to prevent coercion). +\end{itemize} \item[\textbf{3.}] \textbf{Recruitment and Selection Methods} How will subjects be recruited and selected? \vspace{0.3cm} -I will recruit participants through email outreach to department chairs in Psychology, Education, and Sociology, requesting permission to contact faculty and graduate students. I will also use faculty referrals and graduate student networks in relevant departments. All recruitment materials will clearly describe the study purpose, 3-hour time commitment, and emphasize that no prior technical experience is required. Interested participants will complete a brief screening survey to confirm eligibility criteria, then be randomly assigned to either control group (Choregraphe) or experimental group (HRIStudio). Sessions will be scheduled based on participant availability with no deception involved - participants will know they are evaluating research software tools. Participants will be entered into a raffle to win one of four \$25 gift cards. All recruitment materials will emphasize that participation is entirely voluntary, can be withdrawn at any time without consequence, and has no impact on academic standing or department relationships. +\textbf{Wizards:} Recruited via direct email invitations and verbal requests made during department meetings. +\textbf{Test Subjects:} Recruited via general campus announcements (Message Center, flyers) calling for volunteers to "Interact with a robot for 15 minutes." + +Recruitment materials will clearly state the time commitment (75 minutes for Wizards, 15 minutes for Test Subjects) and compensation. Participants will be screened via email for eligibility and randomly assigned to the Control or Experimental condition (for Wizards). \item[\textbf{4.}] \textbf{Detailed Research Methods and Procedures} Describe fully the following: \vspace{0.3cm} -\textbf{a) Research methods and procedures that will be employed in this study} +\textbf{a) Research methods and procedures} -I will conduct a randomized controlled trial comparing two software platforms for robot programming. Each participant will attend a 2-hour training workshop on their assigned platform (HRIStudio or Choregraphe), then complete a 3-hour individual session where they recreate a published HRI experiment using the NAO6 humanoid robot. The task involves programming the robot to perform a standardized greeting interaction sequence. I will measure completion accuracy against the original published protocol and collect user experience data through post-task surveys and brief interviews. +This is a between-subjects user study comparing two software interfaces. +\textbf{For "Wizard" Participants (75 minutes):} +\begin{enumerate} + \item \textbf{Training (15 mins):} Participant receives a standardized tutorial on their assigned software (HRIStudio or Choregraphe) covering speech, motion, and triggers. + \item \textbf{Design Challenge (30 mins):} Participant is given a "Paper Specification" (a storyboard for a 'Geography Quiz Proctor' scenario) and must implement it on the robot. The researcher tracks time-to-completion and help requests. + \item \textbf{Live Trial (15 mins):} A "Test Subject" (Group B participant) enters the room. The Wizard uses their interface to administer the quiz to the Test Subject. + \item \textbf{Debrief (15 mins):} Wizard exports the data and completes the System Usability Scale (SUS) survey. +\end{enumerate} + +\textbf{For "Test Subject" Participants (15 minutes):} +The Test Subject enters the lab, consents to participate, and interacts with the robot for the duration of the "Geography Quiz" (approx. 5-10 minutes). They answer the robot's questions naturally. \vspace{0.2cm} \textbf{b) Approximately how much time each subject is expected to devote to the research} -Each participant will spend approximately 5 hours total: 2 hours in a group training workshop and 3 hours in an individual task session, scheduled within one week of each other. +\begin{itemize} + \item Wizards: 75 minutes (One session). + \item Test Subjects: 15 minutes (One session). +\end{itemize} \vspace{0.2cm} \textbf{c) How data will be collected and recorded} - -Data will be collected with participant identifiers initially (for scheduling and compensation) but will be de-identified for analysis using numerical codes. I will use: (1) Screen recording software to capture participant interactions with the programming interfaces, (2) Timestamped logs from both software platforms showing programming actions, (3) Post-task questionnaires measuring usability, confidence, and satisfaction (Likert scales and open-ended questions), (4) Brief semi-structured interviews (10-15 minutes) about their experience, and (5) Objective scoring rubrics comparing their final robot programs to the target protocol. No audio/video recording of participants themselves, only screen capture of their computer interactions. +\begin{itemize} + \item \textbf{Screen Recording:} Captures the Wizard's workflow and design errors. + \item \textbf{Video Recording:} Captures the robot and the Test Subject during the Live Trial (to measure response timing and script adherence). + \item \textbf{Surveys:} SUS scores and confidence ratings from the Wizards. + \item \textbf{Rubrics:} "Design Fidelity Score" (scored from the saved project file) and "Execution Reliability Score" (scored from the video). +\end{itemize} \vspace{0.2cm} \textbf{d) Methods for obtaining and documenting informed consent of subjects} -I will obtain written informed consent at the beginning of each participant's first session (the training workshop). Participants will receive consent forms via email 24 hours prior to review, and I will review all elements verbally before obtaining signatures. The consent form will clearly explain the study purpose, procedures, time commitment, voluntary nature, right to withdraw, data handling, and compensation details. +Written informed consent will be obtained from ALL participants (Wizards and Test Subjects) prior to their involvement. +\textit{Note:} Wizards will consent to the full 75-minute procedure. Test Subjects will sign a simplified consent form specific to the 15-minute interaction and video recording. \vspace{0.2cm} \textbf{e) Any use of deception in the proposed study and justification for its use} -No deception will be used. Participants will be fully informed that they are evaluating robot programming software tools as part of a comparative study for my honors thesis research. +No deception is used in this study. Both the Wizard and the Test Subject are aware of their roles in the experiment. \vspace{0.2cm} \textbf{f) Methods for preserving confidentiality} - -All data will be stored on password-protected, encrypted university computers in locked offices. Participant names will be replaced with numerical codes within 48 hours of data collection. Screen recordings and interview notes will be stored separately from identifying information. Data will be retained for 3 years following thesis completion, then permanently deleted. Only I and my thesis advisor will have access to identifiable data during the active research period. +All video and screen data will be stored on an encrypted drive kept in the PI's locked lab space. Participant names will be replaced with numerical codes (W-01 for Wizards, S-01 for Subjects). Data will be retained for 3 years following thesis completion. \item[\textbf{5.}] \textbf{Benefits and Payment Arrangements} Indicate any benefits that are expected to accrue to subjects as a result of their participation in the research. In the event that subjects will be paid, describe all payment arrangements, including how much subjects will be paid should they choose to withdraw from the study prior to completion of the research. \vspace{0.3cm} -Participants will be entered into a raffle to win one of four \$25 gift cards as compensation for their time and effort. All participants who complete the informed consent process and attend at least one session will be eligible for the raffle, regardless of whether they complete the full study. The raffle will be conducted after all data collection is complete, and winners will be notified within one week of the final session. The compensation is intended to acknowledge their time contribution rather than serve as an incentive to participate or continue in the study. Participants may also gain exposure to robot programming concepts and tools that could be useful in their own research, though this is not a guaranteed outcome. There are no direct personal benefits beyond the potential raffle compensation and potential learning experience. +\vspace{0.3cm} +Participants will be provided with snacks and refreshments during the session. No monetary compensation will be provided. +Compensation (snacks) is provided immediately upon arrival. If a participant withdraws early, they are still welcome to the refreshments. \item[\textbf{6.}] \textbf{Researcher-Subject Relationships and Coercion Mitigation} Describe any pre-existing relationships between researcher and subjects --- such as teacher--student, superintendent--principal--teacher, employer--employee --- that might impact subjects' ability to participate in the research voluntarily. How will any potential for coercion be mitigated by the researchers? \vspace{0.3cm} -I do not have direct supervisory, teaching, or grading relationships with potential participants since I am recruiting from departments outside of Computer Science. However, as a fellow student at Bucknell University, there may be informal academic or social connections. To mitigate any potential coercion: (1) I will emphasize in all communications that participation is entirely voluntary with no academic or professional consequences for declining or withdrawing, (2) I will not recruit through my own classes or research groups, (3) I will use department chairs and faculty as intermediaries for initial contact rather than direct personal outreach, (4) I will ensure participants understand that their decision to participate will not be shared with faculty in their departments, and (5) I will remind participants at each session that they can withdraw at any time while still remaining eligible for the raffle compensation. +\vspace{0.3cm} +The PI is an undergraduate student in the Computer Science department. To mitigate coercion or undue influence when recruiting faculty: +1. Recruitment will emphasize that participation is voluntary and unrelated to coursework, grading, or departmental politics. +2. Faculty participation will be framed as 'expert review' or 'software evaluation' to minimize power dynamic issues. +3. Test Subjects are screened to ensure they are not current students of the Faculty Wizard they are paired with. + \end{enumerate} \section{PART III - Supporting Documents} \begin{itemize} -\item \textbf{Informed Consent Form} (unless not required) -\item \textbf{Debriefing} (required if using deception) +% \item \textbf{Informed Consent Form} (unless not required) +% \item \textbf{Debriefing} (required if using deception) \item \textbf{Research Materials} +\item \textbf{Informed Consent Form (Wizard)} +\item \textbf{Informed Consent Form (Test Subject)} +\item \textbf{Recruitment Materials} +\item \textbf{Paper Specification (The "Geography Quiz")} +\item \textbf{Post-Study Questionnaire (SUS)} +\item \textbf{CITI Completion Report} \end{itemize} Please note that all materials to which subjects will be exposed should be included (this includes surveys, interview or focus group outlines, visual stimuli, and so on). If using online surveys, please include as a PDF or a text document rather than simply providing a URL. You may upload multiple supporting documents using the 'Add Another' button below.