Resource Type

Dataset

Abstract

RoboGestDB is a multi-modal (RGB-D) dataset of 15 gestures designed for communication between humans and autonomous mobile robots (AMRs) in industrial environments such as warehouses. The release includes 54,000 gesture instances from 50 participants, with each gesture captured simultaneously as an RGB image, a depth image, a 3-second RGB video, and a 3-second depth video using three synchronized Intel RealSense D435 cameras. The gesture set spans four functional categories: signaling human movement intent, issuing AMR motion commands, emergency override controls, and bidirectional human-robot acknowledgments. Recordings were collected under a full factorial design varying interaction distance (3 m, 7 m), lighting level (dim, bright), clothing configuration (regular, loose), and camera viewpoint (frontal, ±30°), with gesture order independently randomized across three repeated trials per condition. All gestures were recorded against a uniform green-screen background, enabling users to generate synthetic background composites for domain-adaptation and robustness studies, in addition to the original captures included. Data collection followed an IRB-approved protocol at the Rochester Institute of Technology (approval no. 01100324) with full informed consent from all participants.

DOI

10.58044/kgcoe-2v52

Publication Date

10-16-2026

Comments

Datasets forthcoming.

Version

1

Creative Commons License

Creative Commons Attribution 4.0 International License
This work is licensed under a Creative Commons Attribution 4.0 International License.

Department, Program, or Center

Industrial and Systems Engineering

College

Kate Gleason College of Engineering

Campus

RIT – Main Campus

Download

Share

COinS