Natural Language Dialogue for Supervisory Control of Autonomous Ground Vehicles

Unmanned Ground Vehicles (UGVs) currently deployed on the battlefield are driven with Operator Control Units (e.g., joystick or point-and-click interface), which requires the full attention of an operator. This means the operator cannot concentrate on maintaining situational awareness, security, or other tasks. To reduce the burden on the operator, SoarTech with our partners Thinking Robots, Brown University, and Southwest Research Institute propose to develop a natural dialogue interface for interacting with UGVs, where an operator can interact with a UGV using speech and gesture. The UGV will be capable of taking high-level commands, engaging in dialogue with the user to ensure that the command is clear and providing feedback to the user during task execution. To accomplish this, we will develop a multi-modal Smart Interaction Device (SID) for UGV control, based on a proven Intelligent User Interface system that we have used for supervisory control of tactical simulation entities. We will extend our Phase I work to focus on gestures and speech for interacting with UGVs during Logistics Connector missions, and implement this on a target ONR platform. This approach will enable natural, supervisory control of one or more UGVs, and greatly reduce the workload of the UGV operator.