SpeakSwarm: A Co-Speech Gesture Generation Pipeline for Conversational Agents using Swarm Robots

Abstract

In interpersonal communication, humans rely on a variety of non-verbal cues such as eye gaze, facial expressions, posture, and gestures to convey information. While most of these often convey broader emotional state, co-speech gestures are aligned with specific words or phrases in speech, reflecting the speaker’s underlying meaning. As a result, extensive research has investigated the application of these gestures in humanoid social robots and chatbots. However, non-humanoid systems remain underexplored. This research aims to bridge that gap by investigating the design and implementation of these gestures using tabletop swarm robots. Prior research has shown that swarm robots can provide a flexible, scalable user interface, making them a promising medium for expressing co-speech gestures.