Hand graph topology selection for skeleton-based sign language recognition

Abstract

State-of-the-art Sign Language Recognition (SLR) frameworks based on Graph Convolutional Networks (GCNs) require a skeleton-based graph topology. Although upper body skeleton configuration is often considered, Sign Languages (SLs) mainly comprises hand shapes, upper body movements, and facial gestures. Notably, the hand plays a major role in performing the sign. This paper investigates optimal choices for the hand graph topology essential for improving the recognition performance. Our experiments on two benchmark Turkish SL datasets, BosphorusSign22k and AUTSL, demonstrate that hand-based topology substantially contributes to performance that is competitive with full body-based topologies.