Dynamic time warping based sign retrieval

Abstract

Sign language, a visual language, is the main mode of communication for the Deaf. A large body of audio-visual material are annotated with sign language for the utilization of the Deaf. Enabling the sign based retrieval of these sources will serve two purposes: (i) Deaf people will be able to make searches using sign language; (ii) People who do not know the meaning of a sign will be able to search that sign in its source, and infer the meaning from accompanying subtitles and speech. In this work, we have developed a baseline system based on dynamic time warping for the sign sample based retrieval in videos. We analyze the effects of using different features, distance measures and temporal reduction techniques on the query performance.