Audio-to-Text Urdu Chatbot using Deep Learning Algorithms RNN and wav2vec2

Abstract

Advancement in technology limited the distances via communication. People globally exchange thoughts in different languages using many ways like text, audio, pictures, and videos to express their ideas. Among many languages Urdu language has more than 100 million people around the world. It is necessities the development of smart applications to facilitate Urdu language users that can communicate via audio instead of only text. A conversation bot system enables individuals and computers to communicate using natural language. Numerous Chabot’s have been developed in English, German, Korean, Spanish, and Chinese languages. Because of the significant language barrier, those who do not speak English, German, Korea, or Spanish well cannot use these chatbots. In this research work we developed a smart chatbot system that can take voice as input for Urdu language using RNN a deep learning model. The proposed system is developed using two datasets UQuaD and custom dataset. A pretrained model is used to convert Urdu audio to text named as “wav2vec2-large-xls-r-300m-Urdu”. The proposed system on UQuaD and custom achieved an accuracy of 68.30% on the UQuaD dataset and 89.6% on the custom dataset.