Generating Human-Like Descriptions for the Given Image Using Deep Learning

¹ MTech Student, Dept of Computer Engg, BVM College, Vallabh Vidyanagar, Anand – 388120, Gujarat, India.
² Prof. & Head of Computer Dept, BVM College, Vallabh Vidyanagar, Anand – 388120, Gujarat, India.
³ Assistant Prof., Dept of Computer Engg, BVM College, Vallabh Vidyanagar, Anand – 388120, Gujarat, India.

^* Corresponding author: tladdha08@gmail.com

Abstract

One of the most prominent applications in the field of computer vision and natural language processing research is image captioner. The paper includes an exhaustive review of the literature on image captioning and the implementation using attention-based encoder-decoder model. The process of depicting an image with textual explanations is known as image captioning. The problem has seen extensive use of encoder-decoder frameworks. In this study, Deep Convolutional Neural Network (CNN) for image classification and Recurrent Neural Network (RNN) for sequence modeling are combined to build a single network that creates descriptions of images using the Microsoft Common Objects in Context Dataset (MSCOCO Dataset). Because of RNNs being computationally expensive to train and assess, memory is often restricted to a few items. By highlighting the most important components of an input image, the Attention model had been used to address this issue. The model was developed using Nvidia Quadro RTX5000 GPU (CUDA), which received the Bleu-1 score of 0.5793 for the 100 generated sentences. The captions generated by the model on the testing dataset labeled nearly all of the objects in the image and were sufficiently like the actual captions in the annotations, even on images outside of the testing dataset.