Generative AI Model For Semantic Image Construction From Audio Prompts

Dept. of ISE Acharya Institute of Technology Bengaluru, India

This email address is being protected from spambots. You need JavaScript enabled to view it.
This email address is being protected from spambots. You need JavaScript enabled to view it.
This email address is being protected from spambots. You need JavaScript enabled to view it.
This email address is being protected from spambots. You need JavaScript enabled to view it.
This email address is being protected from spambots. You need JavaScript enabled to view it.
This email address is being protected from spambots. You need JavaScript enabled to view it.
This email address is being protected from spambots. You need JavaScript enabled to view it.

Abstract

Generative AI is a platform which produces many types of content. It enables users to develop images via description of them in words. Also it is a fact that most of our communication is through speech as opposed to writing. In this study we present what we have named the Audio to Image Generator. We have put together a system that links speech to image creation through a technology that is diffusion. The system has two main components. The first is a feature which we have incorporated from Whisper. Whisper's job is to turn speech into text that includes various accents, languages and also does well with background noise. The second component we have included is Stable Diffusion which is a model for image production. Also it has been made to do this in an accurate and quick way. These elements we put together to what we have which is a bridge between speech and image generation. We tested how well it does by comparing this system to text based methods. We used speech as input for the test. What we found is that it did very well at producing images which matched the intent of the input and also the images that it produced looked very real. Handled unclear prompts better than older techniques.This approach can help with speech-based interactions.It can also help teaching tools to visualize concepts or support projects.This method combines speech recognition with a type of generative modeling.It offers a way to improve AI systems that can handle multiple types of input.It also shows how people and computers can work together to create something focusing on making things accessible.