Computer Vision and Creative Content Generation: Text-to-Sketch Conversion

Within the field of computer vision and creative content generation, the process of combining visual elements based on textual descriptions has emerged as a captivating area of study and advancement. An intriguing application in this field is text-to-sketch conversion, which employs advanced machine learning methods to convert written descriptions into equivalent sketches or drawings. The utilization of visual representation has consistently proven to be a superior method of communication. Therefore, incorporating visualization in any communicative domain will greatly enhance the efficiency of the process. The objective of this paper is to accomplish this aim. This paper presents the development of an image generator that produces a sketch of a picture using the user's provided description. The produced outline is exhibited on an HTML canvas within a website. The generator employs POS tagging to parse the user's description, then utilizes a hidden layer regression neural network and Sketch RNN to generate the desired image based on the parsed description. The Sketch RNN model, which has been pre-trained on the Quick Draw dataset, is employed to draw sketches of the objects mentioned in the user's description. Additionally, a neural network is utilized to position the objects based on the user's instructions. The result consists of a series of brushstrokes that are rendered on the HTML canvas. The software can be utilized by any system user on any computer, allowing them to input their string using the keyboard. The software can operate efficiently and reliably, even on PCs with minimal hardware specifications. A pre-installed browser allows the user to view the user interface of a web page, where they can input information and receive the corresponding output.