In recent years, supervised learning has improved many computer vision problems. However, data scarcity, lack of labeled data, and imbalanced datasets have created issues in adopting this improvement in the medical imaging domain. With the recent advancement in other large language and vision language models(eg: chatgpt, DALL-E) generating synthetic data has become easier. However, this is still cost-prohibitive for large-scale datasets specifically image dataset generation. This approach can also may not be suitable for privacy-first datasets. In this work, the proposed methodology is to generate synthetic images based on available labeled images and then use these generated images along with the existing data to solve above mentioned issues. Chest X-ray datasets are one of the complex datasets that suffer from label imbalance problems and strict data privacy is required for handling any such kind of data. In this work, a simplified generative adversarial network-based solution is used which is cost-effective and provides better results than only using available datasets. This proposed method is especially useful for privacy-first, imbalanced datasets. Finally, this solution was compared with some existing proposals. The promising result obtained using this methodology shows that this proposed solution can be expanded to other domains.