Metal castings are products that are used everywhere. It is used in vehicles, in buildings, for construction and so on. Castings are basically molded shapes formed out of melted metal like iron. The process of making castings, however, can easily be compromised. This gives rise to defects like cracks, flow marks, porosity, and pinhole formation on the surface. Generally, ultra-sonic inspections or simple visual inspections are done to look for defects. But they are time-consuming, expensive and require more labor. In current times, computer vision is used to make the process simpler. Several neural network algorithms were experimented to do image classification. Many convolutional neural network models were experimented to receive good accuracy. But the difficulty faced during training the model is the less availability of actual data of defect goods to train. Since training samples are usually smaller, only a few algorithms like ResNet50 and EfficientNetB 7 gave better accuracy in classifying casting goods as defective or not. It became more important to see how well these algorithms do when the training sample set size becomes even less compared to the testing sample.