Acoustic emission (AE) is a widely used non-destructive test method in structural health monitoring applications to identify the damage type in the material. Usually, the analysis of the AE signal is done by using traditional parameter-based methods. Recently, machine learning methods showed promising results for the analysis of AE signals. However, these machine learning models are complex, slow, and consume significant amounts of energy. To address these limitations and to explore the trade-off between model complexity and the classification accuracy, this paper presents a lightweight artificial neural network model to classify damage types in concrete material using raw acoustic emission signals. The model consists of one hidden layer with four neurons and is trained on a public acoustic emission signal dataset. The created model is deployed to several microcontrollers and the performance of the model is evaluated and compared with a state-of-the-art machine learning model. The model achieves 98.4% accuracy on the test data with only 4019 parameters. In terms of evaluation metrics, the proposed tiny machine learning model outperforms previously proposed models 10 to 1000 times. The proposed model thus enables machine learning in real-time structural health monitoring applications.