Simulation based Evaluation of Fault Detection Algorithms with Applications to Wear Diagnosis in Manipulators

Fault detection algorithms (FDAs) process data to generate a test quantity. Test quantities are used to determine presence of a fault in a monitored system, despite disturbances. Because only limited knowledge of the system can be embedded in an FDA, it is important to evaluate it in scenarios relevant in practice. In this paper, simulation based approaches are proposed in an attempt to determine: i) which disturbances affect the output of an FDA the most; ii) how to compare the performance of different FDAs; and iii) which combinations of fault change size and disturbances variations are allowed to achieve satisfactory performance. The ideas presented are inspired by the literature of design of experiments, surrogate models, sensitivity analysis and change detection. The approaches are illustrated for the problem of wear diagnosis in manipulators where three FDAs are considered. The application study reveals that disturbances caused by variations in temperature and payload mass error affect the FDAs the most. It is also shown how the size of these disturbances delimit the capacity of an FDA to relate to wear changes. Further comparison of the FDAs reveal which performs “best” in average.