Ins Finder: A Practical CPU Undocumented Instruction Detection Framework

As the basic and core component of electronic systems, CPU security is extremely important to network security. Even an unremarkable faulty instruction on the CPU may lead to serious security problems, such as the operating system crashes or privilege increase since it is often considered as a trusted black box. Therefore, CPU instruction detection is particularly crucial to CPU security. However, most existing methods of CPU instruction detection, based on the inconsistency of microarchitecture and instruction set design, suffer from slow speed and low accuracy. Our work is motivated to propose a practical framework for searching CPU undocumented instruction with fast speed and high accuracy. In this paper, we put forward a general framework InsFinder to detect undocumented instruction on CISC and RISC CPU by an efficient and accurate fuzzing method. It makes use of the instruction format to make advanced predictions, which greatly reduces the search space. Moreover, by introducing classification, de-redundancy, and verification, InsFinder greatly improves the detection accuracy. Experiments show that compared with the existing methods, InsFinder is more effective which costs at least 50% less processing time in detecting undocumented instructions on x86-64, ARM64, and RISC-V, and more accurate which divided the detection results into 4 categories. After filtering, the detection results were reduced from millions to less than 10,000.