In this work, we propose and experimentally demonstrate a scheme for generating fast physical random bits (PRBs). For such a scheme, the chaotic signal output from an optoelectronic delay loop (OEDL) pumped by a semiconductor laser (SL) under modulated optical feedback is utilized as the chaotic entropy resource, and the m least significant bits (m-LSBs) extraction and local XOR operation are selected as the post-processing methods. Firstly, through analyzing the influences of some typical parameters on the properties of the chaotic signal, the optimized parameter regions for achieving high quality chaotic signals with weak time-delay signatures and broad bandwidths are determined. Secondly, a high quality chaotic signal under optimized parameters is selected as a chaotic entropy resource, which is sampled and transferred to an original digital bit sequence by an 8-bit analog-to-digital converter (ADC) at a rate of 80 GS/s. Finally, through adopting the m least significant bits (m-LSBs) extraction and logical exclusive OR (XOR) operation to process the original digital bit sequence, a PRB at a rate up to 400 Gbits/s can be generated, which has passed all the NIST tests.