With the ability to convert light energy to electrical energy, photodetectors have great significance in optoelectronics and find numerous applications in the fields of information technology, remote sensing, environmental monitoring, national security, medicines, etc. Semiconducting materials are mainly used in photodetectors due to their superior opto-electronic properties, and it is desirable to design photoactive materials with tunable band-gap, cost-effective manufacturing, and compatibility with flexible device architectures for a wide range of applications. Organic semiconductors are suitable candidates in this regard, and herein, we emphasize the organic-polymer-based photodetectors that are capable of detecting a wide range of optical wavelengths. Compared to inorganic material-based photodetectors, organic-polymer-based photodetectors contain low-bandgap conjugated polymers along with their suitable chemical structures that possess interacting large side groups to be tuned easily and can achieve a broad spectral response and a high ratio of photocurrent to dark current. The parameters that determine the efficacy of a photodetector such as photocurrent, detectivity, and responsivity are also summarized for various organic polymers and their conjugates with the other materials. Additionally, it is interesting to review the underlying photophysical mechanism and various device architectures useful in photodetection.