The acceleration of Deep Neural Networks (DNNs) has attracted much attention in research. Many critical real-time applications benefit from DNN accelerators but are limited by their compute-intensive nature. This work introduces an accelerator for Convolutional Neural Network (CNN), based on a hybrid optoelectronic computing architecture and residue number system (RNS). The RNS reduces the optical critical path and lowers the power requirements. In addition, the wavelength division multiplexing (WDM) allows high-speed operation at the system level by enabling high-level parallelism. The proposed RNS compute modules use one-hot encoding, and thus enable fast switching between the electrical and optical domains. We propose a new architecture that combines residue electrical adders and optical multipliers as the matrix-vector multiplication unit. Moreover, we enhance the implementation of different CNN computational kernels using WDM-enabled RNS based integrated photonics. The area and power efficiency of the proposed accelerator are 0.39 TOPS/s/mm(2) and 3.22 TOPS/s/W, respectively. In terms of computation capability, the proposed chip is 12.7x and 4.02x better than other optical implementation and memristor implementation, respectively. Our experimental evaluation using DNN benchmarks illustrates that our architecture can perform on average more than 72 times faster than GPU under the same power budget. © 2023 ACM, Inc.