In the context of smart aquaculture, real-time Multi-fish Tracking (MFT) poses a significant challenge. Existing Multi-Object Tracking (MOT) methods are often designed for objects with specific shapes or regular motion patterns, such as pedestrians or cars. The unique characteristics of fish, including their uniform appearance and deformation during motion, have been largely overlooked in research. To address this gap, we introduce the Uniform and Deformable Multi-fish Tracking (UD-MFT) benchmark. This dataset not only incorporates challenges related to uniform appearance and diverse deformable shapes of fish during motion in daily activities but also encompasses common MOT challenges like occlusion and disappearance. All sequences are sourced from industrialized aquaculture environments, providing a practical and relevant setting. To understand the distinctiveness of UD-MFT, we quantify the degrees of deformation, appearance, and occlusion levels within the dataset and compare them with tracking targets in existing datasets. Furthermore, to facilitate practical applications, we conduct a comprehensive evaluation of state-of-the-art real-time MOT models on UD-MFT, establishing a comparative baseline for accuracy and computational requirements. Additionally, we perform an in-depth analysis of the impact of deformation and appearance similarity on tracking accuracy. Finally, we provide reflections and recommendations concerning potential avenues for future research in this field. The proposed UD-MFT aims to serve as a robust platform for developing algorithms capable of handling fish with multiple motion patterns, thereby contributing to the advancement of intelligent fish farming. © 2024 Elsevier Ltd