Robust motion tracking and classification of zebrafish with deep learning

Zebrafish serve as a key pre-clinical model for disease research. Studying disease progression requires long-term tracking of fish motion, particularly when considering the function of the musculoskeletal apparatus. While vision-based tracking has been studied, current approaches typically require substantial fine tuning and often lack robustness, especially when multiple fish need to be tracked and motion occurs along all three spatial dimensions. We propose a simplified setup and a 3D tracking method using only two cameras and two convolutional neural networks (CNN) to localize and track multiple fish robustly. We separate the localization of fish and their classification as an individual. This allows for simple training data acquisition by placing fish into the basin individually, without the need for manual annotation. We evaluate the approach on sets of two, five, and ten fish resulting in multiple object tracking accuracies (MOTA) of 0.997, 0.994, and 0.966, respectively. The mean MOTA over all scenarios is 0.99 compared to a mean MOTA of 0.89 for a state-of-the-art tracking algorithm (idtracker.ai). Finally, we present comprehensive data on the motion activity of individual fish in a 30-minute video sequence, demonstrating robust tracking and unique activity profiles of each fish.

Content

Subjects

Deep learning

fish tracking

fish trajectory dataset

locomotion pattern

physical activity

DDC Class

006.3: Artificial Intelligence

570: Life Sciences, Biology

Funding References

Funding(s)

GRK 3144: Multiskalen-Analytik und biomedizinische Bildgebung an Grenzflächen für die Skelettfunktion

Files

Lizenz

https://creativecommons.org/licenses/by/4.0/

Publication version

publishedVersion

Download

Name

Robust_Motion_Tracking_and_Classification_of_Zebrafish_With_Deep_Learning.pdf

Type

Main Article

Size

3.44 MB

Format

Adobe PDF