The current plan is to build a general model able to detect 5 or 9 keypoints using a variety of data from experiments conducted under different conditions and setups. To complete more complex tasks requiring more keypoints, the pretrained model can be finetuned to adapt to the specific downstream task using significantly less data than the pretraining data. Our tracking algorithm tracks identities of the flies based on 5 key points (Figure 1):
Detection of these 5 key points is more robust when compared to wings and legs which move frequently and are occluded in flies’ movement.