PyTorch model evaluation - `per_keypoint_evaluation` and `comparisonbodyparts` by n-poulsen · Pull Request #2810 · DeepLabCut/DeepLabCut

Skip to content

Navigation Menu

Sign in

Appearance settings

Conversation

@n-poulsen

This pull request implements the per_keypoint_evaluation and comparisonbodyparts parameters for PyTorch pose estimation models.

Updates

  • New per_keypoint_evaluation argument for deeplabcut.pose_estimation_pytorch.apis.evaluation.evaluate_network to compute the errors (RMSE and mAP) based on a subset of bodyparts, as done for tensorflow models
  • New comparisonbodyparts argument for deeplabcut.pose_estimation_pytorch.apis.evaluation.evaluate_network to compute RMSE per-bodypart and store the results in a CSV file, as done for tensorflow models
  • Inclusion of unique bodypart RMSE in the "main" RMSE computation to match the tensorflow RMSE computation
  • Additional tests to verify that RMSE computation is correct
  • Updates the functional PyTorch tests to compute per-bodypart RMSE
  • Update docs/pytorch/user_guide.md so that methods where parameters still need to be implemented are labeled orange and not green
  • Update docs/pytorch/user_guide.md to reflect that comparisonbodyparts and per_keypoint_evaluation can now be used with PyTorch models

maximpavliv

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Beside the two comments regarding documentation - this looks good to me!

@n-poulsen n-poulsen deleted the niels/evaluate_network_comparison branch

December 10, 2024 14:52

2 participants

@n-poulsen @maximpavliv