Expand Up @@ -17,7 +17,7 @@ "id": "23v-XAUNQIPY" }, "source": [ "# **DeepLabCut Model Zoo: SuperAnimal models\n", "# DeepLabCut Model Zoo: SuperAnimal models\n", "\n", "\n", "\n", Expand All @@ -31,12 +31,13 @@ "\n", "## **Please consider giving back and labeling a little data to help make each network even better!**\n", "\n", "We have a WebApp, so no need to install anything, just a few clicks! We'd really appreciate your help!\n", "We have a WebApp, so no need to install anything, just a few clicks! We'd really appreciate your help! 🙏\n", " \n", "https://contrib.deeplabcut.org/\n", "\n", "\n", "- **Note, if you performance is less that you would like:** firstly check the labeled_video parameters (i.e. \"pcutoff\" in the config.yaml file that will set the video plotting) - see the end of this notebook. You can also use the model in your own projects locally. Please be sure to cite the papers for the model, and http://modelzoo.deeplabcut.org (paper forthcoming!)\n", "- **Note, if you performance is less that you would like:** firstly check the labeled_video parameters (i.e. \"pcutoff\" that will set the video plotting) - see the end of this notebook.\n", "- You can also use the model in your own projects locally. Please be sure to cite the papers for the model, i.e., [Ye et al. 2023](https://arxiv.org/abs/2203.07436) 🎉\n", "\n", "\n", "\n", Expand All @@ -53,7 +54,7 @@ }, "outputs": [], "source": [ "!pip install \"deeplabcut[tf,modelzoo] @ git+https://github.com/DeepLabCut/DeepLabCut@main\"" "!pip install deeplabcut[tf,modelzoo]" ] }, { Expand All @@ -77,6 +78,15 @@ "import os" ] }, { "cell_type": "markdown", "source": [ "## Please select a video you want to run SuperAnimal-X on:" ], "metadata": { "id": "GXf8N4v28Xqo" } }, { "cell_type": "code", "execution_count": null, Expand All @@ -91,12 +101,24 @@ "for filepath, content in uploaded.items():\n", " print(f'User uploaded file \"{filepath}\" with length {len(content)} bytes')\n", "video_path = os.path.abspath(filepath)\n", "video_name = os.path.splitext(video_path)[0]\n", "\n", "# If this cell fails (e.g., when using Safari in place of Google Chrome),\n", "# manually upload your video via the Files menu to the left\n", "# and define `video_path` yourself with right click > copy path on the video." ] }, { "cell_type": "markdown", "source": [ "## Next select the model you want to use, Quadruped or TopViewMouse\n", "- See http://modelzoo.deeplabcut.org/ for more details on these models\n", "- The pcutoff is for visualization only, namely only keypoints with a value over what you set are shown. 0 is low confidience, 1 is perfect confidience of the model." ], "metadata": { "id": "A8sDYMa08f62" } }, { "cell_type": "code", "execution_count": null, Expand All @@ -105,10 +127,19 @@ }, "outputs": [], "source": [ "supermodel_name = \"superanimal_topviewmouse\" #@param [\"superanimal_topviewmouse\", \"superanimal_quadruped\"]\n", "supermodel_name = \"superanimal_quadruped\" #@param [\"superanimal_topviewmouse\", \"superanimal_quadruped\"]\n", "pcutoff = 0.3 #@param {type:\"slider\", min:0, max:1, step:0.05}" ] }, { "cell_type": "markdown", "source": [ "## Okay, let's go! 🐭🦓🐻" ], "metadata": { "id": "zsB0pGtj9Luq" } }, { "cell_type": "code", "execution_count": null, Expand All @@ -119,6 +150,7 @@ "source": [ "videotype = os.path.splitext(video_path)[1]\n", "scale_list = []\n", "\n", "deeplabcut.video_inference_superanimal(\n", " [video_path],\n", " supermodel_name,\n", Expand All @@ -128,6 +160,38 @@ " pcutoff=pcutoff,\n", ")" ] }, { "cell_type": "markdown", "source": [ "## Let's view the video in Colab:\n", "- otherwise, you can download and look at the video from the left side of your screen! It will end with _labeled.mp4\n", "- If your data doesn't work as well as you'd like, consider fine-tuning our model on your data, changing the pcutoff, changing the scale-range\n", "(pick values smaller and larger than your video image input size). See our repo for more details." ], "metadata": { "id": "gPLZSBpD34Mj" } }, { "cell_type": "code", "source": [ "from base64 import b64encode\n", "from IPython.display import HTML\n", "view_video = open(video_name+'DLC_snapshot-1000_labeled.mp4','rb').read()\n", "\n", "data_url = \"data:video/mp4;base64,\" + b64encode(view_video).decode()\n", "HTML(\"\"\"\n", "<video width=600 controls>\n", " <source src=\"%s\" type=\"video/mp4\">\n", "</video>\n", "\"\"\" % data_url)" ], "metadata": { "id": "ejFJ1Pbg33i6" }, "execution_count": null, "outputs": [] } ], "metadata": { Expand Down