NOAA Upload Script by BryonLewis · Pull Request #1358 · Kitware/dive

This is a script for NOAA to import their mirrored bucket annotations into their system. I'm going to deliver it directly to them but I feel it should be included in the repo as a reference point for others who may want to utilize similar features of girder-client.

  • Allows specifying a GirderId for a root folder and local mounted folder location to mirror. It will then upload the annotations to the mirrored directory structure in the system.
  • ImageSet Structure is :
    • ./bucket-name-images/platform/date/ID/imageType/images/Dir0001
    • ./bucket-name-images/platform/date/ID/imageType/images/Dir0002
    • ./bucket-name-images/platform/date/ID/imageType/images/Dir0003
  • Annotation Struture is:
    • ./bucket-name-annotations/platform/date/ID/imageType/images/ANNOTATIONS
    • ./bucket-name-images/platform/date/ID/imageType/images/ANNOTATIONS/unlabled/annotations-Dir0001.csv
    • ./bucket-name-images/platform/date/ID/imageType/images/ANNOTATIONS/unlabled/annotations-Dir0002.csv
    • ./bucket-name-images/platform/date/ID/imageType/images/ANNOTATIONS/unlabled/annotations-Dir0003.csv

TODO:

  • Modify the current setup to support this new structure. I would probably make it so you input the /images/ directory on girder and reference the ANNOTATIONS folder locally and it will go through and match the annotations and Directory name to the image sequence names.
  • Add capability to clone videos to public folder for easier organization, or possibly clone to a specified folder.
  • Add support for specific keys in the annotation files that will cause it to clone the dataset instead of uploading directly.
  • Create an explanation video about how it works

For reference I also ran our poetry tox -e format ../samples/scripts and that's why the uploadScript.py has been modified

SmallUpload.mp4