Feature/fix nightlight test by NicolasColombi · Pull Request #716 · CLIMADA-project/climada_python

Hi Emanuel, thanks for the suggestions! I might have an idea of why this happens.
If no arguments are provided to load_nighlight_noaa() the function does the following:

  1. Looks for a file with the pattern: '*'2013'*'.stable_lights.avg_vis.p in SYSTEM_DIR. If the file is present the function will return the file name that match the pattern. Example: F182013.v4c_web.stable_lights.avg_vis.p In this example the file present in the folder had satellite F18.

  2. If 1. doesn't apply, it looks for a file with the same pattern but with extention .tif.gz. If it finds one, it uses unzip_tif_to_py() to unzip it and save it as pickle. At this point unzip_tif_to_py() doesn't return the file_name with extension .p but the filename with extension .tif. Consequently so does load_nighlight_noaa().

Second point is that 2) is executed only if 1) is not, meaning that if in the folder there are both a .p and .tif.gz file, load_nighlight_noaa() will run 1) and return the file name ending with .p. What I suspect is that in the Jenkins environment there is already a file F182013.v4c_web.stable_lights.avg_vis.p, meaning that even if we create a fake .tif.gz on the fly in the test, point 2) never gets executed.

To solve this I would suggest two things:

  1. make unzip_tif_to_py() return the file name of the actual .pfile it saved, instead of .tif. But I would like to double check that with you @emanuel-schmid

  2. check if a .p file already exist in SYSTEM_DIR on the Jenkins environment. If yes, I would delate it.

using E99 instead of F18 works if the file is present in the folder (if we create it and put it there), if the file is not present load_nightlight_noaa will download it, but the satellite name on the noaa website are F10-F18. But for our use case it works 👍