Feature/centroids as gdf by chahank · Pull Request #787 · CLIMADA-project/climada_python

and others added 30 commits

April 27, 2023 14:58

@climada-jenkins

# Conflicts:
#	climada/hazard/trop_cyclone.py

@emanuel-schmid

…fined dist_coast in centroids

@emanuel-schmid

@emanuel-schmid

@emanuel-schmid

@idetring

@emanuel-schmid

@emanuel-schmid

# Conflicts:
#	CHANGELOG.md

@emanuel-schmid

…really _are_ failing now

@emanuel-schmid

@emanuel-schmid

@emanuel-schmid

@emanuel-schmid

@emanuel-schmid

@emanuel-schmid

@emanuel-schmid

# Conflicts:
#	CHANGELOG.md

emanuel-schmid added a commit that referenced this pull request

May 8, 2024
* Add geodataframe to centroids: first commit

* Remove raster methods, obsolete methods, and fix some.

* Remove test for removed methods

* Fix from geodataframe method

* Remove set meta to lat lon

* Update all I/O methods

* Remove area pixel

* Add method to compute the area per pixel (not attribute)

* Make get_area_pixel

* Remove not often used dist_coast and elevation

* Remove unused meta

* Change Centroids call to new signature

* Remove vector - raster hazard methods

* Restore legacy excell reader

* Remove clear method

* Remove clear test

* Update read/write centroids in hazard

* Support empty on land or region id

* Check for empty region id

* Set region id correctly

* Remove scheduler

* Add docstring

* Add legacy read hdf5

* Replace incorrect dot product

* Remove meta in assign centroids

* Remove matlab test file

* Update set lat/lon

* Update test with init

* Add note on method

* Add from_meta class method

* Update centroids init

* Update read raster intensity / fraction data

* undo changelog duplications

* Update test for points outside of raster within threshold

* Docstring and cosmetics

* Add to default crs method

* Fix legacy from hdf5 for empty extra values

* Update legacy from_hdf5 to exclude 'latitude'/'longitude' from extra

* Update centroids.select to work properly with mask and indices

* Fix forecast translate bug

* Remove _set_centroids

* Remove reproject raster

* Remove test hazard raster

* Update naming for plot fraction centroids

* Replace HAZ_DEMO_MAT with HAZ_TEST_TC

* Replace mat file with hdf5 test file

* Rewrite write raster method for hazard

* Add a default for sel_cen in mask

* Correct indent typos

* Change duplicate test function name

* Replace mat file with hdf5

* Remove unused mat file import

* Add legacy code to read old centroid hdf5 files.

* Update access to centroids dist coast in gdf

* test_trop_cyclone: get test centroids from data api

* Add method to estimate meta raster from centroids

* Remove unecessary conversion to ne_geom

* Add good lat/lon values for unit test

* centroids.centr.write_hdf5: eliminate side effect

* centroids.centr.write_hdf5: elimination of side effect futile for pandas>=2.1

* centroids.test.TestCentroidsFuns.test_select_pass: adapt to downsized LAT/LON arrays

* test_vec_ras: fix TestCentroids.test_centroids_check_pass

* test_vec_ras: remove TestReader.test_write_read_points_h5 as it is redundant
to and covered by test_centr.TestCentroidsWriter.test_read_write_hdf5

* fix typo in from_exposures method

* centroids.centr.from_csv: write class method

* centroids.centr.from_csv: correct order of xy

* centroids.centr.from_excel: update method for gdf

* centroids.centr. update csv and excel methods

* Update tests in test_vec_ras

* Update more tests and remove some with meta

* centroids.centr.from_excel fix columnn indexing

* Add possibility to NOT recenter crs for countires.

This lead to a bug in the assign region id method because longitude of the countries geometry was shifted, but not the entry lat/lon points.

* Add to_csre convenience method

* Prepare the stage for setting other than admin0 region id

* Remove uneeded import

* cenctroids.centr: fix the obvious

* gencenctroids.centr: change crs of the geodataframe instead of the geometry

* white space cosmetics

* PEP8

* Add solid region id and on land tests

* Auto stash before merge of "feature/centroids_as_gdf" and "origin/feature/centroids_as_gdf"

* Update distance and area tests

* Update set crs in load vector shape file

* centr.Centroids.from_excel: deal with region_id column

* Fix write hazard raster with centroids.get_meta

* Area pixel now correctly uses CEA

* Add actual NE CRS test.

* Make on_land and region_id always part of centroids

* Update docstring

* Update import path

* Add overwrite argument to set region id and on land

* Remove from base grid method

* Fix some linting

* Add some docstrings

* Avoid costly computation in from geodataframe

* Add comment

* Fix typo

* Fix set crs in from geodataframe

* Make properties return arrays and not series

* Remove not needed matlab vars

* Revert return numpy arrays.

* update from excel

* update from_excel and from_csv

* centroids.centr: from_excel fix column renaming and adapt to "new argument names"

* Centroids.from_excel fix: switch key,val in test_storm_europe rather than in centr!

* Return numpy arrays instead of series

* Add some basic tests

* Add tests for meta

* Add tentative changelog.

* Make keywords argument only : name lat/lon : remove set_*

* Auto stash before revert of "Make keywords argument only : name lat/lon : remove set_*"

* remove file

* Fix exposures method test

* Fix setting region id and on land at init.

* Fix typo

* Fix all close values for big numbers

* Improve cosmetics.

* update docstrings, doc cosmetics

* update from_csv method, add test

* remove DEF_VAR_CSV

* Update from_excel method and test

* remove unnecessary import

* fix storm_europe tests

* improve csv and excel tests

* Apply suggestions from code review

Co-authored-by: Lukas Riedel <34276446+peanutfun@users.noreply.github.com>

* add excel, csv write methods and tests

* Replace gdf with _gdf

* Revert "Replace gdf with _gdf"

This reverts commit c7e6ffd.

* Update to_crs methods

* Raise error if wrong exposures

* Make from geodataframe more restrictive.

* Use consistently to_crs method

* Improve read centroids

* Add literal to import

* Update to crs with inplace argument

* improve excel, csv method, test based on review

* Update typing

* Fix equal centroids for different gdf columns ordering

* Make consistent gdf column ordering

* Simplify from excel/csv

* Add support for legacy hazard excell

* Avoid single column loading error for df

* fix path in write_excel, write_csv

* fixing docstrings in centr.py

* Update code cosmetics

* Fix typo

* Allo for kwargs in init and update from exposures

* code linting

* Centroids: test from_meta

* Centroids: refined tests

* Centroids: fix from_exposures

* Centroids: implement get_pixel_shapes

* Centroids: code linting and docstrings

* hazard.test.test_base: remove undefined classes from main

* hazard.base: remove unused imports

* hazard.base: clean up imports

* cosmetics

* hazard.base.write_htdf5: add inline comment.

* Centroids: fix _gdf_from_legacy_hdf signature

* Centroids._gdf_from_legacy_hdf5: static, not classmethod

* hazard.centroids.centr: pylint

* centroids.centr : fix pydoc typos and pylint

* climada.hazard.base: pylint

* hazard.centroids.centr: pylint

* climada.test.test_calicbration: use hzard test file with hdf5 format

* Update climada/hazard/centroids/test/test_vec_ras.py

Co-authored-by: Emanuel Schmid <51439563+emanuel-schmid@users.noreply.github.com>

* Hazard: fix write_raster, and some docstrings

* test.test_calibration: fix test file name

* test_api_client: added test for basic centroids plotting
(just because it fails in PR #787)

* centr.Centroids.plot: all changes reverted
(with the exception of the obsolete `self.set_meta_to_lat_lon()`)

* Hazard tutorial: use centroids.get_meta() instead of the attribute meta

* Hazard tutorial: use Centroids() instead of from_lat_lon()

* hazard tutorial: update Hazard from raster section

* hazard tutorial: fix dist_coast access

* climada.hazard.Hazard: remove broken vector file support

* split hazard.base module into base, base_io and base_plot

* test_base_xarray: logs format changed due to "sub-modules"

* climada.hazard.HazardIO: remove broken vector file support

* hazard.base_io: remove from_mat, move reproject_vector back to base

* rename test_base_xarray

* hazard.test: move TestReadExcel and TestReadHDF5 to test_base_io

* climada.hazard.test.base/_io: fix imports

* climada.hazard.centr: add deprecated methods section

* fix from_tracks method, remove side effect

* climada.hazard.trop_cyclone.TropCytlone.from_tracks: option for predefined dist_coast in centroids

* TropicalCyclone.from_tracks pydoc

* white space cosmetics

* CHANGELOG: list deprecated and removed methods

* Refactor and add to centroids testing

* trop_cyclone.TropCyclone.from_tracks: fix dist_coast type

* climada.hazard.Centroids: raise exception in deprecated methods that really _are_ failing now

* redo lost commits on hazard.base

* changelog: list added Centroids methods

* hazard.centr: depreecated methods: pydoc string instaed of pass

* fixing geodataframe issues

* Revert "fixing geodataframe issues"

This reverts commit d6d2859.

* Changelog and module renaming

* Changelog and module renaming

* module renaming

---------

Co-authored-by: Chahan Kropf <chahan.kropf@posteo.com>
Co-authored-by: Sarah Hülsen <49907095+sarah-hlsn@users.noreply.github.com>
Co-authored-by: Igor Detring <igor.detring@dwd.de>
Co-authored-by: Chahan M. Kropf <chahan.kropf@usys.ethz.ch>
Co-authored-by: Lukas Riedel <34276446+peanutfun@users.noreply.github.com>
Co-authored-by: Thomas Vogt <thomas.vogt@pik-potsdam.de>
Co-authored-by: Thomas Vogt <57705593+tovogt@users.noreply.github.com>

emanuel-schmid added a commit that referenced this pull request

Jul 9, 2024
* Add geodataframe to centroids: first commit

* Remove raster methods, obsolete methods, and fix some.

* Remove test for removed methods

* Fix from geodataframe method

* Remove set meta to lat lon

* Update all I/O methods

* Remove area pixel

* Add method to compute the area per pixel (not attribute)

* Make get_area_pixel

* Remove not often used dist_coast and elevation

* Remove unused meta

* Change Centroids call to new signature

* Remove vector - raster hazard methods

* Restore legacy excell reader

* Remove clear method

* Remove clear test

* Update read/write centroids in hazard

* Support empty on land or region id

* Check for empty region id

* Set region id correctly

* Remove scheduler

* Add docstring

* Add legacy read hdf5

* Replace incorrect dot product

* Remove meta in assign centroids

* Remove matlab test file

* Update set lat/lon

* Update test with init

* Add note on method

* Add from_meta class method

* Update centroids init

* Update read raster intensity / fraction data

* undo changelog duplications

* Update test for points outside of raster within threshold

* Docstring and cosmetics

* Add to default crs method

* Fix legacy from hdf5 for empty extra values

* Update legacy from_hdf5 to exclude 'latitude'/'longitude' from extra

* Update centroids.select to work properly with mask and indices

* Fix forecast translate bug

* Remove _set_centroids

* Remove reproject raster

* Remove test hazard raster

* Update naming for plot fraction centroids

* Replace HAZ_DEMO_MAT with HAZ_TEST_TC

* Replace mat file with hdf5 test file

* Rewrite write raster method for hazard

* Add a default for sel_cen in mask

* Correct indent typos

* Change duplicate test function name

* Replace mat file with hdf5

* Remove unused mat file import

* Add legacy code to read old centroid hdf5 files.

* Update access to centroids dist coast in gdf

* test_trop_cyclone: get test centroids from data api

* Add method to estimate meta raster from centroids

* Remove unecessary conversion to ne_geom

* Add good lat/lon values for unit test

* centroids.centr.write_hdf5: eliminate side effect

* centroids.centr.write_hdf5: elimination of side effect futile for pandas>=2.1

* centroids.test.TestCentroidsFuns.test_select_pass: adapt to downsized LAT/LON arrays

* test_vec_ras: fix TestCentroids.test_centroids_check_pass

* test_vec_ras: remove TestReader.test_write_read_points_h5 as it is redundant
to and covered by test_centr.TestCentroidsWriter.test_read_write_hdf5

* fix typo in from_exposures method

* centroids.centr.from_csv: write class method

* centroids.centr.from_csv: correct order of xy

* centroids.centr.from_excel: update method for gdf

* centroids.centr. update csv and excel methods

* Update tests in test_vec_ras

* Update more tests and remove some with meta

* centroids.centr.from_excel fix columnn indexing

* Add possibility to NOT recenter crs for countires.

This lead to a bug in the assign region id method because longitude of the countries geometry was shifted, but not the entry lat/lon points.

* Add to_csre convenience method

* Prepare the stage for setting other than admin0 region id

* Remove uneeded import

* cenctroids.centr: fix the obvious

* gencenctroids.centr: change crs of the geodataframe instead of the geometry

* white space cosmetics

* PEP8

* Add solid region id and on land tests

* Auto stash before merge of "feature/centroids_as_gdf" and "origin/feature/centroids_as_gdf"

* Update distance and area tests

* Update set crs in load vector shape file

* centr.Centroids.from_excel: deal with region_id column

* Fix write hazard raster with centroids.get_meta

* Area pixel now correctly uses CEA

* Add actual NE CRS test.

* Make on_land and region_id always part of centroids

* Update docstring

* Update import path

* Add overwrite argument to set region id and on land

* Remove from base grid method

* Fix some linting

* Add some docstrings

* Avoid costly computation in from geodataframe

* Add comment

* Fix typo

* Fix set crs in from geodataframe

* Make properties return arrays and not series

* Remove not needed matlab vars

* Revert return numpy arrays.

* update from excel

* update from_excel and from_csv

* centroids.centr: from_excel fix column renaming and adapt to "new argument names"

* Centroids.from_excel fix: switch key,val in test_storm_europe rather than in centr!

* Return numpy arrays instead of series

* Add some basic tests

* Add tests for meta

* Add tentative changelog.

* Make keywords argument only : name lat/lon : remove set_*

* Auto stash before revert of "Make keywords argument only : name lat/lon : remove set_*"

* remove file

* Fix exposures method test

* Fix setting region id and on land at init.

* Fix typo

* Fix all close values for big numbers

* Improve cosmetics.

* update docstrings, doc cosmetics

* update from_csv method, add test

* remove DEF_VAR_CSV

* Update from_excel method and test

* remove unnecessary import

* fix storm_europe tests

* improve csv and excel tests

* Apply suggestions from code review

Co-authored-by: Lukas Riedel <34276446+peanutfun@users.noreply.github.com>

* add excel, csv write methods and tests

* Replace gdf with _gdf

* Revert "Replace gdf with _gdf"

This reverts commit c7e6ffd.

* Update to_crs methods

* Raise error if wrong exposures

* Make from geodataframe more restrictive.

* Use consistently to_crs method

* Improve read centroids

* Add literal to import

* Update to crs with inplace argument

* improve excel, csv method, test based on review

* Update typing

* Fix equal centroids for different gdf columns ordering

* Make consistent gdf column ordering

* Simplify from excel/csv

* Add support for legacy hazard excell

* Avoid single column loading error for df

* fix path in write_excel, write_csv

* fixing docstrings in centr.py

* Update code cosmetics

* Fix typo

* Allo for kwargs in init and update from exposures

* code linting

* Centroids: test from_meta

* Centroids: refined tests

* Centroids: fix from_exposures

* Centroids: implement get_pixel_shapes

* Centroids: code linting and docstrings

* hazard.test.test_base: remove undefined classes from main

* hazard.base: remove unused imports

* hazard.base: clean up imports

* cosmetics

* hazard.base.write_htdf5: add inline comment.

* Centroids: fix _gdf_from_legacy_hdf signature

* Centroids._gdf_from_legacy_hdf5: static, not classmethod

* hazard.centroids.centr: pylint

* centroids.centr : fix pydoc typos and pylint

* climada.hazard.base: pylint

* hazard.centroids.centr: pylint

* climada.test.test_calicbration: use hzard test file with hdf5 format

* Update climada/hazard/centroids/test/test_vec_ras.py

Co-authored-by: Emanuel Schmid <51439563+emanuel-schmid@users.noreply.github.com>

* Hazard: fix write_raster, and some docstrings

* test.test_calibration: fix test file name

* test_api_client: added test for basic centroids plotting
(just because it fails in PR #787)

* centr.Centroids.plot: all changes reverted
(with the exception of the obsolete `self.set_meta_to_lat_lon()`)

* Hazard tutorial: use centroids.get_meta() instead of the attribute meta

* Hazard tutorial: use Centroids() instead of from_lat_lon()

* hazard tutorial: update Hazard from raster section

* hazard tutorial: fix dist_coast access

* climada.hazard.Hazard: remove broken vector file support

* climada.hazard.centr: add deprecated methods section

* fix from_tracks method, remove side effect

* climada.hazard.trop_cyclone.TropCytlone.from_tracks: option for predefined dist_coast in centroids

* TropicalCyclone.from_tracks pydoc

* white space cosmetics

* CHANGELOG: list deprecated and removed methods

* Refactor and add to centroids testing

* trop_cyclone.TropCyclone.from_tracks: fix dist_coast type

* climada.hazard.Centroids: raise exception in deprecated methods that really _are_ failing now

* changelog: list added Centroids methods

* hazard.centr: depreecated methods: pydoc string instaed of pass

* fixing geodataframe issues

* Revert "fixing geodataframe issues"

This reverts commit d6d2859.

* proposed fix for set_geometry warning

* u_coord: use set_geometry instead of direct column assignment for GeoDataFrames

* white space cosmetics

* simplification

---------

Co-authored-by: Chahan Kropf <chahan.kropf@posteo.com>
Co-authored-by: Sarah Hülsen <49907095+sarah-hlsn@users.noreply.github.com>
Co-authored-by: Igor Detring <igor.detring@dwd.de>
Co-authored-by: Chahan M. Kropf <chahan.kropf@usys.ethz.ch>
Co-authored-by: Lukas Riedel <34276446+peanutfun@users.noreply.github.com>
Co-authored-by: Thomas Vogt <thomas.vogt@pik-potsdam.de>
Co-authored-by: Thomas Vogt <57705593+tovogt@users.noreply.github.com>

emanuel-schmid added a commit that referenced this pull request

Jul 17, 2024
* Add geodataframe to centroids: first commit

* Remove raster methods, obsolete methods, and fix some.

* Remove test for removed methods

* Fix from geodataframe method

* Remove set meta to lat lon

* Update all I/O methods

* Remove area pixel

* Add method to compute the area per pixel (not attribute)

* Make get_area_pixel

* Remove not often used dist_coast and elevation

* Remove unused meta

* Change Centroids call to new signature

* Remove vector - raster hazard methods

* Restore legacy excell reader

* Remove clear method

* Remove clear test

* Update read/write centroids in hazard

* Support empty on land or region id

* Check for empty region id

* Set region id correctly

* Remove scheduler

* Add docstring

* Add legacy read hdf5

* Replace incorrect dot product

* Remove meta in assign centroids

* Remove matlab test file

* Update set lat/lon

* Update test with init

* Add note on method

* Add from_meta class method

* Update centroids init

* Update read raster intensity / fraction data

* undo changelog duplications

* Update test for points outside of raster within threshold

* Docstring and cosmetics

* Add to default crs method

* Fix legacy from hdf5 for empty extra values

* Update legacy from_hdf5 to exclude 'latitude'/'longitude' from extra

* Update centroids.select to work properly with mask and indices

* Fix forecast translate bug

* Remove _set_centroids

* Remove reproject raster

* Remove test hazard raster

* Update naming for plot fraction centroids

* Replace HAZ_DEMO_MAT with HAZ_TEST_TC

* Replace mat file with hdf5 test file

* Rewrite write raster method for hazard

* Add a default for sel_cen in mask

* Correct indent typos

* Change duplicate test function name

* Replace mat file with hdf5

* Remove unused mat file import

* Add legacy code to read old centroid hdf5 files.

* Update access to centroids dist coast in gdf

* test_trop_cyclone: get test centroids from data api

* Add method to estimate meta raster from centroids

* Remove unecessary conversion to ne_geom

* Add good lat/lon values for unit test

* centroids.centr.write_hdf5: eliminate side effect

* centroids.centr.write_hdf5: elimination of side effect futile for pandas>=2.1

* centroids.test.TestCentroidsFuns.test_select_pass: adapt to downsized LAT/LON arrays

* test_vec_ras: fix TestCentroids.test_centroids_check_pass

* test_vec_ras: remove TestReader.test_write_read_points_h5 as it is redundant
to and covered by test_centr.TestCentroidsWriter.test_read_write_hdf5

* fix typo in from_exposures method

* centroids.centr.from_csv: write class method

* centroids.centr.from_csv: correct order of xy

* centroids.centr.from_excel: update method for gdf

* centroids.centr. update csv and excel methods

* Update tests in test_vec_ras

* Update more tests and remove some with meta

* centroids.centr.from_excel fix columnn indexing

* Add possibility to NOT recenter crs for countires.

This lead to a bug in the assign region id method because longitude of the countries geometry was shifted, but not the entry lat/lon points.

* Add to_csre convenience method

* Prepare the stage for setting other than admin0 region id

* Remove uneeded import

* cenctroids.centr: fix the obvious

* gencenctroids.centr: change crs of the geodataframe instead of the geometry

* white space cosmetics

* PEP8

* Add solid region id and on land tests

* Auto stash before merge of "feature/centroids_as_gdf" and "origin/feature/centroids_as_gdf"

* Update distance and area tests

* Update set crs in load vector shape file

* centr.Centroids.from_excel: deal with region_id column

* Fix write hazard raster with centroids.get_meta

* Area pixel now correctly uses CEA

* Add actual NE CRS test.

* Make on_land and region_id always part of centroids

* Update docstring

* Update import path

* Add overwrite argument to set region id and on land

* Remove from base grid method

* Fix some linting

* Add some docstrings

* Avoid costly computation in from geodataframe

* Add comment

* Fix typo

* Fix set crs in from geodataframe

* Make properties return arrays and not series

* Remove not needed matlab vars

* Revert return numpy arrays.

* update from excel

* update from_excel and from_csv

* centroids.centr: from_excel fix column renaming and adapt to "new argument names"

* Centroids.from_excel fix: switch key,val in test_storm_europe rather than in centr!

* Return numpy arrays instead of series

* Add some basic tests

* Add tests for meta

* Add tentative changelog.

* Make keywords argument only : name lat/lon : remove set_*

* Auto stash before revert of "Make keywords argument only : name lat/lon : remove set_*"

* remove file

* Fix exposures method test

* Fix setting region id and on land at init.

* Fix typo

* Fix all close values for big numbers

* Improve cosmetics.

* update docstrings, doc cosmetics

* update from_csv method, add test

* remove DEF_VAR_CSV

* Update from_excel method and test

* remove unnecessary import

* fix storm_europe tests

* improve csv and excel tests

* Apply suggestions from code review

Co-authored-by: Lukas Riedel <34276446+peanutfun@users.noreply.github.com>

* add excel, csv write methods and tests

* Replace gdf with _gdf

* Revert "Replace gdf with _gdf"

This reverts commit c7e6ffd.

* Update to_crs methods

* Raise error if wrong exposures

* Make from geodataframe more restrictive.

* Use consistently to_crs method

* Improve read centroids

* Add literal to import

* Update to crs with inplace argument

* improve excel, csv method, test based on review

* Update typing

* Fix equal centroids for different gdf columns ordering

* Make consistent gdf column ordering

* Simplify from excel/csv

* Add support for legacy hazard excell

* Avoid single column loading error for df

* fix path in write_excel, write_csv

* fixing docstrings in centr.py

* Update code cosmetics

* Fix typo

* Allo for kwargs in init and update from exposures

* code linting

* Centroids: test from_meta

* Centroids: refined tests

* Centroids: fix from_exposures

* Centroids: implement get_pixel_shapes

* Centroids: code linting and docstrings

* hazard.test.test_base: remove undefined classes from main

* hazard.base: remove unused imports

* hazard.base: clean up imports

* cosmetics

* hazard.base.write_htdf5: add inline comment.

* Centroids: fix _gdf_from_legacy_hdf signature

* Centroids._gdf_from_legacy_hdf5: static, not classmethod

* hazard.centroids.centr: pylint

* centroids.centr : fix pydoc typos and pylint

* climada.hazard.base: pylint

* hazard.centroids.centr: pylint

* climada.test.test_calicbration: use hzard test file with hdf5 format

* Update climada/hazard/centroids/test/test_vec_ras.py

Co-authored-by: Emanuel Schmid <51439563+emanuel-schmid@users.noreply.github.com>

* Hazard: fix write_raster, and some docstrings

* test.test_calibration: fix test file name

* test_api_client: added test for basic centroids plotting
(just because it fails in PR #787)

* centr.Centroids.plot: all changes reverted
(with the exception of the obsolete `self.set_meta_to_lat_lon()`)

* Hazard tutorial: use centroids.get_meta() instead of the attribute meta

* Hazard tutorial: use Centroids() instead of from_lat_lon()

* hazard tutorial: update Hazard from raster section

* hazard tutorial: fix dist_coast access

* climada.hazard.Hazard: remove broken vector file support

* climada.hazard.centr: add deprecated methods section

* fix from_tracks method, remove side effect

* climada.hazard.trop_cyclone.TropCytlone.from_tracks: option for predefined dist_coast in centroids

* TropicalCyclone.from_tracks pydoc

* white space cosmetics

* CHANGELOG: list deprecated and removed methods

* Refactor and add to centroids testing

* trop_cyclone.TropCyclone.from_tracks: fix dist_coast type

* climada.hazard.Centroids: raise exception in deprecated methods that really _are_ failing now

* changelog: list added Centroids methods

* hazard.centr: depreecated methods: pydoc string instaed of pass

* fixing geodataframe issues

* Revert "fixing geodataframe issues"

This reverts commit d6d2859.

* proposed fix for set_geometry warning

* u_coord: use set_geometry instead of direct column assignment for GeoDataFrames

* white space cosmetics

* skip scheduler in set_df_geometry_points

* simplification

* cosmetics

* lint

Co-authored-by: Lukas Riedel <34276446+peanutfun@users.noreply.github.com>

* ws cosmetics

* Update climada/util/coordinates.py

Co-authored-by: Lukas Riedel <34276446+peanutfun@users.noreply.github.com>

* warn through warnings not through logs, fix deprecation version

* remove obsolete multi_processing test, make test_points_to_raster mutltiprocessing instead

* syntax fix

* coordinates, dask may or not return a Polygon array (could be str)

* changelog and code comments

---------

Co-authored-by: Chahan Kropf <chahan.kropf@posteo.com>
Co-authored-by: Sarah Hülsen <49907095+sarah-hlsn@users.noreply.github.com>
Co-authored-by: Igor Detring <igor.detring@dwd.de>
Co-authored-by: Chahan M. Kropf <chahan.kropf@usys.ethz.ch>
Co-authored-by: Lukas Riedel <34276446+peanutfun@users.noreply.github.com>
Co-authored-by: Thomas Vogt <thomas.vogt@pik-potsdam.de>
Co-authored-by: Thomas Vogt <57705593+tovogt@users.noreply.github.com>

gdeskos pushed a commit to gdeskos/climada_python that referenced this pull request

Sep 16, 2024
* Add geodataframe to centroids: first commit

* Remove raster methods, obsolete methods, and fix some.

* Remove test for removed methods

* Fix from geodataframe method

* Remove set meta to lat lon

* Update all I/O methods

* Remove area pixel

* Add method to compute the area per pixel (not attribute)

* Make get_area_pixel

* Remove not often used dist_coast and elevation

* Remove unused meta

* Change Centroids call to new signature

* Remove vector - raster hazard methods

* Restore legacy excell reader

* Remove clear method

* Remove clear test

* Update read/write centroids in hazard

* Support empty on land or region id

* Check for empty region id

* Set region id correctly

* Remove scheduler

* Add docstring

* Add legacy read hdf5

* Replace incorrect dot product

* Remove meta in assign centroids

* Remove matlab test file

* Update set lat/lon

* Update test with init

* 'Automated update v4.0.0'

* Add note on method

* changelog: for 4.0.1

* 'Automated update v4.0.1'

* Add from_meta class method

* Update centroids init

* Update read raster intensity / fraction data

* undo changelog duplications

* Update test for points outside of raster within threshold

* Docstring and cosmetics

* Add to default crs method

* Fix legacy from hdf5 for empty extra values

* Update legacy from_hdf5 to exclude 'latitude'/'longitude' from extra

* Update centroids.select to work properly with mask and indices

* Fix forecast translate bug

* Remove _set_centroids

* Remove reproject raster

* Remove test hazard raster

* Update naming for plot fraction centroids

* Replace HAZ_DEMO_MAT with HAZ_TEST_TC

* Replace mat file with hdf5 test file

* Rewrite write raster method for hazard

* Add a default for sel_cen in mask

* Correct indent typos

* Change duplicate test function name

* Replace mat file with hdf5

* Remove unused mat file import

* Add legacy code to read old centroid hdf5 files.

* Update access to centroids dist coast in gdf

* test_trop_cyclone: get test centroids from data api

* Add method to estimate meta raster from centroids

* Remove unecessary conversion to ne_geom

* Add good lat/lon values for unit test

* centroids.centr.write_hdf5: eliminate side effect

* centroids.centr.write_hdf5: elimination of side effect futile for pandas>=2.1

* centroids.test.TestCentroidsFuns.test_select_pass: adapt to downsized LAT/LON arrays

* test_vec_ras: fix TestCentroids.test_centroids_check_pass

* test_vec_ras: remove TestReader.test_write_read_points_h5 as it is redundant
to and covered by test_centr.TestCentroidsWriter.test_read_write_hdf5

* fix typo in from_exposures method

* centroids.centr.from_csv: write class method

* centroids.centr.from_csv: correct order of xy

* centroids.centr.from_excel: update method for gdf

* centroids.centr. update csv and excel methods

* Update tests in test_vec_ras

* Update more tests and remove some with meta

* centroids.centr.from_excel fix columnn indexing

* Add possibility to NOT recenter crs for countires.

This lead to a bug in the assign region id method because longitude of the countries geometry was shifted, but not the entry lat/lon points.

* Add to_csre convenience method

* Prepare the stage for setting other than admin0 region id

* Remove uneeded import

* cenctroids.centr: fix the obvious

* gencenctroids.centr: change crs of the geodataframe instead of the geometry

* white space cosmetics

* PEP8

* Add solid region id and on land tests

* Auto stash before merge of "feature/centroids_as_gdf" and "origin/feature/centroids_as_gdf"

* Update distance and area tests

* Update set crs in load vector shape file

* centr.Centroids.from_excel: deal with region_id column

* Fix write hazard raster with centroids.get_meta

* Area pixel now correctly uses CEA

* Add actual NE CRS test.

* Make on_land and region_id always part of centroids

* Update docstring

* Update import path

* Add overwrite argument to set region id and on land

* Remove from base grid method

* Fix some linting

* Add some docstrings

* Avoid costly computation in from geodataframe

* Add comment

* Fix typo

* Fix set crs in from geodataframe

* Make properties return arrays and not series

* Remove not needed matlab vars

* Revert return numpy arrays.

* update from excel

* update from_excel and from_csv

* centroids.centr: from_excel fix column renaming and adapt to "new argument names"

* Centroids.from_excel fix: switch key,val in test_storm_europe rather than in centr!

* Return numpy arrays instead of series

* Add some basic tests

* Add tests for meta

* Add tentative changelog.

* Make keywords argument only : name lat/lon : remove set_*

* Auto stash before revert of "Make keywords argument only : name lat/lon : remove set_*"

* remove file

* Fix exposures method test

* Fix setting region id and on land at init.

* Fix typo

* Fix all close values for big numbers

* Improve cosmetics.

* update docstrings, doc cosmetics

* update from_csv method, add test

* remove DEF_VAR_CSV

* Update from_excel method and test

* remove unnecessary import

* fix storm_europe tests

* improve csv and excel tests

* Apply suggestions from code review

Co-authored-by: Lukas Riedel <34276446+peanutfun@users.noreply.github.com>

* add excel, csv write methods and tests

* Replace gdf with _gdf

* Revert "Replace gdf with _gdf"

This reverts commit c7e6ffd.

* Update to_crs methods

* Raise error if wrong exposures

* Make from geodataframe more restrictive.

* Use consistently to_crs method

* Improve read centroids

* Add literal to import

* Update to crs with inplace argument

* improve excel, csv method, test based on review

* Update typing

* Fix equal centroids for different gdf columns ordering

* Make consistent gdf column ordering

* Simplify from excel/csv

* Add support for legacy hazard excell

* Avoid single column loading error for df

* fix path in write_excel, write_csv

* fixing docstrings in centr.py

* Update code cosmetics

* Fix typo

* Allo for kwargs in init and update from exposures

* code linting

* Centroids: test from_meta

* Centroids: refined tests

* Centroids: fix from_exposures

* Centroids: implement get_pixel_shapes

* Centroids: code linting and docstrings

* hazard.test.test_base: remove undefined classes from main

* hazard.base: remove unused imports

* hazard.base: clean up imports

* cosmetics

* hazard.base.write_htdf5: add inline comment.

* Centroids: fix _gdf_from_legacy_hdf signature

* Centroids._gdf_from_legacy_hdf5: static, not classmethod

* hazard.centroids.centr: pylint

* centroids.centr : fix pydoc typos and pylint

* climada.hazard.base: pylint

* hazard.centroids.centr: pylint

* climada.test.test_calicbration: use hzard test file with hdf5 format

* Update climada/hazard/centroids/test/test_vec_ras.py

Co-authored-by: Emanuel Schmid <51439563+emanuel-schmid@users.noreply.github.com>

* Hazard: fix write_raster, and some docstrings

* test.test_calibration: fix test file name

* test_api_client: added test for basic centroids plotting
(just because it fails in PR CLIMADA-project#787)

* centr.Centroids.plot: all changes reverted
(with the exception of the obsolete `self.set_meta_to_lat_lon()`)

* Hazard tutorial: use centroids.get_meta() instead of the attribute meta

* Hazard tutorial: use Centroids() instead of from_lat_lon()

* hazard tutorial: update Hazard from raster section

* hazard tutorial: fix dist_coast access

* climada.hazard.Hazard: remove broken vector file support

* climada.hazard.centr: add deprecated methods section

* fix from_tracks method, remove side effect

* climada.hazard.trop_cyclone.TropCytlone.from_tracks: option for predefined dist_coast in centroids

* TropicalCyclone.from_tracks pydoc

* white space cosmetics

* CHANGELOG: list deprecated and removed methods

* Refactor and add to centroids testing

* trop_cyclone.TropCyclone.from_tracks: fix dist_coast type

* climada.hazard.Centroids: raise exception in deprecated methods that really _are_ failing now

* changelog: list added Centroids methods

* hazard.centr: depreecated methods: pydoc string instaed of pass

* fixing geodataframe issues

* Revert "fixing geodataframe issues"

This reverts commit d6d2859.

* centroids: deprecation warnings improved

---------

Co-authored-by: Chahan Kropf <chahan.kropf@posteo.com>
Co-authored-by: climada <test.climada@gmail.com>
Co-authored-by: emanuel-schmid <schmide@ethz.ch>
Co-authored-by: Sarah Hülsen <49907095+sarah-hlsn@users.noreply.github.com>
Co-authored-by: Igor Detring <igor.detring@dwd.de>
Co-authored-by: Lukas Riedel <34276446+peanutfun@users.noreply.github.com>
Co-authored-by: Thomas Vogt <thomas.vogt@pik-potsdam.de>
Co-authored-by: Thomas Vogt <57705593+tovogt@users.noreply.github.com>
Co-authored-by: Emanuel Schmid <51439563+emanuel-schmid@users.noreply.github.com>

gdeskos pushed a commit to gdeskos/climada_python that referenced this pull request

Sep 16, 2024
* Add geodataframe to centroids: first commit

* Remove raster methods, obsolete methods, and fix some.

* Remove test for removed methods

* Fix from geodataframe method

* Remove set meta to lat lon

* Update all I/O methods

* Remove area pixel

* Add method to compute the area per pixel (not attribute)

* Make get_area_pixel

* Remove not often used dist_coast and elevation

* Remove unused meta

* Change Centroids call to new signature

* Remove vector - raster hazard methods

* Restore legacy excell reader

* Remove clear method

* Remove clear test

* Update read/write centroids in hazard

* Support empty on land or region id

* Check for empty region id

* Set region id correctly

* Remove scheduler

* Add docstring

* Add legacy read hdf5

* Replace incorrect dot product

* Remove meta in assign centroids

* Remove matlab test file

* Update set lat/lon

* Update test with init

* Add note on method

* Add from_meta class method

* Update centroids init

* Update read raster intensity / fraction data

* undo changelog duplications

* Update test for points outside of raster within threshold

* Docstring and cosmetics

* Add to default crs method

* Fix legacy from hdf5 for empty extra values

* Update legacy from_hdf5 to exclude 'latitude'/'longitude' from extra

* Update centroids.select to work properly with mask and indices

* Fix forecast translate bug

* Remove _set_centroids

* Remove reproject raster

* Remove test hazard raster

* Update naming for plot fraction centroids

* Replace HAZ_DEMO_MAT with HAZ_TEST_TC

* Replace mat file with hdf5 test file

* Rewrite write raster method for hazard

* Add a default for sel_cen in mask

* Correct indent typos

* Change duplicate test function name

* Replace mat file with hdf5

* Remove unused mat file import

* Add legacy code to read old centroid hdf5 files.

* Update access to centroids dist coast in gdf

* test_trop_cyclone: get test centroids from data api

* Add method to estimate meta raster from centroids

* Remove unecessary conversion to ne_geom

* Add good lat/lon values for unit test

* centroids.centr.write_hdf5: eliminate side effect

* centroids.centr.write_hdf5: elimination of side effect futile for pandas>=2.1

* centroids.test.TestCentroidsFuns.test_select_pass: adapt to downsized LAT/LON arrays

* test_vec_ras: fix TestCentroids.test_centroids_check_pass

* test_vec_ras: remove TestReader.test_write_read_points_h5 as it is redundant
to and covered by test_centr.TestCentroidsWriter.test_read_write_hdf5

* fix typo in from_exposures method

* centroids.centr.from_csv: write class method

* centroids.centr.from_csv: correct order of xy

* centroids.centr.from_excel: update method for gdf

* centroids.centr. update csv and excel methods

* Update tests in test_vec_ras

* Update more tests and remove some with meta

* centroids.centr.from_excel fix columnn indexing

* Add possibility to NOT recenter crs for countires.

This lead to a bug in the assign region id method because longitude of the countries geometry was shifted, but not the entry lat/lon points.

* Add to_csre convenience method

* Prepare the stage for setting other than admin0 region id

* Remove uneeded import

* cenctroids.centr: fix the obvious

* gencenctroids.centr: change crs of the geodataframe instead of the geometry

* white space cosmetics

* PEP8

* Add solid region id and on land tests

* Auto stash before merge of "feature/centroids_as_gdf" and "origin/feature/centroids_as_gdf"

* Update distance and area tests

* Update set crs in load vector shape file

* centr.Centroids.from_excel: deal with region_id column

* Fix write hazard raster with centroids.get_meta

* Area pixel now correctly uses CEA

* Add actual NE CRS test.

* Make on_land and region_id always part of centroids

* Update docstring

* Update import path

* Add overwrite argument to set region id and on land

* Remove from base grid method

* Fix some linting

* Add some docstrings

* Avoid costly computation in from geodataframe

* Add comment

* Fix typo

* Fix set crs in from geodataframe

* Make properties return arrays and not series

* Remove not needed matlab vars

* Revert return numpy arrays.

* update from excel

* update from_excel and from_csv

* centroids.centr: from_excel fix column renaming and adapt to "new argument names"

* Centroids.from_excel fix: switch key,val in test_storm_europe rather than in centr!

* Return numpy arrays instead of series

* Add some basic tests

* Add tests for meta

* Add tentative changelog.

* Make keywords argument only : name lat/lon : remove set_*

* Auto stash before revert of "Make keywords argument only : name lat/lon : remove set_*"

* remove file

* Fix exposures method test

* Fix setting region id and on land at init.

* Fix typo

* Fix all close values for big numbers

* Improve cosmetics.

* update docstrings, doc cosmetics

* update from_csv method, add test

* remove DEF_VAR_CSV

* Update from_excel method and test

* remove unnecessary import

* fix storm_europe tests

* improve csv and excel tests

* Apply suggestions from code review

Co-authored-by: Lukas Riedel <34276446+peanutfun@users.noreply.github.com>

* add excel, csv write methods and tests

* Replace gdf with _gdf

* Revert "Replace gdf with _gdf"

This reverts commit c7e6ffd.

* Update to_crs methods

* Raise error if wrong exposures

* Make from geodataframe more restrictive.

* Use consistently to_crs method

* Improve read centroids

* Add literal to import

* Update to crs with inplace argument

* improve excel, csv method, test based on review

* Update typing

* Fix equal centroids for different gdf columns ordering

* Make consistent gdf column ordering

* Simplify from excel/csv

* Add support for legacy hazard excell

* Avoid single column loading error for df

* fix path in write_excel, write_csv

* fixing docstrings in centr.py

* Update code cosmetics

* Fix typo

* Allo for kwargs in init and update from exposures

* code linting

* Centroids: test from_meta

* Centroids: refined tests

* Centroids: fix from_exposures

* Centroids: implement get_pixel_shapes

* Centroids: code linting and docstrings

* hazard.test.test_base: remove undefined classes from main

* hazard.base: remove unused imports

* hazard.base: clean up imports

* cosmetics

* hazard.base.write_htdf5: add inline comment.

* Centroids: fix _gdf_from_legacy_hdf signature

* Centroids._gdf_from_legacy_hdf5: static, not classmethod

* hazard.centroids.centr: pylint

* centroids.centr : fix pydoc typos and pylint

* climada.hazard.base: pylint

* hazard.centroids.centr: pylint

* climada.test.test_calicbration: use hzard test file with hdf5 format

* Update climada/hazard/centroids/test/test_vec_ras.py

Co-authored-by: Emanuel Schmid <51439563+emanuel-schmid@users.noreply.github.com>

* Hazard: fix write_raster, and some docstrings

* test.test_calibration: fix test file name

* test_api_client: added test for basic centroids plotting
(just because it fails in PR CLIMADA-project#787)

* centr.Centroids.plot: all changes reverted
(with the exception of the obsolete `self.set_meta_to_lat_lon()`)

* Hazard tutorial: use centroids.get_meta() instead of the attribute meta

* Hazard tutorial: use Centroids() instead of from_lat_lon()

* hazard tutorial: update Hazard from raster section

* hazard tutorial: fix dist_coast access

* climada.hazard.Hazard: remove broken vector file support

* split hazard.base module into base, base_io and base_plot

* test_base_xarray: logs format changed due to "sub-modules"

* climada.hazard.HazardIO: remove broken vector file support

* hazard.base_io: remove from_mat, move reproject_vector back to base

* rename test_base_xarray

* hazard.test: move TestReadExcel and TestReadHDF5 to test_base_io

* climada.hazard.test.base/_io: fix imports

* climada.hazard.centr: add deprecated methods section

* fix from_tracks method, remove side effect

* climada.hazard.trop_cyclone.TropCytlone.from_tracks: option for predefined dist_coast in centroids

* TropicalCyclone.from_tracks pydoc

* white space cosmetics

* CHANGELOG: list deprecated and removed methods

* Refactor and add to centroids testing

* trop_cyclone.TropCyclone.from_tracks: fix dist_coast type

* climada.hazard.Centroids: raise exception in deprecated methods that really _are_ failing now

* redo lost commits on hazard.base

* changelog: list added Centroids methods

* hazard.centr: depreecated methods: pydoc string instaed of pass

* fixing geodataframe issues

* Revert "fixing geodataframe issues"

This reverts commit d6d2859.

* Changelog and module renaming

* Changelog and module renaming

* module renaming

---------

Co-authored-by: Chahan Kropf <chahan.kropf@posteo.com>
Co-authored-by: Sarah Hülsen <49907095+sarah-hlsn@users.noreply.github.com>
Co-authored-by: Igor Detring <igor.detring@dwd.de>
Co-authored-by: Chahan M. Kropf <chahan.kropf@usys.ethz.ch>
Co-authored-by: Lukas Riedel <34276446+peanutfun@users.noreply.github.com>
Co-authored-by: Thomas Vogt <thomas.vogt@pik-potsdam.de>
Co-authored-by: Thomas Vogt <57705593+tovogt@users.noreply.github.com>