Project

General

Profile

Feature #9534

GeoJson store: systematically use geodataframes

Added by Philippe May almost 2 years ago. Updated almost 2 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Target version:
-
Start date:
06/01/2020
Due date:
% Done:

100%

Close

Description

Currently, there's a mix of:
- (for category/survey models): get data from DB with Geopandas and process using vectorised pandas/numpy operations
- (for custom defined models): get data from DB using gino

-> change all models to allow feature retrieving through GeoPandas.

Benefits:
- unified process
- expected bug speedups when Geopandas, Shapely and PyGeos get integrated, which will provide vectorized operations on geometries (see https://github.com/Toblerity/Shapely/issues/782 and related tickets, eg. https://github.com/pygeos/pygeos/issues/82)

Biggest refactorings:

- popup: implement get_popup_from_df and deprecate caption, get_popup
- properties: implement get_properties_from_df and deprecate get_properties
- styles (mapbox-layout and mapbox-paint): currently, they are added to GeoJson FeatureCollection object, which might not follow the standard: move to HTTP headers


Related issues

Related to Gisaf - Support #9483: Performance: writing gpkg is slower than shapefiles New 23/12/2019

Associated revisions

Revision c78a3cb2 (diff)
Added by Philippe May almost 2 years ago

Refs #9534: generalize GeoDataframes
Use gdf.to_json() (slightly faster, gygeos should bring an order of magnitude speed improvement
Implement:
- model.get_geojson_http_headers
- model.get_popup_from_df
- model.model.get_popup_from_df
- mapbox-paint and mapbox-layout move HTTP headers

Revision 702d99aa (diff)
Added by Philippe May almost 2 years ago

Refs #9534: geodataframe for geojson generation
Implement:
- get_popup_from_df
- get_properties_from_df
Exception: Well

Revision 68c1e319 (diff)
Added by Philippe May almost 2 years ago

Refs #9534: Well: use dataframe for geojson

Revision 775bd7be (diff)
Added by Philippe May almost 2 years ago

Refs #9534: cadastral: use dataframe for geojson

Revision e522ab9b (diff)
Added by Philippe May almost 2 years ago

Refs #9534: Refactoring, cleanups

Revision cfd05cef (diff)
Added by Philippe May almost 2 years ago

Refs #9534: Refactoring, cleanups

Revision f9fbac55 (diff)
Added by Philippe May almost 2 years ago

Refs #9534: update live layers
Mapbox style given in HTTP headers
Use redis stores for mapbox_paint and mapbox_layout
Fix performance goofup

Revision ece2a8e9 (diff)
Added by Philippe May almost 2 years ago

Refs #9534: fix well get_properties

Revision 35dbd145 (diff)
Added by Philippe May almost 2 years ago

Refs #9534: fix cadastral location text on map

Revision fd2a621e (diff)
Added by Philippe May almost 2 years ago

Refs #9534: fix waste water treatment plant style

Revision af95b914 (diff)
Added by Philippe May almost 2 years ago

Refs #9534: cleanup: use nice pandas string ops
instead of dirty numpy

History

#1 Updated by Philippe May almost 2 years ago

TODO: fix:
- CadastralLocation text
- wells
- optical fiber style
- geological formation style

#2 Updated by Philippe May almost 2 years ago

  • % Done changed from 0 to 100
  • Status changed from New to Resolved

#3 Updated by Philippe May almost 2 years ago

  • Related to Support #9483: Performance: writing gpkg is slower than shapefiles added

Also available in: Atom PDF