Gisaf: Integrity checks and database maintenance

Added by Philippe May 3 months ago

Integrity checks

In Gisaf, the structure of the database and the processes (eg. basket imports) ensure that basics of the quality of the data is met. However, the quality of data can be improved further: data integrity.

Each integrity check is defined as a "business rule", like a condition that filters database records. Data matching these rules are tagged, and these tags can eventually be used for correction actions (automatic, semi-automatic, manual), reports, etc.

In Gisaf, this new feature is included in the admin. The first integrity check is for determining the map features which miss some information (surveyor). The integrity check tags the features, which are then easy to spot on the online map.


An other useful feature was added: a button to "vacuum" the database. The experience shows that under some circumstances (eg. uncontrolled shutdown), Postgres internals can be affected. The symptom is very visible in Gisaf's map: the tree of layers looks almost empty, as if all the data had vanished.

No panic! A simple click on a button in the web admin interface should fix that (Manage => Integrity and maintenance => Vacuum DB).

Gisaf: Online administration

Added by Philippe May 3 months ago

The recent releases of Gisaf include online admin/management tools, that were previously dealt with tweaks on a Flask admin sibling site.

As always, the web interface is relatively minimalist and task centered, it should be more intuitive and faster for daily routine work.

The most important of these features are:

  • Baskets: with lots of refactoring and major updates under the hood, adopting more GeoPandas. Note that points based models are now imported automatically, straight from the basket to the database.
  • Survey categories: the edition of the survey categories (addition, updates, deletions) do not require system admin intervention as previously, as Gisaf now creates, updates and drop the tables in the database when categories are changed from the new interface.
  • User, roles, access (ACLs)

With these updates, it is not required anymore to have an instance of Flask-admin running specific code (with annoying duplication of the definition of models, for example) for administration tasks.

Gisaf: Web widgets

Added by Philippe May 5 months ago

Gisaf can now be used to generate widgets to be embedded in other web sites. For now, there's one for rainfall.

It's very easy: just add the following in the web site (adjust the size as needed):

<iframe src='' width="300" height="150" frameborder='0'></iframe>

It's easy to generate these widgets, typically in notebooks.

Gisaf: Speed ups and network bandwidth reduction

Added by Philippe May 5 months ago

Another round of speed ups were achieved by using a combination of several techniques: use of compression and cache.

Loading the application takes now around 1.5MB of network, compared to about 5MB.

Loading layers was also reduced drastically. Typically, compression reduces the network traffic by 4 times. In other words, it's 4 times faster.

Even better, the browser's cache is used optimally: loading the same layer again is now almost instantaneous... unless it was updated in between.

Gisaf: Summary of the updates in January 2020

Added by Philippe May 5 months ago


Any view of the map can be shared with an URL (often called a permalink). That link will open the view (lat, lon, zoom, etc), along with the layers and the base map.

Live layers

The live layers now behave almost the same as any regular (database) layer: one can export using the same menu, view information.
Beside, they also can have a description (displayed on mouse over the live layer in the layer tree), an icon, etc.

Background layer opacity

The styling of vector background wasn't pretty due to the change of opacity. Changing opacity works now as expected.

User editable base map definitions

Any user with the relative role (permission) can now edit base map layer: add new ones, change or delete existing ones. See the Create a base map with the selected layers option of the More... menu in the left panel.


The icon for categories is now displayed correctly (with the GisafSymbol font, rather than a obscure Unicode value)


All known point layers known to date have a proper icon. Beside, the layer tree (left panel) now displays icons according to the geometry type (Point, Line or Polygon) rather than an _?_.

Under the wood improvements

- Huge performance/loading speed improvements for category (survey) and live layers, lesser improvement for the custom (Misc) ones

- Improved the styling of icons in the WebUI (no more css definitions necessary)

- Made the home page in the dashboard configurable

Bug fixes

Many small bugs were fixed in the map view:

- correct resize of the map when the info panel is closed

- layers disappearing on some occasions, eg. on changing the background layer, when network requests were interrupted, etc.

- reprojection issues, causing a wrong calculation of area, length displayed in the info panel

GIS: Major update

Added by Philippe May over 1 year ago

A whole new version of the site is available online: a new look, and a deep revamp under the scene.

Few new features:

  • Dashboards, defined in Jupyter notebooks
  • Search (filter) the element displayed on the map
  • A button on the map which allows to zoom on the selected layers (i.e. eventually, only the filtered items)
  • A new admin, well integrated with the main site, but still work-in-progress: it can currently only display the content of the tables.

Note that the "old" admin, which is still required for much of the work like imports/baskets, is available at another URL: /old-admin

GIS: Data for Residential Zone

Added by Philippe May over 1 year ago

We have included the survey done so far for the Residential Zone. It includes more than 50000 survey points, and the associated line work.


    Also available in: Atom