Wiki » History » Version 69
Version 68 (Giulio Di Anastasio, 16/09/2020 10:23) → Version 69/317 (Giulio Di Anastasio, 16/10/2020 10:58)
h1. Wiki
h2. About the Auroville CSR Geomatics Studio
We are a team working on geographical and related information with an engineering and scientific approach. The concept of *geomatics* is explained here: https://en.wikipedia.org/wiki/Geomatics.
We develop, maintain and publish data on this web site: https://gis.auroville.org.in.
h3. Team
Currently, the team consists of:
* Bala
* Giulio
* Philippe
* Raj
* Ram
* Selvarani
h3. Collaborations
Quick report of the collaborations the Geomatics Team the areas of work: see [[collaborations]].
h2. Our workflow
See [[Workflow]].
h2. Working with Gisaf
h3. Survey data
Raw survey data are contained in CSV files, typically downloaded from surveying instruments.
See more information of the process for the survey data (including a flow diagram): [[Survey data]]
h3. Categories
The categories define the types of the geographical features.
See [[Categories]]
h3. QGis: work on shapefiles
Go to [[shapefiles]].
h2. Data analysis
See [[Data analysis]]
h2. Hosting
The team is located in the CSR of Auroville.
We have setup a server for hosting the software and database. See [[CSR_server]] for technical information about the setup.
h3. Backup and restoration of the database
See [[db-support]]
h2. Gear
h3. Survey equipment
See [[survey equipment]]
h3. Weather station
See [[Ambient_Weather_weather_station]]
h2. Plan for future
Beside living well, see [[plan]].
h2. Other
* [[GDAL (OGR) tools]]
h2. Links and references
[[links]]
h2. Old docs
[[Shapefiles]]
[[Data (measurements, auxiliary tables)]]
h2. Pavneet's docs (imported from gisaf's wiki)
[[Rules of Map making - What all Maps should have!]]
[[Survey Database]]
[[Field to finish]]
[[Survey Data Post-Processing]]
[[Wells Documentation]]
[[Civil 3D useful commands]]
[[Online references for Civil 3D]]
[[connections in QGIS- using browser panel and Add postGIS]]
[[Reconcilation of Raw survey data using pgAdmin]]
[[importing RAW data to GISAF]]
[[Editing Z value of features in Shapefiles in QGIS]]
[[Miscellaneous- Civil 3D]]
[[Documentation- Rain Gauge]]
[[Documentation- Wells Monitoring (Piezometer by Bala)]]
[[Documentation- Flow meter, by Bala]]
[[Documentation- DST- Vegetation Indexing]]
[[Documentation- DST- Interpolation]]
[[Documentation- DST- Survey- Office workflow]]
[[From CAD to GIS, by Giulio]]
[[QGIS- Miscellaneous]]
h2. Giulio's documentation
[[Documentation - Reconciliation of points using Gisaf]]
[[Documentation - Status and Status Changes]]
[[Documentation - Tags retained after re-import of same geometry]]
h1. Access to data
h2. Connection to server directly from CSR
To connect to the server directly without going through Aurinoco server, the correct url is
http://gis.csr.av
h2. Connection to Gisaf via QGis through WFS / OGC API
This works only on QGis from version 3.14.15 onward
In the browser, click on WFS/OGC API, then right-click to create a new connection
Give a name (e.g. OGC API Qgis Gisaf)
Give the url https://gis.auroville.org.in/ogcapi
Under the WFS Options box, on Version dropdown, the default option "Maximum" works just fine
Click on OK
The list of layers will appear in the Browser under WFS/OGC API.
h1. How to create a new projection in QGis
To create a new projection in QGis, go to menu "Settings", and click on "Custom Projections".
A pop-up window appears with a list of all projections defined in QGis projects used by the user so far.
Click on the green "+" sign on the right top part of the window to create a new projection.
In the "Name" box, type "TM CSRAUSPOS SF1" (which means TM = Transverse Mercator projection; CSRAUSPOS = theparameters for this projection are derived from the processing of DGPS raw data by AUSPOS - Online GPS Processing Service - https://www.ga.gov.au/scientific-topics/positioning-navigation/geodesy/auspos; SF1 = Scale Factor is 1).
In the "Format" dropdown list, select "Proj String (legacy - Not Recommended)"
In the "Parameters" box, paste the following "+proj=tmerc +lat_0=12.01605433+lon_0=79.80998934 +k=1 +x_0=370455.630 +y_0=1328608.994 +ellps=WGS84+towgs84=0,0,0,0,0,0,0 +units=m +no_defs".
Finally, click on OK.
In a more explicit way, the parameters mean the following:
Map Projection: TransverseMercator (TM)
False Easting: 370455.6300
False Northing: 1328608.9940
Latitude of Origin: 12°00'57.79560" (DMS) 12.01605433 (DD)
Central Meridian: 79°48'35.96164" (DMS) 79.80998934 (DD)
Scale Factor: 1.00000000
Zone Width: 6.0°
h1. Elimination of Duplicate points – General criteria
It might happen that the same physical feature (e.g. a tree, or a pole) is surveyed more than once: this can happen because there are many physical features in an area, and the survey needs more than one station. So, for example a tree is surveyed from a station, and gets a serial number on that date. When the station is then changed, it might happen that the same tree is resurveyed: another serial number is given, and possibly a different date, if the survey from the second station happened on a different day.
It is clear that the same tree is then represented with two different points, which means that two different trees exist: but only one tree really exist in the physical reality.
It is clear that one of the two points is redundant and needs to be removed. If this is noted by the surveyor directly in the field, then the issue is solved by the surveyor himself during processing time.
If instead, due to various reasons, it was not noted by the surveyor in the field, it will need to be cleaned after the processing, possibly by post-processing staff.
How to identify duplicate points?
The following criteria can be used:
1. The distance between the two points is less than 30 cm (trees are surveyed if their trunk diameter is at least about 20 cm, so in 30 cm cannot exist two of them)
2. The orig_id (serial number) of the points are not in series
3. The survey date is not the same
4. In case of trees, the species of trees is the same
5. 5. In case of trees, the tree type is not TDEF (because TDEF are mapped irrespective of their diameter, so they can actually have a small trunk, and two of them might exist in 30 cm), not OT (many TDEF species are surveyed as OT if not otherwise indicated by a botanist)
6. The context needs to be evaluated: if one tree is deleted in an area where many trees exist in a limited space, then loosing one in the map is not a big error. If instead one tree is deleted where there are very few trees, then it might be a big loss.
h2. About the Auroville CSR Geomatics Studio
We are a team working on geographical and related information with an engineering and scientific approach. The concept of *geomatics* is explained here: https://en.wikipedia.org/wiki/Geomatics.
We develop, maintain and publish data on this web site: https://gis.auroville.org.in.
h3. Team
Currently, the team consists of:
* Bala
* Giulio
* Philippe
* Raj
* Ram
* Selvarani
h3. Collaborations
Quick report of the collaborations the Geomatics Team the areas of work: see [[collaborations]].
h2. Our workflow
See [[Workflow]].
h2. Working with Gisaf
h3. Survey data
Raw survey data are contained in CSV files, typically downloaded from surveying instruments.
See more information of the process for the survey data (including a flow diagram): [[Survey data]]
h3. Categories
The categories define the types of the geographical features.
See [[Categories]]
h3. QGis: work on shapefiles
Go to [[shapefiles]].
h2. Data analysis
See [[Data analysis]]
h2. Hosting
The team is located in the CSR of Auroville.
We have setup a server for hosting the software and database. See [[CSR_server]] for technical information about the setup.
h3. Backup and restoration of the database
See [[db-support]]
h2. Gear
h3. Survey equipment
See [[survey equipment]]
h3. Weather station
See [[Ambient_Weather_weather_station]]
h2. Plan for future
Beside living well, see [[plan]].
h2. Other
* [[GDAL (OGR) tools]]
h2. Links and references
[[links]]
h2. Old docs
[[Shapefiles]]
[[Data (measurements, auxiliary tables)]]
h2. Pavneet's docs (imported from gisaf's wiki)
[[Rules of Map making - What all Maps should have!]]
[[Survey Database]]
[[Field to finish]]
[[Survey Data Post-Processing]]
[[Wells Documentation]]
[[Civil 3D useful commands]]
[[Online references for Civil 3D]]
[[connections in QGIS- using browser panel and Add postGIS]]
[[Reconcilation of Raw survey data using pgAdmin]]
[[importing RAW data to GISAF]]
[[Editing Z value of features in Shapefiles in QGIS]]
[[Miscellaneous- Civil 3D]]
[[Documentation- Rain Gauge]]
[[Documentation- Wells Monitoring (Piezometer by Bala)]]
[[Documentation- Flow meter, by Bala]]
[[Documentation- DST- Vegetation Indexing]]
[[Documentation- DST- Interpolation]]
[[Documentation- DST- Survey- Office workflow]]
[[From CAD to GIS, by Giulio]]
[[QGIS- Miscellaneous]]
h2. Giulio's documentation
[[Documentation - Reconciliation of points using Gisaf]]
[[Documentation - Status and Status Changes]]
[[Documentation - Tags retained after re-import of same geometry]]
h1. Access to data
h2. Connection to server directly from CSR
To connect to the server directly without going through Aurinoco server, the correct url is
http://gis.csr.av
h2. Connection to Gisaf via QGis through WFS / OGC API
This works only on QGis from version 3.14.15 onward
In the browser, click on WFS/OGC API, then right-click to create a new connection
Give a name (e.g. OGC API Qgis Gisaf)
Give the url https://gis.auroville.org.in/ogcapi
Under the WFS Options box, on Version dropdown, the default option "Maximum" works just fine
Click on OK
The list of layers will appear in the Browser under WFS/OGC API.
h1. How to create a new projection in QGis
To create a new projection in QGis, go to menu "Settings", and click on "Custom Projections".
A pop-up window appears with a list of all projections defined in QGis projects used by the user so far.
Click on the green "+" sign on the right top part of the window to create a new projection.
In the "Name" box, type "TM CSRAUSPOS SF1" (which means TM = Transverse Mercator projection; CSRAUSPOS = theparameters for this projection are derived from the processing of DGPS raw data by AUSPOS - Online GPS Processing Service - https://www.ga.gov.au/scientific-topics/positioning-navigation/geodesy/auspos; SF1 = Scale Factor is 1).
In the "Format" dropdown list, select "Proj String (legacy - Not Recommended)"
In the "Parameters" box, paste the following "+proj=tmerc +lat_0=12.01605433+lon_0=79.80998934 +k=1 +x_0=370455.630 +y_0=1328608.994 +ellps=WGS84+towgs84=0,0,0,0,0,0,0 +units=m +no_defs".
Finally, click on OK.
In a more explicit way, the parameters mean the following:
Map Projection: TransverseMercator (TM)
False Easting: 370455.6300
False Northing: 1328608.9940
Latitude of Origin: 12°00'57.79560" (DMS) 12.01605433 (DD)
Central Meridian: 79°48'35.96164" (DMS) 79.80998934 (DD)
Scale Factor: 1.00000000
Zone Width: 6.0°
h1. Elimination of Duplicate points – General criteria
It might happen that the same physical feature (e.g. a tree, or a pole) is surveyed more than once: this can happen because there are many physical features in an area, and the survey needs more than one station. So, for example a tree is surveyed from a station, and gets a serial number on that date. When the station is then changed, it might happen that the same tree is resurveyed: another serial number is given, and possibly a different date, if the survey from the second station happened on a different day.
It is clear that the same tree is then represented with two different points, which means that two different trees exist: but only one tree really exist in the physical reality.
It is clear that one of the two points is redundant and needs to be removed. If this is noted by the surveyor directly in the field, then the issue is solved by the surveyor himself during processing time.
If instead, due to various reasons, it was not noted by the surveyor in the field, it will need to be cleaned after the processing, possibly by post-processing staff.
How to identify duplicate points?
The following criteria can be used:
1. The distance between the two points is less than 30 cm (trees are surveyed if their trunk diameter is at least about 20 cm, so in 30 cm cannot exist two of them)
2. The orig_id (serial number) of the points are not in series
3. The survey date is not the same
4. In case of trees, the species of trees is the same
5. 5. In case of trees, the tree type is not TDEF (because TDEF are mapped irrespective of their diameter, so they can actually have a small trunk, and two of them might exist in 30 cm), not OT (many TDEF species are surveyed as OT if not otherwise indicated by a botanist)
6. The context needs to be evaluated: if one tree is deleted in an area where many trees exist in a limited space, then loosing one in the map is not a big error. If instead one tree is deleted where there are very few trees, then it might be a big loss.