About the Auroville CSR Geomatics Studio¶
We are a team working on geographical and related information with an engineering and scientific approach. The concept of geomatics is explained here: https://en.wikipedia.org/wiki/Geomatics.
We develop, maintain and publish data on this web site: https://gis.auroville.org.in.
Currently, the team consists of:
Quick report of the collaborations the Geomatics Team the areas of work.
Collaborations/coordination with other other groups, etc
|CSR Geomatics Description of Activity / Research / Project||AA||AVF||LB||FAMC||WG||CSR||Talam||DST|
|Topographic Survey of Auroville City Area in coordination with L’avenir d’Auroville: Matrimandir entire compound and Residential Zone Sector 1 completed, Sector 2 half completed||yes||yes|
|Topographic Survey of specific projects area: Vibrance, Wasteless, Cultural Zone along the Crown Road, Baraka, Gardens of Unexpected|
|Collective wastewater treatment systems health check-up survey: 68 plants evaluated||yes||yes||yes||yes|
|Manual weekly monitoring of water level in selected wells on Auroville land on approximately 50 wells (number fluctuates depending on local conditions)||yes||yes||yes|
|Collection of rainfall data through manual raingauges distributed to Aurovilians: data received regularly from at least 7 raingauges||yes||yes|
|Collection of weather data through automatic weather station installed at CSR: data collected every minute, stored in the database, and published online in real time||yes||yes||yes|
|Collaboration with Land Board for survey of identified land boundary stones: collection of coordinates of Government boundary stones for georefering of cadastral maps||yes||yes|
|Collaboration with AV Foundation for compilation of land ownership map: geographic land records as provided by AV Foundation, protected by login access||yes|
|Collaboration with L’avenir d’Auroville for data sharing and coordinated system set-up: organisation of geographic data for unique online platform to enhance planning processes (initial stage)||yes|
|Collaboration with L’avenir d’Auroville and FAMC for database links: structural work on procedures for linking databases already existing with various agencies||yes||yes|
|Study of application of drone topographic survey in Auroville context: pilot test conducted on Residential Zone with outside drone survey agency||yes||yes||yes|
|Study of open source software for drone topographic survey: testing of softwares like OpenDroneMap, MeshLab, CloudeCompare to evaluate work flow, processes and limitations||yes||yes|
|Surface water modelling (ongoing): research on evaluation of surface water collection potential through automatic processing of field data||yes||yes|
|Collaboration with TALAM on research on radio-transmission for automatic wells level monitoring (ongoing): evaluation of application of radio-transmitted signals for water-related monitoring and metering (selected borewells, flow meters)||yes||yes|
|Organisation of data collected in Auroville in the past (ongoing): standardisation of data in geospatial database||yes|
|Provision for basic statistical analyses of geographic-related data (indicators/dashboard)||yes||yes|
|Publication of collected data in standardised format on web portal gis.auroville.org.in||yes||yes|
- AA : L'avenir d'Auroville
- AVF : Auroville Foundation
- LB : Auroville Land Board
- FAMC : Auroville Funds and Assets Management Committee
- WG : Auroville Water Group
- CSR : Auroville Centre for Scientific Research
- TALAM : a project under CSR
- DST : Department of Science and Technology, Ministry of Science and Technology, New Delhi
Most field surveys are executed by our team of surveyors, using advanced DGPS equipment.
Other surveyors might also be contracted.
The survey data are imported to a CAD software (Aurocad/Civil 3D).
See editing shapefiles in Autocad
Working with Gisaf¶
Raw survey data are contained in CSV files, typically downloaded from surveying instruments.
See more information of the process for the survey data (including a flow diagram): Survey data
The categories define the types of the geographical features.
QGis: work on shapefiles¶
Go to shapefiles.
See Data analysis
The team is located in the CSR of Auroville.
We have setup a server for hosting the software and database. See CSR_server for technical information about the setup.
Backup and restoration of the database¶
See survey equipment
Plan for future¶
Beside living well, see plan.
Links and references¶
Pavneet's docs (imported from gisaf's wiki)¶
Rules of Map making - What all Maps should have!
Field to finish
Survey Data Post-Processing
Civil 3D useful commands
Online references for Civil 3D
connections in QGIS- using browser panel and Add postGIS
Reconcilation of Raw survey data using pgAdmin
importing RAW data to GISAF
Editing Z value of features in Shapefiles in QGIS
Miscellaneous- Civil 3D
Documentation- Rain Gauge
Documentation- Wells Monitoring (Piezometer by Bala)
Documentation- Flow meter, by Bala
Documentation- DST- Vegetation Indexing
Documentation- DST- Interpolation
Documentation- DST- Survey- Office workflow
From CAD to GIS, by Giulio
Access to data¶
Connection to server directly from CSR¶
To connect to the server directly without going through Aurinoco server, the correct url is
Connection to Gisaf via QGis through WFS / OGC API¶
This works only on QGis from version 3.14.15 onward
In the browser, click on WFS/OGC API, then right-click to create a new connection
Give a name (e.g. OGC API Qgis Gisaf)
Give the url https://gis.auroville.org.in/ogcapi
Under the WFS Options box, on Version dropdown, the default option "Maximum" works just fine
Click on OK
The list of layers will appear in the Browser under WFS/OGC API.
How to create a new projection in QGis¶
To create a new projection in QGis, go to menu "Settings", and click on "Custom Projections".
A pop-up window appears with a list of all projections defined in QGis projects used by the user so far.
Click on the green "+" sign on the right top part of the window to create a new projection.
In the "Name" box, type "TM CSRAUSPOS SF1" (which means TM = Transverse Mercator projection; CSRAUSPOS = theparameters for this projection are derived from the processing of DGPS raw data by AUSPOS - Online GPS Processing Service - https://www.ga.gov.au/scientific-topics/positioning-navigation/geodesy/auspos; SF1 = Scale Factor is 1).
In the "Format" dropdown list, select "Proj String (legacy - Not Recommended)"
In the "Parameters" box, paste the following "+proj=tmerc +lat_0=12.01605433+lon_0=79.80998934 +k=1 +x_0=370455.630 +y_0=1328608.994 +ellps=WGS84+towgs84=0,0,0,0,0,0,0 +units=m +no_defs".
Finally, click on OK.
In a more explicit way, the parameters mean the following:
Map Projection: TransverseMercator (TM)
False Easting: 370455.6300
False Northing: 1328608.9940
Latitude of Origin: 12°00'57.79560" (DMS) 12.01605433 (DD)
Central Meridian: 79°48'35.96164" (DMS) 79.80998934 (DD)
Scale Factor: 1.00000000
Zone Width: 6.0°
Elimination of Duplicate points – General criteria¶
It might happen that the same physical feature (e.g. a tree, or a pole) is surveyed more than once: this can happen because there are many physical features in an area, and the survey needs more than one station. So, for example a tree is surveyed from a station, and gets a serial number on that date. When the station is then changed, it might happen that the same tree is resurveyed: another serial number is given, and possibly a different date, if the survey from the second station happened on a different day.
It is clear that the same tree is then represented with two different points, which means that two different trees exist: but only one tree really exist in the physical reality.
It is clear that one of the two points is redundant and needs to be removed. If this is noted by the surveyor directly in the field, then the issue is solved by the surveyor himself during processing time.
If instead, due to various reasons, it was not noted by the surveyor in the field, it will need to be cleaned after the processing, possibly by post-processing staff.
How to identify duplicate points?
The following criteria can be used:
1. The distance between the two points is less than 30 cm (trees are surveyed if their trunk diameter is at least about 20 cm, so in 30 cm cannot exist two of them)
2. The orig_id (serial number) of the points are not in series
3. The survey date is not the same
4. In case of trees, the species of trees is the same
5. 5. In case of trees, the tree type is not TDEF (because TDEF are mapped irrespective of their diameter, so they can actually have a small trunk, and two of them might exist in 30 cm), not OT (many TDEF species are surveyed as OT if not otherwise indicated by a botanist)
6. The context needs to be evaluated: if one tree is deleted in an area where many trees exist in a limited space, then loosing one in the map is not a big error. If instead one tree is deleted where there are very few trees, then it might be a big loss.
Linework for the Survey Area¶
1. Creation of Initial Linework in QGIS using Survey points import - (Ram, System 4)¶
Initial Linework in QGIS is started by surveyor with the knowledge from the Field. For this step, points are simply imported into the QGIS from the field text file (.csv or .txt). CRS needs to be TM-AUSPOS. The box of “First record has field names” shall not be ticked. In Point Coordinates, select the correct field for x, for y and for z (usually “field_2” for x, “field_3” for y and “field_4” for z). Points can be styled using the “Categorized” style in “Symbology”, using “Field_5” as value, or using a Rule-based symbology using the category (field 5) as filter.
Linework is created by connecting points having same description and belonging to the same physical feature. All line and polygon features are created as lines.
The Initial Linework for the Survey Area is also stored temporarily in
Note: The line shapefiles / Geopackages shall be in CRS: TM AUSPOS¶
2. Creation of final working drawing Shapefiles / Geopackages - (Selvarani, System 1)¶
Final working drawing Shapefiles / Geopackages are created from the Initial Linework of Survey Area.
As the Surveyor draws all features as lines (both for lines and polygons features), the following actions shall be done:
1. If features are lines:
• Export the shapefile / geopackage into the final working drawing folder (Final WD), in separate folders according to its type (e.g. BLDG, FENC, ROAD, etc).
The CRS for the export shall be EPSG:4326 - WGS 84¶
2. If features are polygons:
• Lines shall be converted into polygons:
to do it, first click on the layer to be converted to make it active (e.g. WD-CZ-01-F-LL84_V-BLDG-MHOL------E), then go to “Vector” Menu, click on Geometry Tools, click on Line to Polygons:
The new window for “Lines to Polygons” conversion will appear:
• Always cross check the input layer, to make sure that the input layer is the active one
• Save the output in a temporary layer
• The temporary layer will be listed in the list of layers, it shall be exported to the saving location as (eg . D: > Survey > GB-01 > Final WD > A-Shp)
The CRS for the export shall be EPSG:4326 - WGS 84¶
Once all the shapefiles / geopackages are exported in Final WD, for each of the newly exported layers the Topology Checker Tool shall be used.
Linework for the whole Survey Zone¶
1. Merging Shapefiles / Geopackages - (Selvarani, System 1)¶
A copy of the Zone Master shapefiles / geopackages are taken from System 4 and stored in Temp Folder on Desktop in System 1.
Master shapefiles / geopackages are merged with the Survey Area shapefiles / geopackages:
• To do it, go to “Vector” Menu, click on Geoprocessing Tools, then click on Union:
The new window for “Union” will appear:
• To make sure that the right geometry is generated by this process (“line” type, not “Multiline”, and similarly “Polygon” type, not “Multipolygon), we need to always keep the Master shapefile (e.g. Final-CZ-01-2021-02-05-LL84_V-BLDG-MHOL------E) as Input layer, and the Survey Area shapefile as Overlay Layer (e.g. WD-CZ-01-F-LL84_V-BLDG-MHOL------E).
• (The output can be saved to a file, as the CRS should already be EPSG4326 – WGS84.)
2. Storing Shapefiles / Geopackages - (Selvarani, System 1)¶
Save the merged shape file in the correct location in Final folder as (eg . D: > Survey > GB-01 > Final)
Date in the name of Final Shapefile / Geopackage needs to be updated.
Once the merging operation is completed, the copy of Master shapefile / geopackage is deleted from the Temp folder.
3. Topology check of merged shapefiles¶
The topology checker is applied again on the merged shapefiles / geopackages.
The “id_field” shall be removed from the attribute table.
4. Archive and replace the Master Shapefiles / Geopackages (Ram, System 4)¶
Archive the previous master shapefiles / geopackages on system 4, and copy the new merged shapefiles / geopackages in its place.
*Then delete the Merged Shapefile / Geopackage folder from System 1. *
5. Note about Shapefiles and Geopackages¶
All the above works are usually done using shapefile format, in QGIS latest version (3.16.3).
The Geopackage export is done in QGis versions older than 3.12 (e.g. 3.4, 3.6, 3.8, 3.10) so that the lines are not saved as “Multilines” but as “Lines”and polygons are not saved as “Multipolygons” but as “Polygons”. This is very important to be notes, as Gisaf database does not accept the Multipolygon and Multiline geometry types.