Project

General

Profile

Wiki » History » Version 278

Giulio Di Anastasio, 06/05/2021 11:18

1 1 Philippe May
h1. %{color:BLUE}  Wiki%
2 227 Giulio Di Anastasio
3 232 Giulio Di Anastasio
h1. [[CSR Geomatics Team]]
4 231 Giulio Di Anastasio
5 229 Giulio Di Anastasio
h1. [[Collaborations]]
6 229 Giulio Di Anastasio
7 240 Giulio Di Anastasio
h1. [[Data Analysis]]
8 234 Giulio Di Anastasio
9 241 Giulio Di Anastasio
h1. [[CSR Geomatics Server Setup]]
10 234 Giulio Di Anastasio
11 245 Giulio Di Anastasio
h1. [[Database Backup and Restoration]]
12 243 Giulio Di Anastasio
13 243 Giulio Di Anastasio
h1. [[CAD to GIS Workflow]] %{color:RED}  - TO BE UPDATED%
14 248 Giulio Di Anastasio
15 246 Giulio Di Anastasio
h1. [[GIS Conventions]]
16 246 Giulio Di Anastasio
 
17 252 Giulio Di Anastasio
h1. [[Working with Gisaf]]
18 96 Selvarani C
19 92 Selvarani C
h1. [[Gisaf Workflow]]
20 252 Giulio Di Anastasio
21 256 Giulio Di Anastasio
h1. [[GIS to CAD Workflow]] %{color:RED}  - TO BE UPDATED%
22 94 Selvarani C
23 257 Giulio Di Anastasio
h1. [[Weather Station]]
24 142 Selvarani C
25 259 Giulio Di Anastasio
h1. [[GDAL - Useful Tips]]
26 5 Philippe May
27 262 Giulio Di Anastasio
h1. [[Gisaf - Auxiliary Data (non-geographical data)]]
28 143 Selvarani C
29 264 Giulio Di Anastasio
h1. [[QGis - Useful Tips]]
30 264 Giulio Di Anastasio
31 266 Giulio Di Anastasio
h1. [[AutoCAD Civil 3D - Useful Tips]]
32 266 Giulio Di Anastasio
33 268 Giulio Di Anastasio
h1. [[Wells documentation]]
34 8 Philippe May
35 272 Giulio Di Anastasio
h1. [[Gisaf Data Modifications]]
36 272 Giulio Di Anastasio
37 274 Giulio Di Anastasio
h1. [[Other Useful Softwares]]
38 272 Giulio Di Anastasio
39 262 Giulio Di Anastasio
40 1 Philippe May
41 274 Giulio Di Anastasio
42 274 Giulio Di Anastasio
43 107 Selvarani C
44 110 Selvarani C
45 110 Selvarani C
46 110 Selvarani C
47 113 Selvarani C
48 270 Giulio Di Anastasio
49 270 Giulio Di Anastasio
50 119 Selvarani C
51 119 Selvarani C
52 127 Selvarani C
h1. Importing point data (TS and RTK) to GISAF 
53 127 Selvarani C
54 127 Selvarani C
# *Gisaf Admin > Basket > Survey data > "Project" > "surveyor" > TS/ RTK > upload > import.*
55 127 Selvarani C
# *Auto import of Raw points data* (changes from Raw to Shapefiles, the point files)
56 1 Philippe May
  In GISAF Admin > Other> Project > select the project for which you want to import the data > with selected > Auto import to GIS...
57 127 Selvarani C
58 172 Selvarani C
59 128 Selvarani C
60 128 Selvarani C
h1. Editing Z value of features in Shapefiles in QGIS
61 128 Selvarani C
62 128 Selvarani C
h2. using vertex editor tool - https://www.youtube.com/watch?v=8V8i1AtcA74&t=256s
63 128 Selvarani C
64 129 Selvarani C
h1. Miscellaneous- Civil 3D
65 129 Selvarani C
66 129 Selvarani C
*Autodesk civil 3D Geotechnical module- for borehole data :- To analyse borehole data, To make profiles and calculate volumes*
67 129 Selvarani C
68 129 Selvarani C
https://www.youtube.com/watch?v=0Be9kShBou0
69 129 Selvarani C
Reference videos
70 129 Selvarani C
Exploring:
71 129 Selvarani C
https://www.youtube.com/watch?v=gr-ISPzLcU0
72 129 Selvarani C
Modelling:
73 129 Selvarani C
https://www.youtube.com/watch?v=Pz0-HOoiBrs
74 129 Selvarani C
solids
75 129 Selvarani C
https://www.youtube.com/watch?v=mQ0Yeh6tZA8
76 129 Selvarani C
https://www.youtube.com/watch?v=4eMsUiYBhuE
77 129 Selvarani C
78 129 Selvarani C
79 129 Selvarani C
*CIVIL 3D Survey*
80 129 Selvarani C
Getting started- Always open a new drawing with a template. 
81 129 Selvarani C
82 129 Selvarani C
*Description key sets*- (till 18:30) https://www.youtube.com/watch?v=mmwkkRyBkS0
83 129 Selvarani C
As when the points come into the drawing, they are going to be filtered with description key sets and the description key sets will assign properties to the points.
84 129 Selvarani C
Tool space > settings > points > description key sets
85 129 Selvarani C
“Description key sets name”> edit key- to view the points in a list: 
86 129 Selvarani C
Automatic linework (18:30- till end)
87 129 Selvarani C
Survey > Linework code sets- for automatic linework
88 129 Selvarani C
89 129 Selvarani C
*Using Master view*
90 129 Selvarani C
Uses-
91 129 Selvarani C
copy styles from one drawing to other
92 129 Selvarani C
copy drawing data from one drawing to other (data referencing)
93 129 Selvarani C
94 129 Selvarani C
*Civil 3d surface model*- from points- break lines
95 129 Selvarani C
https://www.youtube.com/watch?v=wC40rdmDEMo
96 129 Selvarani C
moving for example- point groups under point groups- 
97 129 Selvarani C
98 129 Selvarani C
*Civil 3D Planning and Analysis*
99 129 Selvarani C
Workspace: Planning and Analysis
100 129 Selvarani C
1. Working with Object data (GIS Attributes)
101 129 Selvarani C
Map Setup> Define object data> New table> Define new object data table start defining fields
102 129 Selvarani C
103 129 Selvarani C
104 130 Selvarani C
105 130 Selvarani C
h1. Documentation- Rain Gauge
106 130 Selvarani C
107 130 Selvarani C
h2. Manual Rain Gauge
108 130 Selvarani C
109 130 Selvarani C
*Why it is important to comply to standards?*
110 130 Selvarani C
111 130 Selvarani C
1. Consistency for comparing rainfall data in different places within Auroville since rainfall varies in different parts of Auroville.
112 130 Selvarani C
113 130 Selvarani C
2. Making it possible to use for any kind of scientific analysis by contribution to the primary data in sustainable water management for Auroville. 
114 130 Selvarani C
115 130 Selvarani C
*Proposed standards and ethics:*
116 130 Selvarani C
117 130 Selvarani C
1. The time of taking the reading - 08:30 am. 
118 130 Selvarani C
The rain is recorded over a period of 24 hours- 8:30 of previous day till 8:30 of the present day, and the date is put as the present day. For example, if the rain is measured at 8:30 am on 10 dec 2017, then it is logged in as on 10 dec 2017.
119 130 Selvarani C
120 130 Selvarani C
2. Manual rain gauge typically used- green cylindrical and a collection jar.  
121 130 Selvarani C
  
122 130 Selvarani C
3. Measuring jar: 10 mm corresponding jar
123 130 Selvarani C
124 130 Selvarani C
 
125 130 Selvarani C
Area of the rim of manual rain gauge= 200 square cm
126 130 Selvarani C
Diameter of rim= approximately 16 cm
127 130 Selvarani C
*NOTE:* - If another kind of rain gauge is used, it should be used with the corresponding measuring jar as type of jar and calibration on jar depends on the area of rim. Any cases of different rain gauge other than above should be reported before use for recording purposes.
128 130 Selvarani C
129 130 Selvarani C
4. The minimum record-able unit is 0.2 mm (Least measure on the 10 mm cylinder). T is marked for below 1 mm.
130 130 Selvarani C
131 130 Selvarani C
5. The reading should be preferably submitted in an excel sheet (the data sheet will be provided). The frequency of sending data can be daily or weekly.
132 130 Selvarani C
6. The rain gauge should be kept in a safe and open to sky area clear from tree cover or objects in surrounding so that direct rain falls into it and not from trees/ objects etc. Preferably roof top or a higher place on ground.
133 130 Selvarani C
7. A check on the rain gauge should be kept when there is long period of gap in rainy days to make sure it was emptied out and there is no blockage. 
134 130 Selvarani C
135 130 Selvarani C
8. In case of doubt in reading or incorrect measure, "incorrect measure" should be marked
136 130 Selvarani C
137 130 Selvarani C
138 130 Selvarani C
*Ethics*
139 130 Selvarani C
1. Readings should not be missed. Time to time check on rain gauge is required during long periods of no rain (especially when the rain season is approaching). 
140 130 Selvarani C
This is important because firstly, if it rained in the night and it wasn’t recorded, reading for the day is lost; Secondly, if the jar was not emptied, it gives incorrect subsequent reading. 
141 130 Selvarani C
142 130 Selvarani C
2.  If the person is not going to be available temporarily where the rain gauge is kept, the responsibility should be passed on to someone they can rely on after explaining the standards. 
143 130 Selvarani C
144 130 Selvarani C
*Important-*
145 130 Selvarani C
Data won’t be published in case of non-compliance to standards. This is to maintain the sanctity of proper scientific data collection and to keep it reliable for sharing by publishing. Discussions regarding standards are welcomed.
146 130 Selvarani C
CSR Geomatics Team is placed on first floor in CSR, Auroshilpam.
147 130 Selvarani C
148 130 Selvarani C
*AV rain data publication*: (http://gis.auroville.org.in/measures/raingauge_av).
149 130 Selvarani C
150 130 Selvarani C
Adding Rain gauge to our web portal (GISAF), following information is needed:
151 130 Selvarani C
1. GPS co-ordinates of location of placing the rain gauge (can be obtained using mobile phones easily)
152 130 Selvarani C
2. Name, place of residence & contact number- Mobile & Landline
153 130 Selvarani C
154 130 Selvarani C
h2. Automatic Rain Gauge
155 130 Selvarani C
156 130 Selvarani C
There is a possibility of publishing rain data coming from Automatic rain gauges like (id 15, auro orchard) and Weather stations. 
157 130 Selvarani C
For Automatic rain gauges, we can upload the files coming from the rain gauge directly into GISAF.
158 130 Selvarani C
159 130 Selvarani C
160 130 Selvarani C
_Thank you for contribution towards a sustainable management of Water in Auroville through Data collection._
161 130 Selvarani C
162 130 Selvarani C
163 131 Selvarani C
164 131 Selvarani C
h1. Documentation- Wells Monitoring (Manual and Automatic / Piezometer by Bala)
165 131 Selvarani C
166 131 Selvarani C
h2. Manual- Using tape with a sensor
167 131 Selvarani C
168 131 Selvarani C
Timings: The person assigned the job of monitoring collects the readings in three slots.
169 131 Selvarani C
170 131 Selvarani C
1.	Between 6 am to 7 am till about 9 am to 10 am.
171 131 Selvarani C
2.	Between 11 am and 1 pm
172 131 Selvarani C
3.	Between 2 pm and 5 pm
173 131 Selvarani C
174 131 Selvarani C
Things required:
175 131 Selvarani C
•	Notebook
176 131 Selvarani C
•	Pen
177 131 Selvarani C
•	Measuring tape
178 131 Selvarani C
•	Vehicle for movement
179 131 Selvarani C
180 131 Selvarani C
181 131 Selvarani C
*Ethics*- 
182 131 Selvarani C
•	The monitoring should happen in coordination with the community members/ care taker (assigned by the person in charge from within the community). There should be a clear communication from monitor side as to which days the monitoring of a well has to happen and at around what time so that it can be made sure that the pump is not turned on before monitoring. In case a pump was turned on in a well, the monitor should have a gap of about 5 hours on the same day before going again for monitoring.
183 131 Selvarani C
•	The monitor should have contact numbers of the person in charge/ care taker for any communication.
184 131 Selvarani C
•	If the monitoring is stopped for any reason at any point, the monitor should communicate the same to the person concerned.
185 131 Selvarani C
•	The monitor shall take responsibility to inform any kind of changes in a well in terms of its functioning etc.
186 131 Selvarani C
•	The monitor should report to the CSR Geomatics Team who has responsibility to publish data.
187 131 Selvarani C
•	The monitor published the data to the website and works with the geomatics team.
188 131 Selvarani C
189 131 Selvarani C
190 131 Selvarani C
*Definitions*:
191 131 Selvarani C
A master file is maintained with the records of the wells from the field. It is updated when a new well is located. The following set of information are filled out in the Masterfile. 
192 131 Selvarani C
193 131 Selvarani C
*Well type*
194 131 Selvarani C
1.	Open
195 131 Selvarani C
2.	Borewell
196 131 Selvarani C
3.	Dug cum borewell
197 131 Selvarani C
198 131 Selvarani C
*Well status*
199 131 Selvarani C
1.	In use
200 131 Selvarani C
2.	Not in use
201 131 Selvarani C
3.	Closed
202 131 Selvarani C
203 131 Selvarani C
*Pump Status*
204 131 Selvarani C
1.	Functioning
205 131 Selvarani C
2.	Not functioning
206 131 Selvarani C
207 131 Selvarani C
*Pump Automation*
208 131 Selvarani C
1.	Manual 
209 131 Selvarani C
2.	Automatic
210 131 Selvarani C
211 131 Selvarani C
* Non-accessibility factors*
212 131 Selvarani C
1.	Dogs
213 131 Selvarani C
2.	Heavy slab
214 131 Selvarani C
3.	Narrow casing
215 131 Selvarani C
4.	enclosed/locked
216 131 Selvarani C
5.	permission
217 131 Selvarani C
218 131 Selvarani C
*Data Matching Accuracy (reconnecting with harvest wells data)*
219 131 Selvarani C
1.	High
220 131 Selvarani C
2.	Medium
221 131 Selvarani C
3.	Low
222 131 Selvarani C
223 131 Selvarani C
*Well coordinates*
224 131 Selvarani C
For a new well, coordinates of the well position are taken on a mobile GPS.
225 131 Selvarani C
Person In charge
226 131 Selvarani C
For communication purpose, contact the person in charge as recorded in directory.
227 131 Selvarani C
228 131 Selvarani C
h2. Automatic- Piezometer (by bala, to be edited) 
229 131 Selvarani C
230 131 Selvarani C
The calibrations and setting are already done by Azha
231 131 Selvarani C
The piezometer is taken to the site of unused well. First the depth and Water level  is checked manually using water meter
232 131 Selvarani C
Then the sensor of the piezometer is inserted into the well 1m above the bottom of the well, so the sensor does not get affected from mud or water inside
233 131 Selvarani C
The sensor sends the data to an electronic board which translates the signal into the proper output, and then it sends the signal to the transmitting device above, on the ground surface. The transmitter then sends it to the receiver at CSR, the signal is then routed to Talam office through Interneet, enters the software and the reading is processed.
234 131 Selvarani C
The piezometer sometimes does not have proper signal and so it would not be able to send it
235 131 Selvarani C
236 132 Selvarani C
h1. Documentation- Flow meter by Bala
237 132 Selvarani C
238 132 Selvarani C
Flow meter – for checking the flow of water in pipe
239 132 Selvarani C
240 132 Selvarani C
CSR bought ultrasonic flow meter from Chennai and the company (company name? )trained Bala and Vijai(CSR) on how to set it up.
241 132 Selvarani C
we tested in the west water system pump in csr. then we test in many more place .
242 132 Selvarani C
 ami, aurodam and buddha garden borewell pumps . we got request from the water service 
243 132 Selvarani C
Cross check they flow meters .
244 132 Selvarani C
 so before that  we want to know how our meter works. 
245 132 Selvarani C
so we tested in our tank . the pump pumping from our sump tank so the flow will be stranded.  when you pump in the bore well  the flow goes up and down. 
246 132 Selvarani C
we tested two times in  15 minutes.and one time 30 minutes,so we got variation between this three. 
247 132 Selvarani C
we find 1,5 % error but the flow meret they said  1 % error only.
248 132 Selvarani C
we cross checked the flow meter of the water service we find some error  in they meter also. 
249 132 Selvarani C
we tried they bore well flowmeter and they sump tank flow meter also.and we find some error also in they meters.  
250 132 Selvarani C
then we find some error in (AVWS) meter also.
251 132 Selvarani C
It is used to check the flow of water in pipe
252 132 Selvarani C
tools used 
253 132 Selvarani C
First the outer diameter of the pipe has to be entered in the device. It is measured using Vernier caliper
254 132 Selvarani C
Then the thickness of the pipe is set which is also measured using vernier
255 132 Selvarani C
Also the device asks for the material of the pipe. If the pipe material is know it can be set and if it not know then there is an option which is other (mostly PVC, HDPE, and  iron) 
256 132 Selvarani C
After entering these details the device gives the spacing for the sensors. There are 2 sensors up and down which has to be fixed accordingly
257 132 Selvarani C
When the sensors are fixed the motor is turned on and the sensors send reading to the display device
258 132 Selvarani C
This is noted down once every minute and taken for 15 to 30 minutes. Then the average of this is determined. This is done because of the variation in the flow. This gives the flow rate in 1 hour.
259 1 Philippe May
This was done in different places to check to flow rate
260 132 Selvarani C
261 177 Selvarani C
262 133 Selvarani C
263 133 Selvarani C
h1. Documentation- DST- Vegetation Indexing
264 133 Selvarani C
265 133 Selvarani C
h2. Steps for Dzetsaka Classification tool for Vegetation indexing in QGIS
266 133 Selvarani C
267 133 Selvarani C
 
268 133 Selvarani C
 1. Install the plugin Dzetsaka classfication tool.
269 133 Selvarani C
 2. Open the Raster from the Survey.
270 133 Selvarani C
 3. Create a polygon shapefile for index sampling. Mark polygons and give the ID's (1,2,3 for Tree, grass, bare land etc) Cover the variations in samples as much as possible. 
271 133 Selvarani C
     More the samples, better the indexing.
272 133 Selvarani C
 4. Apply Dzetsaka Classification tool, Select the base raster and the sample- index polygon shapefile created in step 3.
273 133 Selvarani C
 5. The result is a Raster with DN numbers specified in the Shapefile in step 3.
274 133 Selvarani C
 6. Apply the Sieve raster command (Raster> Analysis> Sieve)- Try different threshold numbers and view the results till the noise is removed from the Raster.
275 133 Selvarani C
 7. Polygonise the Raster to Vector (From processing)
276 133 Selvarani C
 8. Run the v.generalise tool on the shapefile. This tool removes the pixelated boundaries of the polygons in the Vector.
277 133 Selvarani C
278 134 Selvarani C
h1. Documentation- DST- Interpolation (Processing toolbox)
279 134 Selvarani C
280 134 Selvarani C
The following 4 tools have been mostly explored and the results were compared. The ones used for quick analysis are 1. Cubic Spline and 2. V.surf.spline . The rest of the tools are for further exploration and used depending on the need of the project. In some tools, the elevation values of points should be stored in the attribute table (Using field calculator and giving command - *Z($Geometry)* )
281 134 Selvarani C
282 134 Selvarani C
# Interpolate (Cubic spline) - SAGA
283 134 Selvarani C
# V.surf.bspline - GRASS. Parameters to set - cell size. Set this parameter above 0.001 ( 0.00001, 0.000001 etc) and check the results.
284 134 Selvarani C
# V.surf.rst - GRASS
285 134 Selvarani C
# Krigging - SAGA
286 1 Philippe May
287 181 Selvarani C
[[Documentation- DST- Survey- Office workflow]]  - *%{color:RED} to be written%*
288 135 Selvarani C
289 135 Selvarani C
h1. From CAD to GIS by Giulio
290 135 Selvarani C
291 135 Selvarani C
*FEATURES CREATION*
292 135 Selvarani C
1.	Assign a CRS to the drawing (TM-AUSPOS) (MAPCSLIBRARY command)
293 135 Selvarani C
2.	Create features in CAD (Points, lines, polygons)
294 135 Selvarani C
3.	Export shapefile (a) from CAD (Output > DWG to SDF) (Convert to LL84 – 3D)
295 135 Selvarani C
296 135 Selvarani C
*FEATURES IMPORT INTO DB FIRST TIME*
297 135 Selvarani C
4.	Create zip file of the shapefile
298 135 Selvarani C
5.	Upload into the GISAF Shapefiles Basket
299 135 Selvarani C
6.	Import the shapefile into DB
300 135 Selvarani C
7.	Save the shapefile on Local Machine
301 135 Selvarani C
302 135 Selvarani C
*FEATURES IMPORT INTO DB EVERYTIME*
303 135 Selvarani C
8.	Combine the new features to corresponding last shape files (Insert the process here).
304 135 Selvarani C
9.	Follow step 4-8 again
305 135 Selvarani C
306 135 Selvarani C
*FEATURES EDITING IN QGIS*
307 135 Selvarani C
10.	Open the table in QGis
308 135 Selvarani C
11.	Save as a shapefile (b) in TM AUSPOS CRS
309 135 Selvarani C
12.	In CAD, open a new drawing and assign AUSPOS CRS
310 135 Selvarani C
13.	Import the shapefile (b) (MapImport) with Object Data (Data tab > Create Object Data > OK), tick “Import polygons as closed polylines”, then press OK
311 135 Selvarani C
14.	Edit features
312 135 Selvarani C
15.	Change workspace into “Planning and analysis”
313 135 Selvarani C
16.	Export shapefile (a) from CAD (Output > DWG to SDF) with ONLY the id selected (Data Tab > Select Attributes > Object Data > Filename > id) (Convert to LL84 – 3D)
314 135 Selvarani C
315 135 Selvarani C
*FEATURES IMPORT INTO DB*
316 135 Selvarani C
17.	Create zip file of the shapefile
317 135 Selvarani C
18.	Upload into the GISAF Shapefiles Basket
318 135 Selvarani C
19.	Import the shapefile into DB
319 135 Selvarani C
20.	Delete the shapefile from Local Machine
320 135 Selvarani C
321 135 Selvarani C
322 182 Selvarani C
323 1 Philippe May
324 136 Selvarani C
h1. QGIS- Miscellaneous
325 136 Selvarani C
326 136 Selvarani C
*QGIS Introduction:*
327 136 Selvarani C
https://www.birdseyeviewgis.com/blog/2018/2/22/my-favorite-features-of-qgis-30to-date
328 136 Selvarani C
3D visualization of raster DEM- https://www.youtube.com/watch?v=2KrCsbP0kUs
329 136 Selvarani C
330 136 Selvarani C
*Spatial query:* 
331 136 Selvarani C
Spatial Query is selection of features that satisfies a certain condition which relates to other features in a space.
332 136 Selvarani C
Using plugin- Spatial query
333 136 Selvarani C
http://www.geodose.com/2018/03/spatial-query-in-qgis-3.html
334 136 Selvarani C
Labelling:
335 136 Selvarani C
Labelling with more than one field names and in different lines
336 136 Selvarani C
|| '\n' ||
337 136 Selvarani C
338 136 Selvarani C
339 136 Selvarani C
340 136 Selvarani C
*Hierarchy of extensions*
341 136 Selvarani C
342 136 Selvarani C
File levels and their uses.
343 136 Selvarani C
https://nathanw.net/2014/03/22/all-the-q-files/
344 136 Selvarani C
345 136 Selvarani C
+The Project file (.qgs)+
346 136 Selvarani C
It contains: Layer source pointer + Style information + Composers + a whole heap of other stuff
347 136 Selvarani C
348 136 Selvarani C
+The Layer Definition file (.qlr)+
349 136 Selvarani C
It contains: Layer source pointer + Style information
350 136 Selvarani C
351 136 Selvarani C
+The QML file (.qml)+
352 136 Selvarani C
It contains: Style information
353 136 Selvarani C
354 59 Giulio Di Anastasio
h2. Giulio's documentation
355 52 Giulio Di Anastasio
356 183 Selvarani C
357 137 Selvarani C
358 137 Selvarani C
h1. Documentation - Reconciliation of points using Gisaf
359 137 Selvarani C
360 137 Selvarani C
Reconciliation of points is a procedure used when a point is stored in a wrong table, because its category was either wrongly recorded in the field by the surveyor, or it has ben reviewed later by the surveyor or the data validator and found to be wrong.
361 137 Selvarani C
362 137 Selvarani C
h2. Definition: Raw points are all points coming from the field survey. Raw points can be points referring to a Point feature (e.g. trees, or elevation points, or floor level), or points measured in the field to draw a line (e.g. the vertices of a fence) or a polygon (e.g. the corners of a building outline).
363 137 Selvarani C
364 137 Selvarani C
In the overall workflow, *+reconciliation affects only raw points+*. It takes place after the field textfile is uploaded into the basket and its points imported (raw points stored in the raw survey tables). Here you can visualize the Workflow diagram: [[Survey_data]].
365 137 Selvarani C
366 137 Selvarani C
367 137 Selvarani C
If a raw point refers to a point feature, reconciling it means moving it to another category/table meant for point features, not for lines/polygons. So, *+only raw points of point features can be reconciled+*. Raw points pertaining to line features and/or polygon features cannot be reconciled, so these raw points will remain in their original wrong table.
368 137 Selvarani C
369 137 Selvarani C
370 137 Selvarani C
PLEASE NOTE: 
371 137 Selvarani C
The attributes of a line/polygon, which are derived from their defining raw points, will not be modified by any reconciliation, because *+only raw points of point features can be reconciled+*.
372 137 Selvarani C
373 137 Selvarani C
h2. How to perform Reconciliation
374 137 Selvarani C
375 137 Selvarani C
To perform reconciliation of points: Login to Gisaf -> click on the G icon on the upper-left corner of the website page -> Manage -> Reconciliation by orig.ID
376 137 Selvarani C
377 137 Selvarani C
On the right end side of the screen, clicking on the field "Destination" a list of all categories will appear: these categories refer not to the Raw survey tables, but to the V_ tables of the database (points, lines, polygons).
378 137 Selvarani C
379 137 Selvarani C
Under it, in the field "Original ID", the original point number of the point to be reconciled is to be entered.
380 137 Selvarani C
381 137 Selvarani C
Clicking on the "Search points" button, the result shows the database unique id of the point, its survey category, its survey date, its geometry type (point, line, polygon), and the Project the point belongs to. In case of multiple points with the same original id (in case of different Projects, the field number of points might be not unique if the numbering of points in the field has restarted) all points having that original id are displayed: thanks to their date or Project or type, it is easy to identify the correct point to be reconciled.
382 137 Selvarani C
383 137 Selvarani C
Once the point to be reconciled is identified, clicking on the button "Reconcile" will run the reconciliation, and a message will appear stating that it has been done successfully. An error message can appear if a reconciliation of a raw point of a line/polygon feature has been attempted: this type of points cannot in fact be reconciled.
384 137 Selvarani C
385 137 Selvarani C
Once a raw point has been reconciled, *+it cannot be reconciled again+*. In case a raw point has been wrongly reconciled, it cannot be reconciled again through the above procedure, but it has to be reconciled manually through QGis or pgadmin software.
386 137 Selvarani C
387 138 Selvarani C
h1. Documentation - Status and Status Changes
388 138 Selvarani C
389 138 Selvarani C
Status have been created to keep track og changes in surveyed features.
390 138 Selvarani C
It is an additional (though provided for in AIA standards), single digit value, at the end of the Gisaf Category/Cad layer name.
391 138 Selvarani C
Each Status need to have a corresponding CAD layer/Gisaf Category, with a short code associatedto it, so that field entries can be done easily.
392 138 Selvarani C
393 138 Selvarani C
Status have been defined as follows:
394 138 Selvarani C
N - New Work
395 138 Selvarani C
E - Existing o remain
396 138 Selvarani C
D - Existing to demolish, Demolished or Changed
397 138 Selvarani C
F- Future work, Proposed feature
398 138 Selvarani C
T - Temporary work
399 138 Selvarani C
M - Item to be moved
400 138 Selvarani C
X - Not in contract
401 138 Selvarani C
402 138 Selvarani C
403 138 Selvarani C
By default, Status is defined as E (Existing). Status can anyhow be changed later using gAdmin, or QGis (through the PostGis connection).
404 138 Selvarani C
It needs to be done manually, one feature (point, line, polygon) at a time.
405 138 Selvarani C
406 138 Selvarani C
407 1 Philippe May
In the future Status changes might be incorporated in the Admin panel.
408 138 Selvarani C
409 185 Selvarani C
410 67 Giulio Di Anastasio
411 139 Selvarani C
h1. Documentation - Tags retained after re-import of same geometry
412 139 Selvarani C
413 139 Selvarani C
Documentation - Tags retained after re-import of same geometry
414 139 Selvarani C
The linework for infrastructure survey carried out by Eric Chacra in May 2020 was originally imported with a problem of ambiguity in the "Accuracy" table and in the "Accuracy" table.
415 139 Selvarani C
The result was that lines did not inherit the attributes survey date, accuracy, equipment, surveyor.
416 139 Selvarani C
Nevertheless lines were displayed on the Gisaf map, without these attributes, and tags were given to some of these lines.
417 139 Selvarani C
418 139 Selvarani C
The values for the two tables ("Accuracy", "Accuracy") have been corrected, ambiguity resolved.
419 139 Selvarani C
The lines in the layers V-ELEC-UGND------E and V-COMM-CABL------E have been then reimported, the attributes have been properly assigned, and the tags have been retained.
420 139 Selvarani C
421 139 Selvarani C
4 August 2020
422 139 Selvarani C
423 67 Giulio Di Anastasio
h1. Access to data
424 67 Giulio Di Anastasio
425 67 Giulio Di Anastasio
h2. Connection to server directly from CSR
426 67 Giulio Di Anastasio
427 67 Giulio Di Anastasio
To connect to the server directly without going through Aurinoco server, the correct url is
428 67 Giulio Di Anastasio
http://gis.csr.av
429 67 Giulio Di Anastasio
430 67 Giulio Di Anastasio
h2. Connection to Gisaf via QGis through WFS / OGC API
431 67 Giulio Di Anastasio
432 67 Giulio Di Anastasio
This works only on QGis from version 3.14.15 onward
433 67 Giulio Di Anastasio
434 67 Giulio Di Anastasio
In the browser, click on WFS/OGC API, then right-click to create a new connection
435 67 Giulio Di Anastasio
Give a name (e.g. OGC API Qgis Gisaf)
436 67 Giulio Di Anastasio
Give the url https://gis.auroville.org.in/ogcapi
437 67 Giulio Di Anastasio
438 67 Giulio Di Anastasio
Under the WFS Options box, on Version dropdown, the default option "Maximum" works just fine
439 67 Giulio Di Anastasio
Click on OK
440 67 Giulio Di Anastasio
The list of layers will appear in the Browser under WFS/OGC API.
441 68 Giulio Di Anastasio
442 68 Giulio Di Anastasio
443 68 Giulio Di Anastasio
h1. How to create a new projection in QGis
444 68 Giulio Di Anastasio
445 68 Giulio Di Anastasio
To create a new projection in QGis, go to menu "Settings", and click on "Custom Projections".
446 68 Giulio Di Anastasio
A pop-up window appears with a list of all projections defined in QGis projects used by the user so far.
447 68 Giulio Di Anastasio
Click on the green "+" sign on the right top part of the window to create a new projection.
448 68 Giulio Di Anastasio
In the "Name" box, type "TM CSRAUSPOS SF1" (which means TM = Transverse Mercator projection; CSRAUSPOS = theparameters for this projection are derived from the processing of DGPS raw data by AUSPOS -  Online GPS Processing Service - https://www.ga.gov.au/scientific-topics/positioning-navigation/geodesy/auspos; SF1 = Scale Factor is 1).
449 68 Giulio Di Anastasio
In the "Format" dropdown list, select "Proj String (legacy - Not Recommended)"
450 68 Giulio Di Anastasio
In the "Parameters" box, paste the following "+proj=tmerc +lat_0=12.01605433+lon_0=79.80998934 +k=1 +x_0=370455.630 +y_0=1328608.994 +ellps=WGS84+towgs84=0,0,0,0,0,0,0 +units=m +no_defs".
451 68 Giulio Di Anastasio
452 68 Giulio Di Anastasio
Finally, click on OK.
453 68 Giulio Di Anastasio
454 68 Giulio Di Anastasio
In a more explicit  way, the parameters mean the following:
455 68 Giulio Di Anastasio
Map Projection:        TransverseMercator (TM)
456 68 Giulio Di Anastasio
False Easting:          370455.6300
457 68 Giulio Di Anastasio
False Northing:        1328608.9940
458 68 Giulio Di Anastasio
Latitude of Origin:    12°00'57.79560" (DMS)   12.01605433 (DD)
459 68 Giulio Di Anastasio
Central Meridian:     79°48'35.96164"  (DMS)   79.80998934 (DD)
460 68 Giulio Di Anastasio
Scale Factor:             1.00000000
461 68 Giulio Di Anastasio
Zone Width:              6.0°
462 69 Giulio Di Anastasio
463 69 Giulio Di Anastasio
h1. Elimination of Duplicate points – General criteria
464 69 Giulio Di Anastasio
465 69 Giulio Di Anastasio
It might happen that the same physical feature (e.g. a tree, or a pole) is surveyed more than once: this can happen because there are many physical features in an area, and the survey needs more than one station. So, for example a tree is surveyed from a station, and gets a serial number on that date. When the station is then changed, it might happen that the same tree is resurveyed: another serial number is given, and possibly a different date, if the survey from the second station happened on a different day.
466 69 Giulio Di Anastasio
It is clear that the same tree is then represented with two different points, which means that two different trees exist: but only one tree really exist in the physical reality.
467 69 Giulio Di Anastasio
It is clear that one of the two points is redundant and needs to be removed. If this is noted by the surveyor directly in the field, then the issue is solved by the surveyor himself during processing time.
468 69 Giulio Di Anastasio
If instead, due to various reasons, it was not noted by the surveyor in the field, it will need to be cleaned after the processing, possibly by post-processing staff.
469 69 Giulio Di Anastasio
How to identify duplicate points?
470 69 Giulio Di Anastasio
The following criteria can be used:
471 69 Giulio Di Anastasio
1.	The distance between the two points is less than 30 cm (trees are surveyed if their trunk diameter is at least about 20 cm, so in 30 cm cannot exist two of them)
472 69 Giulio Di Anastasio
2.	The orig_id (serial number) of the points are not in series
473 69 Giulio Di Anastasio
3.	The survey date is not the same
474 69 Giulio Di Anastasio
4.	In case of trees, the species of trees is the same
475 69 Giulio Di Anastasio
5.	5. In case of trees, the tree type is not TDEF (because TDEF are mapped irrespective of their diameter, so they can actually have a small trunk, and two of them might exist in 30 cm), not OT (many TDEF species are surveyed as OT if not otherwise indicated by a botanist)
476 69 Giulio Di Anastasio
6.	The context needs to be evaluated: if one tree is deleted in an area where many trees exist in a limited space, then loosing one in the map is not a big error. If instead one tree is deleted where there are very few trees, then it might be a big loss.
477 70 Giulio Di Anastasio
478 70 Giulio Di Anastasio
479 70 Giulio Di Anastasio
h1. Linework for the Survey Area
480 70 Giulio Di Anastasio
481 70 Giulio Di Anastasio
h2. 1.	Creation of Initial Linework in QGIS using Survey points import - (Ram, System 4)
482 70 Giulio Di Anastasio
483 70 Giulio Di Anastasio
Initial Linework in QGIS is started by surveyor with the knowledge from the Field. For this step, points are simply imported into the QGIS from the field text file (.csv or .txt). CRS needs to be TM-AUSPOS. The box of “First record has field names” shall not be ticked. In Point Coordinates, select the correct field for x, for y and for z (usually “field_2” for x, “field_3” for y and “field_4” for z). Points can be styled using the “Categorized” style in “Symbology”, using “Field_5” as value, or using a Rule-based symbology using the category (field 5) as filter.
484 74 Selvarani C
Linework is created by connecting points having same description and belonging to the same physical feature. *All line and polygon features are created as lines*.
485 70 Giulio Di Anastasio
The Initial Linework for the Survey Area is also stored temporarily in 
486 70 Giulio Di Anastasio
487 74 Selvarani C
+D: > AVSM > Zone-Survey number (eg RZ-01) > Survey Area (eg J) > Temporary WD+
488 70 Giulio Di Anastasio
489 70 Giulio Di Anastasio
h2. Note: The line shapefiles / Geopackages shall be in CRS: TM AUSPOS
490 70 Giulio Di Anastasio
491 70 Giulio Di Anastasio
h2. 2.	Creation of final working drawing Shapefiles / Geopackages - (Selvarani, System 1)
492 1 Philippe May
493 1 Philippe May
Final working drawing Shapefiles / Geopackages are created from the Initial Linework of Survey Area.
494 70 Giulio Di Anastasio
As the Surveyor draws all features as lines (both for lines and polygons features), the following actions shall be done:
495 74 Selvarani C
1.	*If features are lines:*
496 70 Giulio Di Anastasio
•	Export the shapefile / geopackage into the final working drawing folder (Final WD), in separate folders according to its type (e.g. BLDG, FENC, ROAD, etc).
497 70 Giulio Di Anastasio
498 70 Giulio Di Anastasio
h2. The CRS for the export shall be EPSG:4326 - WGS 84
499 70 Giulio Di Anastasio
500 74 Selvarani C
2.	*If features are polygons:*
501 1 Philippe May
•	Lines shall be converted into polygons:
502 74 Selvarani C
503 1 Philippe May
to do it, first click on the layer to be converted to make it active (e.g. WD-CZ-01-F-LL84_V-BLDG-MHOL------E), then go to “Vector” Menu, click on Geometry Tools, click on Line to Polygons:
504 87 Selvarani C
505 84 Selvarani C
!https://redmine.auroville.org.in/attachments/download/9760/Line%20to%20Polygon%20Menu.png!
506 70 Giulio Di Anastasio
507 86 Selvarani C
508 1 Philippe May
The new window for “Lines to Polygons” conversion will appear:
509 86 Selvarani C
510 86 Selvarani C
511 83 Selvarani C
!https://redmine.auroville.org.in/attachments/download/9762/Lines%20to%20Polygon%20Window.png!
512 1 Philippe May
513 71 Giulio Di Anastasio
•	Always cross check the input layer, to make sure that the input layer is the active one
514 71 Giulio Di Anastasio
•	Save the output in a temporary layer
515 74 Selvarani C
•	The temporary layer will be listed in the list of layers, it shall be exported to the saving location as +D: > Survey > Zone-Survey Number > Final WD > Survey Area SHP+ (eg . D: > Survey > GB-01 > Final WD > A-Shp)
516 72 Giulio Di Anastasio
517 74 Selvarani C
h2. The CRS for the export shall be EPSG:4326 - WGS 84
518 74 Selvarani C
519 72 Giulio Di Anastasio
Once all the shapefiles / geopackages are exported in Final WD, for each of the newly exported layers the Topology Checker Tool shall be used.
520 1 Philippe May
521 74 Selvarani C
h2. Linework for the whole Survey Zone
522 74 Selvarani C
523 74 Selvarani C
h2. 1.	Merging Shapefiles / Geopackages - (Selvarani, System 1)
524 74 Selvarani C
525 72 Giulio Di Anastasio
A copy of the Zone Master shapefiles / geopackages are taken from System 4 and stored in Temp Folder on Desktop in System 1.
526 1 Philippe May
Master shapefiles / geopackages are merged with the Survey Area shapefiles / geopackages:
527 72 Giulio Di Anastasio
•	To do it, go to “Vector” Menu, click on Geoprocessing Tools, then click on Union:
528 72 Giulio Di Anastasio
529 85 Selvarani C
!https://redmine.auroville.org.in/attachments/download/9763/Union%20Menu.png!
530 1 Philippe May
531 1 Philippe May
532 85 Selvarani C
The new window for “Union” will appear:
533 1 Philippe May
534 86 Selvarani C
535 86 Selvarani C
!https://redmine.auroville.org.in/attachments/download/9764/Union%20Window.png!
536 73 Giulio Di Anastasio
537 74 Selvarani C
•	To make sure that the right geometry is generated by this process (“line” type, not “Multiline”, and similarly “Polygon” type, not “Multipolygon), we need to always keep the *Master shapefile* (e.g. Final-CZ-01-2021-02-05-LL84_V-BLDG-MHOL------E) *as Input layer*, and the Survey Area shapefile as Overlay Layer (e.g. WD-CZ-01-F-LL84_V-BLDG-MHOL------E).
538 73 Giulio Di Anastasio
•	(The output can be saved to a file, as the CRS should already be EPSG4326 – WGS84.)
539 73 Giulio Di Anastasio
540 74 Selvarani C
h2. 2.	Storing Shapefiles / Geopackages - (Selvarani, System 1)
541 74 Selvarani C
542 74 Selvarani C
Save the merged shape file in the correct location in Final folder as +D: > Survey > Zone-Survey Number > Final+ (eg . D: > Survey > GB-01 > Final)
543 73 Giulio Di Anastasio
Date in the name of Final Shapefile / Geopackage needs to be updated.
544 73 Giulio Di Anastasio
Once the merging operation is completed, the copy of Master shapefile / geopackage is deleted from the Temp folder.
545 73 Giulio Di Anastasio
546 74 Selvarani C
h2. 3.	Topology check of merged shapefiles
547 74 Selvarani C
548 73 Giulio Di Anastasio
The topology checker is applied again on the merged shapefiles / geopackages.
549 73 Giulio Di Anastasio
The “id_field” shall be removed from the attribute table.
550 73 Giulio Di Anastasio
551 74 Selvarani C
h2. 4.	Archive and replace the Master Shapefiles / Geopackages (Ram, System 4)
552 74 Selvarani C
553 73 Giulio Di Anastasio
Archive the previous master shapefiles / geopackages on system 4, and copy the new merged shapefiles / geopackages in its place.
554 74 Selvarani C
*Then delete the Merged Shapefile / Geopackage folder from System 1.
555 74 Selvarani C
*
556 73 Giulio Di Anastasio
557 74 Selvarani C
h2. 5.	Note about Shapefiles and Geopackages
558 73 Giulio Di Anastasio
559 73 Giulio Di Anastasio
All the above works are usually done using shapefile format, in QGIS latest version (3.16.3).
560 221 Selvarani C
The Geopackage export is done in QGis versions older than 3.12 (e.g. 3.4, 3.6, 3.8, 3.10) so that the lines are not saved as “Multilines” but as “Lines”and polygons are not saved as “Multipolygons” but as “Polygons”. This is very important to be noted, as Gisaf database does not accept the Multipolygon and Multiline geometry types.
561 221 Selvarani C
A different way to create "Polygons" is to use the command Vector -> Geometry Tools -> Multipart to Single Parts and apply it to the layer: from "Multipolygon" it will become "Polygon" (check in layer Properties).
562 222 Selvarani C
As on 13 March 2021, Gisaf can accept Multipolygon layers, because the command "Multipart to Single parts" has been integrated into the importing command (see Redmine ticket #11691)
563 223 Selvarani C
564 223 Selvarani C
h1. Creating 3D Shapefile/Geopackage
565 223 Selvarani C
566 223 Selvarani C
While creating a Shapefile/Geopackage, the File name/Database-table name, Geometry type and CRS have to be entered.
567 223 Selvarani C
In order to create a 3D Shapefile/Geopackage, the additional dimensions *"Z(+M values)"/ "Include Z dimension"* has to be ticked: this way the 3D Shapefile/Geopackage is accepted by Gisaf without errors, otherwise the Shapefile/Geopackage can't be imported in Gisaf because the Z dimension is missing (the geometries in the database are all 3D).
568 224 Giulio Di Anastasio
569 224 Giulio Di Anastasio
h1. Exporting from QGis (shapefiles and/or geopackages) to CAD dxf format
570 224 Giulio Di Anastasio
571 226 Giulio Di Anastasio
An algorithm has been created by Selvarani, to see the whole process click here:
572 226 Giulio Di Anastasio
https://redmine.auroville.org.in/projects/gis/wiki/Automatic_export_from_QGis_to_dxf