Project

General

Profile

Wiki » History » Version 280

Giulio Di Anastasio, 06/05/2021 11:51

1 1 Philippe May
h1. %{color:BLUE}  Wiki%
2 227 Giulio Di Anastasio
3 232 Giulio Di Anastasio
h1. [[CSR Geomatics Team]]
4 231 Giulio Di Anastasio
5 229 Giulio Di Anastasio
h1. [[Collaborations]]
6 229 Giulio Di Anastasio
7 240 Giulio Di Anastasio
h1. [[Data Analysis]]
8 234 Giulio Di Anastasio
9 241 Giulio Di Anastasio
h1. [[CSR Geomatics Server Setup]]
10 234 Giulio Di Anastasio
11 245 Giulio Di Anastasio
h1. [[Database Backup and Restoration]]
12 243 Giulio Di Anastasio
13 243 Giulio Di Anastasio
h1. [[CAD to GIS Workflow]] %{color:RED}  - TO BE UPDATED%
14 248 Giulio Di Anastasio
15 246 Giulio Di Anastasio
h1. [[GIS Conventions]]
16 246 Giulio Di Anastasio
 
17 252 Giulio Di Anastasio
h1. [[Working with Gisaf]]
18 96 Selvarani C
19 92 Selvarani C
h1. [[Gisaf Workflow]]
20 252 Giulio Di Anastasio
21 256 Giulio Di Anastasio
h1. [[GIS to CAD Workflow]] %{color:RED}  - TO BE UPDATED%
22 94 Selvarani C
23 257 Giulio Di Anastasio
h1. [[Weather Station]]
24 142 Selvarani C
25 259 Giulio Di Anastasio
h1. [[GDAL - Useful Tips]]
26 5 Philippe May
27 262 Giulio Di Anastasio
h1. [[Gisaf - Auxiliary Data (non-geographical data)]]
28 143 Selvarani C
29 264 Giulio Di Anastasio
h1. [[QGis - Useful Tips]]
30 264 Giulio Di Anastasio
31 266 Giulio Di Anastasio
h1. [[AutoCAD Civil 3D - Useful Tips]]
32 266 Giulio Di Anastasio
33 280 Giulio Di Anastasio
h1. [[Wells]]
34 8 Philippe May
35 272 Giulio Di Anastasio
h1. [[Gisaf Data Modifications]]
36 272 Giulio Di Anastasio
37 274 Giulio Di Anastasio
h1. [[Other Useful Softwares]]
38 272 Giulio Di Anastasio
39 262 Giulio Di Anastasio
40 1 Philippe May
41 274 Giulio Di Anastasio
42 274 Giulio Di Anastasio
43 107 Selvarani C
44 110 Selvarani C
45 110 Selvarani C
46 110 Selvarani C
47 113 Selvarani C
48 270 Giulio Di Anastasio
49 270 Giulio Di Anastasio
50 119 Selvarani C
51 119 Selvarani C
52 127 Selvarani C
h1. Importing point data (TS and RTK) to GISAF 
53 127 Selvarani C
54 127 Selvarani C
# *Gisaf Admin > Basket > Survey data > "Project" > "surveyor" > TS/ RTK > upload > import.*
55 127 Selvarani C
# *Auto import of Raw points data* (changes from Raw to Shapefiles, the point files)
56 1 Philippe May
  In GISAF Admin > Other> Project > select the project for which you want to import the data > with selected > Auto import to GIS...
57 127 Selvarani C
58 172 Selvarani C
59 128 Selvarani C
60 128 Selvarani C
61 128 Selvarani C
62 129 Selvarani C
h1. Miscellaneous- Civil 3D
63 129 Selvarani C
64 129 Selvarani C
*Autodesk civil 3D Geotechnical module- for borehole data :- To analyse borehole data, To make profiles and calculate volumes*
65 129 Selvarani C
66 129 Selvarani C
https://www.youtube.com/watch?v=0Be9kShBou0
67 129 Selvarani C
Reference videos
68 129 Selvarani C
Exploring:
69 129 Selvarani C
https://www.youtube.com/watch?v=gr-ISPzLcU0
70 129 Selvarani C
Modelling:
71 129 Selvarani C
https://www.youtube.com/watch?v=Pz0-HOoiBrs
72 129 Selvarani C
solids
73 129 Selvarani C
https://www.youtube.com/watch?v=mQ0Yeh6tZA8
74 129 Selvarani C
https://www.youtube.com/watch?v=4eMsUiYBhuE
75 129 Selvarani C
76 129 Selvarani C
77 129 Selvarani C
*CIVIL 3D Survey*
78 129 Selvarani C
Getting started- Always open a new drawing with a template. 
79 129 Selvarani C
80 129 Selvarani C
*Description key sets*- (till 18:30) https://www.youtube.com/watch?v=mmwkkRyBkS0
81 129 Selvarani C
As when the points come into the drawing, they are going to be filtered with description key sets and the description key sets will assign properties to the points.
82 129 Selvarani C
Tool space > settings > points > description key sets
83 129 Selvarani C
“Description key sets name”> edit key- to view the points in a list: 
84 129 Selvarani C
Automatic linework (18:30- till end)
85 129 Selvarani C
Survey > Linework code sets- for automatic linework
86 129 Selvarani C
87 129 Selvarani C
*Using Master view*
88 129 Selvarani C
Uses-
89 129 Selvarani C
copy styles from one drawing to other
90 129 Selvarani C
copy drawing data from one drawing to other (data referencing)
91 129 Selvarani C
92 129 Selvarani C
*Civil 3d surface model*- from points- break lines
93 129 Selvarani C
https://www.youtube.com/watch?v=wC40rdmDEMo
94 129 Selvarani C
moving for example- point groups under point groups- 
95 129 Selvarani C
96 129 Selvarani C
*Civil 3D Planning and Analysis*
97 129 Selvarani C
Workspace: Planning and Analysis
98 129 Selvarani C
1. Working with Object data (GIS Attributes)
99 129 Selvarani C
Map Setup> Define object data> New table> Define new object data table start defining fields
100 129 Selvarani C
101 129 Selvarani C
102 130 Selvarani C
103 130 Selvarani C
h1. Documentation- Rain Gauge
104 130 Selvarani C
105 130 Selvarani C
h2. Manual Rain Gauge
106 130 Selvarani C
107 130 Selvarani C
*Why it is important to comply to standards?*
108 130 Selvarani C
109 130 Selvarani C
1. Consistency for comparing rainfall data in different places within Auroville since rainfall varies in different parts of Auroville.
110 130 Selvarani C
111 130 Selvarani C
2. Making it possible to use for any kind of scientific analysis by contribution to the primary data in sustainable water management for Auroville. 
112 130 Selvarani C
113 130 Selvarani C
*Proposed standards and ethics:*
114 130 Selvarani C
115 130 Selvarani C
1. The time of taking the reading - 08:30 am. 
116 130 Selvarani C
The rain is recorded over a period of 24 hours- 8:30 of previous day till 8:30 of the present day, and the date is put as the present day. For example, if the rain is measured at 8:30 am on 10 dec 2017, then it is logged in as on 10 dec 2017.
117 130 Selvarani C
118 130 Selvarani C
2. Manual rain gauge typically used- green cylindrical and a collection jar.  
119 130 Selvarani C
  
120 130 Selvarani C
3. Measuring jar: 10 mm corresponding jar
121 130 Selvarani C
122 130 Selvarani C
 
123 130 Selvarani C
Area of the rim of manual rain gauge= 200 square cm
124 130 Selvarani C
Diameter of rim= approximately 16 cm
125 130 Selvarani C
*NOTE:* - If another kind of rain gauge is used, it should be used with the corresponding measuring jar as type of jar and calibration on jar depends on the area of rim. Any cases of different rain gauge other than above should be reported before use for recording purposes.
126 130 Selvarani C
127 130 Selvarani C
4. The minimum record-able unit is 0.2 mm (Least measure on the 10 mm cylinder). T is marked for below 1 mm.
128 130 Selvarani C
129 130 Selvarani C
5. The reading should be preferably submitted in an excel sheet (the data sheet will be provided). The frequency of sending data can be daily or weekly.
130 130 Selvarani C
6. The rain gauge should be kept in a safe and open to sky area clear from tree cover or objects in surrounding so that direct rain falls into it and not from trees/ objects etc. Preferably roof top or a higher place on ground.
131 130 Selvarani C
7. A check on the rain gauge should be kept when there is long period of gap in rainy days to make sure it was emptied out and there is no blockage. 
132 130 Selvarani C
133 130 Selvarani C
8. In case of doubt in reading or incorrect measure, "incorrect measure" should be marked
134 130 Selvarani C
135 130 Selvarani C
136 130 Selvarani C
*Ethics*
137 130 Selvarani C
1. Readings should not be missed. Time to time check on rain gauge is required during long periods of no rain (especially when the rain season is approaching). 
138 130 Selvarani C
This is important because firstly, if it rained in the night and it wasn’t recorded, reading for the day is lost; Secondly, if the jar was not emptied, it gives incorrect subsequent reading. 
139 130 Selvarani C
140 130 Selvarani C
2.  If the person is not going to be available temporarily where the rain gauge is kept, the responsibility should be passed on to someone they can rely on after explaining the standards. 
141 130 Selvarani C
142 130 Selvarani C
*Important-*
143 130 Selvarani C
Data won’t be published in case of non-compliance to standards. This is to maintain the sanctity of proper scientific data collection and to keep it reliable for sharing by publishing. Discussions regarding standards are welcomed.
144 130 Selvarani C
CSR Geomatics Team is placed on first floor in CSR, Auroshilpam.
145 130 Selvarani C
146 130 Selvarani C
*AV rain data publication*: (http://gis.auroville.org.in/measures/raingauge_av).
147 130 Selvarani C
148 130 Selvarani C
Adding Rain gauge to our web portal (GISAF), following information is needed:
149 130 Selvarani C
1. GPS co-ordinates of location of placing the rain gauge (can be obtained using mobile phones easily)
150 130 Selvarani C
2. Name, place of residence & contact number- Mobile & Landline
151 130 Selvarani C
152 130 Selvarani C
h2. Automatic Rain Gauge
153 130 Selvarani C
154 130 Selvarani C
There is a possibility of publishing rain data coming from Automatic rain gauges like (id 15, auro orchard) and Weather stations. 
155 130 Selvarani C
For Automatic rain gauges, we can upload the files coming from the rain gauge directly into GISAF.
156 130 Selvarani C
157 130 Selvarani C
158 130 Selvarani C
_Thank you for contribution towards a sustainable management of Water in Auroville through Data collection._
159 130 Selvarani C
160 130 Selvarani C
161 131 Selvarani C
162 131 Selvarani C
h1. Documentation- Wells Monitoring (Manual and Automatic / Piezometer by Bala)
163 131 Selvarani C
164 131 Selvarani C
h2. Manual- Using tape with a sensor
165 131 Selvarani C
166 131 Selvarani C
Timings: The person assigned the job of monitoring collects the readings in three slots.
167 131 Selvarani C
168 131 Selvarani C
1.	Between 6 am to 7 am till about 9 am to 10 am.
169 131 Selvarani C
2.	Between 11 am and 1 pm
170 131 Selvarani C
3.	Between 2 pm and 5 pm
171 131 Selvarani C
172 131 Selvarani C
Things required:
173 131 Selvarani C
•	Notebook
174 131 Selvarani C
•	Pen
175 131 Selvarani C
•	Measuring tape
176 131 Selvarani C
•	Vehicle for movement
177 131 Selvarani C
178 131 Selvarani C
179 131 Selvarani C
*Ethics*- 
180 131 Selvarani C
•	The monitoring should happen in coordination with the community members/ care taker (assigned by the person in charge from within the community). There should be a clear communication from monitor side as to which days the monitoring of a well has to happen and at around what time so that it can be made sure that the pump is not turned on before monitoring. In case a pump was turned on in a well, the monitor should have a gap of about 5 hours on the same day before going again for monitoring.
181 131 Selvarani C
•	The monitor should have contact numbers of the person in charge/ care taker for any communication.
182 131 Selvarani C
•	If the monitoring is stopped for any reason at any point, the monitor should communicate the same to the person concerned.
183 131 Selvarani C
•	The monitor shall take responsibility to inform any kind of changes in a well in terms of its functioning etc.
184 131 Selvarani C
•	The monitor should report to the CSR Geomatics Team who has responsibility to publish data.
185 131 Selvarani C
•	The monitor published the data to the website and works with the geomatics team.
186 131 Selvarani C
187 131 Selvarani C
188 131 Selvarani C
*Definitions*:
189 131 Selvarani C
A master file is maintained with the records of the wells from the field. It is updated when a new well is located. The following set of information are filled out in the Masterfile. 
190 131 Selvarani C
191 131 Selvarani C
*Well type*
192 131 Selvarani C
1.	Open
193 131 Selvarani C
2.	Borewell
194 131 Selvarani C
3.	Dug cum borewell
195 131 Selvarani C
196 131 Selvarani C
*Well status*
197 131 Selvarani C
1.	In use
198 131 Selvarani C
2.	Not in use
199 131 Selvarani C
3.	Closed
200 131 Selvarani C
201 131 Selvarani C
*Pump Status*
202 131 Selvarani C
1.	Functioning
203 131 Selvarani C
2.	Not functioning
204 131 Selvarani C
205 131 Selvarani C
*Pump Automation*
206 131 Selvarani C
1.	Manual 
207 131 Selvarani C
2.	Automatic
208 131 Selvarani C
209 131 Selvarani C
* Non-accessibility factors*
210 131 Selvarani C
1.	Dogs
211 131 Selvarani C
2.	Heavy slab
212 131 Selvarani C
3.	Narrow casing
213 131 Selvarani C
4.	enclosed/locked
214 131 Selvarani C
5.	permission
215 131 Selvarani C
216 131 Selvarani C
*Data Matching Accuracy (reconnecting with harvest wells data)*
217 131 Selvarani C
1.	High
218 131 Selvarani C
2.	Medium
219 131 Selvarani C
3.	Low
220 131 Selvarani C
221 131 Selvarani C
*Well coordinates*
222 131 Selvarani C
For a new well, coordinates of the well position are taken on a mobile GPS.
223 131 Selvarani C
Person In charge
224 131 Selvarani C
For communication purpose, contact the person in charge as recorded in directory.
225 131 Selvarani C
226 131 Selvarani C
h2. Automatic- Piezometer (by bala, to be edited) 
227 131 Selvarani C
228 131 Selvarani C
The calibrations and setting are already done by Azha
229 131 Selvarani C
The piezometer is taken to the site of unused well. First the depth and Water level  is checked manually using water meter
230 131 Selvarani C
Then the sensor of the piezometer is inserted into the well 1m above the bottom of the well, so the sensor does not get affected from mud or water inside
231 131 Selvarani C
The sensor sends the data to an electronic board which translates the signal into the proper output, and then it sends the signal to the transmitting device above, on the ground surface. The transmitter then sends it to the receiver at CSR, the signal is then routed to Talam office through Interneet, enters the software and the reading is processed.
232 131 Selvarani C
The piezometer sometimes does not have proper signal and so it would not be able to send it
233 131 Selvarani C
234 132 Selvarani C
h1. Documentation- Flow meter by Bala
235 132 Selvarani C
236 132 Selvarani C
Flow meter – for checking the flow of water in pipe
237 132 Selvarani C
238 132 Selvarani C
CSR bought ultrasonic flow meter from Chennai and the company (company name? )trained Bala and Vijai(CSR) on how to set it up.
239 132 Selvarani C
we tested in the west water system pump in csr. then we test in many more place .
240 132 Selvarani C
 ami, aurodam and buddha garden borewell pumps . we got request from the water service 
241 132 Selvarani C
Cross check they flow meters .
242 132 Selvarani C
 so before that  we want to know how our meter works. 
243 132 Selvarani C
so we tested in our tank . the pump pumping from our sump tank so the flow will be stranded.  when you pump in the bore well  the flow goes up and down. 
244 132 Selvarani C
we tested two times in  15 minutes.and one time 30 minutes,so we got variation between this three. 
245 132 Selvarani C
we find 1,5 % error but the flow meret they said  1 % error only.
246 132 Selvarani C
we cross checked the flow meter of the water service we find some error  in they meter also. 
247 132 Selvarani C
we tried they bore well flowmeter and they sump tank flow meter also.and we find some error also in they meters.  
248 132 Selvarani C
then we find some error in (AVWS) meter also.
249 132 Selvarani C
It is used to check the flow of water in pipe
250 132 Selvarani C
tools used 
251 132 Selvarani C
First the outer diameter of the pipe has to be entered in the device. It is measured using Vernier caliper
252 132 Selvarani C
Then the thickness of the pipe is set which is also measured using vernier
253 132 Selvarani C
Also the device asks for the material of the pipe. If the pipe material is know it can be set and if it not know then there is an option which is other (mostly PVC, HDPE, and  iron) 
254 132 Selvarani C
After entering these details the device gives the spacing for the sensors. There are 2 sensors up and down which has to be fixed accordingly
255 132 Selvarani C
When the sensors are fixed the motor is turned on and the sensors send reading to the display device
256 132 Selvarani C
This is noted down once every minute and taken for 15 to 30 minutes. Then the average of this is determined. This is done because of the variation in the flow. This gives the flow rate in 1 hour.
257 1 Philippe May
This was done in different places to check to flow rate
258 132 Selvarani C
259 177 Selvarani C
260 133 Selvarani C
261 133 Selvarani C
h1. Documentation- DST- Vegetation Indexing
262 133 Selvarani C
263 133 Selvarani C
h2. Steps for Dzetsaka Classification tool for Vegetation indexing in QGIS
264 133 Selvarani C
265 133 Selvarani C
 
266 133 Selvarani C
 1. Install the plugin Dzetsaka classfication tool.
267 133 Selvarani C
 2. Open the Raster from the Survey.
268 133 Selvarani C
 3. Create a polygon shapefile for index sampling. Mark polygons and give the ID's (1,2,3 for Tree, grass, bare land etc) Cover the variations in samples as much as possible. 
269 133 Selvarani C
     More the samples, better the indexing.
270 133 Selvarani C
 4. Apply Dzetsaka Classification tool, Select the base raster and the sample- index polygon shapefile created in step 3.
271 133 Selvarani C
 5. The result is a Raster with DN numbers specified in the Shapefile in step 3.
272 133 Selvarani C
 6. Apply the Sieve raster command (Raster> Analysis> Sieve)- Try different threshold numbers and view the results till the noise is removed from the Raster.
273 133 Selvarani C
 7. Polygonise the Raster to Vector (From processing)
274 133 Selvarani C
 8. Run the v.generalise tool on the shapefile. This tool removes the pixelated boundaries of the polygons in the Vector.
275 133 Selvarani C
276 134 Selvarani C
h1. Documentation- DST- Interpolation (Processing toolbox)
277 134 Selvarani C
278 134 Selvarani C
The following 4 tools have been mostly explored and the results were compared. The ones used for quick analysis are 1. Cubic Spline and 2. V.surf.spline . The rest of the tools are for further exploration and used depending on the need of the project. In some tools, the elevation values of points should be stored in the attribute table (Using field calculator and giving command - *Z($Geometry)* )
279 134 Selvarani C
280 134 Selvarani C
# Interpolate (Cubic spline) - SAGA
281 134 Selvarani C
# V.surf.bspline - GRASS. Parameters to set - cell size. Set this parameter above 0.001 ( 0.00001, 0.000001 etc) and check the results.
282 134 Selvarani C
# V.surf.rst - GRASS
283 134 Selvarani C
# Krigging - SAGA
284 1 Philippe May
285 181 Selvarani C
[[Documentation- DST- Survey- Office workflow]]  - *%{color:RED} to be written%*
286 135 Selvarani C
287 135 Selvarani C
h1. From CAD to GIS by Giulio
288 135 Selvarani C
289 135 Selvarani C
*FEATURES CREATION*
290 135 Selvarani C
1.	Assign a CRS to the drawing (TM-AUSPOS) (MAPCSLIBRARY command)
291 135 Selvarani C
2.	Create features in CAD (Points, lines, polygons)
292 135 Selvarani C
3.	Export shapefile (a) from CAD (Output > DWG to SDF) (Convert to LL84 – 3D)
293 135 Selvarani C
294 135 Selvarani C
*FEATURES IMPORT INTO DB FIRST TIME*
295 135 Selvarani C
4.	Create zip file of the shapefile
296 135 Selvarani C
5.	Upload into the GISAF Shapefiles Basket
297 135 Selvarani C
6.	Import the shapefile into DB
298 135 Selvarani C
7.	Save the shapefile on Local Machine
299 135 Selvarani C
300 135 Selvarani C
*FEATURES IMPORT INTO DB EVERYTIME*
301 135 Selvarani C
8.	Combine the new features to corresponding last shape files (Insert the process here).
302 135 Selvarani C
9.	Follow step 4-8 again
303 135 Selvarani C
304 135 Selvarani C
*FEATURES EDITING IN QGIS*
305 135 Selvarani C
10.	Open the table in QGis
306 135 Selvarani C
11.	Save as a shapefile (b) in TM AUSPOS CRS
307 135 Selvarani C
12.	In CAD, open a new drawing and assign AUSPOS CRS
308 135 Selvarani C
13.	Import the shapefile (b) (MapImport) with Object Data (Data tab > Create Object Data > OK), tick “Import polygons as closed polylines”, then press OK
309 135 Selvarani C
14.	Edit features
310 135 Selvarani C
15.	Change workspace into “Planning and analysis”
311 135 Selvarani C
16.	Export shapefile (a) from CAD (Output > DWG to SDF) with ONLY the id selected (Data Tab > Select Attributes > Object Data > Filename > id) (Convert to LL84 – 3D)
312 135 Selvarani C
313 135 Selvarani C
*FEATURES IMPORT INTO DB*
314 135 Selvarani C
17.	Create zip file of the shapefile
315 135 Selvarani C
18.	Upload into the GISAF Shapefiles Basket
316 135 Selvarani C
19.	Import the shapefile into DB
317 135 Selvarani C
20.	Delete the shapefile from Local Machine
318 135 Selvarani C
319 135 Selvarani C
320 182 Selvarani C
321 1 Philippe May
322 136 Selvarani C
h1. QGIS- Miscellaneous
323 136 Selvarani C
324 136 Selvarani C
*QGIS Introduction:*
325 136 Selvarani C
https://www.birdseyeviewgis.com/blog/2018/2/22/my-favorite-features-of-qgis-30to-date
326 136 Selvarani C
3D visualization of raster DEM- https://www.youtube.com/watch?v=2KrCsbP0kUs
327 136 Selvarani C
328 136 Selvarani C
*Spatial query:* 
329 136 Selvarani C
Spatial Query is selection of features that satisfies a certain condition which relates to other features in a space.
330 136 Selvarani C
Using plugin- Spatial query
331 136 Selvarani C
http://www.geodose.com/2018/03/spatial-query-in-qgis-3.html
332 136 Selvarani C
Labelling:
333 136 Selvarani C
Labelling with more than one field names and in different lines
334 136 Selvarani C
|| '\n' ||
335 136 Selvarani C
336 136 Selvarani C
337 136 Selvarani C
338 136 Selvarani C
*Hierarchy of extensions*
339 136 Selvarani C
340 136 Selvarani C
File levels and their uses.
341 136 Selvarani C
https://nathanw.net/2014/03/22/all-the-q-files/
342 136 Selvarani C
343 136 Selvarani C
+The Project file (.qgs)+
344 136 Selvarani C
It contains: Layer source pointer + Style information + Composers + a whole heap of other stuff
345 136 Selvarani C
346 136 Selvarani C
+The Layer Definition file (.qlr)+
347 136 Selvarani C
It contains: Layer source pointer + Style information
348 136 Selvarani C
349 136 Selvarani C
+The QML file (.qml)+
350 136 Selvarani C
It contains: Style information
351 136 Selvarani C
352 59 Giulio Di Anastasio
h2. Giulio's documentation
353 52 Giulio Di Anastasio
354 183 Selvarani C
355 137 Selvarani C
356 137 Selvarani C
h1. Documentation - Reconciliation of points using Gisaf
357 137 Selvarani C
358 137 Selvarani C
Reconciliation of points is a procedure used when a point is stored in a wrong table, because its category was either wrongly recorded in the field by the surveyor, or it has ben reviewed later by the surveyor or the data validator and found to be wrong.
359 137 Selvarani C
360 137 Selvarani C
h2. Definition: Raw points are all points coming from the field survey. Raw points can be points referring to a Point feature (e.g. trees, or elevation points, or floor level), or points measured in the field to draw a line (e.g. the vertices of a fence) or a polygon (e.g. the corners of a building outline).
361 137 Selvarani C
362 137 Selvarani C
In the overall workflow, *+reconciliation affects only raw points+*. It takes place after the field textfile is uploaded into the basket and its points imported (raw points stored in the raw survey tables). Here you can visualize the Workflow diagram: [[Survey_data]].
363 137 Selvarani C
364 137 Selvarani C
365 137 Selvarani C
If a raw point refers to a point feature, reconciling it means moving it to another category/table meant for point features, not for lines/polygons. So, *+only raw points of point features can be reconciled+*. Raw points pertaining to line features and/or polygon features cannot be reconciled, so these raw points will remain in their original wrong table.
366 137 Selvarani C
367 137 Selvarani C
368 137 Selvarani C
PLEASE NOTE: 
369 137 Selvarani C
The attributes of a line/polygon, which are derived from their defining raw points, will not be modified by any reconciliation, because *+only raw points of point features can be reconciled+*.
370 137 Selvarani C
371 137 Selvarani C
h2. How to perform Reconciliation
372 137 Selvarani C
373 137 Selvarani C
To perform reconciliation of points: Login to Gisaf -> click on the G icon on the upper-left corner of the website page -> Manage -> Reconciliation by orig.ID
374 137 Selvarani C
375 137 Selvarani C
On the right end side of the screen, clicking on the field "Destination" a list of all categories will appear: these categories refer not to the Raw survey tables, but to the V_ tables of the database (points, lines, polygons).
376 137 Selvarani C
377 137 Selvarani C
Under it, in the field "Original ID", the original point number of the point to be reconciled is to be entered.
378 137 Selvarani C
379 137 Selvarani C
Clicking on the "Search points" button, the result shows the database unique id of the point, its survey category, its survey date, its geometry type (point, line, polygon), and the Project the point belongs to. In case of multiple points with the same original id (in case of different Projects, the field number of points might be not unique if the numbering of points in the field has restarted) all points having that original id are displayed: thanks to their date or Project or type, it is easy to identify the correct point to be reconciled.
380 137 Selvarani C
381 137 Selvarani C
Once the point to be reconciled is identified, clicking on the button "Reconcile" will run the reconciliation, and a message will appear stating that it has been done successfully. An error message can appear if a reconciliation of a raw point of a line/polygon feature has been attempted: this type of points cannot in fact be reconciled.
382 137 Selvarani C
383 137 Selvarani C
Once a raw point has been reconciled, *+it cannot be reconciled again+*. In case a raw point has been wrongly reconciled, it cannot be reconciled again through the above procedure, but it has to be reconciled manually through QGis or pgadmin software.
384 137 Selvarani C
385 138 Selvarani C
h1. Documentation - Status and Status Changes
386 138 Selvarani C
387 138 Selvarani C
Status have been created to keep track og changes in surveyed features.
388 138 Selvarani C
It is an additional (though provided for in AIA standards), single digit value, at the end of the Gisaf Category/Cad layer name.
389 138 Selvarani C
Each Status need to have a corresponding CAD layer/Gisaf Category, with a short code associatedto it, so that field entries can be done easily.
390 138 Selvarani C
391 138 Selvarani C
Status have been defined as follows:
392 138 Selvarani C
N - New Work
393 138 Selvarani C
E - Existing o remain
394 138 Selvarani C
D - Existing to demolish, Demolished or Changed
395 138 Selvarani C
F- Future work, Proposed feature
396 138 Selvarani C
T - Temporary work
397 138 Selvarani C
M - Item to be moved
398 138 Selvarani C
X - Not in contract
399 138 Selvarani C
400 138 Selvarani C
401 138 Selvarani C
By default, Status is defined as E (Existing). Status can anyhow be changed later using gAdmin, or QGis (through the PostGis connection).
402 138 Selvarani C
It needs to be done manually, one feature (point, line, polygon) at a time.
403 138 Selvarani C
404 138 Selvarani C
405 1 Philippe May
In the future Status changes might be incorporated in the Admin panel.
406 138 Selvarani C
407 185 Selvarani C
408 67 Giulio Di Anastasio
409 139 Selvarani C
h1. Documentation - Tags retained after re-import of same geometry
410 139 Selvarani C
411 139 Selvarani C
Documentation - Tags retained after re-import of same geometry
412 139 Selvarani C
The linework for infrastructure survey carried out by Eric Chacra in May 2020 was originally imported with a problem of ambiguity in the "Accuracy" table and in the "Accuracy" table.
413 139 Selvarani C
The result was that lines did not inherit the attributes survey date, accuracy, equipment, surveyor.
414 139 Selvarani C
Nevertheless lines were displayed on the Gisaf map, without these attributes, and tags were given to some of these lines.
415 139 Selvarani C
416 139 Selvarani C
The values for the two tables ("Accuracy", "Accuracy") have been corrected, ambiguity resolved.
417 139 Selvarani C
The lines in the layers V-ELEC-UGND------E and V-COMM-CABL------E have been then reimported, the attributes have been properly assigned, and the tags have been retained.
418 139 Selvarani C
419 139 Selvarani C
4 August 2020
420 139 Selvarani C
421 67 Giulio Di Anastasio
h1. Access to data
422 67 Giulio Di Anastasio
423 67 Giulio Di Anastasio
h2. Connection to server directly from CSR
424 67 Giulio Di Anastasio
425 67 Giulio Di Anastasio
To connect to the server directly without going through Aurinoco server, the correct url is
426 67 Giulio Di Anastasio
http://gis.csr.av
427 67 Giulio Di Anastasio
428 67 Giulio Di Anastasio
h2. Connection to Gisaf via QGis through WFS / OGC API
429 67 Giulio Di Anastasio
430 67 Giulio Di Anastasio
This works only on QGis from version 3.14.15 onward
431 67 Giulio Di Anastasio
432 67 Giulio Di Anastasio
In the browser, click on WFS/OGC API, then right-click to create a new connection
433 67 Giulio Di Anastasio
Give a name (e.g. OGC API Qgis Gisaf)
434 67 Giulio Di Anastasio
Give the url https://gis.auroville.org.in/ogcapi
435 67 Giulio Di Anastasio
436 67 Giulio Di Anastasio
Under the WFS Options box, on Version dropdown, the default option "Maximum" works just fine
437 67 Giulio Di Anastasio
Click on OK
438 67 Giulio Di Anastasio
The list of layers will appear in the Browser under WFS/OGC API.
439 68 Giulio Di Anastasio
440 68 Giulio Di Anastasio
441 68 Giulio Di Anastasio
h1. How to create a new projection in QGis
442 68 Giulio Di Anastasio
443 68 Giulio Di Anastasio
To create a new projection in QGis, go to menu "Settings", and click on "Custom Projections".
444 68 Giulio Di Anastasio
A pop-up window appears with a list of all projections defined in QGis projects used by the user so far.
445 68 Giulio Di Anastasio
Click on the green "+" sign on the right top part of the window to create a new projection.
446 68 Giulio Di Anastasio
In the "Name" box, type "TM CSRAUSPOS SF1" (which means TM = Transverse Mercator projection; CSRAUSPOS = theparameters for this projection are derived from the processing of DGPS raw data by AUSPOS -  Online GPS Processing Service - https://www.ga.gov.au/scientific-topics/positioning-navigation/geodesy/auspos; SF1 = Scale Factor is 1).
447 68 Giulio Di Anastasio
In the "Format" dropdown list, select "Proj String (legacy - Not Recommended)"
448 68 Giulio Di Anastasio
In the "Parameters" box, paste the following "+proj=tmerc +lat_0=12.01605433+lon_0=79.80998934 +k=1 +x_0=370455.630 +y_0=1328608.994 +ellps=WGS84+towgs84=0,0,0,0,0,0,0 +units=m +no_defs".
449 68 Giulio Di Anastasio
450 68 Giulio Di Anastasio
Finally, click on OK.
451 68 Giulio Di Anastasio
452 68 Giulio Di Anastasio
In a more explicit  way, the parameters mean the following:
453 68 Giulio Di Anastasio
Map Projection:        TransverseMercator (TM)
454 68 Giulio Di Anastasio
False Easting:          370455.6300
455 68 Giulio Di Anastasio
False Northing:        1328608.9940
456 68 Giulio Di Anastasio
Latitude of Origin:    12°00'57.79560" (DMS)   12.01605433 (DD)
457 68 Giulio Di Anastasio
Central Meridian:     79°48'35.96164"  (DMS)   79.80998934 (DD)
458 68 Giulio Di Anastasio
Scale Factor:             1.00000000
459 68 Giulio Di Anastasio
Zone Width:              6.0°
460 69 Giulio Di Anastasio
461 69 Giulio Di Anastasio
h1. Elimination of Duplicate points – General criteria
462 69 Giulio Di Anastasio
463 69 Giulio Di Anastasio
It might happen that the same physical feature (e.g. a tree, or a pole) is surveyed more than once: this can happen because there are many physical features in an area, and the survey needs more than one station. So, for example a tree is surveyed from a station, and gets a serial number on that date. When the station is then changed, it might happen that the same tree is resurveyed: another serial number is given, and possibly a different date, if the survey from the second station happened on a different day.
464 69 Giulio Di Anastasio
It is clear that the same tree is then represented with two different points, which means that two different trees exist: but only one tree really exist in the physical reality.
465 69 Giulio Di Anastasio
It is clear that one of the two points is redundant and needs to be removed. If this is noted by the surveyor directly in the field, then the issue is solved by the surveyor himself during processing time.
466 69 Giulio Di Anastasio
If instead, due to various reasons, it was not noted by the surveyor in the field, it will need to be cleaned after the processing, possibly by post-processing staff.
467 69 Giulio Di Anastasio
How to identify duplicate points?
468 69 Giulio Di Anastasio
The following criteria can be used:
469 69 Giulio Di Anastasio
1.	The distance between the two points is less than 30 cm (trees are surveyed if their trunk diameter is at least about 20 cm, so in 30 cm cannot exist two of them)
470 69 Giulio Di Anastasio
2.	The orig_id (serial number) of the points are not in series
471 69 Giulio Di Anastasio
3.	The survey date is not the same
472 69 Giulio Di Anastasio
4.	In case of trees, the species of trees is the same
473 69 Giulio Di Anastasio
5.	5. In case of trees, the tree type is not TDEF (because TDEF are mapped irrespective of their diameter, so they can actually have a small trunk, and two of them might exist in 30 cm), not OT (many TDEF species are surveyed as OT if not otherwise indicated by a botanist)
474 69 Giulio Di Anastasio
6.	The context needs to be evaluated: if one tree is deleted in an area where many trees exist in a limited space, then loosing one in the map is not a big error. If instead one tree is deleted where there are very few trees, then it might be a big loss.
475 70 Giulio Di Anastasio
476 70 Giulio Di Anastasio
477 70 Giulio Di Anastasio
h1. Linework for the Survey Area
478 70 Giulio Di Anastasio
479 70 Giulio Di Anastasio
h2. 1.	Creation of Initial Linework in QGIS using Survey points import - (Ram, System 4)
480 70 Giulio Di Anastasio
481 70 Giulio Di Anastasio
Initial Linework in QGIS is started by surveyor with the knowledge from the Field. For this step, points are simply imported into the QGIS from the field text file (.csv or .txt). CRS needs to be TM-AUSPOS. The box of “First record has field names” shall not be ticked. In Point Coordinates, select the correct field for x, for y and for z (usually “field_2” for x, “field_3” for y and “field_4” for z). Points can be styled using the “Categorized” style in “Symbology”, using “Field_5” as value, or using a Rule-based symbology using the category (field 5) as filter.
482 74 Selvarani C
Linework is created by connecting points having same description and belonging to the same physical feature. *All line and polygon features are created as lines*.
483 70 Giulio Di Anastasio
The Initial Linework for the Survey Area is also stored temporarily in 
484 70 Giulio Di Anastasio
485 74 Selvarani C
+D: > AVSM > Zone-Survey number (eg RZ-01) > Survey Area (eg J) > Temporary WD+
486 70 Giulio Di Anastasio
487 70 Giulio Di Anastasio
h2. Note: The line shapefiles / Geopackages shall be in CRS: TM AUSPOS
488 70 Giulio Di Anastasio
489 70 Giulio Di Anastasio
h2. 2.	Creation of final working drawing Shapefiles / Geopackages - (Selvarani, System 1)
490 1 Philippe May
491 1 Philippe May
Final working drawing Shapefiles / Geopackages are created from the Initial Linework of Survey Area.
492 70 Giulio Di Anastasio
As the Surveyor draws all features as lines (both for lines and polygons features), the following actions shall be done:
493 74 Selvarani C
1.	*If features are lines:*
494 70 Giulio Di Anastasio
•	Export the shapefile / geopackage into the final working drawing folder (Final WD), in separate folders according to its type (e.g. BLDG, FENC, ROAD, etc).
495 70 Giulio Di Anastasio
496 70 Giulio Di Anastasio
h2. The CRS for the export shall be EPSG:4326 - WGS 84
497 70 Giulio Di Anastasio
498 74 Selvarani C
2.	*If features are polygons:*
499 1 Philippe May
•	Lines shall be converted into polygons:
500 74 Selvarani C
501 1 Philippe May
to do it, first click on the layer to be converted to make it active (e.g. WD-CZ-01-F-LL84_V-BLDG-MHOL------E), then go to “Vector” Menu, click on Geometry Tools, click on Line to Polygons:
502 87 Selvarani C
503 84 Selvarani C
!https://redmine.auroville.org.in/attachments/download/9760/Line%20to%20Polygon%20Menu.png!
504 70 Giulio Di Anastasio
505 86 Selvarani C
506 1 Philippe May
The new window for “Lines to Polygons” conversion will appear:
507 86 Selvarani C
508 86 Selvarani C
509 83 Selvarani C
!https://redmine.auroville.org.in/attachments/download/9762/Lines%20to%20Polygon%20Window.png!
510 1 Philippe May
511 71 Giulio Di Anastasio
•	Always cross check the input layer, to make sure that the input layer is the active one
512 71 Giulio Di Anastasio
•	Save the output in a temporary layer
513 74 Selvarani C
•	The temporary layer will be listed in the list of layers, it shall be exported to the saving location as +D: > Survey > Zone-Survey Number > Final WD > Survey Area SHP+ (eg . D: > Survey > GB-01 > Final WD > A-Shp)
514 72 Giulio Di Anastasio
515 74 Selvarani C
h2. The CRS for the export shall be EPSG:4326 - WGS 84
516 74 Selvarani C
517 72 Giulio Di Anastasio
Once all the shapefiles / geopackages are exported in Final WD, for each of the newly exported layers the Topology Checker Tool shall be used.
518 1 Philippe May
519 74 Selvarani C
h2. Linework for the whole Survey Zone
520 74 Selvarani C
521 74 Selvarani C
h2. 1.	Merging Shapefiles / Geopackages - (Selvarani, System 1)
522 74 Selvarani C
523 72 Giulio Di Anastasio
A copy of the Zone Master shapefiles / geopackages are taken from System 4 and stored in Temp Folder on Desktop in System 1.
524 1 Philippe May
Master shapefiles / geopackages are merged with the Survey Area shapefiles / geopackages:
525 72 Giulio Di Anastasio
•	To do it, go to “Vector” Menu, click on Geoprocessing Tools, then click on Union:
526 72 Giulio Di Anastasio
527 85 Selvarani C
!https://redmine.auroville.org.in/attachments/download/9763/Union%20Menu.png!
528 1 Philippe May
529 1 Philippe May
530 85 Selvarani C
The new window for “Union” will appear:
531 1 Philippe May
532 86 Selvarani C
533 86 Selvarani C
!https://redmine.auroville.org.in/attachments/download/9764/Union%20Window.png!
534 73 Giulio Di Anastasio
535 74 Selvarani C
•	To make sure that the right geometry is generated by this process (“line” type, not “Multiline”, and similarly “Polygon” type, not “Multipolygon), we need to always keep the *Master shapefile* (e.g. Final-CZ-01-2021-02-05-LL84_V-BLDG-MHOL------E) *as Input layer*, and the Survey Area shapefile as Overlay Layer (e.g. WD-CZ-01-F-LL84_V-BLDG-MHOL------E).
536 73 Giulio Di Anastasio
•	(The output can be saved to a file, as the CRS should already be EPSG4326 – WGS84.)
537 73 Giulio Di Anastasio
538 74 Selvarani C
h2. 2.	Storing Shapefiles / Geopackages - (Selvarani, System 1)
539 74 Selvarani C
540 74 Selvarani C
Save the merged shape file in the correct location in Final folder as +D: > Survey > Zone-Survey Number > Final+ (eg . D: > Survey > GB-01 > Final)
541 73 Giulio Di Anastasio
Date in the name of Final Shapefile / Geopackage needs to be updated.
542 73 Giulio Di Anastasio
Once the merging operation is completed, the copy of Master shapefile / geopackage is deleted from the Temp folder.
543 73 Giulio Di Anastasio
544 74 Selvarani C
h2. 3.	Topology check of merged shapefiles
545 74 Selvarani C
546 73 Giulio Di Anastasio
The topology checker is applied again on the merged shapefiles / geopackages.
547 73 Giulio Di Anastasio
The “id_field” shall be removed from the attribute table.
548 73 Giulio Di Anastasio
549 74 Selvarani C
h2. 4.	Archive and replace the Master Shapefiles / Geopackages (Ram, System 4)
550 74 Selvarani C
551 73 Giulio Di Anastasio
Archive the previous master shapefiles / geopackages on system 4, and copy the new merged shapefiles / geopackages in its place.
552 74 Selvarani C
*Then delete the Merged Shapefile / Geopackage folder from System 1.
553 74 Selvarani C
*
554 73 Giulio Di Anastasio
555 74 Selvarani C
h2. 5.	Note about Shapefiles and Geopackages
556 73 Giulio Di Anastasio
557 73 Giulio Di Anastasio
All the above works are usually done using shapefile format, in QGIS latest version (3.16.3).
558 221 Selvarani C
The Geopackage export is done in QGis versions older than 3.12 (e.g. 3.4, 3.6, 3.8, 3.10) so that the lines are not saved as “Multilines” but as “Lines”and polygons are not saved as “Multipolygons” but as “Polygons”. This is very important to be noted, as Gisaf database does not accept the Multipolygon and Multiline geometry types.
559 221 Selvarani C
A different way to create "Polygons" is to use the command Vector -> Geometry Tools -> Multipart to Single Parts and apply it to the layer: from "Multipolygon" it will become "Polygon" (check in layer Properties).
560 222 Selvarani C
As on 13 March 2021, Gisaf can accept Multipolygon layers, because the command "Multipart to Single parts" has been integrated into the importing command (see Redmine ticket #11691)
561 223 Selvarani C
562 223 Selvarani C
h1. Creating 3D Shapefile/Geopackage
563 223 Selvarani C
564 223 Selvarani C
While creating a Shapefile/Geopackage, the File name/Database-table name, Geometry type and CRS have to be entered.
565 223 Selvarani C
In order to create a 3D Shapefile/Geopackage, the additional dimensions *"Z(+M values)"/ "Include Z dimension"* has to be ticked: this way the 3D Shapefile/Geopackage is accepted by Gisaf without errors, otherwise the Shapefile/Geopackage can't be imported in Gisaf because the Z dimension is missing (the geometries in the database are all 3D).
566 224 Giulio Di Anastasio
567 224 Giulio Di Anastasio
h1. Exporting from QGis (shapefiles and/or geopackages) to CAD dxf format
568 224 Giulio Di Anastasio
569 226 Giulio Di Anastasio
An algorithm has been created by Selvarani, to see the whole process click here:
570 226 Giulio Di Anastasio
https://redmine.auroville.org.in/projects/gis/wiki/Automatic_export_from_QGis_to_dxf