Project

General

Profile

Wiki » History » Version 281

Giulio Di Anastasio, 06/05/2021 12:02

1 1 Philippe May
h1. %{color:BLUE}  Wiki%
2 227 Giulio Di Anastasio
3 232 Giulio Di Anastasio
h1. [[CSR Geomatics Team]]
4 231 Giulio Di Anastasio
5 229 Giulio Di Anastasio
h1. [[Collaborations]]
6 229 Giulio Di Anastasio
7 240 Giulio Di Anastasio
h1. [[Data Analysis]]
8 234 Giulio Di Anastasio
9 241 Giulio Di Anastasio
h1. [[CSR Geomatics Server Setup]]
10 234 Giulio Di Anastasio
11 245 Giulio Di Anastasio
h1. [[Database Backup and Restoration]]
12 243 Giulio Di Anastasio
13 243 Giulio Di Anastasio
h1. [[CAD to GIS Workflow]] %{color:RED}  - TO BE UPDATED%
14 248 Giulio Di Anastasio
15 246 Giulio Di Anastasio
h1. [[GIS Conventions]]
16 246 Giulio Di Anastasio
 
17 252 Giulio Di Anastasio
h1. [[Working with Gisaf]]
18 96 Selvarani C
19 92 Selvarani C
h1. [[Gisaf Workflow]]
20 252 Giulio Di Anastasio
21 256 Giulio Di Anastasio
h1. [[GIS to CAD Workflow]] %{color:RED}  - TO BE UPDATED%
22 94 Selvarani C
23 257 Giulio Di Anastasio
h1. [[Weather Station]]
24 142 Selvarani C
25 259 Giulio Di Anastasio
h1. [[GDAL - Useful Tips]]
26 5 Philippe May
27 262 Giulio Di Anastasio
h1. [[Gisaf - Auxiliary Data (non-geographical data)]]
28 143 Selvarani C
29 264 Giulio Di Anastasio
h1. [[QGis - Useful Tips]]
30 264 Giulio Di Anastasio
31 266 Giulio Di Anastasio
h1. [[AutoCAD Civil 3D - Useful Tips]]
32 266 Giulio Di Anastasio
33 280 Giulio Di Anastasio
h1. [[Wells]]
34 8 Philippe May
35 272 Giulio Di Anastasio
h1. [[Gisaf Data Modifications]]
36 272 Giulio Di Anastasio
37 274 Giulio Di Anastasio
h1. [[Other Useful Softwares]]
38 272 Giulio Di Anastasio
39 262 Giulio Di Anastasio
40 1 Philippe May
41 274 Giulio Di Anastasio
42 274 Giulio Di Anastasio
43 107 Selvarani C
44 110 Selvarani C
45 110 Selvarani C
46 110 Selvarani C
47 113 Selvarani C
48 270 Giulio Di Anastasio
49 270 Giulio Di Anastasio
50 119 Selvarani C
51 119 Selvarani C
52 127 Selvarani C
h1. Importing point data (TS and RTK) to GISAF 
53 127 Selvarani C
54 127 Selvarani C
# *Gisaf Admin > Basket > Survey data > "Project" > "surveyor" > TS/ RTK > upload > import.*
55 127 Selvarani C
# *Auto import of Raw points data* (changes from Raw to Shapefiles, the point files)
56 1 Philippe May
  In GISAF Admin > Other> Project > select the project for which you want to import the data > with selected > Auto import to GIS...
57 127 Selvarani C
58 172 Selvarani C
59 128 Selvarani C
60 128 Selvarani C
61 128 Selvarani C
62 129 Selvarani C
h1. Miscellaneous- Civil 3D
63 129 Selvarani C
64 129 Selvarani C
*Autodesk civil 3D Geotechnical module- for borehole data :- To analyse borehole data, To make profiles and calculate volumes*
65 129 Selvarani C
66 129 Selvarani C
https://www.youtube.com/watch?v=0Be9kShBou0
67 129 Selvarani C
Reference videos
68 129 Selvarani C
Exploring:
69 129 Selvarani C
https://www.youtube.com/watch?v=gr-ISPzLcU0
70 129 Selvarani C
Modelling:
71 129 Selvarani C
https://www.youtube.com/watch?v=Pz0-HOoiBrs
72 129 Selvarani C
solids
73 129 Selvarani C
https://www.youtube.com/watch?v=mQ0Yeh6tZA8
74 129 Selvarani C
https://www.youtube.com/watch?v=4eMsUiYBhuE
75 129 Selvarani C
76 129 Selvarani C
77 129 Selvarani C
*CIVIL 3D Survey*
78 129 Selvarani C
Getting started- Always open a new drawing with a template. 
79 129 Selvarani C
80 129 Selvarani C
*Description key sets*- (till 18:30) https://www.youtube.com/watch?v=mmwkkRyBkS0
81 129 Selvarani C
As when the points come into the drawing, they are going to be filtered with description key sets and the description key sets will assign properties to the points.
82 129 Selvarani C
Tool space > settings > points > description key sets
83 129 Selvarani C
“Description key sets name”> edit key- to view the points in a list: 
84 129 Selvarani C
Automatic linework (18:30- till end)
85 129 Selvarani C
Survey > Linework code sets- for automatic linework
86 129 Selvarani C
87 129 Selvarani C
*Using Master view*
88 129 Selvarani C
Uses-
89 129 Selvarani C
copy styles from one drawing to other
90 129 Selvarani C
copy drawing data from one drawing to other (data referencing)
91 129 Selvarani C
92 129 Selvarani C
*Civil 3d surface model*- from points- break lines
93 129 Selvarani C
https://www.youtube.com/watch?v=wC40rdmDEMo
94 129 Selvarani C
moving for example- point groups under point groups- 
95 129 Selvarani C
96 129 Selvarani C
*Civil 3D Planning and Analysis*
97 129 Selvarani C
Workspace: Planning and Analysis
98 129 Selvarani C
1. Working with Object data (GIS Attributes)
99 129 Selvarani C
Map Setup> Define object data> New table> Define new object data table start defining fields
100 129 Selvarani C
101 129 Selvarani C
102 130 Selvarani C
103 130 Selvarani C
h1. Documentation- Rain Gauge
104 130 Selvarani C
105 130 Selvarani C
h2. Manual Rain Gauge
106 130 Selvarani C
107 130 Selvarani C
*Why it is important to comply to standards?*
108 130 Selvarani C
109 130 Selvarani C
1. Consistency for comparing rainfall data in different places within Auroville since rainfall varies in different parts of Auroville.
110 130 Selvarani C
111 130 Selvarani C
2. Making it possible to use for any kind of scientific analysis by contribution to the primary data in sustainable water management for Auroville. 
112 130 Selvarani C
113 130 Selvarani C
*Proposed standards and ethics:*
114 130 Selvarani C
115 130 Selvarani C
1. The time of taking the reading - 08:30 am. 
116 130 Selvarani C
The rain is recorded over a period of 24 hours- 8:30 of previous day till 8:30 of the present day, and the date is put as the present day. For example, if the rain is measured at 8:30 am on 10 dec 2017, then it is logged in as on 10 dec 2017.
117 130 Selvarani C
118 130 Selvarani C
2. Manual rain gauge typically used- green cylindrical and a collection jar.  
119 130 Selvarani C
  
120 130 Selvarani C
3. Measuring jar: 10 mm corresponding jar
121 130 Selvarani C
122 130 Selvarani C
 
123 130 Selvarani C
Area of the rim of manual rain gauge= 200 square cm
124 130 Selvarani C
Diameter of rim= approximately 16 cm
125 130 Selvarani C
*NOTE:* - If another kind of rain gauge is used, it should be used with the corresponding measuring jar as type of jar and calibration on jar depends on the area of rim. Any cases of different rain gauge other than above should be reported before use for recording purposes.
126 130 Selvarani C
127 130 Selvarani C
4. The minimum record-able unit is 0.2 mm (Least measure on the 10 mm cylinder). T is marked for below 1 mm.
128 130 Selvarani C
129 130 Selvarani C
5. The reading should be preferably submitted in an excel sheet (the data sheet will be provided). The frequency of sending data can be daily or weekly.
130 130 Selvarani C
6. The rain gauge should be kept in a safe and open to sky area clear from tree cover or objects in surrounding so that direct rain falls into it and not from trees/ objects etc. Preferably roof top or a higher place on ground.
131 130 Selvarani C
7. A check on the rain gauge should be kept when there is long period of gap in rainy days to make sure it was emptied out and there is no blockage. 
132 130 Selvarani C
133 130 Selvarani C
8. In case of doubt in reading or incorrect measure, "incorrect measure" should be marked
134 130 Selvarani C
135 130 Selvarani C
136 130 Selvarani C
*Ethics*
137 130 Selvarani C
1. Readings should not be missed. Time to time check on rain gauge is required during long periods of no rain (especially when the rain season is approaching). 
138 130 Selvarani C
This is important because firstly, if it rained in the night and it wasn’t recorded, reading for the day is lost; Secondly, if the jar was not emptied, it gives incorrect subsequent reading. 
139 130 Selvarani C
140 130 Selvarani C
2.  If the person is not going to be available temporarily where the rain gauge is kept, the responsibility should be passed on to someone they can rely on after explaining the standards. 
141 130 Selvarani C
142 130 Selvarani C
*Important-*
143 130 Selvarani C
Data won’t be published in case of non-compliance to standards. This is to maintain the sanctity of proper scientific data collection and to keep it reliable for sharing by publishing. Discussions regarding standards are welcomed.
144 130 Selvarani C
CSR Geomatics Team is placed on first floor in CSR, Auroshilpam.
145 130 Selvarani C
146 130 Selvarani C
*AV rain data publication*: (http://gis.auroville.org.in/measures/raingauge_av).
147 130 Selvarani C
148 130 Selvarani C
Adding Rain gauge to our web portal (GISAF), following information is needed:
149 130 Selvarani C
1. GPS co-ordinates of location of placing the rain gauge (can be obtained using mobile phones easily)
150 130 Selvarani C
2. Name, place of residence & contact number- Mobile & Landline
151 130 Selvarani C
152 130 Selvarani C
h2. Automatic Rain Gauge
153 130 Selvarani C
154 130 Selvarani C
There is a possibility of publishing rain data coming from Automatic rain gauges like (id 15, auro orchard) and Weather stations. 
155 130 Selvarani C
For Automatic rain gauges, we can upload the files coming from the rain gauge directly into GISAF.
156 130 Selvarani C
157 130 Selvarani C
158 130 Selvarani C
_Thank you for contribution towards a sustainable management of Water in Auroville through Data collection._
159 130 Selvarani C
160 130 Selvarani C
161 131 Selvarani C
162 131 Selvarani C
163 132 Selvarani C
h1. Documentation- Flow meter by Bala
164 132 Selvarani C
165 132 Selvarani C
Flow meter – for checking the flow of water in pipe
166 132 Selvarani C
167 132 Selvarani C
CSR bought ultrasonic flow meter from Chennai and the company (company name? )trained Bala and Vijai(CSR) on how to set it up.
168 132 Selvarani C
we tested in the west water system pump in csr. then we test in many more place .
169 132 Selvarani C
 ami, aurodam and buddha garden borewell pumps . we got request from the water service 
170 132 Selvarani C
Cross check they flow meters .
171 132 Selvarani C
 so before that  we want to know how our meter works. 
172 132 Selvarani C
so we tested in our tank . the pump pumping from our sump tank so the flow will be stranded.  when you pump in the bore well  the flow goes up and down. 
173 132 Selvarani C
we tested two times in  15 minutes.and one time 30 minutes,so we got variation between this three. 
174 132 Selvarani C
we find 1,5 % error but the flow meret they said  1 % error only.
175 132 Selvarani C
we cross checked the flow meter of the water service we find some error  in they meter also. 
176 132 Selvarani C
we tried they bore well flowmeter and they sump tank flow meter also.and we find some error also in they meters.  
177 132 Selvarani C
then we find some error in (AVWS) meter also.
178 132 Selvarani C
It is used to check the flow of water in pipe
179 132 Selvarani C
tools used 
180 132 Selvarani C
First the outer diameter of the pipe has to be entered in the device. It is measured using Vernier caliper
181 132 Selvarani C
Then the thickness of the pipe is set which is also measured using vernier
182 132 Selvarani C
Also the device asks for the material of the pipe. If the pipe material is know it can be set and if it not know then there is an option which is other (mostly PVC, HDPE, and  iron) 
183 132 Selvarani C
After entering these details the device gives the spacing for the sensors. There are 2 sensors up and down which has to be fixed accordingly
184 132 Selvarani C
When the sensors are fixed the motor is turned on and the sensors send reading to the display device
185 132 Selvarani C
This is noted down once every minute and taken for 15 to 30 minutes. Then the average of this is determined. This is done because of the variation in the flow. This gives the flow rate in 1 hour.
186 1 Philippe May
This was done in different places to check to flow rate
187 132 Selvarani C
188 177 Selvarani C
189 133 Selvarani C
190 133 Selvarani C
h1. Documentation- DST- Vegetation Indexing
191 133 Selvarani C
192 133 Selvarani C
h2. Steps for Dzetsaka Classification tool for Vegetation indexing in QGIS
193 133 Selvarani C
194 133 Selvarani C
 
195 133 Selvarani C
 1. Install the plugin Dzetsaka classfication tool.
196 133 Selvarani C
 2. Open the Raster from the Survey.
197 133 Selvarani C
 3. Create a polygon shapefile for index sampling. Mark polygons and give the ID's (1,2,3 for Tree, grass, bare land etc) Cover the variations in samples as much as possible. 
198 133 Selvarani C
     More the samples, better the indexing.
199 133 Selvarani C
 4. Apply Dzetsaka Classification tool, Select the base raster and the sample- index polygon shapefile created in step 3.
200 133 Selvarani C
 5. The result is a Raster with DN numbers specified in the Shapefile in step 3.
201 133 Selvarani C
 6. Apply the Sieve raster command (Raster> Analysis> Sieve)- Try different threshold numbers and view the results till the noise is removed from the Raster.
202 133 Selvarani C
 7. Polygonise the Raster to Vector (From processing)
203 133 Selvarani C
 8. Run the v.generalise tool on the shapefile. This tool removes the pixelated boundaries of the polygons in the Vector.
204 133 Selvarani C
205 134 Selvarani C
h1. Documentation- DST- Interpolation (Processing toolbox)
206 134 Selvarani C
207 134 Selvarani C
The following 4 tools have been mostly explored and the results were compared. The ones used for quick analysis are 1. Cubic Spline and 2. V.surf.spline . The rest of the tools are for further exploration and used depending on the need of the project. In some tools, the elevation values of points should be stored in the attribute table (Using field calculator and giving command - *Z($Geometry)* )
208 134 Selvarani C
209 134 Selvarani C
# Interpolate (Cubic spline) - SAGA
210 134 Selvarani C
# V.surf.bspline - GRASS. Parameters to set - cell size. Set this parameter above 0.001 ( 0.00001, 0.000001 etc) and check the results.
211 134 Selvarani C
# V.surf.rst - GRASS
212 134 Selvarani C
# Krigging - SAGA
213 1 Philippe May
214 181 Selvarani C
[[Documentation- DST- Survey- Office workflow]]  - *%{color:RED} to be written%*
215 135 Selvarani C
216 135 Selvarani C
h1. From CAD to GIS by Giulio
217 135 Selvarani C
218 135 Selvarani C
*FEATURES CREATION*
219 135 Selvarani C
1.	Assign a CRS to the drawing (TM-AUSPOS) (MAPCSLIBRARY command)
220 135 Selvarani C
2.	Create features in CAD (Points, lines, polygons)
221 135 Selvarani C
3.	Export shapefile (a) from CAD (Output > DWG to SDF) (Convert to LL84 – 3D)
222 135 Selvarani C
223 135 Selvarani C
*FEATURES IMPORT INTO DB FIRST TIME*
224 135 Selvarani C
4.	Create zip file of the shapefile
225 135 Selvarani C
5.	Upload into the GISAF Shapefiles Basket
226 135 Selvarani C
6.	Import the shapefile into DB
227 135 Selvarani C
7.	Save the shapefile on Local Machine
228 135 Selvarani C
229 135 Selvarani C
*FEATURES IMPORT INTO DB EVERYTIME*
230 135 Selvarani C
8.	Combine the new features to corresponding last shape files (Insert the process here).
231 135 Selvarani C
9.	Follow step 4-8 again
232 135 Selvarani C
233 135 Selvarani C
*FEATURES EDITING IN QGIS*
234 135 Selvarani C
10.	Open the table in QGis
235 135 Selvarani C
11.	Save as a shapefile (b) in TM AUSPOS CRS
236 135 Selvarani C
12.	In CAD, open a new drawing and assign AUSPOS CRS
237 135 Selvarani C
13.	Import the shapefile (b) (MapImport) with Object Data (Data tab > Create Object Data > OK), tick “Import polygons as closed polylines”, then press OK
238 135 Selvarani C
14.	Edit features
239 135 Selvarani C
15.	Change workspace into “Planning and analysis”
240 135 Selvarani C
16.	Export shapefile (a) from CAD (Output > DWG to SDF) with ONLY the id selected (Data Tab > Select Attributes > Object Data > Filename > id) (Convert to LL84 – 3D)
241 135 Selvarani C
242 135 Selvarani C
*FEATURES IMPORT INTO DB*
243 135 Selvarani C
17.	Create zip file of the shapefile
244 135 Selvarani C
18.	Upload into the GISAF Shapefiles Basket
245 135 Selvarani C
19.	Import the shapefile into DB
246 135 Selvarani C
20.	Delete the shapefile from Local Machine
247 135 Selvarani C
248 135 Selvarani C
249 182 Selvarani C
250 1 Philippe May
251 136 Selvarani C
h1. QGIS- Miscellaneous
252 136 Selvarani C
253 136 Selvarani C
*QGIS Introduction:*
254 136 Selvarani C
https://www.birdseyeviewgis.com/blog/2018/2/22/my-favorite-features-of-qgis-30to-date
255 136 Selvarani C
3D visualization of raster DEM- https://www.youtube.com/watch?v=2KrCsbP0kUs
256 136 Selvarani C
257 136 Selvarani C
*Spatial query:* 
258 136 Selvarani C
Spatial Query is selection of features that satisfies a certain condition which relates to other features in a space.
259 136 Selvarani C
Using plugin- Spatial query
260 136 Selvarani C
http://www.geodose.com/2018/03/spatial-query-in-qgis-3.html
261 136 Selvarani C
Labelling:
262 136 Selvarani C
Labelling with more than one field names and in different lines
263 136 Selvarani C
|| '\n' ||
264 136 Selvarani C
265 136 Selvarani C
266 136 Selvarani C
267 136 Selvarani C
*Hierarchy of extensions*
268 136 Selvarani C
269 136 Selvarani C
File levels and their uses.
270 136 Selvarani C
https://nathanw.net/2014/03/22/all-the-q-files/
271 136 Selvarani C
272 136 Selvarani C
+The Project file (.qgs)+
273 136 Selvarani C
It contains: Layer source pointer + Style information + Composers + a whole heap of other stuff
274 136 Selvarani C
275 136 Selvarani C
+The Layer Definition file (.qlr)+
276 136 Selvarani C
It contains: Layer source pointer + Style information
277 136 Selvarani C
278 136 Selvarani C
+The QML file (.qml)+
279 136 Selvarani C
It contains: Style information
280 136 Selvarani C
281 59 Giulio Di Anastasio
h2. Giulio's documentation
282 52 Giulio Di Anastasio
283 183 Selvarani C
284 137 Selvarani C
285 137 Selvarani C
h1. Documentation - Reconciliation of points using Gisaf
286 137 Selvarani C
287 137 Selvarani C
Reconciliation of points is a procedure used when a point is stored in a wrong table, because its category was either wrongly recorded in the field by the surveyor, or it has ben reviewed later by the surveyor or the data validator and found to be wrong.
288 137 Selvarani C
289 137 Selvarani C
h2. Definition: Raw points are all points coming from the field survey. Raw points can be points referring to a Point feature (e.g. trees, or elevation points, or floor level), or points measured in the field to draw a line (e.g. the vertices of a fence) or a polygon (e.g. the corners of a building outline).
290 137 Selvarani C
291 137 Selvarani C
In the overall workflow, *+reconciliation affects only raw points+*. It takes place after the field textfile is uploaded into the basket and its points imported (raw points stored in the raw survey tables). Here you can visualize the Workflow diagram: [[Survey_data]].
292 137 Selvarani C
293 137 Selvarani C
294 137 Selvarani C
If a raw point refers to a point feature, reconciling it means moving it to another category/table meant for point features, not for lines/polygons. So, *+only raw points of point features can be reconciled+*. Raw points pertaining to line features and/or polygon features cannot be reconciled, so these raw points will remain in their original wrong table.
295 137 Selvarani C
296 137 Selvarani C
297 137 Selvarani C
PLEASE NOTE: 
298 137 Selvarani C
The attributes of a line/polygon, which are derived from their defining raw points, will not be modified by any reconciliation, because *+only raw points of point features can be reconciled+*.
299 137 Selvarani C
300 137 Selvarani C
h2. How to perform Reconciliation
301 137 Selvarani C
302 137 Selvarani C
To perform reconciliation of points: Login to Gisaf -> click on the G icon on the upper-left corner of the website page -> Manage -> Reconciliation by orig.ID
303 137 Selvarani C
304 137 Selvarani C
On the right end side of the screen, clicking on the field "Destination" a list of all categories will appear: these categories refer not to the Raw survey tables, but to the V_ tables of the database (points, lines, polygons).
305 137 Selvarani C
306 137 Selvarani C
Under it, in the field "Original ID", the original point number of the point to be reconciled is to be entered.
307 137 Selvarani C
308 137 Selvarani C
Clicking on the "Search points" button, the result shows the database unique id of the point, its survey category, its survey date, its geometry type (point, line, polygon), and the Project the point belongs to. In case of multiple points with the same original id (in case of different Projects, the field number of points might be not unique if the numbering of points in the field has restarted) all points having that original id are displayed: thanks to their date or Project or type, it is easy to identify the correct point to be reconciled.
309 137 Selvarani C
310 137 Selvarani C
Once the point to be reconciled is identified, clicking on the button "Reconcile" will run the reconciliation, and a message will appear stating that it has been done successfully. An error message can appear if a reconciliation of a raw point of a line/polygon feature has been attempted: this type of points cannot in fact be reconciled.
311 137 Selvarani C
312 137 Selvarani C
Once a raw point has been reconciled, *+it cannot be reconciled again+*. In case a raw point has been wrongly reconciled, it cannot be reconciled again through the above procedure, but it has to be reconciled manually through QGis or pgadmin software.
313 137 Selvarani C
314 138 Selvarani C
h1. Documentation - Status and Status Changes
315 138 Selvarani C
316 138 Selvarani C
Status have been created to keep track og changes in surveyed features.
317 138 Selvarani C
It is an additional (though provided for in AIA standards), single digit value, at the end of the Gisaf Category/Cad layer name.
318 138 Selvarani C
Each Status need to have a corresponding CAD layer/Gisaf Category, with a short code associatedto it, so that field entries can be done easily.
319 138 Selvarani C
320 138 Selvarani C
Status have been defined as follows:
321 138 Selvarani C
N - New Work
322 138 Selvarani C
E - Existing o remain
323 138 Selvarani C
D - Existing to demolish, Demolished or Changed
324 138 Selvarani C
F- Future work, Proposed feature
325 138 Selvarani C
T - Temporary work
326 138 Selvarani C
M - Item to be moved
327 138 Selvarani C
X - Not in contract
328 138 Selvarani C
329 138 Selvarani C
330 138 Selvarani C
By default, Status is defined as E (Existing). Status can anyhow be changed later using gAdmin, or QGis (through the PostGis connection).
331 138 Selvarani C
It needs to be done manually, one feature (point, line, polygon) at a time.
332 138 Selvarani C
333 138 Selvarani C
334 1 Philippe May
In the future Status changes might be incorporated in the Admin panel.
335 138 Selvarani C
336 185 Selvarani C
337 67 Giulio Di Anastasio
338 139 Selvarani C
h1. Documentation - Tags retained after re-import of same geometry
339 139 Selvarani C
340 139 Selvarani C
Documentation - Tags retained after re-import of same geometry
341 139 Selvarani C
The linework for infrastructure survey carried out by Eric Chacra in May 2020 was originally imported with a problem of ambiguity in the "Accuracy" table and in the "Accuracy" table.
342 139 Selvarani C
The result was that lines did not inherit the attributes survey date, accuracy, equipment, surveyor.
343 139 Selvarani C
Nevertheless lines were displayed on the Gisaf map, without these attributes, and tags were given to some of these lines.
344 139 Selvarani C
345 139 Selvarani C
The values for the two tables ("Accuracy", "Accuracy") have been corrected, ambiguity resolved.
346 139 Selvarani C
The lines in the layers V-ELEC-UGND------E and V-COMM-CABL------E have been then reimported, the attributes have been properly assigned, and the tags have been retained.
347 139 Selvarani C
348 139 Selvarani C
4 August 2020
349 139 Selvarani C
350 67 Giulio Di Anastasio
h1. Access to data
351 67 Giulio Di Anastasio
352 67 Giulio Di Anastasio
h2. Connection to server directly from CSR
353 67 Giulio Di Anastasio
354 67 Giulio Di Anastasio
To connect to the server directly without going through Aurinoco server, the correct url is
355 67 Giulio Di Anastasio
http://gis.csr.av
356 67 Giulio Di Anastasio
357 67 Giulio Di Anastasio
h2. Connection to Gisaf via QGis through WFS / OGC API
358 67 Giulio Di Anastasio
359 67 Giulio Di Anastasio
This works only on QGis from version 3.14.15 onward
360 67 Giulio Di Anastasio
361 67 Giulio Di Anastasio
In the browser, click on WFS/OGC API, then right-click to create a new connection
362 67 Giulio Di Anastasio
Give a name (e.g. OGC API Qgis Gisaf)
363 67 Giulio Di Anastasio
Give the url https://gis.auroville.org.in/ogcapi
364 67 Giulio Di Anastasio
365 67 Giulio Di Anastasio
Under the WFS Options box, on Version dropdown, the default option "Maximum" works just fine
366 67 Giulio Di Anastasio
Click on OK
367 67 Giulio Di Anastasio
The list of layers will appear in the Browser under WFS/OGC API.
368 68 Giulio Di Anastasio
369 68 Giulio Di Anastasio
370 68 Giulio Di Anastasio
h1. How to create a new projection in QGis
371 68 Giulio Di Anastasio
372 68 Giulio Di Anastasio
To create a new projection in QGis, go to menu "Settings", and click on "Custom Projections".
373 68 Giulio Di Anastasio
A pop-up window appears with a list of all projections defined in QGis projects used by the user so far.
374 68 Giulio Di Anastasio
Click on the green "+" sign on the right top part of the window to create a new projection.
375 68 Giulio Di Anastasio
In the "Name" box, type "TM CSRAUSPOS SF1" (which means TM = Transverse Mercator projection; CSRAUSPOS = theparameters for this projection are derived from the processing of DGPS raw data by AUSPOS -  Online GPS Processing Service - https://www.ga.gov.au/scientific-topics/positioning-navigation/geodesy/auspos; SF1 = Scale Factor is 1).
376 68 Giulio Di Anastasio
In the "Format" dropdown list, select "Proj String (legacy - Not Recommended)"
377 68 Giulio Di Anastasio
In the "Parameters" box, paste the following "+proj=tmerc +lat_0=12.01605433+lon_0=79.80998934 +k=1 +x_0=370455.630 +y_0=1328608.994 +ellps=WGS84+towgs84=0,0,0,0,0,0,0 +units=m +no_defs".
378 68 Giulio Di Anastasio
379 68 Giulio Di Anastasio
Finally, click on OK.
380 68 Giulio Di Anastasio
381 68 Giulio Di Anastasio
In a more explicit  way, the parameters mean the following:
382 68 Giulio Di Anastasio
Map Projection:        TransverseMercator (TM)
383 68 Giulio Di Anastasio
False Easting:          370455.6300
384 68 Giulio Di Anastasio
False Northing:        1328608.9940
385 68 Giulio Di Anastasio
Latitude of Origin:    12°00'57.79560" (DMS)   12.01605433 (DD)
386 68 Giulio Di Anastasio
Central Meridian:     79°48'35.96164"  (DMS)   79.80998934 (DD)
387 68 Giulio Di Anastasio
Scale Factor:             1.00000000
388 68 Giulio Di Anastasio
Zone Width:              6.0°
389 69 Giulio Di Anastasio
390 69 Giulio Di Anastasio
h1. Elimination of Duplicate points – General criteria
391 69 Giulio Di Anastasio
392 69 Giulio Di Anastasio
It might happen that the same physical feature (e.g. a tree, or a pole) is surveyed more than once: this can happen because there are many physical features in an area, and the survey needs more than one station. So, for example a tree is surveyed from a station, and gets a serial number on that date. When the station is then changed, it might happen that the same tree is resurveyed: another serial number is given, and possibly a different date, if the survey from the second station happened on a different day.
393 69 Giulio Di Anastasio
It is clear that the same tree is then represented with two different points, which means that two different trees exist: but only one tree really exist in the physical reality.
394 69 Giulio Di Anastasio
It is clear that one of the two points is redundant and needs to be removed. If this is noted by the surveyor directly in the field, then the issue is solved by the surveyor himself during processing time.
395 69 Giulio Di Anastasio
If instead, due to various reasons, it was not noted by the surveyor in the field, it will need to be cleaned after the processing, possibly by post-processing staff.
396 69 Giulio Di Anastasio
How to identify duplicate points?
397 69 Giulio Di Anastasio
The following criteria can be used:
398 69 Giulio Di Anastasio
1.	The distance between the two points is less than 30 cm (trees are surveyed if their trunk diameter is at least about 20 cm, so in 30 cm cannot exist two of them)
399 69 Giulio Di Anastasio
2.	The orig_id (serial number) of the points are not in series
400 69 Giulio Di Anastasio
3.	The survey date is not the same
401 69 Giulio Di Anastasio
4.	In case of trees, the species of trees is the same
402 69 Giulio Di Anastasio
5.	5. In case of trees, the tree type is not TDEF (because TDEF are mapped irrespective of their diameter, so they can actually have a small trunk, and two of them might exist in 30 cm), not OT (many TDEF species are surveyed as OT if not otherwise indicated by a botanist)
403 69 Giulio Di Anastasio
6.	The context needs to be evaluated: if one tree is deleted in an area where many trees exist in a limited space, then loosing one in the map is not a big error. If instead one tree is deleted where there are very few trees, then it might be a big loss.
404 70 Giulio Di Anastasio
405 70 Giulio Di Anastasio
406 70 Giulio Di Anastasio
h1. Linework for the Survey Area
407 70 Giulio Di Anastasio
408 70 Giulio Di Anastasio
h2. 1.	Creation of Initial Linework in QGIS using Survey points import - (Ram, System 4)
409 70 Giulio Di Anastasio
410 70 Giulio Di Anastasio
Initial Linework in QGIS is started by surveyor with the knowledge from the Field. For this step, points are simply imported into the QGIS from the field text file (.csv or .txt). CRS needs to be TM-AUSPOS. The box of “First record has field names” shall not be ticked. In Point Coordinates, select the correct field for x, for y and for z (usually “field_2” for x, “field_3” for y and “field_4” for z). Points can be styled using the “Categorized” style in “Symbology”, using “Field_5” as value, or using a Rule-based symbology using the category (field 5) as filter.
411 74 Selvarani C
Linework is created by connecting points having same description and belonging to the same physical feature. *All line and polygon features are created as lines*.
412 70 Giulio Di Anastasio
The Initial Linework for the Survey Area is also stored temporarily in 
413 70 Giulio Di Anastasio
414 74 Selvarani C
+D: > AVSM > Zone-Survey number (eg RZ-01) > Survey Area (eg J) > Temporary WD+
415 70 Giulio Di Anastasio
416 70 Giulio Di Anastasio
h2. Note: The line shapefiles / Geopackages shall be in CRS: TM AUSPOS
417 70 Giulio Di Anastasio
418 70 Giulio Di Anastasio
h2. 2.	Creation of final working drawing Shapefiles / Geopackages - (Selvarani, System 1)
419 1 Philippe May
420 1 Philippe May
Final working drawing Shapefiles / Geopackages are created from the Initial Linework of Survey Area.
421 70 Giulio Di Anastasio
As the Surveyor draws all features as lines (both for lines and polygons features), the following actions shall be done:
422 74 Selvarani C
1.	*If features are lines:*
423 70 Giulio Di Anastasio
•	Export the shapefile / geopackage into the final working drawing folder (Final WD), in separate folders according to its type (e.g. BLDG, FENC, ROAD, etc).
424 70 Giulio Di Anastasio
425 70 Giulio Di Anastasio
h2. The CRS for the export shall be EPSG:4326 - WGS 84
426 70 Giulio Di Anastasio
427 74 Selvarani C
2.	*If features are polygons:*
428 1 Philippe May
•	Lines shall be converted into polygons:
429 74 Selvarani C
430 1 Philippe May
to do it, first click on the layer to be converted to make it active (e.g. WD-CZ-01-F-LL84_V-BLDG-MHOL------E), then go to “Vector” Menu, click on Geometry Tools, click on Line to Polygons:
431 87 Selvarani C
432 84 Selvarani C
!https://redmine.auroville.org.in/attachments/download/9760/Line%20to%20Polygon%20Menu.png!
433 70 Giulio Di Anastasio
434 86 Selvarani C
435 1 Philippe May
The new window for “Lines to Polygons” conversion will appear:
436 86 Selvarani C
437 86 Selvarani C
438 83 Selvarani C
!https://redmine.auroville.org.in/attachments/download/9762/Lines%20to%20Polygon%20Window.png!
439 1 Philippe May
440 71 Giulio Di Anastasio
•	Always cross check the input layer, to make sure that the input layer is the active one
441 71 Giulio Di Anastasio
•	Save the output in a temporary layer
442 74 Selvarani C
•	The temporary layer will be listed in the list of layers, it shall be exported to the saving location as +D: > Survey > Zone-Survey Number > Final WD > Survey Area SHP+ (eg . D: > Survey > GB-01 > Final WD > A-Shp)
443 72 Giulio Di Anastasio
444 74 Selvarani C
h2. The CRS for the export shall be EPSG:4326 - WGS 84
445 74 Selvarani C
446 72 Giulio Di Anastasio
Once all the shapefiles / geopackages are exported in Final WD, for each of the newly exported layers the Topology Checker Tool shall be used.
447 1 Philippe May
448 74 Selvarani C
h2. Linework for the whole Survey Zone
449 74 Selvarani C
450 74 Selvarani C
h2. 1.	Merging Shapefiles / Geopackages - (Selvarani, System 1)
451 74 Selvarani C
452 72 Giulio Di Anastasio
A copy of the Zone Master shapefiles / geopackages are taken from System 4 and stored in Temp Folder on Desktop in System 1.
453 1 Philippe May
Master shapefiles / geopackages are merged with the Survey Area shapefiles / geopackages:
454 72 Giulio Di Anastasio
•	To do it, go to “Vector” Menu, click on Geoprocessing Tools, then click on Union:
455 72 Giulio Di Anastasio
456 85 Selvarani C
!https://redmine.auroville.org.in/attachments/download/9763/Union%20Menu.png!
457 1 Philippe May
458 1 Philippe May
459 85 Selvarani C
The new window for “Union” will appear:
460 1 Philippe May
461 86 Selvarani C
462 86 Selvarani C
!https://redmine.auroville.org.in/attachments/download/9764/Union%20Window.png!
463 73 Giulio Di Anastasio
464 74 Selvarani C
•	To make sure that the right geometry is generated by this process (“line” type, not “Multiline”, and similarly “Polygon” type, not “Multipolygon), we need to always keep the *Master shapefile* (e.g. Final-CZ-01-2021-02-05-LL84_V-BLDG-MHOL------E) *as Input layer*, and the Survey Area shapefile as Overlay Layer (e.g. WD-CZ-01-F-LL84_V-BLDG-MHOL------E).
465 73 Giulio Di Anastasio
•	(The output can be saved to a file, as the CRS should already be EPSG4326 – WGS84.)
466 73 Giulio Di Anastasio
467 74 Selvarani C
h2. 2.	Storing Shapefiles / Geopackages - (Selvarani, System 1)
468 74 Selvarani C
469 74 Selvarani C
Save the merged shape file in the correct location in Final folder as +D: > Survey > Zone-Survey Number > Final+ (eg . D: > Survey > GB-01 > Final)
470 73 Giulio Di Anastasio
Date in the name of Final Shapefile / Geopackage needs to be updated.
471 73 Giulio Di Anastasio
Once the merging operation is completed, the copy of Master shapefile / geopackage is deleted from the Temp folder.
472 73 Giulio Di Anastasio
473 74 Selvarani C
h2. 3.	Topology check of merged shapefiles
474 74 Selvarani C
475 73 Giulio Di Anastasio
The topology checker is applied again on the merged shapefiles / geopackages.
476 73 Giulio Di Anastasio
The “id_field” shall be removed from the attribute table.
477 73 Giulio Di Anastasio
478 74 Selvarani C
h2. 4.	Archive and replace the Master Shapefiles / Geopackages (Ram, System 4)
479 74 Selvarani C
480 73 Giulio Di Anastasio
Archive the previous master shapefiles / geopackages on system 4, and copy the new merged shapefiles / geopackages in its place.
481 74 Selvarani C
*Then delete the Merged Shapefile / Geopackage folder from System 1.
482 74 Selvarani C
*
483 73 Giulio Di Anastasio
484 74 Selvarani C
h2. 5.	Note about Shapefiles and Geopackages
485 73 Giulio Di Anastasio
486 73 Giulio Di Anastasio
All the above works are usually done using shapefile format, in QGIS latest version (3.16.3).
487 221 Selvarani C
The Geopackage export is done in QGis versions older than 3.12 (e.g. 3.4, 3.6, 3.8, 3.10) so that the lines are not saved as “Multilines” but as “Lines”and polygons are not saved as “Multipolygons” but as “Polygons”. This is very important to be noted, as Gisaf database does not accept the Multipolygon and Multiline geometry types.
488 221 Selvarani C
A different way to create "Polygons" is to use the command Vector -> Geometry Tools -> Multipart to Single Parts and apply it to the layer: from "Multipolygon" it will become "Polygon" (check in layer Properties).
489 222 Selvarani C
As on 13 March 2021, Gisaf can accept Multipolygon layers, because the command "Multipart to Single parts" has been integrated into the importing command (see Redmine ticket #11691)
490 223 Selvarani C
491 223 Selvarani C
h1. Creating 3D Shapefile/Geopackage
492 223 Selvarani C
493 223 Selvarani C
While creating a Shapefile/Geopackage, the File name/Database-table name, Geometry type and CRS have to be entered.
494 223 Selvarani C
In order to create a 3D Shapefile/Geopackage, the additional dimensions *"Z(+M values)"/ "Include Z dimension"* has to be ticked: this way the 3D Shapefile/Geopackage is accepted by Gisaf without errors, otherwise the Shapefile/Geopackage can't be imported in Gisaf because the Z dimension is missing (the geometries in the database are all 3D).
495 224 Giulio Di Anastasio
496 224 Giulio Di Anastasio
h1. Exporting from QGis (shapefiles and/or geopackages) to CAD dxf format
497 224 Giulio Di Anastasio
498 226 Giulio Di Anastasio
An algorithm has been created by Selvarani, to see the whole process click here:
499 226 Giulio Di Anastasio
https://redmine.auroville.org.in/projects/gis/wiki/Automatic_export_from_QGis_to_dxf