Project

General

Profile

Wiki » History » Version 295

Giulio Di Anastasio, 06/05/2021 15:07

1 1 Philippe May
h1. %{color:BLUE}  Wiki%
2 227 Giulio Di Anastasio
3 232 Giulio Di Anastasio
h1. [[CSR Geomatics Team]]
4 231 Giulio Di Anastasio
5 229 Giulio Di Anastasio
h1. [[Collaborations]]
6 229 Giulio Di Anastasio
7 240 Giulio Di Anastasio
h1. [[Data Analysis]]
8 234 Giulio Di Anastasio
9 241 Giulio Di Anastasio
h1. [[CSR Geomatics Server Setup]]
10 234 Giulio Di Anastasio
11 245 Giulio Di Anastasio
h1. [[Database Backup and Restoration]]
12 243 Giulio Di Anastasio
13 243 Giulio Di Anastasio
h1. [[CAD to GIS Workflow]] %{color:RED}  - TO BE UPDATED%
14 248 Giulio Di Anastasio
15 246 Giulio Di Anastasio
h1. [[GIS Conventions]]
16 246 Giulio Di Anastasio
 
17 252 Giulio Di Anastasio
h1. [[Working with Gisaf]]
18 96 Selvarani C
19 92 Selvarani C
h1. [[Gisaf Workflow]]
20 252 Giulio Di Anastasio
21 256 Giulio Di Anastasio
h1. [[GIS to CAD Workflow]] %{color:RED}  - TO BE UPDATED%
22 94 Selvarani C
23 285 Giulio Di Anastasio
h1. [[Rain gauges]]
24 285 Giulio Di Anastasio
25 257 Giulio Di Anastasio
h1. [[Weather Station]]
26 142 Selvarani C
27 259 Giulio Di Anastasio
h1. [[GDAL - Useful Tips]]
28 5 Philippe May
29 262 Giulio Di Anastasio
h1. [[Gisaf - Auxiliary Data (non-geographical data)]]
30 143 Selvarani C
31 264 Giulio Di Anastasio
h1. [[QGis - Useful Tips]]
32 264 Giulio Di Anastasio
33 266 Giulio Di Anastasio
h1. [[AutoCAD Civil 3D - Useful Tips]]
34 266 Giulio Di Anastasio
35 280 Giulio Di Anastasio
h1. [[Wells]]
36 8 Philippe May
37 272 Giulio Di Anastasio
h1. [[Gisaf Data Modifications]]
38 272 Giulio Di Anastasio
39 274 Giulio Di Anastasio
h1. [[Other Useful Softwares]]
40 272 Giulio Di Anastasio
41 286 Giulio Di Anastasio
h1. [[Portable Ultrasonic Flow Meter]]
42 262 Giulio Di Anastasio
43 1 Philippe May
44 274 Giulio Di Anastasio
45 274 Giulio Di Anastasio
46 107 Selvarani C
47 110 Selvarani C
48 110 Selvarani C
49 110 Selvarani C
50 113 Selvarani C
51 270 Giulio Di Anastasio
52 270 Giulio Di Anastasio
53 119 Selvarani C
54 119 Selvarani C
55 128 Selvarani C
56 128 Selvarani C
57 129 Selvarani C
58 1 Philippe May
59 132 Selvarani C
60 177 Selvarani C
61 133 Selvarani C
62 133 Selvarani C
63 133 Selvarani C
64 1 Philippe May
65 69 Giulio Di Anastasio
66 69 Giulio Di Anastasio
h1. Elimination of Duplicate points – General criteria
67 69 Giulio Di Anastasio
68 69 Giulio Di Anastasio
It might happen that the same physical feature (e.g. a tree, or a pole) is surveyed more than once: this can happen because there are many physical features in an area, and the survey needs more than one station. So, for example a tree is surveyed from a station, and gets a serial number on that date. When the station is then changed, it might happen that the same tree is resurveyed: another serial number is given, and possibly a different date, if the survey from the second station happened on a different day.
69 69 Giulio Di Anastasio
It is clear that the same tree is then represented with two different points, which means that two different trees exist: but only one tree really exist in the physical reality.
70 69 Giulio Di Anastasio
It is clear that one of the two points is redundant and needs to be removed. If this is noted by the surveyor directly in the field, then the issue is solved by the surveyor himself during processing time.
71 69 Giulio Di Anastasio
If instead, due to various reasons, it was not noted by the surveyor in the field, it will need to be cleaned after the processing, possibly by post-processing staff.
72 69 Giulio Di Anastasio
How to identify duplicate points?
73 69 Giulio Di Anastasio
The following criteria can be used:
74 69 Giulio Di Anastasio
1.	The distance between the two points is less than 30 cm (trees are surveyed if their trunk diameter is at least about 20 cm, so in 30 cm cannot exist two of them)
75 69 Giulio Di Anastasio
2.	The orig_id (serial number) of the points are not in series
76 69 Giulio Di Anastasio
3.	The survey date is not the same
77 69 Giulio Di Anastasio
4.	In case of trees, the species of trees is the same
78 69 Giulio Di Anastasio
5.	5. In case of trees, the tree type is not TDEF (because TDEF are mapped irrespective of their diameter, so they can actually have a small trunk, and two of them might exist in 30 cm), not OT (many TDEF species are surveyed as OT if not otherwise indicated by a botanist)
79 69 Giulio Di Anastasio
6.	The context needs to be evaluated: if one tree is deleted in an area where many trees exist in a limited space, then loosing one in the map is not a big error. If instead one tree is deleted where there are very few trees, then it might be a big loss.
80 70 Giulio Di Anastasio
81 70 Giulio Di Anastasio
82 70 Giulio Di Anastasio
h1. Linework for the Survey Area
83 70 Giulio Di Anastasio
84 70 Giulio Di Anastasio
h2. 1.	Creation of Initial Linework in QGIS using Survey points import - (Ram, System 4)
85 70 Giulio Di Anastasio
86 70 Giulio Di Anastasio
Initial Linework in QGIS is started by surveyor with the knowledge from the Field. For this step, points are simply imported into the QGIS from the field text file (.csv or .txt). CRS needs to be TM-AUSPOS. The box of “First record has field names” shall not be ticked. In Point Coordinates, select the correct field for x, for y and for z (usually “field_2” for x, “field_3” for y and “field_4” for z). Points can be styled using the “Categorized” style in “Symbology”, using “Field_5” as value, or using a Rule-based symbology using the category (field 5) as filter.
87 74 Selvarani C
Linework is created by connecting points having same description and belonging to the same physical feature. *All line and polygon features are created as lines*.
88 70 Giulio Di Anastasio
The Initial Linework for the Survey Area is also stored temporarily in 
89 70 Giulio Di Anastasio
90 74 Selvarani C
+D: > AVSM > Zone-Survey number (eg RZ-01) > Survey Area (eg J) > Temporary WD+
91 70 Giulio Di Anastasio
92 70 Giulio Di Anastasio
h2. Note: The line shapefiles / Geopackages shall be in CRS: TM AUSPOS
93 70 Giulio Di Anastasio
94 70 Giulio Di Anastasio
h2. 2.	Creation of final working drawing Shapefiles / Geopackages - (Selvarani, System 1)
95 1 Philippe May
96 1 Philippe May
Final working drawing Shapefiles / Geopackages are created from the Initial Linework of Survey Area.
97 70 Giulio Di Anastasio
As the Surveyor draws all features as lines (both for lines and polygons features), the following actions shall be done:
98 74 Selvarani C
1.	*If features are lines:*
99 70 Giulio Di Anastasio
•	Export the shapefile / geopackage into the final working drawing folder (Final WD), in separate folders according to its type (e.g. BLDG, FENC, ROAD, etc).
100 70 Giulio Di Anastasio
101 70 Giulio Di Anastasio
h2. The CRS for the export shall be EPSG:4326 - WGS 84
102 70 Giulio Di Anastasio
103 74 Selvarani C
2.	*If features are polygons:*
104 1 Philippe May
•	Lines shall be converted into polygons:
105 74 Selvarani C
106 1 Philippe May
to do it, first click on the layer to be converted to make it active (e.g. WD-CZ-01-F-LL84_V-BLDG-MHOL------E), then go to “Vector” Menu, click on Geometry Tools, click on Line to Polygons:
107 87 Selvarani C
108 84 Selvarani C
!https://redmine.auroville.org.in/attachments/download/9760/Line%20to%20Polygon%20Menu.png!
109 70 Giulio Di Anastasio
110 86 Selvarani C
111 1 Philippe May
The new window for “Lines to Polygons” conversion will appear:
112 86 Selvarani C
113 86 Selvarani C
114 83 Selvarani C
!https://redmine.auroville.org.in/attachments/download/9762/Lines%20to%20Polygon%20Window.png!
115 1 Philippe May
116 71 Giulio Di Anastasio
•	Always cross check the input layer, to make sure that the input layer is the active one
117 71 Giulio Di Anastasio
•	Save the output in a temporary layer
118 74 Selvarani C
•	The temporary layer will be listed in the list of layers, it shall be exported to the saving location as +D: > Survey > Zone-Survey Number > Final WD > Survey Area SHP+ (eg . D: > Survey > GB-01 > Final WD > A-Shp)
119 72 Giulio Di Anastasio
120 74 Selvarani C
h2. The CRS for the export shall be EPSG:4326 - WGS 84
121 74 Selvarani C
122 72 Giulio Di Anastasio
Once all the shapefiles / geopackages are exported in Final WD, for each of the newly exported layers the Topology Checker Tool shall be used.
123 1 Philippe May
124 74 Selvarani C
h2. Linework for the whole Survey Zone
125 74 Selvarani C
126 74 Selvarani C
h2. 1.	Merging Shapefiles / Geopackages - (Selvarani, System 1)
127 74 Selvarani C
128 72 Giulio Di Anastasio
A copy of the Zone Master shapefiles / geopackages are taken from System 4 and stored in Temp Folder on Desktop in System 1.
129 1 Philippe May
Master shapefiles / geopackages are merged with the Survey Area shapefiles / geopackages:
130 72 Giulio Di Anastasio
•	To do it, go to “Vector” Menu, click on Geoprocessing Tools, then click on Union:
131 72 Giulio Di Anastasio
132 85 Selvarani C
!https://redmine.auroville.org.in/attachments/download/9763/Union%20Menu.png!
133 1 Philippe May
134 1 Philippe May
135 85 Selvarani C
The new window for “Union” will appear:
136 1 Philippe May
137 86 Selvarani C
138 86 Selvarani C
!https://redmine.auroville.org.in/attachments/download/9764/Union%20Window.png!
139 73 Giulio Di Anastasio
140 74 Selvarani C
•	To make sure that the right geometry is generated by this process (“line” type, not “Multiline”, and similarly “Polygon” type, not “Multipolygon), we need to always keep the *Master shapefile* (e.g. Final-CZ-01-2021-02-05-LL84_V-BLDG-MHOL------E) *as Input layer*, and the Survey Area shapefile as Overlay Layer (e.g. WD-CZ-01-F-LL84_V-BLDG-MHOL------E).
141 73 Giulio Di Anastasio
•	(The output can be saved to a file, as the CRS should already be EPSG4326 – WGS84.)
142 73 Giulio Di Anastasio
143 74 Selvarani C
h2. 2.	Storing Shapefiles / Geopackages - (Selvarani, System 1)
144 74 Selvarani C
145 74 Selvarani C
Save the merged shape file in the correct location in Final folder as +D: > Survey > Zone-Survey Number > Final+ (eg . D: > Survey > GB-01 > Final)
146 73 Giulio Di Anastasio
Date in the name of Final Shapefile / Geopackage needs to be updated.
147 73 Giulio Di Anastasio
Once the merging operation is completed, the copy of Master shapefile / geopackage is deleted from the Temp folder.
148 73 Giulio Di Anastasio
149 74 Selvarani C
h2. 3.	Topology check of merged shapefiles
150 74 Selvarani C
151 73 Giulio Di Anastasio
The topology checker is applied again on the merged shapefiles / geopackages.
152 73 Giulio Di Anastasio
The “id_field” shall be removed from the attribute table.
153 73 Giulio Di Anastasio
154 74 Selvarani C
h2. 4.	Archive and replace the Master Shapefiles / Geopackages (Ram, System 4)
155 74 Selvarani C
156 73 Giulio Di Anastasio
Archive the previous master shapefiles / geopackages on system 4, and copy the new merged shapefiles / geopackages in its place.
157 74 Selvarani C
*Then delete the Merged Shapefile / Geopackage folder from System 1.
158 74 Selvarani C
*
159 73 Giulio Di Anastasio
160 74 Selvarani C
h2. 5.	Note about Shapefiles and Geopackages
161 73 Giulio Di Anastasio
162 73 Giulio Di Anastasio
All the above works are usually done using shapefile format, in QGIS latest version (3.16.3).
163 221 Selvarani C
The Geopackage export is done in QGis versions older than 3.12 (e.g. 3.4, 3.6, 3.8, 3.10) so that the lines are not saved as “Multilines” but as “Lines”and polygons are not saved as “Multipolygons” but as “Polygons”. This is very important to be noted, as Gisaf database does not accept the Multipolygon and Multiline geometry types.
164 221 Selvarani C
A different way to create "Polygons" is to use the command Vector -> Geometry Tools -> Multipart to Single Parts and apply it to the layer: from "Multipolygon" it will become "Polygon" (check in layer Properties).
165 222 Selvarani C
As on 13 March 2021, Gisaf can accept Multipolygon layers, because the command "Multipart to Single parts" has been integrated into the importing command (see Redmine ticket #11691)
166 223 Selvarani C
167 223 Selvarani C
h1. Creating 3D Shapefile/Geopackage
168 223 Selvarani C
169 223 Selvarani C
While creating a Shapefile/Geopackage, the File name/Database-table name, Geometry type and CRS have to be entered.
170 223 Selvarani C
In order to create a 3D Shapefile/Geopackage, the additional dimensions *"Z(+M values)"/ "Include Z dimension"* has to be ticked: this way the 3D Shapefile/Geopackage is accepted by Gisaf without errors, otherwise the Shapefile/Geopackage can't be imported in Gisaf because the Z dimension is missing (the geometries in the database are all 3D).
171 224 Giulio Di Anastasio
172 224 Giulio Di Anastasio
h1. Exporting from QGis (shapefiles and/or geopackages) to CAD dxf format
173 224 Giulio Di Anastasio
174 226 Giulio Di Anastasio
An algorithm has been created by Selvarani, to see the whole process click here:
175 226 Giulio Di Anastasio
https://redmine.auroville.org.in/projects/gis/wiki/Automatic_export_from_QGis_to_dxf