Project

General

Profile

Wiki » History » Version 294

Giulio Di Anastasio, 06/05/2021 15:05

1 1 Philippe May
h1. %{color:BLUE}  Wiki%
2 227 Giulio Di Anastasio
3 232 Giulio Di Anastasio
h1. [[CSR Geomatics Team]]
4 231 Giulio Di Anastasio
5 229 Giulio Di Anastasio
h1. [[Collaborations]]
6 229 Giulio Di Anastasio
7 240 Giulio Di Anastasio
h1. [[Data Analysis]]
8 234 Giulio Di Anastasio
9 241 Giulio Di Anastasio
h1. [[CSR Geomatics Server Setup]]
10 234 Giulio Di Anastasio
11 245 Giulio Di Anastasio
h1. [[Database Backup and Restoration]]
12 243 Giulio Di Anastasio
13 243 Giulio Di Anastasio
h1. [[CAD to GIS Workflow]] %{color:RED}  - TO BE UPDATED%
14 248 Giulio Di Anastasio
15 246 Giulio Di Anastasio
h1. [[GIS Conventions]]
16 246 Giulio Di Anastasio
 
17 252 Giulio Di Anastasio
h1. [[Working with Gisaf]]
18 96 Selvarani C
19 92 Selvarani C
h1. [[Gisaf Workflow]]
20 252 Giulio Di Anastasio
21 256 Giulio Di Anastasio
h1. [[GIS to CAD Workflow]] %{color:RED}  - TO BE UPDATED%
22 94 Selvarani C
23 285 Giulio Di Anastasio
h1. [[Rain gauges]]
24 285 Giulio Di Anastasio
25 257 Giulio Di Anastasio
h1. [[Weather Station]]
26 142 Selvarani C
27 259 Giulio Di Anastasio
h1. [[GDAL - Useful Tips]]
28 5 Philippe May
29 262 Giulio Di Anastasio
h1. [[Gisaf - Auxiliary Data (non-geographical data)]]
30 143 Selvarani C
31 264 Giulio Di Anastasio
h1. [[QGis - Useful Tips]]
32 264 Giulio Di Anastasio
33 266 Giulio Di Anastasio
h1. [[AutoCAD Civil 3D - Useful Tips]]
34 266 Giulio Di Anastasio
35 280 Giulio Di Anastasio
h1. [[Wells]]
36 8 Philippe May
37 272 Giulio Di Anastasio
h1. [[Gisaf Data Modifications]]
38 272 Giulio Di Anastasio
39 274 Giulio Di Anastasio
h1. [[Other Useful Softwares]]
40 272 Giulio Di Anastasio
41 286 Giulio Di Anastasio
h1. [[Portable Ultrasonic Flow Meter]]
42 262 Giulio Di Anastasio
43 1 Philippe May
44 274 Giulio Di Anastasio
45 274 Giulio Di Anastasio
46 107 Selvarani C
47 110 Selvarani C
48 110 Selvarani C
49 110 Selvarani C
50 113 Selvarani C
51 270 Giulio Di Anastasio
52 270 Giulio Di Anastasio
53 119 Selvarani C
54 119 Selvarani C
55 128 Selvarani C
56 128 Selvarani C
57 129 Selvarani C
58 1 Philippe May
59 132 Selvarani C
60 177 Selvarani C
61 133 Selvarani C
62 133 Selvarani C
63 133 Selvarani C
64 1 Philippe May
65 68 Giulio Di Anastasio
h1. How to create a new projection in QGis
66 68 Giulio Di Anastasio
67 68 Giulio Di Anastasio
To create a new projection in QGis, go to menu "Settings", and click on "Custom Projections".
68 68 Giulio Di Anastasio
A pop-up window appears with a list of all projections defined in QGis projects used by the user so far.
69 68 Giulio Di Anastasio
Click on the green "+" sign on the right top part of the window to create a new projection.
70 68 Giulio Di Anastasio
In the "Name" box, type "TM CSRAUSPOS SF1" (which means TM = Transverse Mercator projection; CSRAUSPOS = theparameters for this projection are derived from the processing of DGPS raw data by AUSPOS -  Online GPS Processing Service - https://www.ga.gov.au/scientific-topics/positioning-navigation/geodesy/auspos; SF1 = Scale Factor is 1).
71 68 Giulio Di Anastasio
In the "Format" dropdown list, select "Proj String (legacy - Not Recommended)"
72 68 Giulio Di Anastasio
In the "Parameters" box, paste the following "+proj=tmerc +lat_0=12.01605433+lon_0=79.80998934 +k=1 +x_0=370455.630 +y_0=1328608.994 +ellps=WGS84+towgs84=0,0,0,0,0,0,0 +units=m +no_defs".
73 68 Giulio Di Anastasio
74 68 Giulio Di Anastasio
Finally, click on OK.
75 68 Giulio Di Anastasio
76 68 Giulio Di Anastasio
In a more explicit  way, the parameters mean the following:
77 68 Giulio Di Anastasio
Map Projection:        TransverseMercator (TM)
78 68 Giulio Di Anastasio
False Easting:          370455.6300
79 68 Giulio Di Anastasio
False Northing:        1328608.9940
80 68 Giulio Di Anastasio
Latitude of Origin:    12°00'57.79560" (DMS)   12.01605433 (DD)
81 68 Giulio Di Anastasio
Central Meridian:     79°48'35.96164"  (DMS)   79.80998934 (DD)
82 68 Giulio Di Anastasio
Scale Factor:             1.00000000
83 68 Giulio Di Anastasio
Zone Width:              6.0°
84 69 Giulio Di Anastasio
85 69 Giulio Di Anastasio
h1. Elimination of Duplicate points – General criteria
86 69 Giulio Di Anastasio
87 69 Giulio Di Anastasio
It might happen that the same physical feature (e.g. a tree, or a pole) is surveyed more than once: this can happen because there are many physical features in an area, and the survey needs more than one station. So, for example a tree is surveyed from a station, and gets a serial number on that date. When the station is then changed, it might happen that the same tree is resurveyed: another serial number is given, and possibly a different date, if the survey from the second station happened on a different day.
88 69 Giulio Di Anastasio
It is clear that the same tree is then represented with two different points, which means that two different trees exist: but only one tree really exist in the physical reality.
89 69 Giulio Di Anastasio
It is clear that one of the two points is redundant and needs to be removed. If this is noted by the surveyor directly in the field, then the issue is solved by the surveyor himself during processing time.
90 69 Giulio Di Anastasio
If instead, due to various reasons, it was not noted by the surveyor in the field, it will need to be cleaned after the processing, possibly by post-processing staff.
91 69 Giulio Di Anastasio
How to identify duplicate points?
92 69 Giulio Di Anastasio
The following criteria can be used:
93 69 Giulio Di Anastasio
1.	The distance between the two points is less than 30 cm (trees are surveyed if their trunk diameter is at least about 20 cm, so in 30 cm cannot exist two of them)
94 69 Giulio Di Anastasio
2.	The orig_id (serial number) of the points are not in series
95 69 Giulio Di Anastasio
3.	The survey date is not the same
96 69 Giulio Di Anastasio
4.	In case of trees, the species of trees is the same
97 69 Giulio Di Anastasio
5.	5. In case of trees, the tree type is not TDEF (because TDEF are mapped irrespective of their diameter, so they can actually have a small trunk, and two of them might exist in 30 cm), not OT (many TDEF species are surveyed as OT if not otherwise indicated by a botanist)
98 69 Giulio Di Anastasio
6.	The context needs to be evaluated: if one tree is deleted in an area where many trees exist in a limited space, then loosing one in the map is not a big error. If instead one tree is deleted where there are very few trees, then it might be a big loss.
99 70 Giulio Di Anastasio
100 70 Giulio Di Anastasio
101 70 Giulio Di Anastasio
h1. Linework for the Survey Area
102 70 Giulio Di Anastasio
103 70 Giulio Di Anastasio
h2. 1.	Creation of Initial Linework in QGIS using Survey points import - (Ram, System 4)
104 70 Giulio Di Anastasio
105 70 Giulio Di Anastasio
Initial Linework in QGIS is started by surveyor with the knowledge from the Field. For this step, points are simply imported into the QGIS from the field text file (.csv or .txt). CRS needs to be TM-AUSPOS. The box of “First record has field names” shall not be ticked. In Point Coordinates, select the correct field for x, for y and for z (usually “field_2” for x, “field_3” for y and “field_4” for z). Points can be styled using the “Categorized” style in “Symbology”, using “Field_5” as value, or using a Rule-based symbology using the category (field 5) as filter.
106 74 Selvarani C
Linework is created by connecting points having same description and belonging to the same physical feature. *All line and polygon features are created as lines*.
107 70 Giulio Di Anastasio
The Initial Linework for the Survey Area is also stored temporarily in 
108 70 Giulio Di Anastasio
109 74 Selvarani C
+D: > AVSM > Zone-Survey number (eg RZ-01) > Survey Area (eg J) > Temporary WD+
110 70 Giulio Di Anastasio
111 70 Giulio Di Anastasio
h2. Note: The line shapefiles / Geopackages shall be in CRS: TM AUSPOS
112 70 Giulio Di Anastasio
113 70 Giulio Di Anastasio
h2. 2.	Creation of final working drawing Shapefiles / Geopackages - (Selvarani, System 1)
114 1 Philippe May
115 1 Philippe May
Final working drawing Shapefiles / Geopackages are created from the Initial Linework of Survey Area.
116 70 Giulio Di Anastasio
As the Surveyor draws all features as lines (both for lines and polygons features), the following actions shall be done:
117 74 Selvarani C
1.	*If features are lines:*
118 70 Giulio Di Anastasio
•	Export the shapefile / geopackage into the final working drawing folder (Final WD), in separate folders according to its type (e.g. BLDG, FENC, ROAD, etc).
119 70 Giulio Di Anastasio
120 70 Giulio Di Anastasio
h2. The CRS for the export shall be EPSG:4326 - WGS 84
121 70 Giulio Di Anastasio
122 74 Selvarani C
2.	*If features are polygons:*
123 1 Philippe May
•	Lines shall be converted into polygons:
124 74 Selvarani C
125 1 Philippe May
to do it, first click on the layer to be converted to make it active (e.g. WD-CZ-01-F-LL84_V-BLDG-MHOL------E), then go to “Vector” Menu, click on Geometry Tools, click on Line to Polygons:
126 87 Selvarani C
127 84 Selvarani C
!https://redmine.auroville.org.in/attachments/download/9760/Line%20to%20Polygon%20Menu.png!
128 70 Giulio Di Anastasio
129 86 Selvarani C
130 1 Philippe May
The new window for “Lines to Polygons” conversion will appear:
131 86 Selvarani C
132 86 Selvarani C
133 83 Selvarani C
!https://redmine.auroville.org.in/attachments/download/9762/Lines%20to%20Polygon%20Window.png!
134 1 Philippe May
135 71 Giulio Di Anastasio
•	Always cross check the input layer, to make sure that the input layer is the active one
136 71 Giulio Di Anastasio
•	Save the output in a temporary layer
137 74 Selvarani C
•	The temporary layer will be listed in the list of layers, it shall be exported to the saving location as +D: > Survey > Zone-Survey Number > Final WD > Survey Area SHP+ (eg . D: > Survey > GB-01 > Final WD > A-Shp)
138 72 Giulio Di Anastasio
139 74 Selvarani C
h2. The CRS for the export shall be EPSG:4326 - WGS 84
140 74 Selvarani C
141 72 Giulio Di Anastasio
Once all the shapefiles / geopackages are exported in Final WD, for each of the newly exported layers the Topology Checker Tool shall be used.
142 1 Philippe May
143 74 Selvarani C
h2. Linework for the whole Survey Zone
144 74 Selvarani C
145 74 Selvarani C
h2. 1.	Merging Shapefiles / Geopackages - (Selvarani, System 1)
146 74 Selvarani C
147 72 Giulio Di Anastasio
A copy of the Zone Master shapefiles / geopackages are taken from System 4 and stored in Temp Folder on Desktop in System 1.
148 1 Philippe May
Master shapefiles / geopackages are merged with the Survey Area shapefiles / geopackages:
149 72 Giulio Di Anastasio
•	To do it, go to “Vector” Menu, click on Geoprocessing Tools, then click on Union:
150 72 Giulio Di Anastasio
151 85 Selvarani C
!https://redmine.auroville.org.in/attachments/download/9763/Union%20Menu.png!
152 1 Philippe May
153 1 Philippe May
154 85 Selvarani C
The new window for “Union” will appear:
155 1 Philippe May
156 86 Selvarani C
157 86 Selvarani C
!https://redmine.auroville.org.in/attachments/download/9764/Union%20Window.png!
158 73 Giulio Di Anastasio
159 74 Selvarani C
•	To make sure that the right geometry is generated by this process (“line” type, not “Multiline”, and similarly “Polygon” type, not “Multipolygon), we need to always keep the *Master shapefile* (e.g. Final-CZ-01-2021-02-05-LL84_V-BLDG-MHOL------E) *as Input layer*, and the Survey Area shapefile as Overlay Layer (e.g. WD-CZ-01-F-LL84_V-BLDG-MHOL------E).
160 73 Giulio Di Anastasio
•	(The output can be saved to a file, as the CRS should already be EPSG4326 – WGS84.)
161 73 Giulio Di Anastasio
162 74 Selvarani C
h2. 2.	Storing Shapefiles / Geopackages - (Selvarani, System 1)
163 74 Selvarani C
164 74 Selvarani C
Save the merged shape file in the correct location in Final folder as +D: > Survey > Zone-Survey Number > Final+ (eg . D: > Survey > GB-01 > Final)
165 73 Giulio Di Anastasio
Date in the name of Final Shapefile / Geopackage needs to be updated.
166 73 Giulio Di Anastasio
Once the merging operation is completed, the copy of Master shapefile / geopackage is deleted from the Temp folder.
167 73 Giulio Di Anastasio
168 74 Selvarani C
h2. 3.	Topology check of merged shapefiles
169 74 Selvarani C
170 73 Giulio Di Anastasio
The topology checker is applied again on the merged shapefiles / geopackages.
171 73 Giulio Di Anastasio
The “id_field” shall be removed from the attribute table.
172 73 Giulio Di Anastasio
173 74 Selvarani C
h2. 4.	Archive and replace the Master Shapefiles / Geopackages (Ram, System 4)
174 74 Selvarani C
175 73 Giulio Di Anastasio
Archive the previous master shapefiles / geopackages on system 4, and copy the new merged shapefiles / geopackages in its place.
176 74 Selvarani C
*Then delete the Merged Shapefile / Geopackage folder from System 1.
177 74 Selvarani C
*
178 73 Giulio Di Anastasio
179 74 Selvarani C
h2. 5.	Note about Shapefiles and Geopackages
180 73 Giulio Di Anastasio
181 73 Giulio Di Anastasio
All the above works are usually done using shapefile format, in QGIS latest version (3.16.3).
182 221 Selvarani C
The Geopackage export is done in QGis versions older than 3.12 (e.g. 3.4, 3.6, 3.8, 3.10) so that the lines are not saved as “Multilines” but as “Lines”and polygons are not saved as “Multipolygons” but as “Polygons”. This is very important to be noted, as Gisaf database does not accept the Multipolygon and Multiline geometry types.
183 221 Selvarani C
A different way to create "Polygons" is to use the command Vector -> Geometry Tools -> Multipart to Single Parts and apply it to the layer: from "Multipolygon" it will become "Polygon" (check in layer Properties).
184 222 Selvarani C
As on 13 March 2021, Gisaf can accept Multipolygon layers, because the command "Multipart to Single parts" has been integrated into the importing command (see Redmine ticket #11691)
185 223 Selvarani C
186 223 Selvarani C
h1. Creating 3D Shapefile/Geopackage
187 223 Selvarani C
188 223 Selvarani C
While creating a Shapefile/Geopackage, the File name/Database-table name, Geometry type and CRS have to be entered.
189 223 Selvarani C
In order to create a 3D Shapefile/Geopackage, the additional dimensions *"Z(+M values)"/ "Include Z dimension"* has to be ticked: this way the 3D Shapefile/Geopackage is accepted by Gisaf without errors, otherwise the Shapefile/Geopackage can't be imported in Gisaf because the Z dimension is missing (the geometries in the database are all 3D).
190 224 Giulio Di Anastasio
191 224 Giulio Di Anastasio
h1. Exporting from QGis (shapefiles and/or geopackages) to CAD dxf format
192 224 Giulio Di Anastasio
193 226 Giulio Di Anastasio
An algorithm has been created by Selvarani, to see the whole process click here:
194 226 Giulio Di Anastasio
https://redmine.auroville.org.in/projects/gis/wiki/Automatic_export_from_QGis_to_dxf