Project

General

Profile

Wiki » History » Version 250

Giulio Di Anastasio, 05/05/2021 09:29

1 1 Philippe May
h1. %{color:BLUE}  Wiki%
2 227 Giulio Di Anastasio
3 232 Giulio Di Anastasio
h1. [[CSR Geomatics Team]]
4 231 Giulio Di Anastasio
5 2 Philippe May
6 229 Giulio Di Anastasio
h1. [[Collaborations]]
7 229 Giulio Di Anastasio
8 48 Philippe May
9 240 Giulio Di Anastasio
h1. [[Data Analysis]]
10 234 Giulio Di Anastasio
11 237 Giulio Di Anastasio
12 241 Giulio Di Anastasio
h1. [[CSR Geomatics Server Setup]]
13 234 Giulio Di Anastasio
14 88 Selvarani C
15 245 Giulio Di Anastasio
h1. [[Database Backup and Restoration]]
16 243 Giulio Di Anastasio
17 243 Giulio Di Anastasio
18 248 Giulio Di Anastasio
h1. [[CAD to GIS Workflow]] %{color:RED}  - TO BE UPDATED%
19 246 Giulio Di Anastasio
20 246 Giulio Di Anastasio
21 250 Giulio Di Anastasio
h1. [[GIS Conventions]]
22 96 Selvarani C
 
23 92 Selvarani C
24 94 Selvarani C
25 188 Selvarani C
h1. Working with Gisaf
26 3 Philippe May
27 6 Philippe May
h3. Survey data
28 6 Philippe May
29 12 Philippe May
Raw survey data are contained in CSV files, typically downloaded from surveying instruments.
30 12 Philippe May
31 144 Selvarani C
See more information of the process for the survey data (including a flow diagram)
32 12 Philippe May
33 97 Selvarani C
h1. Survey data
34 97 Selvarani C
35 97 Selvarani C
h2. Workflow summary
36 97 Selvarani C
37 190 Selvarani C
p=. !https://redmine.auroville.org.in/attachments/download/9839/Survey_data.jpg!
38 97 Selvarani C
39 97 Selvarani C
h2. Import basket
40 97 Selvarani C
41 97 Selvarani C
Gisaf provides an "import basket" dedicated for raw survey data, which is generated by land survey equipment (Leica's Total Station and RTK). 
42 97 Selvarani C
43 97 Selvarani C
These are CSV files, like:
44 97 Selvarani C
<pre>
45 97 Selvarani C
100081,370633.969,1327742.157,51.187,,,
46 97 Selvarani C
100083,370628.876,1327702.913,51.565,T52,,
47 97 Selvarani C
100082,370628.729,1327720.019,51.261,T52,,
48 97 Selvarani C
100081,370633.969,1327742.154,51.179,,,
49 97 Selvarani C
100083,370628.876,1327702.913,51.565,T52,,
50 97 Selvarani C
20800,370633.969,1327742.154,51.180,,,
51 97 Selvarani C
20801,370618.795,1327713.172,52.817,E30,,
52 97 Selvarani C
20802,370623.674,1327711.436,51.283,B35,,
53 97 Selvarani C
20803,370619.314,1327713.407,51.383,B35,,
54 97 Selvarani C
</pre>
55 97 Selvarani C
56 97 Selvarani C
Each category (5th column) must be defined in the Category table (see [[Categories]]).
57 97 Selvarani C
58 97 Selvarani C
59 97 Selvarani C
h2. Organization of the raw survey data basket
60 97 Selvarani C
61 97 Selvarani C
The basket should be organized in a directory structure:
62 97 Selvarani C
63 97 Selvarani C
- Project name (these can be themselves put in a hierarchy of (sub)directories)
64 97 Selvarani C
65 97 Selvarani C
- Surveyor's organization
66 97 Selvarani C
67 97 Selvarani C
- Equipment (eg. TS, RTK)
68 97 Selvarani C
69 97 Selvarani C
- Survey files (eg. @Our_project-Some_comment-2018-02-23.txt@)
70 97 Selvarani C
71 97 Selvarani C
h3. Format of the survey file names
72 97 Selvarani C
73 97 Selvarani C
<pre>
74 97 Selvarani C
Our_project-Some_comment-2018-02-23.txt
75 97 Selvarani C
</pre>
76 97 Selvarani C
77 97 Selvarani C
The date of the survey follows the ISO date standard: @YYYY-MM-DD@.
78 97 Selvarani C
79 97 Selvarani C
80 97 Selvarani C
h2. Import to the database
81 97 Selvarani C
82 97 Selvarani C
When importing raw survey data files to the database, Gisaf does 2 steps as described below.
83 97 Selvarani C
It's worth noting that, in this process, no reprojection is done.
84 97 Selvarani C
85 97 Selvarani C
86 97 Selvarani C
h3. Feed the raw_survey table
87 97 Selvarani C
88 97 Selvarani C
Each point of the imported raw survey data file is inserted to the raw_survey table:
89 97 Selvarani C
90 97 Selvarani C
# Creation of a Point geometry: the raw_survey table has a geometry column for a single point (@geom@) with x,y and z coordinates
91 97 Selvarani C
# Save the @id@ of the original point *to the @orig_id@ column*
92 97 Selvarani C
# *A unique @id@ is computed* from the following fields: @id@, @project@, @equipment@, @date@
93 97 Selvarani C
# The project is saved in the @project_id@ column
94 97 Selvarani C
# The surveyor identification in @srvyr_id@
95 97 Selvarani C
# The date of survey is saved in the @date@ column
96 97 Selvarani C
# The accuracy is tagged in the @accur_id@, according to a mapping defined in the @accuracyequimentsurveyormapping@ table, which depends on the surveyor and equipment
97 97 Selvarani C
# The category of the point
98 97 Selvarani C
99 97 Selvarani C
100 97 Selvarani C
h3. Feed the @RAW_V_*@ tables
101 97 Selvarani C
102 97 Selvarani C
From the @raw_survey@ table, each point is then copied to its respective @RAW_V_@ table, with basically the same information.
103 97 Selvarani C
104 97 Selvarani C
These tables (which should be created manually or with the admin notebook called @create_tables@, as of today's writing), only contain points.
105 97 Selvarani C
106 97 Selvarani C
The project is saved along: see below.
107 97 Selvarani C
108 97 Selvarani C
109 97 Selvarani C
h2. Import the points
110 97 Selvarani C
111 97 Selvarani C
For categories that define points (opposite to lines and polygons, which require _line work_ carried over in CAD or in a GIS software, see [[Line work]]), the points can be imported automatically to their final destination: the @V_*@ tables.
112 97 Selvarani C
113 97 Selvarani C
Note: in this process, the geometries are reprojected.
114 97 Selvarani C
115 191 Selvarani C
h2. Auto import of the points
116 97 Selvarani C
117 97 Selvarani C
The points found in the @RAW_V_*@ tables can be imported automatically, project per project, the project page of the admin interface.
118 97 Selvarani C
119 192 Selvarani C
h2. Import of the line work (lines and polygons)
120 97 Selvarani C
121 142 Selvarani C
h2. See [[Line work]] - *%{color:RED} to be written%*
122 97 Selvarani C
123 97 Selvarani C
The shapefiles generated manually (line work) should be put in the project's basket, and imported from it.
124 97 Selvarani C
125 148 Selvarani C
h1. Categories
126 12 Philippe May
127 102 Selvarani C
The categories define the types of the geographical features and they are mapped according to ISO standard layer naming conventions: see https://www.nationalcadstandard.org/ncs5/pdfs/ncs5_clg_lnf.pdf
128 101 Selvarani C
129 101 Selvarani C
Gisaf uses:
130 101 Selvarani C
131 101 Selvarani C
* a table @category@ where the layers are defined
132 101 Selvarani C
* a table per category
133 101 Selvarani C
134 101 Selvarani C
h2. Fields for the categories
135 101 Selvarani C
136 194 Selvarani C
*%{color:RED} to be written%* - TODO
137 101 Selvarani C
138 101 Selvarani C
h2. Creation of the RAW_* tables
139 101 Selvarani C
140 101 Selvarani C
This step must be done manually (as of today's date of writing).
141 101 Selvarani C
142 195 Selvarani C
h1. QGis: work on shapefiles
143 1 Philippe May
144 142 Selvarani C
h2. Go to [[shapefiles]] - *%{color:RED} to be written%*
145 5 Philippe May
146 201 Selvarani C
147 5 Philippe May
148 104 Selvarani C
149 17 Philippe May
h2. Gear
150 17 Philippe May
151 17 Philippe May
h3. Survey equipment
152 17 Philippe May
153 143 Selvarani C
h2. See [[survey equipment]] - *%{color:RED} to be written%*
154 17 Philippe May
155 152 Selvarani C
h1. Weather station
156 17 Philippe May
157 152 Selvarani C
158 11 Philippe May
159 106 Selvarani C
h1. Ambient Weather weather station
160 106 Selvarani C
161 106 Selvarani C
We have purchased a WS2902A weather station (https://www.ambientweather.com/amws2902.html).
162 106 Selvarani C
Firmware version: 4.0.2.
163 106 Selvarani C
164 106 Selvarani C
h2. Manual
165 106 Selvarani C
166 106 Selvarani C
The operating manual of the weather station can be found at https://p10.secure.hostingprod.com/@site.ambientweatherstore.com/ssl/Manuals/WS-2902C.pdf
167 106 Selvarani C
168 106 Selvarani C
h2. Connection
169 106 Selvarani C
170 106 Selvarani C
h3. Wifi
171 106 Selvarani C
172 106 Selvarani C
Set up the wifi of the console using the "Ambient Tool" phone application. IP address given by DHCP on the router is: 192.168.1.101
173 106 Selvarani C
174 106 Selvarani C
h3. Local communication
175 106 Selvarani C
176 106 Selvarani C
Fail so far: the only exposed port is TCP/45000. Telnet doesn't show any activity. Nothing found on Internet on this protocol.
177 106 Selvarani C
178 106 Selvarani C
One interesting project may come, hijacking the connection to cloud services: https://www.wxforum.net/index.php?topic=35033.0
179 106 Selvarani C
180 106 Selvarani C
181 106 Selvarani C
h3. Cloud connection
182 106 Selvarani C
183 106 Selvarani C
We'll create an account on AmbientWeather.net (and eventually on WUnderground.net and/or weathercloud.net), and:
184 106 Selvarani C
185 106 Selvarani C
* have the console upload data to there
186 106 Selvarani C
* gisaf to retrieve our WS data from there
187 106 Selvarani C
188 106 Selvarani C
h1. Notes on Console
189 106 Selvarani C
190 106 Selvarani C
The daily rainfall data displayed in  the console resets at 00.30 every night
191 11 Philippe May
192 7 Philippe May
h2. Plan for future
193 7 Philippe May
194 154 Selvarani C
Beside living well, 
195 8 Philippe May
196 107 Selvarani C
h1. Plan
197 107 Selvarani C
198 107 Selvarani C
Some interesting projects that might be integrated:
199 107 Selvarani C
200 107 Selvarani C
* https://github.com/Oslandia/albion : Build 3D geological model from wells information
201 107 Selvarani C
202 49 Philippe May
h2. Other
203 1 Philippe May
204 155 Selvarani C
205 49 Philippe May
206 108 Selvarani C
h1. GDAL (OGR) tools
207 108 Selvarani C
208 108 Selvarani C
"GDAL":https://gdal.org/ is a translator library for raster and vector geospatial data formats. It is used by many software (including QGIS and many other open source ones, including Gisaf). Some command line utilities are supplied, like:
209 108 Selvarani C
210 108 Selvarani C
* @ogr2ogr@ can easily convert one data format to another
211 108 Selvarani C
* @ogrinfo@ displays information about files.
212 108 Selvarani C
213 108 Selvarani C
h2. Using Windows
214 108 Selvarani C
215 108 Selvarani C
On a computer with Windows and GQIS installed:
216 108 Selvarani C
217 108 Selvarani C
1. Open a command line console (eg. <Windows Key> to display the Start menu, then just type @cmd@ and <Enter>)
218 108 Selvarani C
2. In the console window, type (adjust with the QGIS version and location, this seems to be the standard one):
219 108 Selvarani C
220 108 Selvarani C
<pre>
221 108 Selvarani C
"c:\Program Files\QGis 3.10\OSGeo4W.bat"
222 108 Selvarani C
</pre>
223 108 Selvarani C
224 108 Selvarani C
3. GDAL utilities can be used: @ogr2ogr@, etc.
225 108 Selvarani C
226 108 Selvarani C
h2. Example: convert Geopackage to Shapefiles
227 108 Selvarani C
228 108 Selvarani C
Output the content of the geopackage @9wdoogfr_2019-11-13_12_26_07.gpkg@ to the folder @shapefiles@:
229 108 Selvarani C
230 108 Selvarani C
<pre>
231 108 Selvarani C
ogr2ogr -progress -f "ESRI Shapefile" shapefiles 9wdoogfr_2019-11-13_12_26_07.gpkg
232 108 Selvarani C
</pre>
233 108 Selvarani C
234 108 Selvarani C
To output the content of the geopackage @9wdoogfr_2019-11-13_12_26_07.gpkg@ to the @root@ folder:
235 108 Selvarani C
236 108 Selvarani C
<pre>
237 108 Selvarani C
ogr2ogr -progress -f "ESRI Shapefile" c:\shapefiles 9wdoogfr_2019-11-13_12_26_07.gpkg
238 108 Selvarani C
</pre>
239 108 Selvarani C
240 108 Selvarani C
h2. With reprojection
241 108 Selvarani C
242 108 Selvarani C
Same as above, reprojecting to UTM44N: 
243 108 Selvarani C
244 108 Selvarani C
<pre>
245 108 Selvarani C
ogr2ogr -progress -f "ESRI Shapefile" -t_srs EPSG:32644 c:\shapefiles 9wdoogfr_2019-11-13_12_26_07.gpkg
246 108 Selvarani C
</pre>
247 108 Selvarani C
248 8 Philippe May
h2. Links and references
249 8 Philippe May
250 156 Selvarani C
251 16 Philippe May
252 109 Selvarani C
h1. Links
253 109 Selvarani C
254 109 Selvarani C
h2. Water management
255 109 Selvarani C
256 109 Selvarani C
h3. Modflow
257 109 Selvarani C
258 109 Selvarani C
* https://water.usgs.gov/ogw/modflow/
259 109 Selvarani C
260 109 Selvarani C
The reference software for underground water modelling and simulation.
261 109 Selvarani C
In conjunction with flopy (https://water.usgs.gov/ogw/flopy/) and Jupyter (https://jupyter.org/), it provides a _relatively easy_ to use interface.
262 109 Selvarani C
263 109 Selvarani C
264 109 Selvarani C
h3. Freewat
265 109 Selvarani C
266 109 Selvarani C
* http://www.freewat.eu/project
267 109 Selvarani C
268 109 Selvarani C
This project is partly based on modflow, and integrates with QGis.
269 109 Selvarani C
270 109 Selvarani C
h2. QGIS
271 109 Selvarani C
272 109 Selvarani C
* https://www.qgis.org/en/site/
273 109 Selvarani C
274 109 Selvarani C
* Tools for Geology
275 109 Selvarani C
Construction of geological cross sections in QGIS - http://www.geokincern.com/?p=1452
276 109 Selvarani C
277 109 Selvarani C
278 109 Selvarani C
h2. Autocad
279 109 Selvarani C
280 109 Selvarani C
* Overview of Converting Geospatial Data to Drawing Objects: 
281 109 Selvarani C
http://docs.autodesk.com/CIV3D/2013/ENU/index.html?url=filesMAPC3D/GUID-C38FD485-3CC2-4B52-8264-0D8C0F45422B.htm,topicNumber=MAPC3Dd30e41809
282 109 Selvarani C
283 109 Selvarani C
* CAD-DB connection:
284 109 Selvarani C
https://knowledge.autodesk.com/support/autocad-civil-3d/learn-explore/caas/video/youtube/watch-v-AQoB--nyUJA.html
285 109 Selvarani C
286 211 Selvarani C
h1. Orfeo
287 109 Selvarani C
288 109 Selvarani C
* https://www.orfeo-toolbox.org/
289 109 Selvarani C
290 109 Selvarani C
Remote sensing
291 16 Philippe May
292 16 Philippe May
h2. Old docs
293 16 Philippe May
294 157 Selvarani C
[[Shapefiles]] - *%{color:RED} to be written%*
295 158 Selvarani C
296 110 Selvarani C
h1. Data (measurements auxiliary tables)
297 110 Selvarani C
298 110 Selvarani C
Besides the importation of [[shapefiles]], Gisaf can import non-geophical information: auxiliary data (typically categories like the list of locations names, well types, etc), and temporal informations (well levels, etc).
299 110 Selvarani C
300 110 Selvarani C
h2. Command line
301 110 Selvarani C
302 110 Selvarani C
The @import_to_db.py@ script imports files, fetched from a set of URLs (typically, in the Redmine Files section of this project), formats and pre-process, and imports to the database.
303 110 Selvarani C
304 110 Selvarani C
@import_to_db.py@ is a support tool, that is planned to be integrated with the web interface.
305 110 Selvarani C
306 110 Selvarani C
Import all with:
307 110 Selvarani C
308 110 Selvarani C
<pre>
309 110 Selvarani C
phil@phil-mbp:~/BlueLight/gisaf_src/gisaf$ python import_to_db.py
310 110 Selvarani C
</pre>
311 110 Selvarani C
312 110 Selvarani C
The script currently accepts an argument for filtering the URLs to import.
313 110 Selvarani C
314 212 Selvarani C
h1. Pavneet's docs (imported from gisaf's wiki)
315 111 Selvarani C
316 111 Selvarani C
h1. Basic Rules of Map making
317 111 Selvarani C
318 111 Selvarani C
Regardless of the cartographic style or content, most maps have the following common elements.
319 111 Selvarani C
320 111 Selvarani C
321 111 Selvarani C
h2. TITLE
322 111 Selvarani C
323 111 Selvarani C
The title should be in a large font, easily identifiable as the title of the map and should include descriptive text as to the location and purpose of the map. If the map is thematic, the theme should be included in the title. For example: Corn Production in Washington, 1990. The title is usually the largest font size of all lettering on the layout, however, it should not dominate the map graphic itself. The title may or may not be in a box and does not need to be at the top of the page (though it often is). For published materials (e.g., books or articles) the title may be included in a figure caption instead.
324 111 Selvarani C
325 111 Selvarani C
 
326 111 Selvarani C
327 111 Selvarani C
h2. SCALE INDICATOR
328 111 Selvarani C
329 111 Selvarani C
The scale of the map is typically indicated by a graphic bar scale, a representative fraction or a verbal scale. The reader must be able to determine the relationship between a unit of measure on the map and a unit of measure in the real world.
330 111 Selvarani C
331 111 Selvarani C
 
332 111 Selvarani C
333 111 Selvarani C
h2. ORIENTATION
334 111 Selvarani C
335 111 Selvarani C
A map should indicate which way is north (and/or south, east and west). Commonly this is done by a north arrow or compass rose. Orientation may also be shown by graticule or grid marks (e.g. lines of latitude and longitude). By convention north is towards the top of the page (thus some maps do not have north arrows), but the orientation must still be given for a 'proper' map. North does not have to be at the top of the page and a north arrow is essential in maps where it is not.
336 111 Selvarani C
337 111 Selvarani C
h2. BORDER(s)
338 111 Selvarani C
339 111 Selvarani C
A border identifies exactly where the mapped area stops. The border is often the thickest line on the map and should be close to the edges of the mapped area. The distance between the map and the border should be the same on all sides (balanced).
340 111 Selvarani C
341 111 Selvarani C
There can also be a border around the entire map layout (enclosing and grouping the title, legend, text boxes, etc.). 
342 111 Selvarani C
343 111 Selvarani C
Both of these borders are sometimes referred to as a 'neatline.' In addition, there is sometimes a thin additional line just outside of a border (accentuating it and ideally making it more visually appealing) that may also be referred to as a neatline.
344 111 Selvarani C
345 111 Selvarani C
h2. LEGEND
346 111 Selvarani C
347 111 Selvarani C
A legend defines the symbols or colors (including shades of gray and patterns) used on the map. Maps do not need legends if the symbology is so common or simple as to be easily understood by the reader. However, it must be clear what each marker or line type, weight and pattern represents. The legend does not need to be labeled "Legend." The more complicated the symbology on a map the more important the legend becomes.
348 111 Selvarani C
349 111 Selvarani C
h2. MAP CREDITS
350 111 Selvarani C
351 111 Selvarani C
* SOURCE OF DATA (especially on thematic maps)
352 111 Selvarani C
* NAME of the cartographer  
353 111 Selvarani C
* DATE of the map creation/publication 
354 111 Selvarani C
* DATE of the map data 
355 111 Selvarani C
* PROJECTION of the map (especially small-scale maps)
356 111 Selvarani C
357 111 Selvarani C
h2. LOCATOR MAP (INSET)
358 111 Selvarani C
359 111 Selvarani C
A locator map is needed if the area of the map is not easily recognizable or is of large scale. For example, if you map Whatcom County, there should be an inset map of Washington, showing the location of Whatcom County.  Inset DETAIL map(s) may also be used to show an area of the map in greater detail (larger scale).
360 111 Selvarani C
361 111 Selvarani C
h2. EFFECTIVE GRAPHICAL DESIGN
362 111 Selvarani C
363 111 Selvarani C
The layout design is as important as effective sentence structure is to written text. Layout design refers to the planning and decision making processes involved in the visual display of the spatial data. You can achieve balance by rearranging the map elements (north arrow, legend, scale, title, etc.) and changing size of the text, border. etc. The map and map elements should be:
364 111 Selvarani C
365 111 Selvarani C
* Neatly drawn
366 111 Selvarani C
* Appropriately and consistently generalized
367 111 Selvarani C
* Symmetrically balanced (avoid crowding or large blank areas)
368 111 Selvarani C
* Without unnecessary clutter (keep it simple, be wary of 'artistic' details)
369 111 Selvarani C
370 111 Selvarani C
h2. VISUAL HIERARCHY
371 111 Selvarani C
372 111 Selvarani C
A hierarchy of symbology should be used for the lettering, line weights and shading. More important features are typically larger and/or darker, less important/background information should be smaller and/or lighter. At the same time, do not "over weight" or "under weight" features.
373 111 Selvarani C
374 111 Selvarani C
h2. PURPOSE
375 111 Selvarani C
376 111 Selvarani C
All maps have a purpose which should influence every element of the map and the map layout. A cartographer should be able to clearly articulate the purpose of their map and should keep the audience (who the map is going to be used by) and the client (who the maps is being produced for) in mind.
377 1 Philippe May
378 214 Selvarani C
h3. NOTE: Any, or all, of the above 'rules' can be (and frequently have been) violated at the discretion of the cartographer IF doing so produces a better map (better serving its purpose and audience).
379 111 Selvarani C
380 111 Selvarani C
*In general, with cartography, less is more (avoid excessive clutter).*
381 1 Philippe May
382 161 Selvarani C
h1. GIS - Survey_Database
383 112 Selvarani C
384 112 Selvarani C
DB- Database
385 112 Selvarani C
C3D- Civil 3D
386 112 Selvarani C
387 215 Selvarani C
h1. How to create Survey database in Civil 3D
388 112 Selvarani C
389 215 Selvarani C
h2. 1. Setting up the Working Folder
390 112 Selvarani C
391 112 Selvarani C
A working folder needs to be created where the survey databases gets stored. In the *Toolbox* > Right click on *Survey databases* > Set the working folder - Save the folder in the desired location. We are setting the Working Folder for the Civil 3D databases in C:
392 112 Selvarani C
393 117 Selvarani C
!https://redmine.auroville.org.in/attachments/download/9783/working%20folder%20image.jpg!
394 116 Selvarani C
395 215 Selvarani C
h2. 2. Creating a database
396 112 Selvarani C
397 112 Selvarani C
Right click on *Survey databases* > *New local Survey database* - Enter the name> click ok. A survey DB is created (It is in bold which signifies it is the current DB under use). Multiple DB's can be created. A survey DB can be opened for editing by right click > *open for edit*. It can be closed by right click > *close for edits*. A survey DB can also be opened as *read-only* by same procedures.
398 112 Selvarani C
399 215 Selvarani C
h2. 3. Setting up the database
400 112 Selvarani C
401 112 Selvarani C
Survey Database(abc) > Right click + Survey Database Settings > Specify co-ordinate Zone > Distance- Metre > Temperature - Celsius > Pressure - Millibars > Distance type - Horizontal > Vertical type - Vertical Distance.
402 112 Selvarani C
403 112 Selvarani C
Then the next step is to create the Networks under which the point data gets uploaded. For example in our case we have - TS and RTK
404 112 Selvarani C
405 112 Selvarani C
h2. Components of Survey Database
406 112 Selvarani C
407 112 Selvarani C
Survey database in civil 3D has the following components when expanded. 
408 112 Selvarani C
# Import Events
409 112 Selvarani C
# Survey Queries
410 112 Selvarani C
# Networks
411 112 Selvarani C
# Network Groups
412 112 Selvarani C
# Figures
413 112 Selvarani C
# Figure Groups
414 112 Selvarani C
# Survey Points
415 112 Selvarani C
# Survey point Groups
416 112 Selvarani C
417 118 Selvarani C
!https://redmine.auroville.org.in/attachments/download/9782/Survey_db.png!
418 163 Selvarani C
419 113 Selvarani C
420 113 Selvarani C
h1. Survey- Field to Finish
421 113 Selvarani C
422 113 Selvarani C
h2. Steps from field work 
423 113 Selvarani C
424 113 Selvarani C
# ?? To be added by Raj and Ram
425 113 Selvarani C
# Copy the .txt files from the equipment- Controller/Total station, to system using a pendrive (Storing the data- "D:"> "AVSM"> "Water Projects"> Respective project folder)> "TS"/"RTK")
426 113 Selvarani C
# Cleaning up the .txt files/removing errors (If any)
427 113 Selvarani C
428 113 Selvarani C
h2. Feeding the survey data into Civil 3D- by Surveyor
429 113 Selvarani C
430 113 Selvarani C
# After cleaning up of files, the files are stored in a desired location, to be used as import events into the Civil 3D file
431 113 Selvarani C
# Open up project in Civil 3D
432 113 Selvarani C
# Importing of Events (explain the step)
433 113 Selvarani C
434 113 Selvarani C
h2. Processing- by Surveyor
435 113 Selvarani C
436 113 Selvarani C
Initial line work is generated from survey points in the drawing. Points for reconciliation (Changing the point codes) are reported, if any. It is important that there is a fair knowledge of Autodesk Civil 3D for this work.
437 113 Selvarani C
438 113 Selvarani C
h2. Post-Processing
439 113 Selvarani C
440 113 Selvarani C
Final linework is generated on top of the processed linework. At this stage, a drawing is cleaned and prepared for sharing. problems like - overlaps in line-work, proper assigning of layers etc. Points for reconciliation are reported, if any. Post-Processing is crucial to generate correct and standardised survey drawings (in .dwg format) keeping in mind that those drawings will be used to generate shape files (.shp format). Therefore, it becomes important to follow a certain workflow based on compatibility of .dwg elements and .shp elements - for example for creating a shapefile of "Polygon" type, the elements in .dwg should be all "Polylines" of "closed" nature. All features should be in zero " 0 " elevation and the shapefile generated should be 3d type. This has been elaborated here < insert > At this stage, a good knowledge of Autodesk Civil 3D and GIS (to a certain extent) is a must.
441 113 Selvarani C
442 113 Selvarani C
h2. Creating shapefiles in AutoCAD
443 113 Selvarani C
444 217 Selvarani C
?? add by pavneet - *%{color:RED} to be written%*
445 113 Selvarani C
446 113 Selvarani C
h2. Sharing on WebGIS - "GISAF-  +gis.auroville.org.in+" 
447 113 Selvarani C
448 113 Selvarani C
After creating the shp files from AutoCAD, the shp files are ready to be shared on WebGIS platform. Following are the steps to do so.
449 113 Selvarani C
450 113 Selvarani C
451 113 Selvarani C
# *Upload and import Raw survey points to GISAF* - Before uploading and importing the shapefiles, it is necessary to upload and import the corresponding Raw survey points into GISAF.
452 113 Selvarani C
# *Auto-import of point shapefiles* - After the import of point files (RTK and TS), the next step is to Auto-import the points which are point type shp files. Go to Admin> Others> Projects> Select the corresponding project > With selected > Auto- import to GIS Database ( GISAF)
453 113 Selvarani C
# *Upload shp to basket* - Zip together all the types of files obtained after saving a shapefile on the system (shp, prj, dbf etc). Follow the same naming standards when creating this. Upload it to the basket in GISAF- Admin> Basket> Shapefiles> click on relevant project> click on relevant sub-project (if any)> upload file> select the zipped file from system> select the category from directory >save.
454 113 Selvarani C
# *Import to GISAF (Only with Authorization)* - Click on the import arrow.
455 113 Selvarani C
456 113 Selvarani C
h2. Adding new codes
457 113 Selvarani C
458 220 Selvarani C
ADD !! - *%{color:RED} to be written%*
459 113 Selvarani C
460 113 Selvarani C
h2. Reconciliation of points
461 113 Selvarani C
462 220 Selvarani C
The points are noted with their codes from and codes to reconcile into. ADD!! - *%{color:RED} to be written%*
463 119 Selvarani C
464 119 Selvarani C
h1. Survey Data Post-Processing
465 119 Selvarani C
466 119 Selvarani C
+Softwares used: Civil 3D, QGIS+
467 119 Selvarani C
468 119 Selvarani C
AutoCAD Civil 3D - C3D, 
469 119 Selvarani C
470 119 Selvarani C
C3D is being used for post processing of survey data. C3D offers a BIM solution for Land Surveyors. Basic knowledge about C3D can be obtained by following tutorials online though a pre-acquired knowledge of Autodesk AutoCAD is a plus point and sufficient enough to start working with C3D.
471 119 Selvarani C
472 119 Selvarani C
QGIS is an Open source GIS.
473 119 Selvarani C
474 119 Selvarani C
Getting Started:
475 119 Selvarani C
+Standards and Workflow for C3D+
476 119 Selvarani C
The layers (Nomenclature of layers is coming from standards- U.S. National CAD Standard Version-AIA) are pre-assigned a geometry type (Point, Line and Polygon) in their layers description (Use layer manager to check). This is done keeping in mind inter-operability with GIS (In shapefiles format), geometry type Shape files are of three kinds- points, lines and polygons.
477 119 Selvarani C
478 119 Selvarani C
+Geometry types-+
479 119 Selvarani C
Based on the description of Layers, if the layer Geometry is specified as 'Point', the raw survey data can be auto imported to webGIS - *GISAF*, no post processing is needed for 'Points'. For lines and polygons, We use Polylines (for 2D lines/Polygons) and 3Dpolylines (for 3D lines/polygons). In case of curved lines in 2D, Polylines command (PLINE/PL) is enough but its not possible to create curved lines in 3D using the 3Dpolylines, in such cases feature lines come into the picture. (*Important:* 2D polylines, splines, ellipses and circles are not exportable to shapefile format, they have to be always converted into Polylines).
480 119 Selvarani C
481 119 Selvarani C
*We are at the moment generating shapefiles (lines and polygons) in 2D (z=0) but keeping the format of files as 3D.
482 119 Selvarani C
483 119 Selvarani C
+Using Feature lines for creating Curved geometries in 3D-+
484 119 Selvarani C
There are a series of steps involved when creating curved lines using feature lines. You can follow two methods depending on the situation :-
485 119 Selvarani C
486 119 Selvarani C
Method 1(For curved objects in 3D using elevation of points):- Create the object using 3DPOLY. Now use the create feature lines from objects and select the objects to be converted to feature lines. You will not notice any difference in the geometry on the screen as such but in properties you will see the object type description of selected object as feature lines.
487 119 Selvarani C
488 119 Selvarani C
Method 2(For curved objects in 3D by draping the lines/curves on a surface):- Create the object using PLINE. You will notice all the lines are straight. Now use the create feature lines from objects and select the objects to be converted to feature lines. You will not notice any difference in the geometry on the screen as such but in properties you will see the object type description of selected object as feature lines.
489 119 Selvarani C
490 119 Selvarani C
Now modify tab is used to convert the straight lines to curved lines. There are many ways to do it and different methods can be adopted in different situations. The easiest and the most commonly used method is by using the smooth command in the modify tab. After using the smooth command, a curve would be visible. This is a curve in 3D. To view it, object viewer option can be used by selecting the object and right clicking.
491 119 Selvarani C
More complex methods have to be studied and each case has to be taken into consideration separatly when using - FITCURVEFEATURE In some cases to obtain a desired curve(Meaning more fragments) more PI's (Point of insertion) need to be added into the feature line. PI are the points where the feature lines gets fragmented when exploded.This step often becomes difficult to manage due to complexities involved in mathematical functions behind creation of feature lines.But it should be explored by all means.
492 119 Selvarani C
Because the feature lines are not exportable to shapefiles, the feature line has to converted back into a 3D polyline. This can be done simply by explolding the object. Once exploded into 3D polylines, you will see that the curve gets fragmented into smaller segments of straight lines. You can repeat the process of creating the 3Dpolyline object into feature line and then smoothening it till you get a desired geometry resembling the curve.
493 119 Selvarani C
Situations of sharing of points with different layers and proceeding with linework in such circumstances-
494 119 Selvarani C
There is always a best way to take the survey points taken in the field by the surveyor. For instance, in case of a road and a curb adjacent to it, the surveyor takes the points only once. It is only during the post processing that the lines are generated (You can create the lines in field while taking the survey points as well, but that is not the most feasible method majorly due to time constraints) and in this case, two line on the same position shall be generated, one under layer of roads and one under layer of curbs. 
495 119 Selvarani C
The line work becomes complex in case of generating 3d lines and 2d lines on the same place. This can arise due to having some layers as 2D (for example building outlines) and some as 3D (paved surfaces- <taking into consideration the future use of the layers. In this case, for the purpose of water management, it becomes essential to have survey of such surfaces as 3D. Survey of building outlines doesn't need to be 3D because the elevation points are not recorded while surveying them due to feasibility).
496 119 Selvarani C
So, in case of generating 3D linework from survey points with no elevation, Feature lines and SURFACES come into picture. The surfaces are generated by an interpolation method in C3D by giving a set of Point Groups. We have selected the triangulation method as the appropriate method due to availability of a dense set of points. 
497 119 Selvarani C
While creating the feature lines from objects (as explained above, check the box "assign elevation" and a dialog box appears to select the surface you want to select). Surfaces will have to be generated prior to using this. (add wiki for generating surface). Surface should be generated including the points of the layer in the point group of it. 
498 119 Selvarani C
499 119 Selvarani C
*+Purpose of Survey+*
500 119 Selvarani C
501 119 Selvarani C
# The survey is conducted for generating a Base Map which is representative of Topography, Infrastructure, Drainage, Buildings etc, with a focus on Water Management in Auroville. Furthermore this survey can be used as base map to build upon more detailed surveys for the various purposes such as Town planning, Land Surveys, Transportation Planning etc.
502 119 Selvarani C
503 119 Selvarani C
504 119 Selvarani C
+Use of Layers+
505 119 Selvarani C
506 119 Selvarani C
# *V-BLDG-SHED* (Shed) layer is used when the structure is not entirely enclosed, if At least one side is open (No walls/partition etc). One thumb rule is to answer the question if the building can be locked safely or not. If not, then it is a shed.
507 119 Selvarani C
# *V-BLDG-HUT* (Hut) is used when the building (For human activity/living space) is single storey with roof made up of perishable material such as keet, straw etc.
508 119 Selvarani C
# Use of *V-BLDG-ROOF* (Roof) is for buildings with permanent roofs extending out about more than 1 meter from the outline at ground level. It can also be used for complex roof structures for example Matrimandir Petals.
509 119 Selvarani C
# Use of *V-BLDG-OTLN* (Building outline) is for depicting the outer line of walls of a structure on ground level. This is used when the roof of the building is more or less of the same profile.
510 119 Selvarani C
# *V-BLDG-RTWL* (Retaining Wall) is for walls with the function of retaining either earth/water and the side of the wall should be visible. In case of very thin walls and no side visible, it comes as *V-BLDG-RWLL* (Retaining wall line). This is depicted as a single line in drawing.
511 119 Selvarani C
# Use of *V-ROAD-CYCP* is for designated cycle path as a single line.
512 119 Selvarani C
# Use of *V-ROAD-FPAT* is for pedestrian paths as a single line. It is used when there is a path of very small width. If the need is to represent a path/road which is unpaved in nature, *V-ROAD-UPVD* is used, which is polygon type. To represent a path/road of paved nature *V-ROAD-PAVD* is used, which is also a polygon type.
513 119 Selvarani C
# *V-WMNG-PIPP* for pipe points, *V-WMNG-PIPL*- for pipe lines.
514 119 Selvarani C
# *V-BLDG-BMRK* is for unidentified bench marks and *V-BLDG-MHOL* is for unidentified manholes.
515 119 Selvarani C
# *V-BLDG-PLTF* is for impervious surfaces. for example sand and gravel will not come under this layer. cemented paths etc can come.
516 119 Selvarani C
# *V-WMNG-POOL* is for water bodies made for leisure activities- like swimming pool etc.
517 119 Selvarani C
# *V-WMNG-SUMP* is collectively for artificial water retention structures.
518 119 Selvarani C
# *V-WATR-POND* is for top of natural water retention.
519 119 Selvarani C
# *V-WATR-PONB* is for bottom of natural water retention.
520 119 Selvarani C
# *V-WMNG-DRBA* is for bottom of artificial drains.
521 119 Selvarani C
# *V-WMNG-DRTA* is for top of artificial drains.
522 119 Selvarani C
# *V-WATR-DRBN* is for bottom of Natural drains.
523 119 Selvarani C
# *V-WATR-DRTN* is for top of Natural drains.
524 119 Selvarani C
# *V-WMNG-DRNC* is for drain covers.
525 1 Philippe May
# *V-WATR-DRAS* is for indicating slope of drainage, these are arrows.
526 119 Selvarani C
527 120 Selvarani C
h1. Wells Documentation
528 120 Selvarani C
529 120 Selvarani C
Documentation of the wells in Auroville started afresh in September 2017 by Bala working with the *Water Group*. He used a mobile GPS to record co-ordinates and took pictures along with other data related to a well. An effort was made to reconcile data by identification  of the wells with existing data from *Auroville Water Harvest* which ceased to exist around 2007/2009. In some cases the codes on the well on location (sometimes there is a code on the pump, on the casing or on a nearby wall) helped in reconciliation but mostly by spatial mapping using QGIS. Some wells could not be reconciled due to absence of any nearby well in old data. However, we are publishing all the wells we have documented so far.
530 120 Selvarani C
531 120 Selvarani C
*Terms of reference*
532 120 Selvarani C
533 120 Selvarani C
+Well status+ 
534 120 Selvarani C
	
535 120 Selvarani C
# *In use*      Well is equipped with functioning a pump	
536 120 Selvarani C
# *Not in use*  Well is not equipped with a functioning pump. (Special case- well is equipped with a non functioning pump)
537 120 Selvarani C
# *Closed*      Well is closed/ sealed/ abandoned, (Historical reference)
538 120 Selvarani C
539 1 Philippe May
Note: Wells with pump under repair (temporary measure) at the time of survey are treated as *In use*.
540 120 Selvarani C
541 167 Selvarani C
542 121 Selvarani C
543 121 Selvarani C
h1. Civil 3D useful commands
544 121 Selvarani C
545 121 Selvarani C
546 121 Selvarani C
h2. Making Feature line to polyline-
547 121 Selvarani C
548 121 Selvarani C
Select the feature line> go to elevation editor (under feature line tab> edit elevations)> select all the points in table, give elevation(this will give same elevation to all the points thereby making it possible to retain the curves in the polyline) > explode the feature line (use X enter)
549 121 Selvarani C
550 121 Selvarani C
h2. Converting Circle to polyline-
551 121 Selvarani C
552 121 Selvarani C
BR > enter > break the circle at two points to obtain a part of circle. 
553 121 Selvarani C
PEDIT> enter> select the part of leftover circle> J > enter> Close> enter.
554 121 Selvarani C
555 121 Selvarani C
h2. Converting 2DPOLYLINE to POLYLINE
556 121 Selvarani C
557 121 Selvarani C
Explode the 2D polyline, and use the PEDIT command to convert the segments to polylines. Then join the polylines.
558 121 Selvarani C
559 121 Selvarani C
h2. Viewing only the used layer in ACAD-
560 121 Selvarani C
561 121 Selvarani C
Set the value of SHOWLAYERUSAGE from 0 to 1
562 121 Selvarani C
563 121 Selvarani C
h2. Convert 3d polyline to polyline- 
564 121 Selvarani C
565 121 Selvarani C
COVERT3DPOLYS (Change the elevation of the polyline using properties manager to 0 in our case)
566 121 Selvarani C
567 121 Selvarani C
h2. Generating contours from a Surface in Civil 3D
568 121 Selvarani C
569 121 Selvarani C
# In prospector, Create a point group of points needed to generate the contours from for example- TOPO elevation points or BLDG floor levels. If the point group is already present, then proceed to next step.
570 121 Selvarani C
# Create a surface in the prospector> when creating the new surface, give the name and styling in the dialog box- styling used by us is "Triangles and surface- 0.1 and 0.5"> Assign the point groups in the surface as created in previous step.
571 121 Selvarani C
# Go to the surface created in prospector and right click > Edit surface style> Turn on the Major and Minor contour.
572 121 Selvarani C
# Turn on the layers for Major and Minor contours in Layers Manager (LA > enter)
573 121 Selvarani C
574 121 Selvarani C
h2. Shortcuts:
575 121 Selvarani C
576 121 Selvarani C
• Properties manager - ctrl + 1
577 121 Selvarani C
• To view 3D- Select and right click> object Viewer.
578 121 Selvarani C
• To copy the Line - Copy and select the line enter 
579 121 Selvarani C
• To convert spline to polyline >Splinedit
580 121 Selvarani C
• To export shapefile - Mapexport > follow the process.
581 121 Selvarani C
• To assign coordinate system to the drawing - MAPCSLIBRARY
582 121 Selvarani C
• To check coordinate system of the drawing- > TOOLSPACE> Settings> Right click on the drawing name > Edit settings
583 121 Selvarani C
584 122 Selvarani C
h1. Online references for Civil 3D
585 122 Selvarani C
586 123 Selvarani C
*Description Key Sets*
587 122 Selvarani C
588 122 Selvarani C
https://www.youtube.com/watch?v=Von8oCwYcTk
589 122 Selvarani C
590 124 Selvarani C
*Autodesk civil 3D Geotechnical module- for borehole data*
591 122 Selvarani C
592 122 Selvarani C
https://www.youtube.com/watch?v=0Be9kShBou0
593 122 Selvarani C
594 123 Selvarani C
*Civil 3D for Surveyors*
595 122 Selvarani C
596 122 Selvarani C
https://www.youtube.com/playlist?list=PL1EzH8XlwSxuxCMdeLvzqfgsAlmeeHMv2
597 122 Selvarani C
598 123 Selvarani C
*Civil 3D: Survey - Survey Database*
599 1 Philippe May
600 122 Selvarani C
https://www.youtube.com/watch?v=VPWdAfYJt5Y 
601 123 Selvarani C
602 169 Selvarani C
603 125 Selvarani C
604 125 Selvarani C
h1. Connections in QGIS- Using browser panel and Add postGIS
605 125 Selvarani C
606 125 Selvarani C
Working using QGIS as interface using PostGIS connections.
607 125 Selvarani C
608 125 Selvarani C
* Server: @gisdb.csr.av@
609 125 Selvarani C
610 125 Selvarani C
* Database: @avgis@
611 125 Selvarani C
612 125 Selvarani C
* Leave @Service@ empty
613 125 Selvarani C
614 125 Selvarani C
615 125 Selvarani C
h2. Adding tables (With geometries) in form of shape files from database using Browser panel-
616 125 Selvarani C
617 125 Selvarani C
Go to QGIS> view> Panels> Browser panel> PostGIS> Expand the connection> give credentials> add the desired file by double clicking on it.
618 125 Selvarani C
619 125 Selvarani C
620 125 Selvarani C
h2. Adding tables (For non-geometry type) using PostGIS connections-
621 125 Selvarani C
622 125 Selvarani C
Add PostGIS Layers > Give credentials > select "Also list tables with no geometry" > expand public > click on the desired table> add.
623 125 Selvarani C
624 125 Selvarani C
h2. Joining tables
625 125 Selvarani C
626 125 Selvarani C
Right click/double click on file> go to Join > perform the desired joins - add/subtract the joins. 
627 125 Selvarani C
628 125 Selvarani C
629 170 Selvarani C
h1. *Reconcilation of Raw survey data using pgAdmin*
630 126 Selvarani C
631 126 Selvarani C
+For changing the layer codes, to be done using Pgadmin.+
632 126 Selvarani C
633 126 Selvarani C
h3.  Enter Schema> AVSM RAW Survey > Tables > select the table> Right click - View edit data > All rows. Apply filter to the original id and note the Database id's to be reconciled.
634 126 Selvarani C
635 126 Selvarani C
h3.  GISAf Admin> Other > Reconciliation. Create > Add the database (point) id and give the new target corresponding to the database id to be changed. (Use the Layer name in target not Raw layer name).
636 126 Selvarani C
637 126 Selvarani C
h3.  Other > Project > select the project > with selected> reconcile RAW survey points.
638 126 Selvarani C
639 127 Selvarani C
h1. Importing point data (TS and RTK) to GISAF 
640 127 Selvarani C
641 127 Selvarani C
# *Gisaf Admin > Basket > Survey data > "Project" > "surveyor" > TS/ RTK > upload > import.*
642 127 Selvarani C
# *Auto import of Raw points data* (changes from Raw to Shapefiles, the point files)
643 1 Philippe May
  In GISAF Admin > Other> Project > select the project for which you want to import the data > with selected > Auto import to GIS...
644 127 Selvarani C
645 172 Selvarani C
646 128 Selvarani C
647 128 Selvarani C
h1. Editing Z value of features in Shapefiles in QGIS
648 128 Selvarani C
649 128 Selvarani C
h2. using vertex editor tool - https://www.youtube.com/watch?v=8V8i1AtcA74&t=256s
650 128 Selvarani C
651 129 Selvarani C
h1. Miscellaneous- Civil 3D
652 129 Selvarani C
653 129 Selvarani C
*Autodesk civil 3D Geotechnical module- for borehole data :- To analyse borehole data, To make profiles and calculate volumes*
654 129 Selvarani C
655 129 Selvarani C
https://www.youtube.com/watch?v=0Be9kShBou0
656 129 Selvarani C
Reference videos
657 129 Selvarani C
Exploring:
658 129 Selvarani C
https://www.youtube.com/watch?v=gr-ISPzLcU0
659 129 Selvarani C
Modelling:
660 129 Selvarani C
https://www.youtube.com/watch?v=Pz0-HOoiBrs
661 129 Selvarani C
solids
662 129 Selvarani C
https://www.youtube.com/watch?v=mQ0Yeh6tZA8
663 129 Selvarani C
https://www.youtube.com/watch?v=4eMsUiYBhuE
664 129 Selvarani C
665 129 Selvarani C
666 129 Selvarani C
*CIVIL 3D Survey*
667 129 Selvarani C
Getting started- Always open a new drawing with a template. 
668 129 Selvarani C
669 129 Selvarani C
*Description key sets*- (till 18:30) https://www.youtube.com/watch?v=mmwkkRyBkS0
670 129 Selvarani C
As when the points come into the drawing, they are going to be filtered with description key sets and the description key sets will assign properties to the points.
671 129 Selvarani C
Tool space > settings > points > description key sets
672 129 Selvarani C
“Description key sets name”> edit key- to view the points in a list: 
673 129 Selvarani C
Automatic linework (18:30- till end)
674 129 Selvarani C
Survey > Linework code sets- for automatic linework
675 129 Selvarani C
676 129 Selvarani C
*Using Master view*
677 129 Selvarani C
Uses-
678 129 Selvarani C
copy styles from one drawing to other
679 129 Selvarani C
copy drawing data from one drawing to other (data referencing)
680 129 Selvarani C
681 129 Selvarani C
*Civil 3d surface model*- from points- break lines
682 129 Selvarani C
https://www.youtube.com/watch?v=wC40rdmDEMo
683 129 Selvarani C
moving for example- point groups under point groups- 
684 129 Selvarani C
685 129 Selvarani C
*Civil 3D Planning and Analysis*
686 129 Selvarani C
Workspace: Planning and Analysis
687 129 Selvarani C
1. Working with Object data (GIS Attributes)
688 129 Selvarani C
Map Setup> Define object data> New table> Define new object data table start defining fields
689 129 Selvarani C
690 129 Selvarani C
691 130 Selvarani C
692 130 Selvarani C
h1. Documentation- Rain Gauge
693 130 Selvarani C
694 130 Selvarani C
h2. Manual Rain Gauge
695 130 Selvarani C
696 130 Selvarani C
*Why it is important to comply to standards?*
697 130 Selvarani C
698 130 Selvarani C
1. Consistency for comparing rainfall data in different places within Auroville since rainfall varies in different parts of Auroville.
699 130 Selvarani C
700 130 Selvarani C
2. Making it possible to use for any kind of scientific analysis by contribution to the primary data in sustainable water management for Auroville. 
701 130 Selvarani C
702 130 Selvarani C
*Proposed standards and ethics:*
703 130 Selvarani C
704 130 Selvarani C
1. The time of taking the reading - 08:30 am. 
705 130 Selvarani C
The rain is recorded over a period of 24 hours- 8:30 of previous day till 8:30 of the present day, and the date is put as the present day. For example, if the rain is measured at 8:30 am on 10 dec 2017, then it is logged in as on 10 dec 2017.
706 130 Selvarani C
707 130 Selvarani C
2. Manual rain gauge typically used- green cylindrical and a collection jar.  
708 130 Selvarani C
  
709 130 Selvarani C
3. Measuring jar: 10 mm corresponding jar
710 130 Selvarani C
711 130 Selvarani C
 
712 130 Selvarani C
Area of the rim of manual rain gauge= 200 square cm
713 130 Selvarani C
Diameter of rim= approximately 16 cm
714 130 Selvarani C
*NOTE:* - If another kind of rain gauge is used, it should be used with the corresponding measuring jar as type of jar and calibration on jar depends on the area of rim. Any cases of different rain gauge other than above should be reported before use for recording purposes.
715 130 Selvarani C
716 130 Selvarani C
4. The minimum record-able unit is 0.2 mm (Least measure on the 10 mm cylinder). T is marked for below 1 mm.
717 130 Selvarani C
718 130 Selvarani C
5. The reading should be preferably submitted in an excel sheet (the data sheet will be provided). The frequency of sending data can be daily or weekly.
719 130 Selvarani C
6. The rain gauge should be kept in a safe and open to sky area clear from tree cover or objects in surrounding so that direct rain falls into it and not from trees/ objects etc. Preferably roof top or a higher place on ground.
720 130 Selvarani C
7. A check on the rain gauge should be kept when there is long period of gap in rainy days to make sure it was emptied out and there is no blockage. 
721 130 Selvarani C
722 130 Selvarani C
8. In case of doubt in reading or incorrect measure, "incorrect measure" should be marked
723 130 Selvarani C
724 130 Selvarani C
725 130 Selvarani C
*Ethics*
726 130 Selvarani C
1. Readings should not be missed. Time to time check on rain gauge is required during long periods of no rain (especially when the rain season is approaching). 
727 130 Selvarani C
This is important because firstly, if it rained in the night and it wasn’t recorded, reading for the day is lost; Secondly, if the jar was not emptied, it gives incorrect subsequent reading. 
728 130 Selvarani C
729 130 Selvarani C
2.  If the person is not going to be available temporarily where the rain gauge is kept, the responsibility should be passed on to someone they can rely on after explaining the standards. 
730 130 Selvarani C
731 130 Selvarani C
*Important-*
732 130 Selvarani C
Data won’t be published in case of non-compliance to standards. This is to maintain the sanctity of proper scientific data collection and to keep it reliable for sharing by publishing. Discussions regarding standards are welcomed.
733 130 Selvarani C
CSR Geomatics Team is placed on first floor in CSR, Auroshilpam.
734 130 Selvarani C
735 130 Selvarani C
*AV rain data publication*: (http://gis.auroville.org.in/measures/raingauge_av).
736 130 Selvarani C
737 130 Selvarani C
Adding Rain gauge to our web portal (GISAF), following information is needed:
738 130 Selvarani C
1. GPS co-ordinates of location of placing the rain gauge (can be obtained using mobile phones easily)
739 130 Selvarani C
2. Name, place of residence & contact number- Mobile & Landline
740 130 Selvarani C
741 130 Selvarani C
h2. Automatic Rain Gauge
742 130 Selvarani C
743 130 Selvarani C
There is a possibility of publishing rain data coming from Automatic rain gauges like (id 15, auro orchard) and Weather stations. 
744 130 Selvarani C
For Automatic rain gauges, we can upload the files coming from the rain gauge directly into GISAF.
745 130 Selvarani C
746 130 Selvarani C
747 130 Selvarani C
_Thank you for contribution towards a sustainable management of Water in Auroville through Data collection._
748 130 Selvarani C
749 130 Selvarani C
750 131 Selvarani C
751 131 Selvarani C
h1. Documentation- Wells Monitoring (Manual and Automatic / Piezometer by Bala)
752 131 Selvarani C
753 131 Selvarani C
h2. Manual- Using tape with a sensor
754 131 Selvarani C
755 131 Selvarani C
Timings: The person assigned the job of monitoring collects the readings in three slots.
756 131 Selvarani C
757 131 Selvarani C
1.	Between 6 am to 7 am till about 9 am to 10 am.
758 131 Selvarani C
2.	Between 11 am and 1 pm
759 131 Selvarani C
3.	Between 2 pm and 5 pm
760 131 Selvarani C
761 131 Selvarani C
Things required:
762 131 Selvarani C
•	Notebook
763 131 Selvarani C
•	Pen
764 131 Selvarani C
•	Measuring tape
765 131 Selvarani C
•	Vehicle for movement
766 131 Selvarani C
767 131 Selvarani C
768 131 Selvarani C
*Ethics*- 
769 131 Selvarani C
•	The monitoring should happen in coordination with the community members/ care taker (assigned by the person in charge from within the community). There should be a clear communication from monitor side as to which days the monitoring of a well has to happen and at around what time so that it can be made sure that the pump is not turned on before monitoring. In case a pump was turned on in a well, the monitor should have a gap of about 5 hours on the same day before going again for monitoring.
770 131 Selvarani C
•	The monitor should have contact numbers of the person in charge/ care taker for any communication.
771 131 Selvarani C
•	If the monitoring is stopped for any reason at any point, the monitor should communicate the same to the person concerned.
772 131 Selvarani C
•	The monitor shall take responsibility to inform any kind of changes in a well in terms of its functioning etc.
773 131 Selvarani C
•	The monitor should report to the CSR Geomatics Team who has responsibility to publish data.
774 131 Selvarani C
•	The monitor published the data to the website and works with the geomatics team.
775 131 Selvarani C
776 131 Selvarani C
777 131 Selvarani C
*Definitions*:
778 131 Selvarani C
A master file is maintained with the records of the wells from the field. It is updated when a new well is located. The following set of information are filled out in the Masterfile. 
779 131 Selvarani C
780 131 Selvarani C
*Well type*
781 131 Selvarani C
1.	Open
782 131 Selvarani C
2.	Borewell
783 131 Selvarani C
3.	Dug cum borewell
784 131 Selvarani C
785 131 Selvarani C
*Well status*
786 131 Selvarani C
1.	In use
787 131 Selvarani C
2.	Not in use
788 131 Selvarani C
3.	Closed
789 131 Selvarani C
790 131 Selvarani C
*Pump Status*
791 131 Selvarani C
1.	Functioning
792 131 Selvarani C
2.	Not functioning
793 131 Selvarani C
794 131 Selvarani C
*Pump Automation*
795 131 Selvarani C
1.	Manual 
796 131 Selvarani C
2.	Automatic
797 131 Selvarani C
798 131 Selvarani C
* Non-accessibility factors*
799 131 Selvarani C
1.	Dogs
800 131 Selvarani C
2.	Heavy slab
801 131 Selvarani C
3.	Narrow casing
802 131 Selvarani C
4.	enclosed/locked
803 131 Selvarani C
5.	permission
804 131 Selvarani C
805 131 Selvarani C
*Data Matching Accuracy (reconnecting with harvest wells data)*
806 131 Selvarani C
1.	High
807 131 Selvarani C
2.	Medium
808 131 Selvarani C
3.	Low
809 131 Selvarani C
810 131 Selvarani C
*Well coordinates*
811 131 Selvarani C
For a new well, coordinates of the well position are taken on a mobile GPS.
812 131 Selvarani C
Person In charge
813 131 Selvarani C
For communication purpose, contact the person in charge as recorded in directory.
814 131 Selvarani C
815 131 Selvarani C
h2. Automatic- Piezometer (by bala, to be edited) 
816 131 Selvarani C
817 131 Selvarani C
The calibrations and setting are already done by Azha
818 131 Selvarani C
The piezometer is taken to the site of unused well. First the depth and Water level  is checked manually using water meter
819 131 Selvarani C
Then the sensor of the piezometer is inserted into the well 1m above the bottom of the well, so the sensor does not get affected from mud or water inside
820 131 Selvarani C
The sensor sends the data to an electronic board which translates the signal into the proper output, and then it sends the signal to the transmitting device above, on the ground surface. The transmitter then sends it to the receiver at CSR, the signal is then routed to Talam office through Interneet, enters the software and the reading is processed.
821 131 Selvarani C
The piezometer sometimes does not have proper signal and so it would not be able to send it
822 131 Selvarani C
823 132 Selvarani C
h1. Documentation- Flow meter by Bala
824 132 Selvarani C
825 132 Selvarani C
Flow meter – for checking the flow of water in pipe
826 132 Selvarani C
827 132 Selvarani C
CSR bought ultrasonic flow meter from Chennai and the company (company name? )trained Bala and Vijai(CSR) on how to set it up.
828 132 Selvarani C
we tested in the west water system pump in csr. then we test in many more place .
829 132 Selvarani C
 ami, aurodam and buddha garden borewell pumps . we got request from the water service 
830 132 Selvarani C
Cross check they flow meters .
831 132 Selvarani C
 so before that  we want to know how our meter works. 
832 132 Selvarani C
so we tested in our tank . the pump pumping from our sump tank so the flow will be stranded.  when you pump in the bore well  the flow goes up and down. 
833 132 Selvarani C
we tested two times in  15 minutes.and one time 30 minutes,so we got variation between this three. 
834 132 Selvarani C
we find 1,5 % error but the flow meret they said  1 % error only.
835 132 Selvarani C
we cross checked the flow meter of the water service we find some error  in they meter also. 
836 132 Selvarani C
we tried they bore well flowmeter and they sump tank flow meter also.and we find some error also in they meters.  
837 132 Selvarani C
then we find some error in (AVWS) meter also.
838 132 Selvarani C
It is used to check the flow of water in pipe
839 132 Selvarani C
tools used 
840 132 Selvarani C
First the outer diameter of the pipe has to be entered in the device. It is measured using Vernier caliper
841 132 Selvarani C
Then the thickness of the pipe is set which is also measured using vernier
842 132 Selvarani C
Also the device asks for the material of the pipe. If the pipe material is know it can be set and if it not know then there is an option which is other (mostly PVC, HDPE, and  iron) 
843 132 Selvarani C
After entering these details the device gives the spacing for the sensors. There are 2 sensors up and down which has to be fixed accordingly
844 132 Selvarani C
When the sensors are fixed the motor is turned on and the sensors send reading to the display device
845 132 Selvarani C
This is noted down once every minute and taken for 15 to 30 minutes. Then the average of this is determined. This is done because of the variation in the flow. This gives the flow rate in 1 hour.
846 1 Philippe May
This was done in different places to check to flow rate
847 132 Selvarani C
848 177 Selvarani C
849 133 Selvarani C
850 133 Selvarani C
h1. Documentation- DST- Vegetation Indexing
851 133 Selvarani C
852 133 Selvarani C
h2. Steps for Dzetsaka Classification tool for Vegetation indexing in QGIS
853 133 Selvarani C
854 133 Selvarani C
 
855 133 Selvarani C
 1. Install the plugin Dzetsaka classfication tool.
856 133 Selvarani C
 2. Open the Raster from the Survey.
857 133 Selvarani C
 3. Create a polygon shapefile for index sampling. Mark polygons and give the ID's (1,2,3 for Tree, grass, bare land etc) Cover the variations in samples as much as possible. 
858 133 Selvarani C
     More the samples, better the indexing.
859 133 Selvarani C
 4. Apply Dzetsaka Classification tool, Select the base raster and the sample- index polygon shapefile created in step 3.
860 133 Selvarani C
 5. The result is a Raster with DN numbers specified in the Shapefile in step 3.
861 133 Selvarani C
 6. Apply the Sieve raster command (Raster> Analysis> Sieve)- Try different threshold numbers and view the results till the noise is removed from the Raster.
862 133 Selvarani C
 7. Polygonise the Raster to Vector (From processing)
863 133 Selvarani C
 8. Run the v.generalise tool on the shapefile. This tool removes the pixelated boundaries of the polygons in the Vector.
864 133 Selvarani C
865 134 Selvarani C
h1. Documentation- DST- Interpolation (Processing toolbox)
866 134 Selvarani C
867 134 Selvarani C
The following 4 tools have been mostly explored and the results were compared. The ones used for quick analysis are 1. Cubic Spline and 2. V.surf.spline . The rest of the tools are for further exploration and used depending on the need of the project. In some tools, the elevation values of points should be stored in the attribute table (Using field calculator and giving command - *Z($Geometry)* )
868 134 Selvarani C
869 134 Selvarani C
# Interpolate (Cubic spline) - SAGA
870 134 Selvarani C
# V.surf.bspline - GRASS. Parameters to set - cell size. Set this parameter above 0.001 ( 0.00001, 0.000001 etc) and check the results.
871 134 Selvarani C
# V.surf.rst - GRASS
872 134 Selvarani C
# Krigging - SAGA
873 1 Philippe May
874 181 Selvarani C
[[Documentation- DST- Survey- Office workflow]]  - *%{color:RED} to be written%*
875 135 Selvarani C
876 135 Selvarani C
h1. From CAD to GIS by Giulio
877 135 Selvarani C
878 135 Selvarani C
*FEATURES CREATION*
879 135 Selvarani C
1.	Assign a CRS to the drawing (TM-AUSPOS) (MAPCSLIBRARY command)
880 135 Selvarani C
2.	Create features in CAD (Points, lines, polygons)
881 135 Selvarani C
3.	Export shapefile (a) from CAD (Output > DWG to SDF) (Convert to LL84 – 3D)
882 135 Selvarani C
883 135 Selvarani C
*FEATURES IMPORT INTO DB FIRST TIME*
884 135 Selvarani C
4.	Create zip file of the shapefile
885 135 Selvarani C
5.	Upload into the GISAF Shapefiles Basket
886 135 Selvarani C
6.	Import the shapefile into DB
887 135 Selvarani C
7.	Save the shapefile on Local Machine
888 135 Selvarani C
889 135 Selvarani C
*FEATURES IMPORT INTO DB EVERYTIME*
890 135 Selvarani C
8.	Combine the new features to corresponding last shape files (Insert the process here).
891 135 Selvarani C
9.	Follow step 4-8 again
892 135 Selvarani C
893 135 Selvarani C
*FEATURES EDITING IN QGIS*
894 135 Selvarani C
10.	Open the table in QGis
895 135 Selvarani C
11.	Save as a shapefile (b) in TM AUSPOS CRS
896 135 Selvarani C
12.	In CAD, open a new drawing and assign AUSPOS CRS
897 135 Selvarani C
13.	Import the shapefile (b) (MapImport) with Object Data (Data tab > Create Object Data > OK), tick “Import polygons as closed polylines”, then press OK
898 135 Selvarani C
14.	Edit features
899 135 Selvarani C
15.	Change workspace into “Planning and analysis”
900 135 Selvarani C
16.	Export shapefile (a) from CAD (Output > DWG to SDF) with ONLY the id selected (Data Tab > Select Attributes > Object Data > Filename > id) (Convert to LL84 – 3D)
901 135 Selvarani C
902 135 Selvarani C
*FEATURES IMPORT INTO DB*
903 135 Selvarani C
17.	Create zip file of the shapefile
904 135 Selvarani C
18.	Upload into the GISAF Shapefiles Basket
905 135 Selvarani C
19.	Import the shapefile into DB
906 135 Selvarani C
20.	Delete the shapefile from Local Machine
907 135 Selvarani C
908 135 Selvarani C
909 182 Selvarani C
910 1 Philippe May
911 136 Selvarani C
h1. QGIS- Miscellaneous
912 136 Selvarani C
913 136 Selvarani C
*QGIS Introduction:*
914 136 Selvarani C
https://www.birdseyeviewgis.com/blog/2018/2/22/my-favorite-features-of-qgis-30to-date
915 136 Selvarani C
3D visualization of raster DEM- https://www.youtube.com/watch?v=2KrCsbP0kUs
916 136 Selvarani C
917 136 Selvarani C
*Spatial query:* 
918 136 Selvarani C
Spatial Query is selection of features that satisfies a certain condition which relates to other features in a space.
919 136 Selvarani C
Using plugin- Spatial query
920 136 Selvarani C
http://www.geodose.com/2018/03/spatial-query-in-qgis-3.html
921 136 Selvarani C
Labelling:
922 136 Selvarani C
Labelling with more than one field names and in different lines
923 136 Selvarani C
|| '\n' ||
924 136 Selvarani C
925 136 Selvarani C
926 136 Selvarani C
927 136 Selvarani C
*Hierarchy of extensions*
928 136 Selvarani C
929 136 Selvarani C
File levels and their uses.
930 136 Selvarani C
https://nathanw.net/2014/03/22/all-the-q-files/
931 136 Selvarani C
932 136 Selvarani C
+The Project file (.qgs)+
933 136 Selvarani C
It contains: Layer source pointer + Style information + Composers + a whole heap of other stuff
934 136 Selvarani C
935 136 Selvarani C
+The Layer Definition file (.qlr)+
936 136 Selvarani C
It contains: Layer source pointer + Style information
937 136 Selvarani C
938 136 Selvarani C
+The QML file (.qml)+
939 136 Selvarani C
It contains: Style information
940 136 Selvarani C
941 59 Giulio Di Anastasio
h2. Giulio's documentation
942 52 Giulio Di Anastasio
943 183 Selvarani C
944 137 Selvarani C
945 137 Selvarani C
h1. Documentation - Reconciliation of points using Gisaf
946 137 Selvarani C
947 137 Selvarani C
Reconciliation of points is a procedure used when a point is stored in a wrong table, because its category was either wrongly recorded in the field by the surveyor, or it has ben reviewed later by the surveyor or the data validator and found to be wrong.
948 137 Selvarani C
949 137 Selvarani C
h2. Definition: Raw points are all points coming from the field survey. Raw points can be points referring to a Point feature (e.g. trees, or elevation points, or floor level), or points measured in the field to draw a line (e.g. the vertices of a fence) or a polygon (e.g. the corners of a building outline).
950 137 Selvarani C
951 137 Selvarani C
In the overall workflow, *+reconciliation affects only raw points+*. It takes place after the field textfile is uploaded into the basket and its points imported (raw points stored in the raw survey tables). Here you can visualize the Workflow diagram: [[Survey_data]].
952 137 Selvarani C
953 137 Selvarani C
954 137 Selvarani C
If a raw point refers to a point feature, reconciling it means moving it to another category/table meant for point features, not for lines/polygons. So, *+only raw points of point features can be reconciled+*. Raw points pertaining to line features and/or polygon features cannot be reconciled, so these raw points will remain in their original wrong table.
955 137 Selvarani C
956 137 Selvarani C
957 137 Selvarani C
PLEASE NOTE: 
958 137 Selvarani C
The attributes of a line/polygon, which are derived from their defining raw points, will not be modified by any reconciliation, because *+only raw points of point features can be reconciled+*.
959 137 Selvarani C
960 137 Selvarani C
h2. How to perform Reconciliation
961 137 Selvarani C
962 137 Selvarani C
To perform reconciliation of points: Login to Gisaf -> click on the G icon on the upper-left corner of the website page -> Manage -> Reconciliation by orig.ID
963 137 Selvarani C
964 137 Selvarani C
On the right end side of the screen, clicking on the field "Destination" a list of all categories will appear: these categories refer not to the Raw survey tables, but to the V_ tables of the database (points, lines, polygons).
965 137 Selvarani C
966 137 Selvarani C
Under it, in the field "Original ID", the original point number of the point to be reconciled is to be entered.
967 137 Selvarani C
968 137 Selvarani C
Clicking on the "Search points" button, the result shows the database unique id of the point, its survey category, its survey date, its geometry type (point, line, polygon), and the Project the point belongs to. In case of multiple points with the same original id (in case of different Projects, the field number of points might be not unique if the numbering of points in the field has restarted) all points having that original id are displayed: thanks to their date or Project or type, it is easy to identify the correct point to be reconciled.
969 137 Selvarani C
970 137 Selvarani C
Once the point to be reconciled is identified, clicking on the button "Reconcile" will run the reconciliation, and a message will appear stating that it has been done successfully. An error message can appear if a reconciliation of a raw point of a line/polygon feature has been attempted: this type of points cannot in fact be reconciled.
971 137 Selvarani C
972 137 Selvarani C
Once a raw point has been reconciled, *+it cannot be reconciled again+*. In case a raw point has been wrongly reconciled, it cannot be reconciled again through the above procedure, but it has to be reconciled manually through QGis or pgadmin software.
973 137 Selvarani C
974 138 Selvarani C
h1. Documentation - Status and Status Changes
975 138 Selvarani C
976 138 Selvarani C
Status have been created to keep track og changes in surveyed features.
977 138 Selvarani C
It is an additional (though provided for in AIA standards), single digit value, at the end of the Gisaf Category/Cad layer name.
978 138 Selvarani C
Each Status need to have a corresponding CAD layer/Gisaf Category, with a short code associatedto it, so that field entries can be done easily.
979 138 Selvarani C
980 138 Selvarani C
Status have been defined as follows:
981 138 Selvarani C
N - New Work
982 138 Selvarani C
E - Existing o remain
983 138 Selvarani C
D - Existing to demolish, Demolished or Changed
984 138 Selvarani C
F- Future work, Proposed feature
985 138 Selvarani C
T - Temporary work
986 138 Selvarani C
M - Item to be moved
987 138 Selvarani C
X - Not in contract
988 138 Selvarani C
989 138 Selvarani C
990 138 Selvarani C
By default, Status is defined as E (Existing). Status can anyhow be changed later using gAdmin, or QGis (through the PostGis connection).
991 138 Selvarani C
It needs to be done manually, one feature (point, line, polygon) at a time.
992 138 Selvarani C
993 138 Selvarani C
994 1 Philippe May
In the future Status changes might be incorporated in the Admin panel.
995 138 Selvarani C
996 185 Selvarani C
997 67 Giulio Di Anastasio
998 139 Selvarani C
h1. Documentation - Tags retained after re-import of same geometry
999 139 Selvarani C
1000 139 Selvarani C
Documentation - Tags retained after re-import of same geometry
1001 139 Selvarani C
The linework for infrastructure survey carried out by Eric Chacra in May 2020 was originally imported with a problem of ambiguity in the "Accuracy" table and in the "Accuracy" table.
1002 139 Selvarani C
The result was that lines did not inherit the attributes survey date, accuracy, equipment, surveyor.
1003 139 Selvarani C
Nevertheless lines were displayed on the Gisaf map, without these attributes, and tags were given to some of these lines.
1004 139 Selvarani C
1005 139 Selvarani C
The values for the two tables ("Accuracy", "Accuracy") have been corrected, ambiguity resolved.
1006 139 Selvarani C
The lines in the layers V-ELEC-UGND------E and V-COMM-CABL------E have been then reimported, the attributes have been properly assigned, and the tags have been retained.
1007 139 Selvarani C
1008 139 Selvarani C
4 August 2020
1009 139 Selvarani C
1010 67 Giulio Di Anastasio
h1. Access to data
1011 67 Giulio Di Anastasio
1012 67 Giulio Di Anastasio
h2. Connection to server directly from CSR
1013 67 Giulio Di Anastasio
1014 67 Giulio Di Anastasio
To connect to the server directly without going through Aurinoco server, the correct url is
1015 67 Giulio Di Anastasio
http://gis.csr.av
1016 67 Giulio Di Anastasio
1017 67 Giulio Di Anastasio
h2. Connection to Gisaf via QGis through WFS / OGC API
1018 67 Giulio Di Anastasio
1019 67 Giulio Di Anastasio
This works only on QGis from version 3.14.15 onward
1020 67 Giulio Di Anastasio
1021 67 Giulio Di Anastasio
In the browser, click on WFS/OGC API, then right-click to create a new connection
1022 67 Giulio Di Anastasio
Give a name (e.g. OGC API Qgis Gisaf)
1023 67 Giulio Di Anastasio
Give the url https://gis.auroville.org.in/ogcapi
1024 67 Giulio Di Anastasio
1025 67 Giulio Di Anastasio
Under the WFS Options box, on Version dropdown, the default option "Maximum" works just fine
1026 67 Giulio Di Anastasio
Click on OK
1027 67 Giulio Di Anastasio
The list of layers will appear in the Browser under WFS/OGC API.
1028 68 Giulio Di Anastasio
1029 68 Giulio Di Anastasio
1030 68 Giulio Di Anastasio
h1. How to create a new projection in QGis
1031 68 Giulio Di Anastasio
1032 68 Giulio Di Anastasio
To create a new projection in QGis, go to menu "Settings", and click on "Custom Projections".
1033 68 Giulio Di Anastasio
A pop-up window appears with a list of all projections defined in QGis projects used by the user so far.
1034 68 Giulio Di Anastasio
Click on the green "+" sign on the right top part of the window to create a new projection.
1035 68 Giulio Di Anastasio
In the "Name" box, type "TM CSRAUSPOS SF1" (which means TM = Transverse Mercator projection; CSRAUSPOS = theparameters for this projection are derived from the processing of DGPS raw data by AUSPOS -  Online GPS Processing Service - https://www.ga.gov.au/scientific-topics/positioning-navigation/geodesy/auspos; SF1 = Scale Factor is 1).
1036 68 Giulio Di Anastasio
In the "Format" dropdown list, select "Proj String (legacy - Not Recommended)"
1037 68 Giulio Di Anastasio
In the "Parameters" box, paste the following "+proj=tmerc +lat_0=12.01605433+lon_0=79.80998934 +k=1 +x_0=370455.630 +y_0=1328608.994 +ellps=WGS84+towgs84=0,0,0,0,0,0,0 +units=m +no_defs".
1038 68 Giulio Di Anastasio
1039 68 Giulio Di Anastasio
Finally, click on OK.
1040 68 Giulio Di Anastasio
1041 68 Giulio Di Anastasio
In a more explicit  way, the parameters mean the following:
1042 68 Giulio Di Anastasio
Map Projection:        TransverseMercator (TM)
1043 68 Giulio Di Anastasio
False Easting:          370455.6300
1044 68 Giulio Di Anastasio
False Northing:        1328608.9940
1045 68 Giulio Di Anastasio
Latitude of Origin:    12°00'57.79560" (DMS)   12.01605433 (DD)
1046 68 Giulio Di Anastasio
Central Meridian:     79°48'35.96164"  (DMS)   79.80998934 (DD)
1047 68 Giulio Di Anastasio
Scale Factor:             1.00000000
1048 68 Giulio Di Anastasio
Zone Width:              6.0°
1049 69 Giulio Di Anastasio
1050 69 Giulio Di Anastasio
h1. Elimination of Duplicate points – General criteria
1051 69 Giulio Di Anastasio
1052 69 Giulio Di Anastasio
It might happen that the same physical feature (e.g. a tree, or a pole) is surveyed more than once: this can happen because there are many physical features in an area, and the survey needs more than one station. So, for example a tree is surveyed from a station, and gets a serial number on that date. When the station is then changed, it might happen that the same tree is resurveyed: another serial number is given, and possibly a different date, if the survey from the second station happened on a different day.
1053 69 Giulio Di Anastasio
It is clear that the same tree is then represented with two different points, which means that two different trees exist: but only one tree really exist in the physical reality.
1054 69 Giulio Di Anastasio
It is clear that one of the two points is redundant and needs to be removed. If this is noted by the surveyor directly in the field, then the issue is solved by the surveyor himself during processing time.
1055 69 Giulio Di Anastasio
If instead, due to various reasons, it was not noted by the surveyor in the field, it will need to be cleaned after the processing, possibly by post-processing staff.
1056 69 Giulio Di Anastasio
How to identify duplicate points?
1057 69 Giulio Di Anastasio
The following criteria can be used:
1058 69 Giulio Di Anastasio
1.	The distance between the two points is less than 30 cm (trees are surveyed if their trunk diameter is at least about 20 cm, so in 30 cm cannot exist two of them)
1059 69 Giulio Di Anastasio
2.	The orig_id (serial number) of the points are not in series
1060 69 Giulio Di Anastasio
3.	The survey date is not the same
1061 69 Giulio Di Anastasio
4.	In case of trees, the species of trees is the same
1062 69 Giulio Di Anastasio
5.	5. In case of trees, the tree type is not TDEF (because TDEF are mapped irrespective of their diameter, so they can actually have a small trunk, and two of them might exist in 30 cm), not OT (many TDEF species are surveyed as OT if not otherwise indicated by a botanist)
1063 69 Giulio Di Anastasio
6.	The context needs to be evaluated: if one tree is deleted in an area where many trees exist in a limited space, then loosing one in the map is not a big error. If instead one tree is deleted where there are very few trees, then it might be a big loss.
1064 70 Giulio Di Anastasio
1065 70 Giulio Di Anastasio
1066 70 Giulio Di Anastasio
h1. Linework for the Survey Area
1067 70 Giulio Di Anastasio
1068 70 Giulio Di Anastasio
h2. 1.	Creation of Initial Linework in QGIS using Survey points import - (Ram, System 4)
1069 70 Giulio Di Anastasio
1070 70 Giulio Di Anastasio
Initial Linework in QGIS is started by surveyor with the knowledge from the Field. For this step, points are simply imported into the QGIS from the field text file (.csv or .txt). CRS needs to be TM-AUSPOS. The box of “First record has field names” shall not be ticked. In Point Coordinates, select the correct field for x, for y and for z (usually “field_2” for x, “field_3” for y and “field_4” for z). Points can be styled using the “Categorized” style in “Symbology”, using “Field_5” as value, or using a Rule-based symbology using the category (field 5) as filter.
1071 74 Selvarani C
Linework is created by connecting points having same description and belonging to the same physical feature. *All line and polygon features are created as lines*.
1072 70 Giulio Di Anastasio
The Initial Linework for the Survey Area is also stored temporarily in 
1073 70 Giulio Di Anastasio
1074 74 Selvarani C
+D: > AVSM > Zone-Survey number (eg RZ-01) > Survey Area (eg J) > Temporary WD+
1075 70 Giulio Di Anastasio
1076 70 Giulio Di Anastasio
h2. Note: The line shapefiles / Geopackages shall be in CRS: TM AUSPOS
1077 70 Giulio Di Anastasio
1078 70 Giulio Di Anastasio
h2. 2.	Creation of final working drawing Shapefiles / Geopackages - (Selvarani, System 1)
1079 1 Philippe May
1080 1 Philippe May
Final working drawing Shapefiles / Geopackages are created from the Initial Linework of Survey Area.
1081 70 Giulio Di Anastasio
As the Surveyor draws all features as lines (both for lines and polygons features), the following actions shall be done:
1082 74 Selvarani C
1.	*If features are lines:*
1083 70 Giulio Di Anastasio
•	Export the shapefile / geopackage into the final working drawing folder (Final WD), in separate folders according to its type (e.g. BLDG, FENC, ROAD, etc).
1084 70 Giulio Di Anastasio
1085 70 Giulio Di Anastasio
h2. The CRS for the export shall be EPSG:4326 - WGS 84
1086 70 Giulio Di Anastasio
1087 74 Selvarani C
2.	*If features are polygons:*
1088 1 Philippe May
•	Lines shall be converted into polygons:
1089 74 Selvarani C
1090 1 Philippe May
to do it, first click on the layer to be converted to make it active (e.g. WD-CZ-01-F-LL84_V-BLDG-MHOL------E), then go to “Vector” Menu, click on Geometry Tools, click on Line to Polygons:
1091 87 Selvarani C
1092 84 Selvarani C
!https://redmine.auroville.org.in/attachments/download/9760/Line%20to%20Polygon%20Menu.png!
1093 70 Giulio Di Anastasio
1094 86 Selvarani C
1095 1 Philippe May
The new window for “Lines to Polygons” conversion will appear:
1096 86 Selvarani C
1097 86 Selvarani C
1098 83 Selvarani C
!https://redmine.auroville.org.in/attachments/download/9762/Lines%20to%20Polygon%20Window.png!
1099 1 Philippe May
1100 71 Giulio Di Anastasio
•	Always cross check the input layer, to make sure that the input layer is the active one
1101 71 Giulio Di Anastasio
•	Save the output in a temporary layer
1102 74 Selvarani C
•	The temporary layer will be listed in the list of layers, it shall be exported to the saving location as +D: > Survey > Zone-Survey Number > Final WD > Survey Area SHP+ (eg . D: > Survey > GB-01 > Final WD > A-Shp)
1103 72 Giulio Di Anastasio
1104 74 Selvarani C
h2. The CRS for the export shall be EPSG:4326 - WGS 84
1105 74 Selvarani C
1106 72 Giulio Di Anastasio
Once all the shapefiles / geopackages are exported in Final WD, for each of the newly exported layers the Topology Checker Tool shall be used.
1107 1 Philippe May
1108 74 Selvarani C
h2. Linework for the whole Survey Zone
1109 74 Selvarani C
1110 74 Selvarani C
h2. 1.	Merging Shapefiles / Geopackages - (Selvarani, System 1)
1111 74 Selvarani C
1112 72 Giulio Di Anastasio
A copy of the Zone Master shapefiles / geopackages are taken from System 4 and stored in Temp Folder on Desktop in System 1.
1113 1 Philippe May
Master shapefiles / geopackages are merged with the Survey Area shapefiles / geopackages:
1114 72 Giulio Di Anastasio
•	To do it, go to “Vector” Menu, click on Geoprocessing Tools, then click on Union:
1115 72 Giulio Di Anastasio
1116 85 Selvarani C
!https://redmine.auroville.org.in/attachments/download/9763/Union%20Menu.png!
1117 1 Philippe May
1118 1 Philippe May
1119 85 Selvarani C
The new window for “Union” will appear:
1120 1 Philippe May
1121 86 Selvarani C
1122 86 Selvarani C
!https://redmine.auroville.org.in/attachments/download/9764/Union%20Window.png!
1123 73 Giulio Di Anastasio
1124 74 Selvarani C
•	To make sure that the right geometry is generated by this process (“line” type, not “Multiline”, and similarly “Polygon” type, not “Multipolygon), we need to always keep the *Master shapefile* (e.g. Final-CZ-01-2021-02-05-LL84_V-BLDG-MHOL------E) *as Input layer*, and the Survey Area shapefile as Overlay Layer (e.g. WD-CZ-01-F-LL84_V-BLDG-MHOL------E).
1125 73 Giulio Di Anastasio
•	(The output can be saved to a file, as the CRS should already be EPSG4326 – WGS84.)
1126 73 Giulio Di Anastasio
1127 74 Selvarani C
h2. 2.	Storing Shapefiles / Geopackages - (Selvarani, System 1)
1128 74 Selvarani C
1129 74 Selvarani C
Save the merged shape file in the correct location in Final folder as +D: > Survey > Zone-Survey Number > Final+ (eg . D: > Survey > GB-01 > Final)
1130 73 Giulio Di Anastasio
Date in the name of Final Shapefile / Geopackage needs to be updated.
1131 73 Giulio Di Anastasio
Once the merging operation is completed, the copy of Master shapefile / geopackage is deleted from the Temp folder.
1132 73 Giulio Di Anastasio
1133 74 Selvarani C
h2. 3.	Topology check of merged shapefiles
1134 74 Selvarani C
1135 73 Giulio Di Anastasio
The topology checker is applied again on the merged shapefiles / geopackages.
1136 73 Giulio Di Anastasio
The “id_field” shall be removed from the attribute table.
1137 73 Giulio Di Anastasio
1138 74 Selvarani C
h2. 4.	Archive and replace the Master Shapefiles / Geopackages (Ram, System 4)
1139 74 Selvarani C
1140 73 Giulio Di Anastasio
Archive the previous master shapefiles / geopackages on system 4, and copy the new merged shapefiles / geopackages in its place.
1141 74 Selvarani C
*Then delete the Merged Shapefile / Geopackage folder from System 1.
1142 74 Selvarani C
*
1143 73 Giulio Di Anastasio
1144 74 Selvarani C
h2. 5.	Note about Shapefiles and Geopackages
1145 73 Giulio Di Anastasio
1146 73 Giulio Di Anastasio
All the above works are usually done using shapefile format, in QGIS latest version (3.16.3).
1147 221 Selvarani C
The Geopackage export is done in QGis versions older than 3.12 (e.g. 3.4, 3.6, 3.8, 3.10) so that the lines are not saved as “Multilines” but as “Lines”and polygons are not saved as “Multipolygons” but as “Polygons”. This is very important to be noted, as Gisaf database does not accept the Multipolygon and Multiline geometry types.
1148 221 Selvarani C
A different way to create "Polygons" is to use the command Vector -> Geometry Tools -> Multipart to Single Parts and apply it to the layer: from "Multipolygon" it will become "Polygon" (check in layer Properties).
1149 222 Selvarani C
As on 13 March 2021, Gisaf can accept Multipolygon layers, because the command "Multipart to Single parts" has been integrated into the importing command (see Redmine ticket #11691)
1150 223 Selvarani C
1151 223 Selvarani C
h1. Creating 3D Shapefile/Geopackage
1152 223 Selvarani C
1153 223 Selvarani C
While creating a Shapefile/Geopackage, the File name/Database-table name, Geometry type and CRS have to be entered.
1154 223 Selvarani C
In order to create a 3D Shapefile/Geopackage, the additional dimensions *"Z(+M values)"/ "Include Z dimension"* has to be ticked: this way the 3D Shapefile/Geopackage is accepted by Gisaf without errors, otherwise the Shapefile/Geopackage can't be imported in Gisaf because the Z dimension is missing (the geometries in the database are all 3D).
1155 224 Giulio Di Anastasio
1156 224 Giulio Di Anastasio
h1. Exporting from QGis (shapefiles and/or geopackages) to CAD dxf format
1157 224 Giulio Di Anastasio
1158 226 Giulio Di Anastasio
An algorithm has been created by Selvarani, to see the whole process click here:
1159 226 Giulio Di Anastasio
https://redmine.auroville.org.in/projects/gis/wiki/Automatic_export_from_QGis_to_dxf