47 lines
2 KiB
Markdown
47 lines
2 KiB
Markdown
|
# Geo Processing Tools — Cortex
|
||
|
|
|
||
|
|
## Overview
|
||
|
|
|
||
|
|
DEM processing, vector tile generation, and PMTiles manipulation tools installed on cortex for compute-tier geospatial batch jobs. Cortex is compute-only — no persistent operational data lives here. Data flows: pi-nas (archive) → cortex (compute) → pi-nas + VM 1130 (outputs).
|
||
|
|
|
||
|
|
## Installed Tools
|
||
|
|
|
||
|
|
| Tool | Version | Package | Purpose |
|
||
|
|
|------|---------|---------|---------|
|
||
|
|
| gdal_contour | GDAL 3.8.4 | gdal-bin (apt) | Generate contour lines from DEM rasters |
|
||
|
|
| gdalinfo | GDAL 3.8.4 | gdal-bin (apt) | Raster metadata inspection |
|
||
|
|
| gdal_translate | GDAL 3.8.4 | gdal-bin (apt) | Raster format conversion |
|
||
|
|
| gdalbuildvrt | GDAL 3.8.4 | gdal-bin (apt) | Virtual raster mosaics |
|
||
|
|
| ogr2ogr | GDAL 3.8.4 | gdal-bin (apt) | Vector format conversion |
|
||
|
|
| tippecanoe | v2.49.0 | tippecanoe (apt) | GeoJSON → vector tiles (MBTiles/PMTiles) |
|
||
|
|
| pmtiles | v1.30.2 | go-pmtiles (binary) | PMTiles inspection, conversion, tile extraction |
|
||
|
|
|
||
|
|
Installed: 2026-04-22
|
||
|
|
|
||
|
|
## Ephemeral Scratch Convention
|
||
|
|
|
||
|
|
All temporary/staging data goes in `/home/zvx/scratch/`. This directory is cleaned after batch jobs complete. Never treat scratch as persistent storage.
|
||
|
|
|
||
|
|
Typical layout:
|
||
|
|
```
|
||
|
|
/home/zvx/scratch/
|
||
|
|
└── nav-processing/ # Current: staged DEM for contour generation
|
||
|
|
└── hillshade-na.pmtiles # 93 GB terrarium DEM (from pi-nas archive)
|
||
|
|
```
|
||
|
|
|
||
|
|
## Terrarium DEM Decode
|
||
|
|
|
||
|
|
The Mapzen Terrarium format encodes elevation in RGB PNG tiles:
|
||
|
|
```
|
||
|
|
elevation_meters = (R * 256 + G + B/256) - 32768
|
||
|
|
```
|
||
|
|
|
||
|
|
To use with GDAL: decode RGB → Float32 elevation array → write as GeoTIFF via VRT with `subClass="VRTRawRasterBand"`.
|
||
|
|
|
||
|
|
## Data Tier Architecture
|
||
|
|
|
||
|
|
| Tier | Host | Role | Example |
|
||
|
|
|------|------|------|---------|
|
||
|
|
| Archive | pi-nas (192.168.1.245) | Source of truth, recovery | `/export/data/nav/hillshade-na.pmtiles` |
|
||
|
|
| Compute | cortex (192.168.1.150) | Batch processing, scratch | `/home/zvx/scratch/nav-processing/` |
|
||
|
|
| Operational | VM 1130 (192.168.1.130) | Serving tiles via nginx | `/mnt/nav/tiles/` |
|