Backend / Geospatial data pipeline

MonLogement — French Administrative Boundaries to Mapbox Vector Tiles

A Laravel job pipeline that turns the official French administrative dataset into regions, departments and communes in the database, and publishes their borders as Mapbox vector tiles.

Role: Backend Engineer

Overview

MonLogement lets users search listings by region, department and commune and see those areas as borders on a map. I built the pipeline that extracts the official French boundary dataset, converts and cleans it with GDAL, stores each area with its bounding box, and pushes the shapes to Mapbox as a tileset so borders render as fast vector tiles instead of heavy GeoJSON.

Tech Stack

backend

PHPLaravelLaravel Queues

geospatial

GDAL (ogr2ogr, ogrinfo)GeoPackageGeoJSON / GeoJSONSeqEPSG:2154 to EPSG:4326

maps

Mapbox Tiling Service (Tilesets API)Mapbox vector tilesMapbox GL JS

data

MySQLCSV import

source

IGN Admin Express (official French administrative boundaries)

Key Numbers

3 (regions, departments, communes)

Administrative levels published

EPSG:2154 to EPSG:4326

Coordinate system

6 decimals

Coordinate precision

The Problem

Area search and area borders need authoritative geometry for roughly 35,000 communes plus departments and regions. The official source is a large national GeoPackage in the Lambert-93 projection, while Mapbox expects WGS84 lon/lat. Serving the raw polygons to the browser as GeoJSON would be far too heavy, and the work is too long-running to do inside a web request.

  • Official data ships as one large GeoPackage with many layers in EPSG:2154 (Lambert-93), not the EPSG:4326 that web maps use
  • Commune polygons are too large to send to the browser as plain GeoJSON
  • The app needs each area's name, code, level and bounding box for search and for fitting the map view
  • Mapbox tileset sources expect line-delimited GeoJSON, not a single FeatureCollection
  • Conversion, upload and tile publishing are long-running and must be retryable without blocking the app
  • Credentials for the Mapbox API must stay out of code and logs

The Solution

  1. 1Treat the dataset as an ETL pipeline of small queued Laravel jobs, each wrapping one GDAL or HTTP step, so every stage can fail, retry and be re-run on its own.
  2. 2Reproject once at extraction (EPSG:2154 to EPSG:4326, XY only, 6-decimal coordinate precision) and keep everything downstream in WGS84.
  3. 3Split the output by purpose: attributes and bounding boxes go to the database for search; shapes go to Mapbox as vector tiles for rendering.

Layer extraction from the official GeoPackage

ogr2ogr loops over the layers of the IGN Admin Express GeoPackage (region, departement, commune, epci, arrondissement, canton and their chef-lieu point layers) and writes one GeoJSON per layer, reprojected to EPSG:4326 with a fixed coordinate precision to shrink file size.

Layer inspection before processing

ogrinfo summary mode is used to check feature counts, fields and geometry types of each file before the import jobs rely on them.

Bounding boxes in the database

An ogr2ogr SQLite-dialect query exports name, code, level and ST_MinX / ST_MinY / ST_MaxX / ST_MaxY per area to CSV, which the importer loads so the map can fit to an area and search can filter spatially without reading geometries.

Line-delimited GeoJSON for tiling

Each level (regions, departments, communes) is converted to GeoJSONSeq, one feature per line, which is the format a Mapbox Tiling Service tileset source accepts.

Tileset source upload

The line-delimited file is uploaded as a multipart request to the Mapbox Tilesets API (sources endpoint) from a Laravel job, one source per level.

Vector tile borders on the map

Published tilesets are drawn as line and fill layers in Mapbox GL, so region, department and commune borders stream as tiles at the zoom levels the user is viewing.

Architecture

Sequential queued job pipeline wrapping GDAL command-line tools and the Mapbox Tilesets API

Boundary refresh pipeline
  1. 1Download the new Admin Express GeoPackage
  2. 2Extract each layer to GeoJSON in EPSG:4326
  3. 3Inspect layers with ogrinfo
  4. 4Export name, code, level and bounding box to CSV
  5. 5Import areas and bounding boxes into the database
  6. 6Convert each level to line-delimited GeoJSON
  7. 7Upload each file as a Mapbox tileset source
  8. 8Publish the tileset and use it as a vector source in Mapbox GL

Components

Source data

IGN Admin Express, GeoPackage, Lambert-93

Official edition dated 2025-09-16, containing administrative layers and their chef-lieu points.

Extraction job

Laravel job, ogr2ogr (GDAL)

Iterates the layer list and writes one reprojected GeoJSON per layer.

Inspection and bbox job

ogrinfo, ogr2ogr SQLite dialect

Verifies each file and exports name, code, level and bounding box to CSV.

Import job

Laravel queue, MySQL

Loads areas and bounding boxes into the area tables used by search and map fitting.

Conversion job

ogr2ogr GeoJSONSeq

Produces line-delimited GeoJSON for each level.

Publish job

Mapbox Tilesets API (Tiling Service)

Uploads each level as a tileset source over HTTP so Mapbox can build vector tiles from it.

Map rendering

Mapbox GL JS

Vector tile source with line layers for borders and highlight on the selected area.

  • Database holds attributes and bounding boxes for search; Mapbox holds geometry for rendering. The app never loads polygons just to draw a border.
  • GDAL steps run as external processes behind jobs, so a conversion failure is a failed job with a log, not a failed web request.

Queues

default

Key Decisions

Vector tiles instead of serving GeoJSON to the browser

Why: Commune borders for the whole country are far too large as GeoJSON. Tiles load only what the current view needs and simplify automatically by zoom level.

Trade-off: Depends on an external tiling service and a publish step whenever the boundaries change.

Reproject once with ogr2ogr at extraction

Why: Every later stage works in EPSG:4326, so the database, the CSV and the tiles all agree and no runtime reprojection is needed.

Trade-off: Source geometry in Lambert-93 is not kept in the app; re-extract from the GeoPackage if it is ever needed.

Fixed coordinate precision (6 decimals)

Why: About 0.1 m of precision is more than a border needs and keeps files and tiles much smaller.

Trade-off: Not suitable for survey-grade use.

Store bounding boxes in the database

Why: Fit-to-area and spatial filtering use four numbers per area instead of loading a polygon.

Trade-off: Boxes must be regenerated with each dataset refresh.

Small queued jobs per step

Why: Each GDAL or HTTP call can be retried or re-run on its own, and the long work stays off the web request path.

Trade-off: More job classes and a need to keep stage order explicit.

Technical Challenges

National dataset in a projected coordinate system

Solution: Reproject to EPSG:4326 and drop Z/M dimensions at extraction, with a fixed coordinate precision.

Outcome: Files load directly into Mapbox and the database without further conversion.

Tileset source rejects a single large FeatureCollection

Solution: Convert each level to line-delimited GeoJSON (GeoJSONSeq) before upload.

Outcome: Large communes layer uploads as a streamable file of one feature per line.

Getting search and map fitting without reading geometry

Solution: Compute min/max X and Y per area with an SQLite-dialect query in ogr2ogr and export to CSV.

Outcome: Bounding boxes load into the database with name, code and level.

Secrets in command lines

Solution: Mapbox secret token read from environment configuration by the job, never written in code, logs or the repository.

Outcome: Tokens can be rotated without code changes.

Engineering Practices

  • One job per pipeline step, each wrapping a single GDAL command or API call
  • Reproducible layer extraction driven by an explicit list of layers
  • Inspection step with ogrinfo before data is imported
  • Secrets supplied through environment configuration
  • Dataset edition recorded in the source file name for traceability

Next Steps

  • Automate dataset refresh when IGN publishes a new Admin Express edition
  • Add tileset recipe publishing and job status polling to the pipeline
  • Record feature counts per run and alert on unexpected drops