Storing map tiles in a single file is a common way to load basemaps on a map client.
There are a few formats available to do this, with different use cases.
Offline
mbtiles
- A format innovated by Mapbox, but is a fully open spec.
- Essentially an SQLite database linking to embedded tiled images.
- The client interfaces with the database and loads each tile as required by the basemap.
OSMAnd SQLite
- Based on BigPlanet SQLite format.
- Basically the same as mbtiles, but a slightly different database schema.
A small aside.
Sometimes it’s necessary to generate both mbtiles and OsmAnd format to view in different software, which is a pain.
There is an open issue in OsmAnd to support mbtiles format, but it’s not a priority for now.
Knowing that they are very similar file formats, I considered the possibility of accessing one SQLite database via another ‘wrapper’ SQLite database in a custom view. This view would map tables and fields from one database schema to the other, eliminating the need to store both tilesets for the same data.
Assuming you have an MBTiles table with the following schema: CREATE TABLE mbtiles_table ( zoom_level INTEGER, tile_column INTEGER, tile_row INTEGER, tile_data BLOB );
And you want to create a view for an OsmAnd SQLite table with a schema like: CREATE TABLE osmand_table ( _id INTEGER PRIMARY KEY AUTOINCREMENT, x INTEGER, y INTEGER, z INTEGER, tile_data BLOB );
You can create a view to convert between them like this: CREATE VIEW osmand_mbtiles_view AS SELECT NULL AS _id, -- Use NULL for auto-increment _id tile_column AS x, tile_row AS y, zoom_level AS z, tile_data FROM mbtiles_table;
I couldn’t get this to work when testing, however (it may warrant further investigation).
If you find a solution, please do let me know!
Online
PMTiles
- A neat new format specifically aimed at cloud-optimising vector tile access (accessing an mbtile file over the web is very inefficient, full details).
- Easily handles both large planet-scale datasets with millions of tiles and small-scale datasets. As a single file it is perfect for S3 object storage.
- Uses HTTP RANGE requests to only download the tiles specified in a BBOX (not the entire file).
- Compression, tile deduplication (no need to repeat that blue ocean tile…), an optimised internal structure to minimise size and number of requests when panning or zooming, and minimal overhead when requesting tiles (tiny initial request).
- For public deployments it is recommended to run behind a CDN to both cache tile requests, and act as a proxy to a private S3 bucket (anonymous direct file download from S3 may incur large costs).
So What Should I Choose?
If the layer (likely a basemap) needs to work offline, then:
- SQLite if the tool/app supports it, e.g. OSMAnd.
- mbtiles for tools that require it, such as ODKCollect.
If working online, then PMTiles may be best:
- A replacement for XYZ basemap tile servers (great for reducing load on the OpenStreetMap servers 🙏).
- Creating custom basemaps from imagery and/or OSM exports.
Converting Between Formats
mbtiles –> OSMAnd
- This is relatively easy due to both being SQLite files at core.
- The excellent Python utility by @tarwirdur mbtiles2osmand does this quite efficiently.
python mbtiles2osmand.py INPUT.mbtiles OUTPUT.sqlite3
- I also ported this to Golang mb2osm, but have some work to do on improving performance. Feel free to contribute!
The advantage of using Golang here is to produce a statically compiled binary. This means that the single file does not require any external dependencies, or interpreter to run (unlike Python), making it more portable.
mbtiles –> pmtiles
- The best choice for this would be go-pmtiles, by the creator of PMTiles.
- Again, a single file binary program that can convert in one command.
pmtiles convert INPUT.mbtiles OUTPUT.pmtiles
Other formats –> pmtiles
- In cases where you have other formats to convert first, e.g. directly from a database, GeoJSON, etc, tippecanoe (>v2.17) is recommended tool.
- The official example:
ogr2ogr -t_srs EPSG:4326 cb_2018_us_zcta510_500k.json cb_2018_us_zcta510_500k.shp
Creates a layer in the vector tiles named “zcta”
tippecanoe -zg --projection=EPSG:4326 -o cb_2018_us_zcta510_500k_nolimit.pmtiles -l zcta cb_2018_us_zcta510_500k.json