OS Maps app tools

Extracting and converting maps from the OS Maps app.

Free the maps you've bought from the OS Maps app

Summary

• Paper Ordnance Survey maps come with a code that allows you to add the map to your OS Account.
• In the OS Maps app you can download these maps without the need for a subscription.
• This tool allows you to extract these maps from the OS Maps app and work with them in MBTiles format.
• This allows you to use the digital OS Maps that you've bought with other mapping apps (such as AlpineQuest, Cartograph Maps, Locus Map, etc.).

⚠️ Disclaimer

The tools and techniques described here should only be used in accordance with the Ordance Survey terms of use and UK law. Be aware that these terms of service may explicitly prohibit the extraction of maps from the app. Don't do anything illegal with the knowledge you learn here; if you do that's entirely your fault.

Please don't share any OS Maps - purchasing their excellent maps funds the work put into making them.

Directions for use

Run with the --help flag for the command line help text:

> python .\ostools.py --help
usage: ostools.py [-h] [-verbose] command ...

Tools for working with maps from the OS Maps app.

positional arguments:
  command
    extract   Extract maps from mbgl-offline.db
    convert   Convert MBTiles files from png to webp
    dedupe    Dedupe tiles from overlapping MBTiles files

optional arguments:
  -h, --help  show this help message and exit
  -verbose    run in verbose mode

Requirements

You should not need to install additional packages to extract or dedupe, but the convert command requires Pillow and other undocumented commands require requests. You should be able to use the requirements.txt file to install these; i.e. pip install -r ./requirements.txt.

Extracting maps (extract)

You'll need a rooted android device to extract. If you don't have a rooted physical device then you can use the emulator built into the Android Studio SDK with a system image that supports elevated privileges.

1. Within the OS Maps app, download the maps that you want to extract

2. From the /data/data/uk.co.ordnancesurvey.osmaps/ directory on the rooted android device copy files/.mapbox/map_data.db and databases/customOfflineMaps.db to your computer. You can do this using ADB via your terminal:

   adb shell
   su
   cp -R /data/data/uk.co.ordnancesurvey.osmaps/files/.mapbox/map_data.db /sdcard/
   cp -R /data/data/uk.co.ordnancesurvey.osmaps/databases/customOfflineMaps.db /sdcard/
   exit
   exit
   adb pull /sdcard/map_data.db
   adb pull /sdcard/customOfflineMaps.db

3. Run python ostools.py extract

Each region downloaded in the OS Maps app will be extracted into its own .mbtiles file.

You can adjust the extraction process using the -tiledb, -infodb, -regions, and -zoom flags:

> python ostools.py extract --help
usage: ostools.py extract [-h] [-tiledb TILEDB] [-infodb INFODB] [-regions [REGIONS ...]] [-zoom ZOOM]

options:
  -h, --help              show this help message and exit
  -tiledb TILEDB          path to tile database (default: './map_data.db')
  -infodb INFODB          path to info database (default: './customOfflineMaps.db')
  -regions [REGIONS ...]  regions to extract from the database
  -zoom ZOOM              MBTiles tile zoom level (default: 16)

Converting maps (convert)

Once you have your maps in MBTiles format you may wish to convert the tiles from PNG to WEBP format to reduce the file size without significantly impacting quality:

1. Run python ostools.py convert {filename}

A new file, {filename}_webp.mbtiles, will be created in which the tile image format is WEBP.

You can adjust the WEBP compression parameters in order to obtain your desired balance of image quality and size:

> python ostools.py convert --help
usage: ostools.py convert [-h] [-quality [QUALITY]] file

positional arguments:
  file                path to MBTiles file

optional arguments:
  -h, --help          show this help message and exit
  -quality [QUALITY]  quality of WEBP compression (0-100, default: 50)

Reference the Pillow documentation for an explanation of how the -quality parameter behaves. This is an illustration of the compression performance on a single map tile:

PNG	WEBP (100)	WEBP (75)	WEBP (50) default	WEBP (25)	WEBP (0)
26,912 bytes	23,668 bytes	16,726 bytes	14,476 bytes	8,820 bytes	2,362 bytes

Deduping tiles (dedupe)

Adjacent OS maps overlap at the edges, but when using digital maps there is no need for this overlap, so storing these duplicated tiles is just a waste of space. As such the dedupe command allows you to remove tiles from one MBTiles file that appear in another.

Note that these tiles are removed from the MBTiles file passed as the first argument, and that this removal is performed in place.

Once the duplicate tiles are removed the MBTiles database is vacuumed so that the file size saving may be obtained. Note that this process can require as much as twice the original file size as free disk space.

There are no optional arguments for this command:

> python .\ostools.py dedupe --help
usage: ostools.py dedupe [-h] file1 file2

positional arguments:
  file1       path to MBTile to dedupe (in place)
  file2       path to MBTile to compare against

optional arguments:
  -h, --help  show this help message and exit

Other commands

Other commands are unsupported and should not be used.

Example usage

A simple example of extracting, converting, and deduping two maps from the OS Maps app database:

> python .\ostools.py -verbose extract

  Loading './customOfflineMaps.db': Done
  Identifying regions: Explorer OL7, Explorer OL6
  Closing './customOfflineMaps.db': Done
  Loading 'map_data.db': Done
  Creating outfile 'Explorer OL7.mbtiles': Done
  Preparing MBTiles structure: Done
  Decompressing and copying map tiles: Done
  Closing 'Explorer OL7.mbtiles': Done
  Creating outfile 'Explorer OL6.mbtiles': Done
  Preparing MBTiles structure: Done
  Decompressing and copying map tiles: Done
  Closing 'Explorer OL6.mbtiles': Done
  Extract completed.

> python .\ostools.py -verbose convert 'Explorer OL6.mbtiles'

  Loading MBTiles file 'Explorer OL6.mbtiles': Done
  Creating outfile 'Explorer OL6_webp.mbtiles': Done
  Preparing MBTiles structure: Done
  Converting tiles to WEBP (be patient!): Done
  Closing 'Explorer OL6.mbtiles': Done
  Closing 'Explorer OL6_webp.mbtiles': Done
  Convert completed.

> python .\ostools.py -verbose dedupe 'Explorer OL6_webp.mbtiles' 'Explorer OL7.mbtiles'

  Loading databases: Done
  Removing tiles from 'Explorer OL6_webp.mbtiles' that appear in 'Explorer OL7.mbtiles': 696 duplicates
  Resizing MBTiles database: Done
  Dedupe completed.

> ls *.mbtiles

  Directory: .\OS-Maps-app-tools
  Mode         LastWriteTime       Length Name
  ----         -------------       ------ ----
  -a----    00/00/0000 00:00    147337216 Explorer OL7.mbtiles
  -a----    00/00/0000 00:00    128802816 Explorer OL6.mbtiles
  -a----    00/00/0000 00:00     76795904 Explorer OL6_webp.mbtiles

Note the difference in file size between the originally extracted Explorer OL6 map (128.8Mb) and the same once converted to WEBP and with duplicate tiles from the adjacent Explorer OL7 map removed (76.8Mb).