HNNewShowAskJobs
Built with Tanstack Start
Pbf2sqlite: Reading OpenStreetMap into a SQLite Database(github.com)
62 points by amadeuspagel 6 days ago | 9 comments
  • juliansimioni3 days ago

    This is super cool. As part of the Pelias geocoder(https://pelias.io/) we use both OSM and SQLite heavily. Currently we've written our own pbf2json tool in Golang (https://github.com/pelias/pbf2json). But creating intermediate databases in SQLite could enable more powerful manipulation of OSM data before we eventually import it.

  • tjridesbikes3 days ago

    This is AWESOME! I'm doing some volunteer work to make a map of cycling routes in my city, and using OSM data to annotate features. Having a SQLite db of all the Way tags would make my work a lot easier! Thanks!

  • milliams5 days ago

    Does SQLite have GIS capabilities (like PostGIS provides for Postgres), or is this storing the data as traditional database primitives?

    • fros1y5 days ago |parent

      Check out Spatialite. https://www.gaia-gis.it/fossil/libspatialite/index

    • Scaevolus5 days ago |parent

      SQLite can be built with the R*tree module, which supports efficiently looking up all bounding boxes that contain a point: https://www.sqlite.org/rtree.html

      PostGIS similarly provides an R*Tree index mode, as well as a heap of functions for doing GIS calculations directly. To do that in SQLite, you'd implement and inject custom functions as appropriate.

    • simonw3 days ago |parent

      From the schema it looks like this is storing nodes and lines: https://github.com/osmzoso/pbf2sqlite/blob/main/doc/pbf2sqli...

      There's an option to enable RTree indexes to speed up queries like this one:

        SELECT way_id
        FROM rtree_way
        WHERE max_lon>= 7.851 AND min_lon<= 7.854
         AND  max_lat>=47.995 AND min_lat<=47.996;
    • 0cf8612b2e1e5 days ago |parent

      DuckDB has put some effort into GIS, so if SQLite does not immediately fit the bill, I would go there.

      https://motherduck.com/blog/getting-started-gis-duckdb/

    • wiredfool3 days ago |parent

      SQLite + spatial (and specific metadata tables) is essentially the Geopackage format, which can be considered the modern equivalent of a shapefile.

      Which is to say, yes, SQLite has geospatial operations and they’re well supported by the open source gis stack.

  • teruakohatu3 days ago

    Could this be used for reverse geocoding?