feat: Go walking skeleton for cmd/ingestion#57
Conversation
|
Claude finished @koinsaari's task in 3m 12s —— View job Review
No must-fix issues found. The PR is clean and ready to merge. A few observations for completeness:
|
Replace the placeholder cmd/ingestion with a working Go binary that reads a .osm.pbf file, filters to POIs, transforms them into models.Place records, and bulk-upserts them into the places table. - Add internal/osm package: PBF streaming via paulmach/osm, allowlist-based POI filter, node-to-place transformer, rank derivation - Add internal/place repository with UpsertBatch using (osm_id, osm_type) conflict key - Add migration 000007 with partial unique index on (osm_id, osm_type) WHERE osm_id <> 0 - Expand models.Category to cover OSM POI taxonomy (Bar, Healthcare, Education, Finance, Entertainment, Government, Transport, Social, Worship) - Replace cmd/ingestion placeholder with full-import dispatcher; diff-sync stays unimplemented - Add end-to-end integration test against an Andorra fixture PBF - Add testhelpers.StartPostgresWithConnInfo so cmd/ingestion tests can build their own configs
9533413 to
6581f48
Compare
|
Claude encountered an error —— View job Tasks
|
What
Replaces the placeholder
cmd/ingestionwith a working Go binary that reads a.osm.pbffile, filters to POIs, transforms them intomodels.Placerecords, and bulk-upserts them into theplacestable.Changes
internal/osmpackage: PBF streaming viapaulmach/osm, allowlist-based POI filter, node-to-place transformer, rank derivationinternal/placerepository withUpsertBatchusing(osm_id, osm_type)conflict key000007with partial unique index on(osm_id, osm_type) WHERE osm_id <> 0models.Categoryto cover the OSM POI taxonomy (Bar,Healthcare,Education,Finance,Entertainment,Government,Transport,Social,Worship)cmd/ingestionplaceholder with afull-importdispatcher;diff-syncis wired up to print "not implemented" for nowtesthelpers.StartPostgresWithConnInfosocmd/ingestiontests can build their own configs from a testcontainers PostGIS instanceCLAUDE.mdto describe the newinternal/osmandinternal/placepackages and the realcmd/ingestionroleNotes
docs/superpowers/migration-roadmap.md. Accessibility ingestion, diff-sync, and place soft-vs-hard-delete logic come in later sub-projects.WHERE osm_id <> 0) lets existingcmd/apitests keep creating places without OSM IDs. In production every place is OSM-sourced and has a non-zero osm_id, so the predicate covers every real row. A follow-up issue should settle whetherPOST /places(the endpoint that justifies the partial predicate) should be removed.testdata/; gives the smoke test ~492k nodes to process and ~976 places to import.Test plan
go test ./...— all unit tests passgo test -tags integration -timeout 300s ./...— all integration tests pass (including the pre-existingcmd/apiones that rely on the partial-index predicate)go vet ./...clean