Update to planetiler 0.10.2

[neodescis]

planetiler 0.10.2 has been released, and it incorporates an updated MLT encoder that fixes an issue creating tiles that MapLibre GL couldn't handle (see original issue). I was hoping that updating this project to the latest planetiler would be a simple update to the POM file, but unfortunately the update also causes a bunch of the tests to fail with the error "Too many possible values: 6281600" (all with getSortKey in the call stack). I did confirm this is not just a test problem; generating tiles fails with the same error.

So, I'm hoping somebody might be able to take a look and bump the planetiler dependency to pick up the MLT encoder fixes.

[neodescis]

Dug a bit further, and I think it might be caused by this change, made as part of adding support for zoom level 16 to planetiler:

onthegomap/planetiler@1fc95a2#diff-0c8b587e302b29afb0ba37f061c873b0dd3ac0ca25421c64706ff9069767aab5R56

It looks like the change limits the sort key enough that it's now crashing in this project.

[neodescis]

I've been able to fix it locally by changing the number of population levels in Places.java's getSortKey() from 100 to 65, but I'm not sure if that's a "good" change. @wipfli, looks like you were the last one to touch this... what do you think?

[wipfli]

Uff interesting. I ported some logic from the old syntax to the new Matcher syntax in Places.java, but I actually do not have a feeling for what the cartographic consequences would be regarding population levels.

[wipfli]

Can you outline again the consequences of zoom 16 support in Planetiler for this basemaps repo? I don't understand what sortKey has to do with maxzoom...

[neodescis]

From the PR, it seems the key for a feature in FeatureGroup is (now) assembled like this:

[tile: 33 bits (63-31)][layer: 8 bits (30-23)][sortKey: 22 bits (22-1)][hasGroup: 1 bit (0)]

Before the PR, the tile was represented by 32 bits, and the sort key by 23 bits. The extra bit for tiles is needed to support the expanded address space of the additional zoom level. So, there's now one less bit available for the sort key. I didn't see that aspect of the change mentioned in the PR, but I assume @msbarry is aware of it.

[msbarry]

Ah yes we discussed this beforehand as a necessary tradeoff to get an extra bit for the tile IDs since it seemed unlikely to break any existing use case. I was planning to move "has group" out of the sort key into the first byte of the tile contents so we'd be back to 23 bits for sort key at the expense of an extra few gb of temp disk space required but haven't made that change yet.

I assume the error is thrown from here?

basemaps/tiles/src/main/java/com/protomaps/basemap/layers/Places.java

Lines 200 to 217 in cd50047

	static int getSortKey(double minZoom, int kindRank, long population, String name) {
	return SortKey
	// (nvkelso 20230803) floats with significant single decimal precision
	// but results in "Too many possible values"
	// Order ASCENDING (smaller manually curated Natural Earth min_zoom win over larger values, across kinds)
	// minZoom is a float with 1 significant digit for manually curated places
	.orderByInt((int) (minZoom * 10), 0, 150)
	// Order ASCENDING (smaller values win, countries then locality then neighbourhood, breaks ties for same minZoom)
	.thenByInt(kindRank, 0, 12)
	// Order DESCENDING (larger values win, San Francisco rank 11 wins over Oakland rank 10)
	// Disabled to allow population log to have larger range
	//.thenByInt(populationRank, 15, 0)
	// Order DESCENDING (larger values win, Millbrea 40k wins over San Bruno 20k, both rank 7)
	.thenByLog(population, 40000000, 1, 100)
	// Order ASCENDING (shorter strings are better than longer strings for map display and adds predictability)
	.thenByInt(name == null ? 0 : name.length(), 0, 31)
	.get();
	}

As a short term fix reducing the resolution of the log(population) portion of the sort key sounds like it should work?

[neodescis]

Correct, that's where the error is thrown. And yes, reducing the log(population) resolution to 65 levels does work; I'm doing a planet-level run right now with that in place. I'd be happy to make a pull request if that's the desired fix to enable the update to 0.10.2.

[1Ninad]

I was planning to move "has group" out of the sort key into the first byte of the tile contents so we'd be back to 23 bits

I'd love to help with the "has group" migration on the Planetiler side to get the 23 sort key bits back. Let me know if that's something I can take on.