Limit the row list allocation based on the row count by franz1981 · Pull Request #3756

Limit the row list allocation based on the row count by franz1981 · Pull Request #3756 · h2database/h2database

Profiling https://github.com/topicusonderwijs/tribe-krd-quarkus startup it reports a huge amount of allocations due to this:

and zooming in (the orange ones are "out of TLAB allocations":

Configuration of H2 is mem and the ORM policy is drop and create, but still, with this PR it would save a huge amount of allocations (it's a quite large model, actually).

franz1981 changed the title ~~Lazy-allocate the Row list to speedup drop or create use cases~~ Limit the row list allocation based on the row count

Mar 14, 2023

I've forgot to mention @katzyn
re

Configuration of H2 is mem

In the reproducer linked, the H2 version is 1.4.197 that, despite being configured as mem isn't using the in-memory branch i.e. rebuildIndexBuffered, but rebuildIndexBlockMerge, see:

	if (session.getDatabase().getMvStore() == null \|\|
	index instanceof MVSpatialIndex) {
	// in-memory
	rebuildIndexBuffered(session, index);
	} else {
	rebuildIndexBlockMerge(session, index);
	}

It seems suspicious to me, but this patch still apply here, I think.
Let me know if I should fill a separate issue for it

In the reproducer linked, the H2 version is 1.4.197 that, despite being configured as mem isn't using the in-memory branch i.e. rebuildIndexBuffered, but rebuildIndexBlockMerge, see:

if (session.getDatabase().getMvStore() == null ||

index instanceof MVSpatialIndex) {

// in-memory

rebuildIndexBuffered(session, index);

} else {

rebuildIndexBlockMerge(session, index);

}

It seems suspicious to me, but this patch still apply here, I think. Let me know if I should fill a separate issue for it

Does using rebuildIndexBuffered for your example make a difference?

thanks @grandinj to reach out! I'll send the licence statement ASAP!

Does using rebuildIndexBuffered for your example make a difference?

I'm sure it will because it contains

int bufferSize = (int) Math.min(total, database.getMaxMemoryRows());

that's the key optimization to save huge arraylists to be created when no rows are found

Sole problem is that even by setting a mem store 1.4.197 isn't not able to hit that code path (that looks like a different issue to me eh!)

On current GIT master, that line of code reads

 if (!session.getDatabase().isPersistent() || index instanceof MVSpatialIndex) {

so it should work better for your case.

@grandinj waiting subscription to the google group to be accepted and ready to send the first msg :)

@grandinj email sent; given that this is not a huge patch I didn't add the full statement, let me know if I should, instead.

many thanks for the quick review!!! @grandinj