Multithreaded replication WIP by meiji163 · Pull Request #1454

Multithreaded replication WIP by meiji163 · Pull Request #1454 · github/gh-ost

and others added 30 commits

September 30, 2024 09:15

Multiple `TableMap` events can happen inside of a single transaction, so we can't really use them to filter out events early or anything like that. Instead, we'll just skip them.

Also adds some basic test cases for the transaction streaming.

Co-authored-by: Daniel Joos <danieljoos@github.com>

* Remove error return value since we don't use it.

* Lock the mutex whenever we plan to update the low watermark to avoid a race condition.

* Check for data races in our unit tests.

* Still return an error from ProcessEventsUntilDrained but actually check it in our code.

* Make coordinator_test.go to check the err from  ProcessEventsUntilDrained again

* Remove unreachable return in ProcessEventsUntilDrained

…ark (#1531)

* Notify waiting channels on completed transaction, not just the watermark.

* Add checksum validation to coordinator test

* Use errgroup to perform transactions concurrently in coordinator_test.go

* Configure concurrency separate from total number of transactions.

* Run similar number of txs to previous test and ignore context.

* Have at least 1 child in a transaction.

* Notify waiting channels for the current sequence number.