gh-ost hang on INFO Waiting for ghost table to be migrated. Current lag is 0s in MySQL 8.0.21
gh-ost run is not passing the stage
INFO Waiting for ghost table to be migrated. Current lag is 0s
| @@version |
+-----------+
| 8.0.21 |
- multi-master replication 2 masters with slave per master
/usr/bin/gh-ost --max-load=Threads_running=500 --critical-load=Threads_running=1000 --initially-drop-ghost-table --initially-drop-old-table --chunk-size=600 --max-lag-millis=3000 --user=dbschema --password=**** --assume-master-host=pdb-mysql-pii-sdl-playground0a.42.net --throttle-control-replicas=pdb-mysql-pii-sdl-playground0a.96.net --host=pdb-mysql-pii-sdl-playground1b.42.net --database=test --table=T1 --verbose --alter="add e varchar(2),add index e_idx (e)" --assume-rbr --allow-master-master --cut-over=default --concurrent-rowcount --default-retries=120 --panic-flag-file=/tmp/ghost.panic.flag --exact-rowcount --postpone-cut-over-flag-file=/tmp/ghost.postpone.flag --debug --execute
2020-09-13 19:18:55 INFO starting gh-ost 1.0.49
2020-09-13 19:18:55 INFO Migrating `test`.`T1`
2020-09-13 19:18:55 INFO connection validated on pdb-mysql-pii-sdl-playground1b.42.net:3306
2020-09-13 19:18:55 INFO User has REPLICATION CLIENT, REPLICATION SLAVE privileges, and has ALL privileges on `test`.*
2020-09-13 19:18:55 INFO binary logs validated on pdb-mysql-pii-sdl-playground1b.42.net:3306
2020-09-13 19:18:55 INFO Inspector initiated on pdb-mysql-pii-sdl-playground1b.42.net:3306, version 8.0.21
2020-09-13 19:18:55 INFO Table found. Engine=InnoDB
2020-09-13 19:18:55 DEBUG Estimated number of rows via STATUS: 80
2020-09-13 19:18:55 DEBUG Validated no foreign keys exist on table
2020-09-13 19:18:55 DEBUG Validated no triggers exist on table
2020-09-13 19:18:55 INFO Estimated number of rows via EXPLAIN: 80
2020-09-13 19:18:55 DEBUG Potential unique keys in T1: [PRIMARY (auto_increment): [ID]; has nullable: false]
2020-09-13 19:18:55 INFO Master forced to be pdb-mysql-pii-sdl-playground0a.42.net:3306
2020-09-13 19:18:55 INFO log_slave_updates validated on pdb-mysql-pii-sdl-playground1b.42.net:3306
2020-09-13 19:18:55 INFO connection validated on pdb-mysql-pii-sdl-playground1b.42.net:3306
2020-09-13 19:18:55 DEBUG Streamer binlog coordinates: mysql-bin.000003:1539502
2020-09-13 19:18:55 INFO Connecting binlog streamer at mysql-bin.000003:1539502
[2020/09/13 19:18:55] [info] binlogsyncer.go:133 create BinlogSyncer with config {99999 mysql pdb-mysql-pii-sdl-playground1b.42.net 3306 dbschema false false <nil> false UTC true 0 0s 0s 0 false}
[2020/09/13 19:18:55] [info] binlogsyncer.go:354 begin to sync binlog from position (mysql-bin.000003, 1539502)
[2020/09/13 19:18:55] [info] binlogsyncer.go:203 register slave for master server pdb-mysql-pii-sdl-playground1b.42.net:3306
2020-09-13 19:18:55 DEBUG Beginning streaming
2020-09-13 19:18:55 INFO rotate to next log from mysql-bin.000003:0 to mysql-bin.000003
[2020/09/13 19:18:55] [info] binlogsyncer.go:723 rotate to (mysql-bin.000003, 1539502)
2020-09-13 19:18:55 INFO connection validated on pdb-mysql-pii-sdl-playground0a.42.net:3306
2020-09-13 19:18:55 INFO connection validated on pdb-mysql-pii-sdl-playground0a.42.net:3306
2020-09-13 19:18:55 INFO will use time_zone='SYSTEM' on applier
2020-09-13 19:18:55 INFO Examining table structure on applier
2020-09-13 19:18:55 INFO Applier initiated on pdb-mysql-pii-sdl-playground0a.42.net:3306, version 8.0.21
2020-09-13 19:18:55 INFO Dropping table `test`.`_T1_gho`
2020-09-13 19:18:55 INFO Table dropped
2020-09-13 19:18:55 INFO Dropping table `test`.`_T1_del`
2020-09-13 19:18:55 INFO Table dropped
2020-09-13 19:18:55 INFO Dropping table `test`.`_T1_ghc`
2020-09-13 19:18:55 INFO Table dropped
2020-09-13 19:18:55 INFO Creating changelog table `test`.`_T1_ghc`
2020-09-13 19:18:55 INFO Changelog table created
2020-09-13 19:18:55 INFO Creating ghost table `test`.`_T1_gho`
2020-09-13 19:18:55 INFO Ghost table created
2020-09-13 19:18:55 INFO Altering ghost table `test`.`_T1_gho`
2020-09-13 19:18:55 DEBUG ALTER statement: alter /* gh-ost */ table `test`.`_T1_gho` add e varchar(2),add index e_idx (e)
2020-09-13 19:18:55 INFO Ghost table altered
2020-09-13 19:18:55 INFO Waiting for ghost table to be migrated. Current lag is 0s