[Feature][chunjun-core] Supports capturing dirty data from the source and when the source sends it downstream #1901 by david-gao1 · Pull Request #1902 · DTStack/chunjun
问题再现:
CREATE` TABLE source ( `ID` int, `FloatColumn` string, `BinaryColumn` bytes, `VarBinaryColumn` bytes, `LongBinaryColumn` bytes ) WITH ( 'connector' = 'xxx-x', ); CREATE TABLE sink ( `ID` int, `FloatColumn` int, `BinaryColumn` string, `VarBinaryColumn` string, `LongBinaryColumn` string ) WITH ( 'connector' = 'stream-x' ); insert into sink select `ID` as `ID`, CAST(`FloatColumn` AS int) as `FloatColumn`, --比如这里数据源来一条脏数据为:111aa, 数据发送到下游算子时会报错,但此时脏数据无法捕获,脏数据管理器的能力就发挥不出来 CAST(`BinaryColumn` AS string) as `BinaryColumn`, CAST(`VarBinaryColumn` AS string) as `VarBinaryColumn`, CAST(`LongBinaryColumn` AS string) as `LongBinaryColumn` from source ;