[feat][starrocks] add starrocks sync/sql connector by Paddy0523 · Pull Request #1237 · DTStack/chunjun

@Paddy0523

Purpose of this pull request

  1. sync : add starrocks source/sink connector
  2. sql : add source/sink/lookup connector

Which issue you fix

Fixes # (issue).

Checklist:

  • I have executed the 'mvn spotless:apply' command to format my code.
  • I have a meaningful commit message (including the issue id, the template of commit message is '[label-type-#issue-id][fixed-module] a meaningful commit message.')
  • I have performed a self-review of my own code.
  • I have commented my code, particularly in hard-to-understand areas.
  • I have made corresponding changes to the documentation.
  • I have added tests that prove my fix is effective or that my feature works.
  • New and existing unit tests pass locally with my changes.
  • I have checked my code and corrected any misspellings.
  • My commit is only one. (If there are multiple commits, you can use 'git squash' to compress multiple commits into one.)

libailin

@libailin

@Paddy0523

FlechazoW

FlechazoW

FlechazoW

import com.starrocks.shade.org.apache.thrift.protocol.TProtocol;
import com.starrocks.shade.org.apache.thrift.transport.TSocket;
import com.starrocks.shade.org.apache.thrift.transport.TTransportException;
import com.starrocks.thrift.*;

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do not import *.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

FlechazoW


import java.io.IOException;
import java.io.Serializable;
import java.util.*;

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do not import *.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

FlechazoW

<version>5.1.49</version>
</dependency>

<!-- <dependency>-->

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove useless code.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@Paddy0523

FlechazoW


<dependency>
<groupId>com.alibaba</groupId>
<artifactId>fastjson</artifactId>

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Duplicate dependencies are depended on the project, and we have other json-processed dependencies, like Gson, in the root pom.

FlechazoW

<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<version>5.1.49</version>

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this version same as other connector?

FlechazoW

FlechazoW

}

@Override
protected void closeInternal() {}

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing to do here?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'close()' method should be called in 'finally' block.

if (offsetOfBatchForRead < flinkRowsCount) {
return true;
}
this.close();

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why call the 'close()' method here ?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed:close in finally

.getPartitions()
.forEach(
(tabletId, tablet) -> {
int tabletCount = Integer.MAX_VALUE;

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this code block should be called in a method.

@FlechazoW

@Paddy0523