NIFI-15568: Fix Iceberg timestamp handling and add S3 storage class support by NirYanay2005 · Pull Request #10877 · apache/nifi

Thanks for making the initial adjustments @NirYanay2005.

The new Storage Class property looks good. If you are interested in getting that merged more quickly, it would be worth breaking that out to a separate Jira issue and pull request.

Regarding the Record field conversion, the problem makes sense, but the proposed solution does not appear to be the best way forward. Introducing a new IcebergRecordConverter in the Parquet module does not seem like the right location for shared Record formatting, although that is worth further consideration. More importantly, running each Record through a conversion process places additional overhead on the entire process.

Instead of this approach, constructing the Iceberg Record object as need in the PutIcebergRecord Processor seems like the optimal place for changes. The initial implementation of the delegating Record depends heavily on NiFi Record field conversion. That would be the first potential place to make a change for handling type conversion in an optimal way.

If you are willing to work through an alternative approach, I can review options. Alternatively, I may take a closer look at the problem and could propose an alternative solution.

Thanks again for working on these issues and feel free to follow up on how you would like to proceed.