NIFI-7307 Adding support for setting and retrieving Azure Blob Storag… by pkelly-nifi · Pull Request #10939 · apache/nifi

NIFI-7307 Adding support for setting and retrieving Azure Blob Storag… by pkelly-nifi · Pull Request #10939 · apache/nifi

Conversation

…e metadata and tags

Summary

NIFI-7307 Adds support for setting and retrieving Azure Blob Storage object metadata and tags. Modeled after similar functionality which exists for S3.

Tracking

Please complete the following tracking steps prior to pull request creation.

Issue Tracking

Apache NiFi Jira issue created

Pull Request Tracking

Pull Request title starts with Apache NiFi Jira issue number, such as NIFI-00000
Pull Request commit message starts with Apache NiFi Jira issue number, as such NIFI-00000
Pull request contains commits signed with a registered key indicating Verified status

Pull Request Formatting

Pull Request based on current revision of the main branch
Pull Request refers to a feature branch with one commit containing changes

Verification

Please indicate the verification steps performed prior to pull request creation.

Build

Build completed using ./mvnw clean install -P contrib-check
- JDK 21
- JDK 25

Licensing

New dependencies are compatible with the Apache License 2.0 according to the License Policy
New dependencies are documented in applicable LICENSE and NOTICE files

Documentation

Documentation formatting appears as expected in rendered files

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left some initial comments. Will be also important to add some test coverage.

		@SeeAlso({ ListAzureBlobStorage_v12.class, FetchAzureBlobStorage_v12.class, PutAzureBlobStorage_v12.class,
		CopyAzureBlobStorage_v12.class, DeleteAzureBlobStorage_v12.class, GetAzureBlobStorageMetadata_v12.class })
		@CapabilityDescription("Retrieves user metadata and/or tags from the specified blob from Azure Blob Storage. The processor uses Azure Blob Storage client library v12.")
		@InputRequirement(Requirement.INPUT_REQUIRED)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

write attributes annotation

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added. Please let me know if you have a better suggestion for annotating these dynamic attributes.

		newAttributes.put(ATTRIBUTE_FORMAT_TAG.formatted(key), value);
		});

		flowFile = session.putAllAttributes(flowFile, newAttributes);

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

missing provenance events

		});

		flowFile = session.putAllAttributes(flowFile, newAttributes);
		session.transfer(flowFile, REL_FOUND);

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

missing provenance events

		CopyAzureBlobStorage_v12.class, DeleteAzureBlobStorage_v12.class, GetAzureBlobStorageMetadata_v12.class })
		@CapabilityDescription("Retrieves user metadata and/or tags from the specified blob from Azure Blob Storage. The processor uses Azure Blob Storage client library v12.")
		@InputRequirement(Requirement.INPUT_REQUIRED)
		public class GetAzureBlobStorageTags_v12 extends AbstractAzureBlobProcessor_v12 {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a lot of code duplication for those two new processors. Could we have a layer of abstraction to avoid code duplication?

Thank you for your review and feedback, @pvillard31. I have made all of the changes you requested and will look into adding tests today. It looks like all of the current Azure blob storage tests only focus on property migration, so it might take me a little time to add for these processors.