Send Files to Azure Blob storage

raavisreeram · ‎2024-11-25

Hi Team ,

I am able to connect to the Azure blob storage but could not able to place the files or list the files in the azure container, could you please help here.

tAzure_Storage_connection -- > tAzure_StorageList

Error:

Exception in component tAzureStorageList_1 (AzureMFTBlob)

java.lang.IllegalArgumentException: The String is not a valid Base64-encoded string.

tAzure_Storage_connection -- > tAzure_StoragePut

Getting the below error

tAzureStoragePut_1 The String is not a valid Base64-encoded string.

raavisreeram · ‎2024-11-25

Hi,

Could you please support in resolving the issue

Madhav_Nalla · ‎2024-11-25

try to remove connection component and provide connection details in the Storage list component, also run the job in Trace log mode and attach the logs here for review

raavisreeram · ‎2024-12-01

As per the guidelines i have configured the details in storage list and run the job
Please find the below similar error

Starting job AzureMFTBlob at 03:47 02/12/2024.
 
[statistics] connecting to socket on port 3766
[statistics] connected
[trace] connecting to socket on port 4509
[trace] connected
[FATAL] 03:47:42 pef_git.azuremftblob_0_1.AzureMFTBlob- tAzureStorageList_2 The String is not a valid Base64-encoded string.
java.lang.IllegalArgumentException: The String is not a valid Base64-encoded string.
 at com.microsoft.azure.storage.core.Base64.decode(Base64.java:88) ~[azure-storage-8.6.0.jar:?]
 at com.microsoft.azure.storage.StorageCredentialsAccountAndKey.<init>(StorageCredentialsAccountAndKey.java:81) ~[azure-storage-8.6.0.jar:?]
 at org.talend.components.azurestorage.AzureConnectionWithKeyService.getCloudStorageAccount(AzureConnectionWithKeyService.java:44) ~[components-azurestorage-0.37.35.jar:?]
 at org.talend.components.azurestorage.blob.AzureStorageBlobService.listBlobs(AzureStorageBlobService.java:124) ~[components-azurestorage-0.37.35.jar:?]
 at org.talend.components.azurestorage.blob.runtime.AzureStorageListReader.start(AzureStorageListReader.java:79) ~[components-azurestorage-0.37.35.jar:?]
 at org.talend.codegen.flowvariables.runtime.FlowVariablesReader.start(FlowVariablesReader.java:73) ~[talend-codegen-utils-0.35.0.jar:?]
 at pef_git.azuremftblob_0_1.AzureMFTBlob.tAzureStorageList_2Process(AzureMFTBlob.java:993) [classes/:?]
 at pef_git.azuremftblob_0_1.AzureMFTBlob.runJobInTOS(AzureMFTBlob.java:2009) [classes/:?]
 at pef_git.azuremftblob_0_1.AzureMFTBlob.main(AzureMFTBlob.java:1663) [classes/:?]
Exception in component tAzureStorageList_2 (AzureMFTBlob)
java.lang.IllegalArgumentException: The String is not a valid Base64-encoded string.
 at com.microsoft.azure.storage.core.Base64.decode(Base64.java:88)
 at com.microsoft.azure.storage.StorageCredentialsAccountAndKey.<init>(StorageCredentialsAccountAndKey.java:81)
 at org.talend.components.azurestorage.AzureConnectionWithKeyService.getCloudStorageAccount(AzureConnectionWithKeyService.java:44)
 at org.talend.components.azurestorage.blob.AzureStorageBlobService.listBlobs(AzureStorageBlobService.java:124)
 at org.talend.components.azurestorage.blob.runtime.AzureStorageListReader.start(AzureStorageListReader.java:79)
 at org.talend.codegen.flowvariables.runtime.FlowVariablesReader.start(FlowVariablesReader.java:73)
 at pef_git.azuremftblob_0_1.AzureMFTBlob.tAzureStorageList_2Process(AzureMFTBlob.java:993)
 at pef_git.azuremftblob_0_1.AzureMFTBlob.runJobInTOS(AzureMFTBlob.java:2009)
 at pef_git.azuremftblob_0_1.AzureMFTBlob.main(AzureMFTBlob.java:1663)
[statistics] disconnected
[trace] disconnected
 
Job AzureMFTBlob ended at 03:47 02/12/2024. [Exit code = 1]

raavisreeram · ‎2024-12-02

Tried using T Java
Getting the same error

Starting job AzureMFTBlob at 12:29 02/12/2024.
 
[INFO ] 12:29:54 pef_git.azuremftblob_0_1.AzureMFTBlob- Project name: PEF_GIT Job name: AzureMFTBlob GIT Commit ID: Talend Version: 8.0.1.20240820_0946-patch
[INFO ] 12:29:54 pef_git.azuremftblob_0_1.AzureMFTBlob- TalendJob: 'AzureMFTBlob' - Start.
[INFO ] 12:29:54 pef_git.azuremftblob_0_1.AzureMFTBlob- TalendJob: 'AzureMFTBlob' - Started.
[statistics] connecting to socket on port 4064
[statistics] connected
[DEBUG] 12:29:55 org.talend.components.azurestorage.blob.runtime.AzureStorageRuntime- RefComponentId: null
[DEBUG] 12:29:55 org.talend.components.azurestorage.blob.runtime.AzureStorageRuntime- RefComponentId: null
[INFO ] 12:29:55 org.talend.components.azure.runtime.token.EndpointUtil- Using endpoint:core.windows.net
[FATAL] 12:29:55 pef_git.azuremftblob_0_1.AzureMFTBlob- tAzureStorageList_2 The String is not a valid Base64-encoded string.
java.lang.IllegalArgumentException: The String is not a valid Base64-encoded string.
 at com.microsoft.azure.storage.core.Base64.decode(Base64.java:88) ~[azure-storage-8.6.0.jar:?]
 at com.microsoft.azure.storage.StorageCredentialsAccountAndKey.<init>(StorageCredentialsAccountAndKey.java:81) ~[azure-storage-8.6.0.jar:?]
 at org.talend.components.azurestorage.AzureConnectionWithKeyService.getCloudStorageAccount(AzureConnectionWithKeyService.java:44) ~[components-azurestorage-0.37.35.jar:?]
 at org.talend.components.azurestorage.blob.AzureStorageBlobService.listBlobs(AzureStorageBlobService.java:124) ~[components-azurestorage-0.37.35.jar:?]
 at org.talend.components.azurestorage.blob.runtime.AzureStorageListReader.start(AzureStorageListReader.java:79) ~[components-azurestorage-0.37.35.jar:?]
 at org.talend.codegen.flowvariables.runtime.FlowVariablesReader.start(FlowVariablesReader.java:73) ~[talend-codegen-utils-0.35.0.jar:?]
 at pef_git.azuremftblob_0_1.AzureMFTBlob.tAzureStorageList_2Process(AzureMFTBlob.java:586) [classes/:?]
 at pef_git.azuremftblob_0_1.AzureMFTBlob.runJobInTOS(AzureMFTBlob.java:1462) [classes/:?]
 at pef_git.azuremftblob_0_1.AzureMFTBlob.main(AzureMFTBlob.java:1130) [classes/:?]
Exception in component tAzureStorageList_2 (AzureMFTBlob)
java.lang.IllegalArgumentException: The String is not a valid Base64-encoded string.
 at com.microsoft.azure.storage.core.Base64.decode(Base64.java:88)
 at com.microsoft.azure.storage.StorageCredentialsAccountAndKey.<init>(StorageCredentialsAccountAndKey.java:81)
 at org.talend.components.azurestorage.AzureConnectionWithKeyService.getCloudStorageAccount(AzureConnectionWithKeyService.java:44)
 at org.talend.components.azurestorage.blob.AzureStorageBlobService.listBlobs(AzureStorageBlobService.java:124)
 at org.talend.components.azurestorage.blob.runtime.AzureStorageListReader.start(AzureStorageListReader.java:79)
 at org.talend.codegen.flowvariables.runtime.FlowVariablesReader.start(FlowVariablesReader.java:73)
 at pef_git.azuremftblob_0_1.AzureMFTBlob.tAzureStorageList_2Process(AzureMFTBlob.java:586)
 at pef_git.azuremftblob_0_1.AzureMFTBlob.runJobInTOS(AzureMFTBlob.java:1462)
 at pef_git.azuremftblob_0_1.AzureMFTBlob.main(AzureMFTBlob.java:1130)
[statistics] disconnected
[INFO ] 12:29:55 pef_git.azuremftblob_0_1.AzureMFTBlob- TalendJob: 'AzureMFTBlob' - Finished - status: failure returnCode: 1
 
Job AzureMFTBlob ended at 12:29 02/12/2024. [Exit code = 1]

Azure

Studio