topic Re: Send Files to Azure Blob storage in Talend Studio

Send Files to Azure Blob storage

raavisreeram — Mon, 25 Nov 2024 13:00:57 GMT

Hi Team ,

I am able to connect to the Azure blob storage but could not able to place the files or list the files in the azure container, could you please help here.

tAzure_Storage_connection -- > tAzure_StorageList

Error:

Exception in component tAzureStorageList_1 (AzureMFTBlob)

java.lang.IllegalArgumentException: The String is not a valid Base64-encoded string.

tAzure_Storage_connection -- > tAzure_StoragePut

Getting the below error

tAzureStoragePut_1 The String is not a valid Base64-encoded string.

Re: Send Files to Azure Blob storage

raavisreeram — Mon, 25 Nov 2024 13:00:23 GMT

Hi,

Could you please support in resolving the issue

Re: Send Files to Azure Blob storage

Madhav_Nalla — Mon, 25 Nov 2024 16:38:37 GMT

try to remove connection component and provide connection details in the Storage list component, also run the job in Trace log mode and attach the logs here for review

Re: Send Files to Azure Blob storage

raavisreeram — Mon, 02 Dec 2024 03:50:04 GMT

As per the guidelines i have configured the details in storage list and run the job
Please find the below similar error

Starting job AzureMFTBlob at 03:47 02/12/2024.
 
[statistics] connecting to socket on port 3766
[statistics] connected
[trace] connecting to socket on port 4509
[trace] connected
[FATAL] 03:47:42 pef_git.azuremftblob_0_1.AzureMFTBlob- tAzureStorageList_2 The String is not a valid Base64-encoded string.
java.lang.IllegalArgumentException: The String is not a valid Base64-encoded string.
 at com.microsoft.azure.storage.core.Base64.decode(Base64.java:88) ~[azure-storage-8.6.0.jar:?]
 at com.microsoft.azure.storage.StorageCredentialsAccountAndKey.<init>(StorageCredentialsAccountAndKey.java:81) ~[azure-storage-8.6.0.jar:?]
 at org.talend.components.azurestorage.AzureConnectionWithKeyService.getCloudStorageAccount(AzureConnectionWithKeyService.java:44) ~[components-azurestorage-0.37.35.jar:?]
 at org.talend.components.azurestorage.blob.AzureStorageBlobService.listBlobs(AzureStorageBlobService.java:124) ~[components-azurestorage-0.37.35.jar:?]
 at org.talend.components.azurestorage.blob.runtime.AzureStorageListReader.start(AzureStorageListReader.java:79) ~[components-azurestorage-0.37.35.jar:?]
 at org.talend.codegen.flowvariables.runtime.FlowVariablesReader.start(FlowVariablesReader.java:73) ~[talend-codegen-utils-0.35.0.jar:?]
 at pef_git.azuremftblob_0_1.AzureMFTBlob.tAzureStorageList_2Process(AzureMFTBlob.java:993) [classes/:?]
 at pef_git.azuremftblob_0_1.AzureMFTBlob.runJobInTOS(AzureMFTBlob.java:2009) [classes/:?]
 at pef_git.azuremftblob_0_1.AzureMFTBlob.main(AzureMFTBlob.java:1663) [classes/:?]
Exception in component tAzureStorageList_2 (AzureMFTBlob)
java.lang.IllegalArgumentException: The String is not a valid Base64-encoded string.
 at com.microsoft.azure.storage.core.Base64.decode(Base64.java:88)
 at com.microsoft.azure.storage.StorageCredentialsAccountAndKey.<init>(StorageCredentialsAccountAndKey.java:81)
 at org.talend.components.azurestorage.AzureConnectionWithKeyService.getCloudStorageAccount(AzureConnectionWithKeyService.java:44)
 at org.talend.components.azurestorage.blob.AzureStorageBlobService.listBlobs(AzureStorageBlobService.java:124)
 at org.talend.components.azurestorage.blob.runtime.AzureStorageListReader.start(AzureStorageListReader.java:79)
 at org.talend.codegen.flowvariables.runtime.FlowVariablesReader.start(FlowVariablesReader.java:73)
 at pef_git.azuremftblob_0_1.AzureMFTBlob.tAzureStorageList_2Process(AzureMFTBlob.java:993)
 at pef_git.azuremftblob_0_1.AzureMFTBlob.runJobInTOS(AzureMFTBlob.java:2009)
 at pef_git.azuremftblob_0_1.AzureMFTBlob.main(AzureMFTBlob.java:1663)
[statistics] disconnected
[trace] disconnected
 
Job AzureMFTBlob ended at 03:47 02/12/2024. [Exit code = 1]

Re: Send Files to Azure Blob storage

raavisreeram — Mon, 02 Dec 2024 12:32:31 GMT

Tried using T Java
Getting the same error

Starting job AzureMFTBlob at 12:29 02/12/2024.
 
[INFO ] 12:29:54 pef_git.azuremftblob_0_1.AzureMFTBlob- Project name: PEF_GIT Job name: AzureMFTBlob GIT Commit ID: Talend Version: 8.0.1.20240820_0946-patch
[INFO ] 12:29:54 pef_git.azuremftblob_0_1.AzureMFTBlob- TalendJob: 'AzureMFTBlob' - Start.
[INFO ] 12:29:54 pef_git.azuremftblob_0_1.AzureMFTBlob- TalendJob: 'AzureMFTBlob' - Started.
[statistics] connecting to socket on port 4064
[statistics] connected
[DEBUG] 12:29:55 org.talend.components.azurestorage.blob.runtime.AzureStorageRuntime- RefComponentId: null
[DEBUG] 12:29:55 org.talend.components.azurestorage.blob.runtime.AzureStorageRuntime- RefComponentId: null
[INFO ] 12:29:55 org.talend.components.azure.runtime.token.EndpointUtil- Using endpoint:core.windows.net
[FATAL] 12:29:55 pef_git.azuremftblob_0_1.AzureMFTBlob- tAzureStorageList_2 The String is not a valid Base64-encoded string.
java.lang.IllegalArgumentException: The String is not a valid Base64-encoded string.
 at com.microsoft.azure.storage.core.Base64.decode(Base64.java:88) ~[azure-storage-8.6.0.jar:?]
 at com.microsoft.azure.storage.StorageCredentialsAccountAndKey.<init>(StorageCredentialsAccountAndKey.java:81) ~[azure-storage-8.6.0.jar:?]
 at org.talend.components.azurestorage.AzureConnectionWithKeyService.getCloudStorageAccount(AzureConnectionWithKeyService.java:44) ~[components-azurestorage-0.37.35.jar:?]
 at org.talend.components.azurestorage.blob.AzureStorageBlobService.listBlobs(AzureStorageBlobService.java:124) ~[components-azurestorage-0.37.35.jar:?]
 at org.talend.components.azurestorage.blob.runtime.AzureStorageListReader.start(AzureStorageListReader.java:79) ~[components-azurestorage-0.37.35.jar:?]
 at org.talend.codegen.flowvariables.runtime.FlowVariablesReader.start(FlowVariablesReader.java:73) ~[talend-codegen-utils-0.35.0.jar:?]
 at pef_git.azuremftblob_0_1.AzureMFTBlob.tAzureStorageList_2Process(AzureMFTBlob.java:586) [classes/:?]
 at pef_git.azuremftblob_0_1.AzureMFTBlob.runJobInTOS(AzureMFTBlob.java:1462) [classes/:?]
 at pef_git.azuremftblob_0_1.AzureMFTBlob.main(AzureMFTBlob.java:1130) [classes/:?]
Exception in component tAzureStorageList_2 (AzureMFTBlob)
java.lang.IllegalArgumentException: The String is not a valid Base64-encoded string.
 at com.microsoft.azure.storage.core.Base64.decode(Base64.java:88)
 at com.microsoft.azure.storage.StorageCredentialsAccountAndKey.<init>(StorageCredentialsAccountAndKey.java:81)
 at org.talend.components.azurestorage.AzureConnectionWithKeyService.getCloudStorageAccount(AzureConnectionWithKeyService.java:44)
 at org.talend.components.azurestorage.blob.AzureStorageBlobService.listBlobs(AzureStorageBlobService.java:124)
 at org.talend.components.azurestorage.blob.runtime.AzureStorageListReader.start(AzureStorageListReader.java:79)
 at org.talend.codegen.flowvariables.runtime.FlowVariablesReader.start(FlowVariablesReader.java:73)
 at pef_git.azuremftblob_0_1.AzureMFTBlob.tAzureStorageList_2Process(AzureMFTBlob.java:586)
 at pef_git.azuremftblob_0_1.AzureMFTBlob.runJobInTOS(AzureMFTBlob.java:1462)
 at pef_git.azuremftblob_0_1.AzureMFTBlob.main(AzureMFTBlob.java:1130)
[statistics] disconnected
[INFO ] 12:29:55 pef_git.azuremftblob_0_1.AzureMFTBlob- TalendJob: 'AzureMFTBlob' - Finished - status: failure returnCode: 1
 
Job AzureMFTBlob ended at 12:29 02/12/2024. [Exit code = 1]