We are trying to upload a very large file (200 GB) to Azure Storage using tAzureStorageConnection and tAzureStoragePut. We configured tAzureStorageConnection to use Azure Active Directory and the tAzureStoragePut is configured to send just the 1 very large file. After exactly 1 hour, we get the failure below. This is because the security token is timing out at 1 hour, a recommend configuration within Azure. We do not have this issue with any other files that transfer under 1 hour. For this to work, the connection with Azure from the Talend components would need to re-authenticate and refresh the token so it can continue its transfer. Has anyone else had this issue?
In the interest of best security practices, we do not want to change the Azure setting that expires a token after 1 hour. We are able to successfully transfer the large file using an Azure Shared Access Signature, but we would prefer to accomplish this task using Azure Active Directory.
------ Error stack
------
:
Caused by: java.io.IOException: Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature. Please see the cause for further information.
at com.microsoft.azure.storage.core.Utility.initIOException(Utility.java:769) ~[azure-storage-8.6.0.jar:?]
at com.microsoft.azure.storage.blob.BlobOutputStreamInternal.writeBlock(BlobOutputStreamInternal.java:462) ~[azure-storage-8.6.0.jar:?]
at com.microsoft.azure.storage.blob.BlobOutputStreamInternal.access$000(BlobOutputStreamInternal.java:47) ~[azure-storage-8.6.0.jar:?]
at com.microsoft.azure.storage.blob.BlobOutputStreamInternal$1.call(BlobOutputStreamInternal.java:406) ~[azure-storage-8.6.0.jar:?]
at com.microsoft.azure.storage.blob.BlobOutputStreamInternal$1.call(BlobOutputStreamInternal.java:403) ~[azure-storage-8.6.0.jar:?]
at java.util.concurrent.FutureTask.run(Unknown Source) ~[?:1.8.0_291]
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ~[?:1.8.0_291]
at java.util.concurrent.FutureTask.run(Unknown Source) ~[?:1.8.0_291]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) ~[?:1.8.0_291]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[?:1.8.0_291]
at java.lang.Thread.run(Unknown Source) ~[?:1.8.0_291]
Caused by: com.microsoft.azure.storage.StorageException: Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
at com.microsoft.azure.storage.StorageException.translateException(StorageException.java:87) ~[azure-storage-8.6.0.jar:?]
at com.microsoft.azure.storage.core.StorageRequest.materializeException(StorageRequest.java:305) ~[azure-storage-8.6.0.jar:?]
at com.microsoft.azure.storage.core.ExecutionEngine.executeWithRetry(ExecutionEngine.java:196) ~[azure-storage-8.6.0.jar:?]
at com.microsoft.azure.storage.blob.CloudBlockBlob.uploadBlockInternal(CloudBlockBlob.java:1243) ~[azure-storage-8.6.0.jar:?]
at com.microsoft.azure.storage.blob.CloudBlockBlob.uploadBlock(CloudBlockBlob.java:1215) ~[azure-storage-8.6.0.jar:?]
at com.microsoft.azure.storage.blob.BlobOutputStreamInternal.writeBlock(BlobOutputStreamInternal.java:456) ~[azure-storage-8.6.0.jar:?]
at com.microsoft.azure.storage.blob.BlobOutputStreamInternal.access$000(BlobOutputStreamInternal.java:47) ~[azure-storage-8.6.0.jar:?]
at com.microsoft.azure.storage.blob.BlobOutputStreamInternal$1.call(BlobOutputStreamInternal.java:406) ~[azure-storage-8.6.0.jar:?]
at com.microsoft.azure.storage.blob.BlobOutputStreamInternal$1.call(BlobOutputStreamInternal.java:403) ~[azure-storage-8.6.0.jar:?]
at java.util.concurrent.FutureTask.run(Unknown Source) ~[?:1.8.0_291]
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ~[?:1.8.0_291]
at java.util.concurrent.FutureTask.run(Unknown Source) ~[?:1.8.0_291]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) ~[?:1.8.0_291]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[?:1.8.0_291]
at java.lang.Thread.run(Unknown Source) ~[?:1.8.0_291]
