Write files iteratively to Azure Blob Storage

[ANK1656078988]

Hi,

I am looking to randomly generate some data using the tRowGenerator component, write this into a file and upload this data to Azure Blob Storage container, at a 10 second interval

What is the best way to do this?

I am currently stuck at the point where I cant connect the "main" line of tfileoutoutdelimited and write into Azure.

The tAzureStoragePut component requires the all the files be stored in a directory before it can be uploaded to Azure. However I want the upload to happen iteratively, one file at a time.

Any suggestions?

[InfoCraft]

Hi @Aravind NK​,

*** tAzureStoragePut can send one or more file in the same time.

So, you have 2 solutions:

1. Send Files on by one(for each iteration)
2. 

Example: I'm able to send all files starting with test0 to the Azure Storage...

===> I don't understand why you iterate every 10 seconds!

Kind regards,

😷19.

[ANK1656078988]

Hi @not specified not specified​ Thank you for the response.

The reason behind iterating every 10 seconds is to simulate a streaming use case, where each file would represent a stream of data.

I am clear on the first option.

For the second one, will the tLoop component OnSubJobOk connector trigger tAzurePutStorage during every iteration?

[InfoCraft · Accepted solution]

Hi @Aravind NK​,

1. OnComponentOK: will trigger the tAzureStoragePut after each itération.
2. OnSubjobOK triggers tAzureStoragePut at the end of all iterations

+++ gift_reminder,

OnSubjobOK and OnComponentOK are trigger links, which can link to another subjob. The main difference between them lies in the execution order of the linked subjob.

• With OnSubjobOK, the linked subjob starts only when the previous subjob completely finishes.
• With OnComponentOK, the linked subjob starts when the previous component finishes.

Kind regards,

😷19.