Talend 8 Job reading all messages in Azure Storage Queue

[GeoNL]

In a Talend 8 job I need to read messages form an Azure Storage Queue.

The options for this are tAzureStorageQueueInput and tAzureStorageQueueInputLoop

The problem with the tAzureStorageQueueInput is that it can retrieve a maximum of 32 messages per run instead of just collecting all that is there in the queue.

The problem with the tAzureStorageQueueInputLoop is that it never breaks out of the loop, how would I know it can safely terminate? In the documentation there is nothing mentioned on how to stop this infinite loop.

Documentation used: https://help.talend.com/r/en-US/8.0/azure-storage-queue/tazurestoragequeueinputloop

[GeoNL]

Workaround used:

• PreJob define a globalVar boolean 'done' as false,
• Start with a tLoop while 'done' is false and iterate over tAzureStorageQueueInput + rest of job,
• tAzureStorageQueueInput get's an onComponentOk where you check the NB_LINE
  • If that is 0 you set the variable 'done' to true
  • theoretically if it is less than the __numberOfMessages__ you could already set 'done' true

[Xiaodi_Shi]

Hello,

Thanks for sharing your suggestion and good solution with us on Community. We will send it to DOCT team to update the documentation if necessary.

Best regards

Sabrina

[Xiaodi_Shi]

Hello,

Thanks for sharing your suggestion and good solution with us on Community. We will send it to DOCT team to update the documentation if necessary.

Best regards

Sabrina