How can i avoid the 429 response from API Call

[kakooo16]

Hello , 

Im facing a problem that i couldn't solve . 

Here is my Job 

Eventually , i know what is causing the problem (mylist)  --> because im iterating 22000 items : these items are stocked in mylist 

The question is how could i break down my list which is composite of 22000 to 200 items so that i could iterate the first 200 items until the list of 22000 items is over . 

Thanks in advance . 

[Anonymous]

Hi
My idea is to split the items to several files (eg: 200 items in one file) on disk, and then iterate each file. eg:
bababa---tfileOutputDelimited
|onsubjobok
tFileList--iterate--tfileinputDelimited--tFlowToIterate--iterate---tRest...bababa--oncomponento--tSleep( if needed)
|onsubjobok
tFileDelete

On tFileOutputDelimited: check the 'split the output to several files' box in the advanced settings panel.

tFileDelete: delete the file folder in the end.

Can you try if it could avoid the 429 response from API Call.

Regards
Shong

[kakooo16]

Hello Shong , 

I really don't get how to put every 200 in a tfileOutputDelimited when we are talking about 22000 items .

I've been reading about ''OFFSET" yesterday and if i m not wrong this will create a sort of pagination  (which seems a great idea) . 

So i've changed my query to be like this : 

Im no longer getting the 429 for a list of 50 items . 

Now how could i iterate the other items . here is my main workflow job .

However , do i have to add a tloop before my tpostgresInput query to iterate the rest of the items . 

Hope that will be a great idea . 

And im waiting for a reply or an other idea because im still stuck in this . 

[kakooo16]

Is there any suggestions ?