Iterate on an array and pass it to a tRestClient component

[HungryOctopus]

Dear community,

I am struggling with an issue in Talend.

I want to process a string of numbers separated with a comma in a tRestClient component. Since the REST API has a limitation of 100 characters, I want to check if it's longer, than create batches of the request, then iterate on it.

I built the job like this, but it doesn't iterate.

How I built the job

This is the code in the tjavarow:

// Get the string from global map (only on first execution)
if (globalMap.get("batchList") == null) {
String linkedPimArticlesFilterParam = (String) globalMap.get("linkedPimArticlesFilterParam");

// Split the string by comma
String[] numbers = linkedPimArticlesFilterParam.split(",");

// Create batches
java.util.List<String> batches = new java.util.ArrayList<String>();
StringBuilder currentBatch = new StringBuilder();

for (String number : numbers) {
String trimmedNumber = number.trim();

String potentialBatch = currentBatch.length() == 0 ?
trimmedNumber : currentBatch.toString() + "," + trimmedNumber;

if (potentialBatch.length() <= 100) {
if (currentBatch.length() > 0) {
currentBatch.append(",");
}
currentBatch.append(trimmedNumber);
} else {
if (currentBatch.length() > 0) {
batches.add(currentBatch.toString());
}
currentBatch = new StringBuilder(trimmedNumber);
}
}

if (currentBatch.length() > 0) {
batches.add(currentBatch.toString());
}

globalMap.put("batchList", batches);
globalMap.put("currentBatchIndex", 0);
}

// Get current batch
java.util.List<String> batches = (java.util.List<String>)globalMap.get("batchList");
Integer currentIndex = (Integer)globalMap.get("currentBatchIndex");

if (currentIndex < batches.size()) {
String currentBatch = batches.get(currentIndex);
globalMap.put("currentBatchData", currentBatch);
globalMap.put("currentBatchIndex", currentIndex + 1);

System.out.println("Processing batch " + (currentIndex + 1) + " of " + batches.size() + ": " + currentBatch);
} else {
// No more batches to process
globalMap.put("currentBatchData", null);
}

System.out.println("Current batch for REST call: " + (String)globalMap.get("currentBatchData"));

This is the key in the tFlowToIterate: (String)globalMap.get("currentBatch")

The tjavaflex does nothing, it is just to have something to link to the trest with a main row.

It seems I can't directly link any iteration component to the tRest.

I'm grateful for any help! 

[HungryOctopus]

Hey together, 

Is there anyone that can help me on this? The AI couldn't help me so fellow humans, it's your time to shine 😉

Thank you,

Best regards,

[Dave_Simo]

hello @HungryOctopus ,

Could you give an example of what you want to do as an iterative loop? It's a loop with a maximum length of 100 characters in the parameter when calling tResClient if I understand correctly?

Best Regards

[HungryOctopus · Accepted solution]

Hi @Dave_Simo , thank you so much for your response.

I get a string with article numbers from globalMap.get("linkedPimArticlesFilterParam").

It looks like this:

0927000004,0927100004,0927150004,09271500041,0940000004,0940100004,0940150004,0941000005,09410000051,0941000085,09410000A3,0941100005,09411000051,0941100085,09411000851,0941100092,09411000A3,0941150005,0941150085,09411500A3,0944000005,09440000051,0944000085,09440000851,0944003200,0944003205,0944003285,0944000002,09440000021,0944100005,09441000051,0944100085,09441000851,0944103200,0944103285,0944103205,09441000021,0944100002,0944150005,0944150085,09441500851,09441500051,09441500941,0944150094,0944153205,09530000852,09530000052,09530000022,09531000852,09531000052,09531000022,09531500852,09531500052,0953150094,095700AC85,095700CB00,0957003200,0957000000,095700AM85,095700CB85,0957003285,095700AA85,0957000085,095700AC05,095700AA05,095700AM05,0957103285,095710CB85,095710AM85,0957100000,095710CB00,0957103200,095710AC85,095710AA85,0957100085,095710AC05,095710AA05,095710AM05,095715AM85,

I want to pass them as a Query parameter in my tRESTClient like this:

The problem is that the REST API has a limitation for the numbers of characters that you can send at the same time. 

So I want to cut it in batches of a defined length, then pass it batch to batch. 

So my talend job should count if the original string has more than 100 characters, if yes pass it 

If no, prepare batches of 100 characters or less without cutting an article number in the middle.

Then pass it to the tRestClient one batch after the other. 

I hope this makes sense, if you have any questions please let me know.

Many thanks,

[Dave_Simo · Accepted solution]

hello @HungryOctopus 

I think I now understand what you want to do and I suggest the design of the job below to cut into batches of 100 and group with a tAggregateRow then loop with a tFlowToIterate to call the tResclient as many as necessary.

1) 

2) 

3) 

4) 

Best Regards

[HungryOctopus]

Hi @Dave_Simo 

I made some slight changes, at the end my job looks like this: 

The tRestClient reads the data from row44. And it's working! 

Thank you so much 🙂

Best regards,