tLoop using While condition on Context variable

[Anonymous]

Hello,

I've been trying to get an API call that returns JSON to loop through result pages and am not having any luck. The API allows 250 records returned and at the end indicates a 'vidoffset' that is used to determine starting point of next API call and 'hasmore' indicating if there are more records (true/false).

I am able to iterate through the vidoffsets and get additional data but when I hit the last one (where hasmore = false) my tLoop keeps going...and clearly won't stop.

The parent job (tLoop - > tRunJob) doesn't seem to be picking up the context variable I've set in the child job for 'hasmore'. I'm attempting to use the While loop type and both my Declaration and Iteration values are blank. I've only set the Condition value (context.hasmore != "true") in an attempt to stop when hasmore no longer = 'true'

In the child job I am establishing the context variable of context.hasmore:

- sql that stores last iteration values -> TFlowToIterate -> tFixedFlowInput -> tSplitRow - > tContextLoad). This works fine.

I pass the vidoffset and other parameters into the next 2 subjobs:

- tRestClient -> tmap -> tExtractJSONFields -> tMap -> SQL output  -> client data I need in reporting. This works fine

- tRestClient -> tmap -> tExtractJSONFields -> tMap -> SQL output -> data that I need for next iteration. This works fine.

If someone has advice on passing the context variable to tLoop in a manner which tLoop will recognize and stop iterating I would greatly appreciate it. I've tried to attach images of both the parent and child jobs.

Thanks,

Ryan

[Anonymous]

Hello,

I was hoping you could respond to the question I had earlier. I am able to return the subjob variable to the parent and see it's changed value. The only hold up here is in the tLoop while condition:

- On the first iteration I don't believe either 'hasmore' or 'hasmoreChildJob' have yet been assigned. And after making the suggested changes I'm getting a NullPointerException on this node. So on the first iteration do I need to set 'hasmore' and 'hasmoreChildJob' to true prior to entering the loop to ensure at least one iteration?

[TRF]

Hi,
To avoid the null pointer exception, add a tSetGlobalVar to initialize your variables.
Hope you can now mark your case as solved.
Kudos also accepted.

[Anonymous]

Thanks! I've been trying that but getting additional errors. Below you see how I'm setting hasmore and Childhasmore (changed the name of subjob return value).

The error I get on tLoop is "java.lang.Boolean cannot be cast to java.util.Map".

If you recall my while condition is: !((Boolean) ((java.util.Map) globalMap.get("hasmore")).get("Childhasmore"))

[TRF]

Hi,

Sorry, wasn't a good idea to initialize those global variables.

You just have to check if "Childasmore" is null or not:

((Boolean)((java.util.Map)globalMap.get("hasmore")).get("Childasmore")) == null || 
((Boolean)((java.util.Map)globalMap.get("hasmore")).get("Childasmore"))

At the 1st ireration "Childasmore" is not defined (null), so the child job is called.

The loop will stop as soon as the child job return false for "Childasmore".

Finally, you must remove tSetGlobalVar and you can remove tJava2 too if you declare the hashmap using tLoop Declaration parameter.

Here is how it is one my side:

Hope this one is the good one!

[TRF]

Hi,
Does this solve your case.
If so, please select the better answer to close it.
Kudos also accepted.

[Anonymous]

It's very close. I was working on it yesterday and first implemented the suggestions you had above. I quickly realized that the Childhasmore variable I'm returning from the subjob was not of type Boolean. I was returning a String which would cause a Java error (java.lang.String cannot be cast to java.lang.Boolean) in tLoop where you suggested my condition be:

((Boolean)((java.util.Map)globalMap.get("sharedMap")).get("Childhasmore")) == null || ((Boolean)((java.util.Map)globalMap.get("sharedMap")).get("Childhasmore"))

I tried changing the subjob to return a Boolean for Childhasmore but am not having luck getting that to update sharedMap. So I need to either convert it to Boolean in order for it to work in tLoop condition, or I need to compare Strings in tLoop instead of Boolean. 

But I'm also going to have to change the tLoop logic because you mentioned on the first iteration (where Childhasmore is null) the loop will run, which it does. Then once Childhasmore has been set (not null) it will break the loop. However, it will be set to True in many cases and the loop needs to run again if that is the case. I think (once I'm able to pass Childhasmore to the condition properly) I will need to adjust the logic because once it's not null I believe the first check will cause the loop to break:

((Boolean)((java.util.Map)globalMap.get("sharedMap")).get("Childhasmore")) == null

I'm thinking I could conditionally set Childhasmore in the subjob (if True then don't update it) and the loop will run again?

((java.util.Map) context.sharedMap).put("Childhasmore",((String)globalMap.get("row2.Childhasmore")));

So open questions now

- Better to get subjob to return Boolean and update Childhasmore OR in tLoop while condition somehow compare Strings instead of Boolean?

- Since subjob will return True and need to run tLoop again how to handle it not being null and stopping the loop with condition of Childhasmore == null.

Thanks again for thinking through this. I'm learning a lot in the process ;')

[Anonymous]

Here is what I'm trying to do in the variable assignment in the subjob. It's not assigning though when Childhasmore = false

if (((String)globalMap.get("row2.Childhasmore")) == "false")
{
((java.util.Map) context.sharedMap).put("Childhasmore",((String)globalMap.get("row2.Childhasmore")));
}

[Anonymous · Accepted solution]

Alas...one of the first things you taught me! I've changed the conditional assignment above to:

if (((String)globalMap.get("row2.Childhasmore")).equals("false"))
{
((java.util.Map) context.sharedMap).put("Childhasmore",((String)globalMap.get("row2.Childhasmore")));
}

That did it! Thank you!