[resolved] Loop a tFileFetch

[Anonymous]

I am hitting a link (via https) using tFileFetch component. The response that comes in is in XML format. I am then using an XPath (tFileInputXML) to extract one particular element from that XML. It actually contains a link, which I need to use to fetch in the next set of response, again using tFileFetch component.
If there is some response, then I need to hit that link again and again (using tLoop?), otherwise quit the moment that element is not present. After that I need to do some post processing on the combined output.
How should I be able to achieve this?
Also, is it possible that I can get all these outputs bundled into one single file as I need to perform some transformation on that.

[Anonymous]

Hi,
Could you please elaborate your case with an example with input and expected output values?
Best regards
Sabrina

[alevy]

To do your recursion, use tLoop -iterate-> tFileFetch -OnComponentOK-> tFileInputXML --> tJavaRow
Set tLoop as:
Loop type: While
Declaration: "boolean goagain = true"
Condition: "goagain"
Iteration: ""
In tJavaRow you can set goagain to false when you don't retrieve the link e.g.
if (input_row.link==null) {goagain = false};
Then simply OnSubjobOK from tLoop to your processing. To process all the files you retrieved just iterate over them using tFileList.

[Anonymous]

I did somewhat very close to this that didn't work. I wasn't able to set the goagain to false if there was no value returned.
This resolves it. Thanks Alevy.

[Anonymous]

I am getting a strange scenario, when the tLoop runs, after fetching the last file, it keeps fetching the same file again and again. I have used a Iterate -> tJava from the tFileInpuXML in addition to the above solution.
When I try to display the value, it doesn't reach that code. But it shows 0 rows processed. In the tJava component also, no output comes. Can anyone please help me out ?
While using XPath in the tFileInputXML, does Talend not support XML files having <!DOCTYPE..> at the start? Do I need to filter that out using tFileInputFullRow -> tFilter?

[alevy · Accepted solution]

This appears to be an issue with the XPath loop in your tFileInputXML. If the loop XPath is not found then the component will not generate any rows and thus the tJavaRow cannot set the goagain variable to false. If the loop XPath exists but the XPath query does not then it will work.
If you can't rejig the loop XPath and the XPath query to ensure a row is always generated, then you will need to add a RunIf trigger from tFileInputXML (condition (Integer)globalMap.get("tFileInputXML_2_NB_LINE")==0) to tJava where you can also set goagain to false.

[Anonymous]

Well, this scenario may be quite familiar but I couldn't find the solution. 
Scenario: I am making a REST API call through tFileFetch and I get a json out of it. I parse it to get paging token and moreresult through tflowtoiterate. Now if more_result is equals true, I have to call the same tFileFetch component to get the new set of json using new pagination result.
I have to loop through tFileFetch until the 'more result' is false.
My approach:
Access token-pagination-tFileFetch_1->JSON->tflowtoIterate->more_result=true->IF ->tFileFetch_2->JSON->tFlowtoIterate->more_result=true->tLOOP [moreresult.equals{true)->tFileFetch_2->
After tFileFetch2 I have used tSetGlobalVar to put pagination as common var to pass to tFileFetch2
I am not sure whether this approach is appreciable or not, please suggest any improvements if any?