Not able to iterate files in Talend Spark using tLoop

[pragna_k]

Is there a way to iterate files in Spark ? Like tfilelist . I know we can give multiple files as input in tfiledelimited input by giving wild card char "*". But my requirement is to iterate each xml file and create corresponding output file with same name. May i know if u have any suggestions on how to achieve this in spark ?

Here is the solution (Screenshot below) that i tried with tLoop which also not working. 

In the below code, context.FileList= concatenated list of files with ",", eg: file1,file2,file3 and so on...

tJava component Code ==>

List<String> FileList = Arrays.asList(context.FileList.split(","));
context.FileListN = context.FileList;

System.out.println("FileList -->"+ context.FileListN);

//String FileListArray[] = context.FileList.split(",");
int listsize = FileList.size();
int i;

globalMap.put("SizeofList", listsize);
globalMap.put("FileList", FileList);

System.out.println("FileList -->"+ globalMap.get("FileList"));
System.out.println("Size of List -->"+ globalMap.get("SizeofList"));

for(i=0;i < listsize; i++){
String filenm = ((List<String>)globalMap.get("FileList")).get(i);
System.out.println("Processing File -->"+ filenm);
Until_ConvertXMLtoJSON_V1.XmltoJson(context.FileDir, filenm);
}

tLoop ==> 

While loop

Declaration == int i=0

condition =i <((Integer)globalMap.get("SizeofList"))

Iteration =i++

Let me know if i am missing anything!

[Anonymous]

Hello,

tLoop component in spark job has the same feature of the DI one, except the parallelization, which does not exist on Spark.

Best regards

Sabrina

[pragna_k]

The tloop component did not work for me. Can you post a solution if u have
with tloop. I want same file names for input and output both.

[badri-nair]

HI, 

Do you have any solution for the same? I have the same requirement hence the same issue .

Thanks

BAdri Nair 

---

@xdshi wrote:

Hello,

tLoop component in spark job has the same feature of the DI one, except the parallelization, which does not exist on Spark.

Best regards

Sabrina

---